Statement of Purpose Essay - Johns Hopkins

Statement of Purpose Yash Mehta It was only after my first taste of research, close to the end of my undergraduate program, that I really found my spark. During my final year thesis at NTU Singapore, I worked on deep learning-based personality prediction [6-9] with Dr. Erik Cambria, and fell in love with the research process. I had always enjoyed the logic of code, but now I could see the adaptability and broad applicability of the tool. I realized that research was my true calling and left the cushy confines of my position as a software developer at Amazon to dive head-first into a life of research. I aim to become a group leader, working alongside passionate researchers leveraging advances in deep learning to uncover fundamental principles that drive learning in the brain. Fascinated by the basic mechanisms of learning in neural networks, I reached out to Dr. Peter Latham at the Gatsby Computational Neuroscience Unit to work on biologically plausible credit assignment algorithms. At Gatsby (1y 6m) in collaboration with Dr. Tim Lillicrap (DeepMind), I sought to understand the potential of perturbation-based algorithms as candidates for learning in the brain. I created an efficient implementation of the node perturbation algorithm and compared the learning dynamics to backpropagation when the network was trained to perform object classification tasks. With Drs. Latham and Lillicrap, I developed a novel method that leverages the auto-differentiation functionality of modern deep learning libraries, allowing for running large-scale experiments with node perturbation on convolutional architectures for the first time. We also discovered and investigated a unique failure mode during training, where we saw a sudden and difficult-to-reverse collapse in the performance of the network. Our results suggested that node perturbation is unlikely to be fast enough to enable effective learning of some complex goal-oriented tasks that animals perform. We summarized these findings in two papers, one of which was accepted at NeurIPS’22 [4,5]. I also worked on a project on biologically plausible alternatives to convolutional networks which was published in NeurIPS’21 [3]. We showed that adding lateral connectivity and allowing learning via local plasticity enabled these networks to achieve nearly convolutional performance while also improving their fit to ventral stream data. With these results, it became increasingly clear to me that learning algorithms are significantly affected by the underlying neural network architecture. Intrigued by the role of underlying network topology, I wondered if we could learn specific neural architectures that were better suited for certain tasks. Can we make intelligent algorithms that evolve an optimal underlying neural circuit itself? I then joined Dr. Frank Hutter’s Machine Learning Lab (1y 3m) to explore these questions and worked on developing efficient algorithms for automated neural network design. This stint at the University of Freiburg helped me gain a good understanding of the recent advances in deep learning. I created an open-source library that enabled researchers to develop novel evolution-based Neural Architecture Search (NAS) algorithms. It also allowed for fair benchmarking comparisons of various existing NAS algorithms, resulting in a first-author publication at ICLR’22 [2]. To apply my deep learning experience to neuroscience modeling and "close the loop" between model and experimental design, I joined Drs. James Fitzgerald and Jan Funke’s labs at the HHMI Janelia Research Campus (1y). Presently, I am leveraging recent advances in deep learning, specifically, meta-learning, to infer the underlying synaptic plasticity rules solely by observing the neuronal activity trajectories during learning. In silico, I’ve modeled this with an artificial neural network constrained by the Drosophila Mushroom Body connectome. I’m closely collaborating with the experimental labs of Drs. Yoshi Aso and Glenn Turner, and intend to fit the model to recordings from Kenyon Cells (KCs), Mushroom Body Output Neurons (MBONs), and cross-compartment dopamine measurements as flies undergo first-order conditioning. As a first step, I am conducting an initial exploratory study of this approach using simulated data, with the aim of recovering back a known plasticity rule (e.g. Oja’s). To understand the robustness of this approach to noise and sparseness of neuronal recordings, I am collaborating with Dr. Larry Abbott’s lab as a visiting researcher at the Zuckerman Institute. In addition to noise and sparseness, we found that our approach is also robust to other factors, including length of measured trajectory, number of trainable parameters, weight initialization, regularization, and network size. We have submitted these promising results on synthetic data to Cosyne [1]. In both academic and non-academic settings, I enjoy teaching, mentoring, and collaborating with others. During my undergraduate studies, I was the head Teaching Assistant for the Design and Analysis of Algorithms course where I was responsible for creating and teaching tutorials for a class of 250 students. At the University of Freiburg, I volunteered as a Teaching Assistant for the Deep Learning Lab and Deep Learning courses. Together with Dr. Frank Hutter, I designed and created a new Attention and Transformers course module from scratch. I also co-supervised 2 Msc. thesis and 3 research technicians. Presently, I teach an introductory machine learning class at a local high school close to Janelia, that I helped design. I believe science is best done when researchers teach and learn from each other, building fruitful interdisciplinary insights. I’m excited by the vast potential of the interdisciplinary field of “NeuroAI”. During my graduate studies, I would like to make strides toward reverse engineering the brain to uncover the fundamental principles that underpin learning and sensorimotor intelligence. By garnering a deeper understanding of biological learning, I believe we can bring machine-learning algorithms closer to those in the brain – leading to better AI systems as well as more powerful theoretical paradigms to understand human cognition. My goal is to work at the interface of academia and industry to construct fruitful collaborations while taking an active role in mentoring the next generation of scientists. I’m confident that the Cognitive Science Programs’s focus on interdisciplinary approaches in concert with my active collaborative style and enthusiasm for research would provide an environment where I would thrive. I would be honored to leverage my prior experience in biologically plausible credit assignment to work with xxxx on modeling-based approaches to understand neuronal circuits and algorithms implemented by the visual system. I am intrigued by xxx‘s work on theories of neural computation and believe it would be promising to develop novel network models and simulations to test them empirically. Given my experience with Drosophila connectome-constrained modeling, I would like to work with xxx to uncover links between variations in circuit features and variations in behavior. [1] Mehta, Y., Tyulmankov, D., Aso, Y., Turner, G., Fitzgerald, J., Funke, J. (2023). Model Based Inference of Synaptic Plasticity Rules from Neural Activity. Cosyne (under review) [2] Mehta, Y., White, C., Zela, A., Krishnakumar, A., Zabergja, G., Moradian, S., ... & Hutter, F. (2022). NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easy. International Conference on Learning Representations (ICLR) [3] Pogodin, R., Mehta, Y., Lillicrap, T., & Latham, P. E. (2021). Towards biologically plausible convolutional networks. Advances in Neural Information Processing Systems (NeurIPS) [4] Hiratani, N., Mehta, Y., Lillicrap, T., & Latham, P. E. (2022). On the Stability and Scalability of Node Perturbation Learning. Advances in Neural Information Processing Systems (NeurIPS) [5] Mehta, Y., Hiratani, N., Humphreys, P., Latham, P. E. & Lillicrap, T. (2022). On the Limitations of Perturbation-Based Methods for Training Deep Neural Networks. (in preparation) [6] Mehta, Y., Majumder, N., Gelbukh, A., & Cambria, E. (2020). Recent trends in deep learning-based personality detection. Artificial Intelligence Review [7] Mehta, Y., Fatehi, S., Kazameini, A., Stachl, C., Cambria, E. and Eetemadi, S., Bottom-up and top-down: Predicting personality with psycholinguistic and language model features. In 2020 IEEE International Conference on Data Mining (ICDM) [8] Li, Y., Kazameini, A., Mehta, Y. and Cambria, E., 2021. Multitask learning for emotion and personality detection. Neurocomputing [9] Mehta, Y., Stachl, C., Markov, K., Yun, J.T. and Schuller, B.W., 2022. Future-generation personality prediction from digital footprints. FGCS