Statement of Purpose Essay - University of Southern California

Motivation The effectiveness of neural networks (NNs) has allowed for “deep learning” to have far reaching applications in fields such as drug discovery in pharmacology, diagnosis in medicine, computer vision in defense, etc. However, there is a gulf between the incredible practical performance and the fairly weak theoretical guarantees of NNs. It is not well understood how to best analyze NNs, why certain training methods allow NNs to “generalize” outside of their training set well, or how a NN implicitly reasons when placed within a decision-making context like reinforcement learning. When applying AI systems more broadly to critical domains like healthcare or defense, the existence of theoretical guarantees addressing these issues may be paramount for trust to exist between users and systems. Research Career Goal My research goal is twofold: (1) I will create novel ML algorithms which retain theoretical optimality guarantees while offering performance competitive to similar applications of NNs; (2) I will propose and implement improvements motivated by recent theoretical advancements to systems utilizing NNs. Both directly ameliorate well-described pitfalls of machine and deep learning by introducing a theoretical component to improve and broaden the use of ML and AI. To achieve these goals, I will first obtain a PhD in Computer Science (CS). In addition to receiving valuable mentorship, graduate study will allow me to build knowledge and intuition for the theoretical tools which I intend to create and apply while improving ML and AI systems. To prepare for graduate study, I am in the UT Dallas (UTD) CS honors program which readies a group of 30 students for research through theoretical courses. I am also pursuing a second degree in math, and was the only undergraduate in a graduate course on optimization in ML. After my PhD, I will enter into academia or a government lab to lead a team of researchers describing new theoretical tools for learning guarantees in ML/AI while simultaneously teaching the subject at the university level. Research Background: Theoretically Grounded Algorithms In order to prepare myself for my goal (1), I have participated in a variety of research projects allowing me to study and apply useful theoretical techniques from online learning, optimization, and CS theory. The summer before my senior year, I participated in an NSF REU with Prof. Brendan Juba from Washington University in St. Louis on reinforcement learning (RL). A fundamental result in RL theory is an upper bound on the learning speed of an agent, typically in terms of the number of states that the RL environment may take. Yet many practical RL application domains—such as resource allocation tasks or the board game “Go”—have an exponential number of distinct states, which indeed means that learning could take intractably long. Our work presents a practical, theoretically grounded approach for RL in these large and difficult application domains. Previous approaches assume the existence of a potentially impossible general polynomial time “oracle planner” for a family of exponential sized environments. In the case the long-term values of states can be described by a basis of linear functions, we construct this oracle by showing fast convergence of a cutting plane method with large input, and prove competitive polynomial time learning bounds. Working remotely (COVID-19), I started the project with a graduate student and Prof. Juba and met with them twice a week to discuss directions and proofs. In addition to learning effective habits of theoretical collaboration over many hours of discussion, I independently studied, derived, and recorded over 30 pages of proof details. I also wrote a majority of our submitted AISTATS 2021 paper, gaining valuable feedback in framing theory to be accessible and useful to the broader scientific community. I have also collaborated with Prof. Nicholas Ruozzi and Prof. Benjamin Raichel at UTD to bridge the gap between classical and deep RL with optimization and computational geometry. We are developing a method for fitting convex functions over a convex hull of data observed during the Q-learning RL algorithm. We may be able to achieve high performance and show correctness guarantees which do not exist for all NN approaches, as our function behaves well everywhere (unlike NNs). I extended existing and developed new convex optimization procedures, and have also coded many prototypes testing our various approaches. Working on this problem for almost two years has allowed me to develop the grit necessary for the theoretical work that my research career will focus on. I recently obtained exciting preliminary RL results that further spurred my motivation and passion for the project, and am now preparing competitive results for publication. Research Background: Theory and Application In line with my goal (2), I have pushed myself to participate in a variety of interdisciplinary application domains where NNs are commonplace to gain practical insight and apply improvements rooted in theory. The summer before my junior year, I interned at the Johns Hopkins Applied Physics Labs (APL), implementing recent deep learning methods to classified computer vision defense projects. Collaborating with APL research scientists, I learned firsthand the difficulty in using deep learning for applications where reliability was paramount, since NNs infamously do not provide performance guarantees. This reaffirmed to me the importance of theoretical approaches. In my second year of university, I joined the UTD Advanced Networks Research Lab (ANRL) under the supervision of Prof. Jason Jue to apply theory and ML to various networking problems. With a graduate student, I worked on a project distributing NNs layer-wise over computing nodes in such a way that inference accuracy could still be retained during partial node failure. Motivated by recent theoretical work, I proposed, prototyped, and implemented a system for NN layer distribution using a NN training technique we term failout. Failout incorporates simulated inference-time environment failure within NN training. We showed that this empirically allows NNs to be resistant to true inference-time layer failure. Eventually, our work appeared at ICML 2020 Workshop on Federated Learning for User Privacy and Data Confidentiality, and is currently under submission at AAAI 2021. In addition, I also worked on a resource allocation problem called “progressive recovery”. After failures such as natural disasters or cyber-attacks, we want to recover communication networks in a prioritized order. With a graduate student and Prof. Jue, we first proved general progressive recovery is NP-hard which resulted in a conference publication at IEEE Globecom. Next, we proved that in many practical cases, optimal orders can be found by searching a smaller solution space. This allowed me to apply deep RL and obtain state-of-the-art results on US network topologies, leading to a journal publication in IEEE Journal on Selected Areas in Communication (JSAC), issue on AI/ML for Networking. Lastly, an ANRL graduate student and I are currently using online convex optimization (OCO) to allocate resources in a network bandwidth management problem in a way that is fair to each customer. I proposed utilizing OCO, and spent many hours in theoretical discussion analyzing fairness in an online setting. This revealed to me the importance of online algorithms and motivated me to continue researching them in my future work. CS @ USC I am excited to work with multiple professors within the theory and ML groups at USC, including Prof. Haipeng Luo, Prof. Vatsal Sharan, and Prof. Mahdi Soltanolkotabi. I would like to work on online learning, reinforcement learning, and bandit problems with Prof. Luo. With Prof. Sharan, I am interested in working on theoretical concepts within optimization and ML more generally, especially in the limited data setting. For example, I am particularly interested in Prof. Sharan’s most recent work on theoretically grounded sample amplification: I have seen numerous heuristics employed in practice but would be excited to further the theoretical foundations of algorithms in this setting. Finally, I am interested in researching the foundations of deep learning, theoretical properties of non-convex optimization procedures, and learning theory more broadly with Prof. Soltanolkotabi. I believe that my focused, high impact research goals combined with my strong history of theoretical collaboration will allow me to effectively contribute to research within USC’s growing ML and theory groups.