Statement of Purpose Essay - University of Southern California

Statement of Purpose Qinyuan Ye “You can't connect the dots looking forward; you can only connect them looking backwards.” said Steve Jobs in his famous commencement speech. Now looking back at my undergraduate study, I have experienced the excitement of connecting dots many times, from lectures, literature, and most importantly, from my own research experience. It leads me to the thought of pursuing a PhD degree, to create the missing edges between dots in the boundless ocean of knowledge, in the field of natural language processing, text mining and their application to social network analysis. The idea of connecting dots first struck me in a lecture when the speaker said, “Exploration and exploitation in reinforcement learning can be regarded as some sort of trading, as related to finance.” I was surprised, a little confused, and all of a sudden, I grasped the idea behind this. What I felt during this process was extraordinary. The same thing happened when I read in the latest conference proceedings about how stochastic gradient descent with momentum was mathematically equivalent to PID tuning in control theory. However, in those examples I was guided by others. The greater joy came from my own research experience, where I created the links independently. My first research project was to analyze large-scale financial data, predict the performance of individual stocks, and construct a portfolio to minimize risks. We were working on top of a widely-used financial framework called multi-factor modeling, which consists of feature engineering and linear regression. I tried to improve the performance in two separate ways. The first was to select informative features. I implemented some features according to technical reports from industry and tested their significance. The second was to adopt advanced machine learning models to replace simple linear regression. It was the first time that I got hands-on experience with advanced machine learning models such as boosted trees and LSTMs. Surprised at how powerful machine learning can be, I also learned about how every detail matters in the implementation of these algorithms. This summer, intending to explore more aspects that machine learning can cover, I joined INK Lab at University of Southern California as an intern and did a systematic study on distantly supervised relation extraction, a task in natural language processing. What surprised me at the beginning was that there also exists a type of feature-based models for this task. Even though natural language does not have the numerical nature as financial data do, and thus uses embeddings instead to map words into continuous space, I still see connection between the two totally different fields. Basic skills such as ablation study and significance test, apply to both cases. In this project, our team mainly focused on an interesting observation that state-of-the-art neural models designed for clean data (annotated manually), may only yield performance comparable with simple feature-based baselines on noisy datasets (annotated with distant supervision). Digging deep into the details of implementation, I found that performance could be greatly boosted with a heuristic threshold trick. Unveiling the mechanism behind threshold, we further related this observation to shifted label distributions between train and test set, which is common due to the labeling process of distant supervision. While lots of efforts were made on reducing label noise in previous work, we pointed out label distribution is an important yet long-overlooked issue, and discussed several methods to mitigate its influence. Our work is currently in submission, and I am the co-first author. My undergraduate thesis will be on the topic of identifying relations between news events, and analyzing their diffusion process in social networks. This is a topic I chose with careful consideration. At that time, I was already quite interested in natural language processing. I also wanted to see how NLP could be applied to real-world applications. Current research in information diffusion over social networks is rarely context-aware. I believe this project itself is very meaningful, and I can utilize my experience in my previous project, as I will also be dealing with huge text corpora in lack of labels. Later when I was reading related literature, I found out that the link is more than I expected. I was amazed at how word embedding in natural language processing is similar to user embedding in social networks. In language, it is co-occurrences that link the words together; in social networks, it is user interactions that link the users together. They can both be formulated into a graph structure. Algorithms will then embed the nodes to represent their characteristics. I am certainly not the first one to find out about this, but I am still very excited since I discovered this connection by myself. My background may seem diverse, but I won’t consider it as a disadvantage. On the one hand, a broader sense in different fields will increase my chance of “connecting the dots” in the future. On the other hand, pursuing a PhD degree is a lasting commitment, and I have to determine what my interests really are before I take the next move. I once consulted a professor about how to find one’s “true love” in research. He suggested that I should try different things, and make decisions based on what I feel. So here I am, with my past diverse research experience, determined to devote myself to natural language processing, text mining and their application to social network analysis. I arrived at this conclusion because I came to realize that one of the most primitive yet amazing achievements of human intelligence is natural language. Enabling machines to understand natural language is an indispensable step towards ultimate machine intelligence. Holding this belief, I feel motivated to explore more about this field. It’s more like a calling, instead of an interest, to a student like me who witnesses the rise of artificial intelligence and wants to make a difference. I’m particularly interested in the following directions. (a) Low resource natural language processing: A majority of current methods rely heavily on large-scale annotated data. However, we don’t have enough data of minor languages or domain-specific text, nor do we have the time and energy to label large-scale corpora manually. Advances in unsupervised and weakly-supervised methods can help utilize unlabeled data. Solutions of this fundamental problem can be applied to generic scenarios. (b) Knowledge acquired from and applied to NLP: It is knowledge accumulated and passed from generation to generation that distinguishes human intelligence from artificial intelligence. Consider what sparks will be lighted when the two intelligences work together. Relation extraction with distant supervision, which I’ve already worked on, is a perfect example that utilizes existing knowledge base, and also generates reliable new entries for knowledge base. Introducing knowledge to NLP is still at an initial stage with blanks to be filled. (c) Bridging NLP and social science: Interdisciplinary study is always fascinating. Data-driven methods will infuse fresh energy to traditional methods in social science. They will also bring new insights to traditional understandings of our society. What I’m anticipating is that NLP becomes more than just “processing”. It can, and will be a powerful tool for some greater good, which in this case is to help social science study such as social network analysis. There’re several professors at University of Southern California whose research interests align with mine well: Professor *** (weakly-supervised methods in NLP) and Professor *** (low-resource information extraction). I am also interested in Professor ***’s work on statistical text analysis and social network analysis. I believe with their guidance and support, as well as the lively cultural and academic atmosphere at USC, I will be given the great opportunity to work on meaningful research projects, and enjoy the journey by “connecting the dots” along the way.