Statement of Purpose Essay - University of Washington

I am seeking to pursue a PhD in computer science to build reliable, interpretable systems for natural language processing (NLP). The objective of my undergraduate research has been to create machines that can process language and understand the auditory world with the same accuracy and efficiency as humans. In this direction, I have worked on research projects relating to common sense question answering, interpreting word relationships, audio source separation, and music generation. This has culminated in multiple publications and promising ongoing work. Humans communicate effectively because we assume that other people have immense prior knowledge and reasoning abilities. For example, if a friend says that she discovered that her plant died upon returning from vacation, we can automatically infer that this was due to prolonged dehydration in the friend’s absence. However, this type of commonsense reasoning is often not written explicitly, making it difficult for natural language understanding systems to learn. When large pretrained language models achieved superhuman performance on common sense question answering (QA) benchmarks, my colleagues and I were motivated to construct a dataset that would pose new challenges to these systems. Under Prof. Doug Downey’s guidance, we introduced a novel procedure for question acquisition in which participants submit questions through an online platform and receive the model’s response in real-time, allowing them to explore its areas of weakness. Our dataset was found to be a challenging extension to the SWAG dataset that we built on, and was published in the RepEval Workshop in conjunction with NAACL 2019. In my summer research project with Prof. Downey, I began to explore another interesting phenomenon: people are able to automatically infer the relationship between adjacent nouns, even in instances not encountered before. For example, we can readily interpret the rare “parsley cake” as a cake made of parsley, rather than cake to celebrate parsley (as in “birthday cake”) or cake that is parsley-themed (as in “superheroes cake”). The relationship between these words is subtle and therefore challenging for machines to decipher. Because existing datasets for this task were limited in size and confined to naive prepositional paraphrases, I developed heuristics to identify noun compounds in text with high accuracy, and scraped online encyclopedias to create a novel dataset of 400,000 examples annotated with definitions. I trained a sequence-to-sequence transformer language model on this dataset to generate paraphrases and definitions for noun compounds. I then collected a separate dataset of millions of unlabeled noun compounds from unstructured text, in order to investigate how we can improve the model by obtaining definitions for this dataset. To this end, I developed several effective methods of automatically evaluating model uncertainty on generated definitions, in order to strategically seek human-written definitions for the most difficult noun compounds. I am currently preparing this work for conference submission. My work in NLP has led me to explore how humans process audio with the same ease and intuition with which they understand language. As human listeners, we can automatically distinguish different sound sources in an auditory scene — in a bustling coffee shop we can selectively listen to a friend, and when we move outside we can distinguish the sounds of a car driving by and birds chirping. Moreover, we can switch seamlessly among these different auditory environments. However, existing deep learning algorithms for source separation are unable to do this because they are trained on, and work exclusively on, specific audio domains. Working with Prof. Bryan Pardo and Dr. Prem Seetharaman, we developed a general model for audio source separation that can handle mixtures of unknown sound types. I derived an automatic measure of separation quality based on analysis of the learned embedding space and performed a series of experiments to demonstrate its correlation with ground-truth. I used this confidence measure to mediate among domain-specific models, creating an ensemble model which significantly outperformed each constituent model on general mixtures. I submitted a first-author paper to ICASSP 2020 on this work. Most recently, a classmate and I took a dive into the realm of music generation, which challenges machines to mimic human creativity while respecting the rules and patterns governing composition. While deep learning has become state-of-the-art for music generation, systems trained on specific music domains, such as Bach chorales, are limited by inherently finite training data. Existing dataset augmentation techniques like segmentation and transposition prevent the model from learning musically important properties like long-term coherence or the expressive qualities of each key signature. Therefore, I proposed an alternative method for data augmentation, where we iteratively generate output from the model, filter it by a handcrafted score function, and feed it back to the model as new training data. I applied this method to the domain of Bach chorales, and we designed the score function to capture known voice-leading rules. Preliminary results suggest that training on high-quality generations instead of transpositions of original music improves model performance when evaluated through the human discrimination task. This project has provided an exciting opportunity for me to combine my research interests and hobby in music, and represents a truly independent research experience from conception to execution. While the work arose out of a class project, we are now working with Prof. Pardo to prepare it for conference submission. In thinking about how to automatically evaluate model uncertainty across a variety of tasks in both language and audio, I have become interested in building more models that are capable of estimating their own performance in reliable and interpretable ways. I believe that this is crucial to responsibly deploying systems in the wild — imagine if medical diagnosis systems could recognize an anomalous case and defer to a human doctor, or a summarization system that can determine when its summary is factually incorrect and refrain from publishing falsities. In addition to ensuring deployment-time improvements, reliable measures of confidence would enable models to learn from a teacher in low-confidence settings, prioritize training on more useful examples, and work effectively in an ensemble. Aside from this research, I am eager to learn and work on a variety of exciting work within NLP. I wish to continue my studies at the University of Washington due to its strong NLP program and the diversity of interesting research directions being explored. In particular, I am excited by Prof. Yejin Choi’s work to build natural language understanding systems that are grounded in the rich environments where language occurs. My previous research has given me an intuition of the limitations of learning through traditional textual corpora, and I am excited to leverage structured prior knowledge or information from visual domains. I believe this is necessary for sophisticated language tasks such as commonsense reasoning, where her pioneering work has inspired my own past research (the CODAH dataset built on SWAG). Similarly, I hope to work with Prof. Noah Smith to interpret and evaluate what NLP systems truly learn, and to develop methods that incorporate context and are robust to artifacts. In addition, I am deeply excited by his work investigating how language reflects and reinforces societal institutions. I am also interested in Prof. Luke Zettlemoyer’s innovative approaches to language modeling, and I hope to learn from his experience combining linguistic and computational approaches. In addition to the unparalleled academic opportunities at UW, I am undoubtedly drawn to its location in my hometown of Seattle. Having returned to the city every summer for internships in both industry and research, I am excited to learn from the university’s unique collaborations with IT industries and research institutes. For these reasons, I am confident that UW is a program unlike any others – one where I will truly feel at home, and which I will contribute to whole-heartedly.