Back to All Essays

Statement of Purpose Essay - Cornell

Program:Phd ML, Responsible AI
Type:PHD
License:CC_BY_NC_SA_4_0
Source: Public Success Story (Rajiv Movva)View Original

**Overview.** AI promises a lot, and I want it to bring everyone forward equitably. Many applications of AI have transformative potential, but predictive systems can also reinforce discrimination. I want to make algorithms fairer by centering the interests of groups that have been historically excluded by technology. I’m interested in quantitative analyses of bias in ML, and how social context mediates real-world algorithmic harms. For my PhD, I’m excited about the following areas of inquiry: - How do intrinsic biases originate in foundational ML models, and how do these biases create downstream representational or allocative harms? - By bringing together technical fairness methods and an intersectional feminist lens, can we co-design algorithms that empower marginalized groups? In short, I’d like to characterize predictive biases in ML, study where they cause harm, and then use these findings to design AI systems for more equitable outcomes. **Motivation.** At MIT, my research interests have gradually shifted from purely technical problems to more society-facing ones. I built research foundations in fields like computational genomics and NLP, and orthogonally, I started to learn more about structural inequities in the US. I began feeling skeptical of the techno-solutionism that was so ingrained in me, realizing that AI doesn’t exist in a vacuum. My Women’s and Gender Studies coursework helped me consider how practical uses of technology might create or reinforce power differentials. I wanted to bring a more critical lens to my machine learning research. For my final project in a Feminist Theory class, I decided to delve into the literature on algorithmic fairness, with particular focus on criminal justice. Most papers boiled down to equalizing a proposed metric across different subgroups, but was that really enough? Once I looked past CS, I found legal scholars and sociologists explaining how people ⁠— especially minorities ⁠— could be exploited even if an algorithm was deemed fair. Some fairness researchers ignored the social context around the data they used to define their metrics, and I wrote an essay sharing this perspective (arXiv). That disconnect stuck with me: as a technologist, I needed to do a better job considering the real-world benefits and harms of my work. Since then, I’ve been working on applying my empirical research style to audit existing algorithms and use technology for justice. **Bias in ML/NLP.** Of course, machine learning doesn’t only pose risk in criminal justice. I’ve worked in NLP, where Large Language Models (LLMs) consistently learn damaging associations about gender and race from large text corpora. These stereotypes can cause direct harm to marginalized users who are trying to interact with a language system, or propagate existing harm by representing minority groups in a negative light. But how is bias actually learned? Past works have identified biased predictions, but it remains unclear how this behavior arises. Datasets certainly play a role: can we alter datasets to reduce representational harm, but without silencing or excluding minority dialects (as has been shown for naive filtering)? Beyond data, how do other aspects of neural network training, like model compression, impact bias? In my talk at the BlackboxNLP workshop (selected as Best Paper), I showed that weight pruning in Transformers significantly alters internal representations, even when accuracy stays the same. Preliminary work finds that pruning may affect subgroup accuracy, and I’d like to investigate this interaction for other common compression techniques in NLP (e.g. knowledge distillation, quantization). I continued to build strong scientific research practices in my work with Prof. Mike Carbin last year, which has prepared me to carry out these types of ML experiments with rigor. **Intersectional Analysis.** Bias is more than a single concept, though: there are different types of bias that a model can display, like predictive disparities or stereotyping. Further, there are several axes of oppression on which bias can act, such as race, gender, or class. Existing audits tend to focus on one manifestation of bias along one axis at a time. I would like to take an intersectional approach, where we study the specific experiences of users with multiple marginalized identities. Black women, for example, may be representationally harmed by ML models due to different stereotypes than those of Black men or white women. Further, intersectional minorities may be affected by more than one type of harm, e.g., occupational stereotypes and toxic hate speech. These issues underscore the importance of expanding our evaluations of bias to consider multiple identifiers and potential harms simultaneously. To capture intersectional specificity, algorithmic audits may be better suited to carefully-designed challenge sets rather than large, general-purpose bias benchmarks. **Participatory Design of AI Systems.** It’s hard to say whether a predictive system is fair or not without social context. Do our academic notions of fairness align with reducing the harms users are most concerned about? To account for lived experience, I’m interested in participatory machine learning, where we explicitly learn from groups that are impacted by an algorithm. I’ve started on this type of work with Prof. Catherine D’Ignazio at MIT’s Data + Feminism Lab. We're working with activist organizations who track gender-based killings (femicides), which tend to be viewed as one-off incidents rather than part of a structural pattern of violence. More comprehensive and centralized logging of victims can help memorialize them and raise awareness, and we've been co-designing NLP models with activists to aid in their victim-finding process (in preparation, FAccT 2022). My modeling decisions have been directly informed by activists’ needs, and the intersectional victim groups that they work with. The project has been a promising look into combining feminist principles with technical methods to build responsible AI. I’d like to expound on this intersection during my PhD, especially by bringing in research methods from HCI to practically assess how algorithms help and harm their users. **Why Cornell?** I appreciate Cornell’s strong commitment to considering the impacts of AI on society, with several faculty combining CS with Information Science and Economics, for example. While my background is mostly in empirical NLP, I see my PhD as a chance to mix theory and practice and enter new disciplines, like law or medicine. I see this interdisciplinary ethos in several Cornell professors, and I’m especially inspired by the research of Emma Pierson, Nicola Dell, and David Mimno. I’d also love to collaborate with neighboring departments (Information Science in particular); Nikhil Garg, Karen Levy, and Tapan Parikh are doing exciting societal and participatory work.