CAREER: Robust Machine Learning via Principled Defenses against Adversaries and Distribution Shifts

About This Grant

Machine learning (ML) systems have achieved impressive capabilities across domains such as vision, language, and planning. However, even state-of-the-art models can fail dramatically in unpredictable ways, including under minor shifts in data distribution. These failures are exacerbated in the presence of adversaries (i.e., other actors or systems that attempt to undermine or attack the ML system) These vulnerabilities present serious concerns for deploying ML in high-stakes settings. This project aims to develop principled defenses that make robustness a core design property of ML systems rather than an afterthought. The approach is to bridge rigorous analysis with practical experimentation in order to understand, predict, and ultimately fix brittleness in modern models. Through new algorithmic tools and conceptual insights, this work will lead to the development of reliable and trustworthy AI systems that can operate safely and reliably in complex, real-world environments. This project will establish a framework of robustness via analysis for building machine learning systems. This approach interleaves simplified theoretical models with real-world experiments to derive actionable insights that improve robustness in practice. The project proceeds along three technical thrusts. First, it develops robust finetuning techniques for large pretrained models (foundation models) by minimizing forgetting and preserving generalization across domains. Second, it introduces defenses against adversarial attacks on language models, including novel finetuning paradigms that enable models to robustly self-correct and consistently enforce safety guidelines, even under complex and varied jailbreak scenarios (malicious attempts to trick the system into generating undesirable output). Third, the project studies robustness in autonomous AI agents composed of multiple subsystems, identifying weakest links, developing methods to enforce trust hierarchies (i.e. what can be trusted and when), and introducing mechanisms to provide formal safety guarantees across the system. Together, these efforts aim to shift the foundations of robust ML from heuristic patchwork to theoretically informed, systematically validated design. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.