Collaborative Research: SaTC: CORE: Small: Detecting and Localizing Non-Functional Vulnerabilities in Machine Learning Libraries

About This Grant

This project aims to improve security and resilience of machine learning (ML) software. Machine learning has been deployed in many critical domains such as drug discovery, financial planning, autonomous driving, and malware detection. This makes it crucial for ML-based software solutions to function properly even when attacked by malicious actors, leading to a line of research focused on functional vulnerabilities, attacks that attempt to make ML systems produce incorrect results. Less studied, however, are other kinds of vulnerabilities that don't attack the core prediction functionality but still pose security risks. These "non-functional" vulnerabilities include denial of service attacks, which attempt to render the system unusable through overloading it; and side-channel attacks, which analyze features like response time to infer sensitive information about the models or data they are trained on. This project will develop methods for detecting and correcting these kinds of non-functional vulnerabilities and make those methods widely available, as well as disseminate educational materials to help security researchers and ML software developers be more aware of these risks. Despite a growing number of reported denial-of-service (DoS) and side channel (SC) vulnerabilities in core ML libraries such as NumPy and TensorFlow, a systematic approach to identifying and debugging them has not been explored due to multiple technical challenges: i) non-functional behaviors are not explicitly encoded in the syntax or semantics of ML code; ii) existing fault localization methods often fail to establish causal relationships; and iii) automatic DoS/SC mitigation is largely lacking for ML applications. This project will develop a novel methodology that combines evolutionary algorithms with a gradient-based guidance to detect DoS and quantify the strengths of SC vulnerabilities. For debugging, the project explores causally guided statistical methods to localize the root causes and guide an optimal mitigation policy. The project team will make a concerted effort to increase participation of women, Hispanic, and other underrepresented communities via special topic courses, research experiences for undergraduates, and summer camps for K-12 students. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.