Collaborative Research: EAGER: NAIRR Pilot: Demystifying the Black Box: Algorithm-Informed Neural Networks

About This Grant

Today's artificial intelligence (AI) systems are powerful but often operate as opaque "black boxes," making decisions without clear explanations. This lack of transparency limits trust in AI, particularly in critical domains such as healthcare, finance, and autonomous systems, where understanding the reasoning behind decisions is essential. At the same time, decades of research have produced mature, well-established, and theoretically proven algorithms. This project introduces Algorithm-Informed Neural Networks (AINNs), a new approach that integrates these proven algorithmic principles into the design of neural networks. By embedding logical steps into AI architectures, AINNs enhance explainability, reliability, and efficiency, making AI systems more interpretable and reducing their dependence on large datasets. This advance is particularly beneficial in fields where data is scarce or sensitive, such as medical diagnostics or regulatory decision-making. By addressing these challenges, the project contributes to the development of trustworthy, transparent, and efficient AI technologies that can drive scientific progress and benefit society. To achieve these goals, the project is structured around two key research tasks. First, it focuses on algorithm-mapped neural models, which construct neural networks by systematically integrating well-established algorithmic logic. Instead of relying solely on training data, these models leverage predefined logical rules — ranging from pseudocode to flowcharts — to ensure reliability and trustworthiness in AI decision-making. This approach reduces training data requirements while improving generalization and interpretability. Second, the research develops latent behavior analysis of neural blocks, a novel debugging tool that enables AI systems to be systematically inspected for correctness. By analyzing the execution patterns of neural subnetworks, this method detects input-specific anomalies and traces them back to logical inconsistencies, facilitating targeted debugging and improving model robustness. The project will evaluate AINNs across diverse tasks, from algorithmic reasoning to perception-based applications, using key metrics such as data efficiency, error localization accuracy, and generalization performance. Expected outcomes include AI systems with greater transparency, lower data dependency, and enhanced reliability, making them more effective in real-world applications. The project will publicly release datasets, models, and tools to promote broader adoption of algorithm-informed AI across multiple domains. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.