Collaborative Research: RI: Small: Towards Assured Robust Intelligence in Multi-Agent Systems with Runtime Safety

About This Grant

Autonomous agents like warehouse robots, sidewalk-operating delivery robots, drones, and robo-taxi fleets are being increasingly deployed in the real world. These agents incorporate artificial intelligence (AI)-driven sensing and estimation, and planning and control. The real-world performance of these agents, however, is far from satisfactory. It is not uncommon to see a delivery robot being confused by an obstacle in its path on the sidewalk, a drone failing to achieve its objectives, or a robo-car driving irratically. Despite significant advances in AI-based design for such agents, there remain critical challenges in making them robust for performance in the real world. Firstly, autonomous agents must obey operational rules (e.g., traffic rules) and maintain safety at all times. Secondly, these agents rely on various sensors, such as cameras, radar, and proximity sensors, which provide only a partial or imperfect observation of the state of the environment. Thirdly, since such agents rarely operate in isolation, they need to coordinate with other autonomous agents or humans. Finally, to ensure robust and resilient performance upon deployment, it is vital to develop methods for runtime monitoring and adaptation for these agents. This project aims to address these critical challenges for achieving robust performance by autonomous agents in the real world. The research conducted in this project can significantly impact the science of safety-assured autonomy with potential applications in autonomous transportation systems, robotics, and smart manufacturing systems. This project will bring together formal methods, reinforcement learning, and multi-agent control theory to develop a scalable framework for high-assurance design of safety-critical and mission-critical cooperative interacting agents. The research conducted in this project will aim to develop model-based and model-free algorithms for control synthesis using data generated in high-fidelity simulators to optimize a performance criterion subject to the satisfaction of specifications expressed in signal temporal logic. The logic will capture safety requirements, operational constraints, and complex environment behaviors. This project will also develop control algorithms for an autonomous agent operating in a mixed-agent environment where human agents may be present, and for teams of cooperative autonomous agents with system-wide objectives and specifications. It will investigate the design and analysis of robust offline and online monitoring algorithms for high-assurance design under uncertainty. It will also aim to design scalable, hierarchical verification algorithms that leverage data and a new formal model based on graphs of assume-guarantee specifications to reason about system correctness and safety both at design-time and runtime. The research in this project will contribute to the foundations of robust intelligence for multi-agent systems that can operate in complex, real-world environments. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.