CAREER: Modeling Team Situational Awareness and Designing Adaptive Interfaces to Support Multi-human-multi-agent Teams

About This Grant

There is a growing demand for technologies like semi-autonomous unmanned craft (drones) to augment human capabilities for infrastructure inspection, disaster management, and search and rescue operations. To realize their potential, such technologies must integrate seamlessly into teams of multiple people interacting with multiple drones. Effective collaboration between people and semi-autonomous drones requires team members to be aware of surroundings, the placement and activity of other team members, anticipating what may happen next, assessing potential risks and hazards. In short, team members must be able to accurately perceive the situation in order to make informed decision about actions --- so-called situational awareness. This project aims to understand team situational awareness and develop computer-implemented tools that support real-time team situational awareness for tasks that involve both people and semi-autonomous drones. By fostering a comprehensive understanding of team dynamics, the project will significantly benefit practitioners by enhancing team collaboration and coordination, thereby improving efficiency and safety. The project will address education and workforce development by collaborating with government, industry, and community stakeholders to create new course modules, K-12 workshops, public safety training programs, and initiatives promoting broad participation, This project will develop an integrated framework to understand, predict, and support team situational awareness in multi-human-multi-agent teams operating in uncertain and dynamic environments. The project has three thrusts. Thrust 1 focuses on identifying optimal information for operators in specialized roles across diverse team structures and environmental settings. Participatory design and simulation studies will determine what information to display and how to present it, offering insights into how information types and team structures influence team situational awareness. Thrust 2 will establish novel metrics and computational models for team situational awareness. It will integrate cognitive theory and machine learning models, incorporate neural response data, and employ uncertainty-guided training to improve performance. Thrust 3 will design adaptive interfaces to guide operators to critical but overlooked information for individual situational awareness and visualize real-time shared situational awareness. These interfaces will improve situational awareness, enhance team collaboration and effectiveness, and help realize the promise of semi-autonomous agent technologies. Comprehensive evaluations in simulator and field studies will assess interface usability, team situational awareness, and performance impacts. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.