Intelligent Simulation-based Learning About Natural Disasters

About This Grant

While simulations are powerful tools for scientific inquiry, most students need scaffolding to engage productively in simulation-based inquiry. This project will develop and study an automated feedback system designed to support middle school students' simulation-based inquiry into wildfires, floods, and hurricanes. The system, called Hazbot, will leverage advanced artificial intelligence (AI) technologies--including machine learning and large language models (LLMs)--to provide timely, personalized feedback as students investigate the three different natural hazards. Hazbot will guide students to collect, analyze, and interpret data from simulations and develop scientific arguments based on that data. Hazbot will also synthesize the automated performance diagnosis and feedback information provided to students and offer teachers targeted instructional suggestions to support individual students and the whole class. The project will research the automated scoring methods, the automated feedback system, the combinations of teacher facilitation and automated feedback needed to support students' simulation-based inquiry, and the impact of Hazbot-integrated wildfire, flood, and hurricane modules on student learning outcomes. The materials generated through design and development will be made available for free to all future students, teachers, and researchers beyond the participants outlined in the project. ISLAND (Intelligent Simulation-based Learning About Natural Disasters) is a five-year Level III Design and Development project aimed at advancing middle school students' understanding of wildfires, floods, and hurricanes--and their ability to construct evidence-based arguments about these hazards--through simulation-based inquiry supported by automated feedback. The project will design a fully integrated AI-enhanced two-tier pedagogical agent to (1) diagnose student performance in simulation-based scientific inquiry and respond in real time to their evolving needs and (2) support teachers by synthesizing student learning in an actionable teacher dashboard. In the first three years, the project will employ design-based research to develop and integrate the Hazbot system into three modules in collaboration with 9 teachers and their 900 students across geographically and demographically diverse schools. This phase will investigate how Hazbot's automated scoring models capture students' simulation-based inquiry behaviors and performance; how its feedback supports students in collecting, analyzing, and interpreting data and constructing evidence-based arguments; and what combinations of teacher facilitation and automated feedback are most effective. In the final two years, the project will conduct three randomized controlled trials (RCTs)--one for each hazard module (wildfires, floods, and hurricanes)--to measure the impact of the Hazbot system on students' understanding of natural hazards and risk, as well as their ability to construct scientific arguments. These RCTs will involve a nationally recruited sample of 72 teachers and 3,600 students with half of the teachers randomly assigned to implement a Hazbot-integrated version of the module and the other half implementing the same module without Hazbot integration. This project will generate critical insights for designing LLM-based feedback systems that can (1) be trained to uphold disciplinary standards, (2) systematically scaffold simulation-based inquiry, and (3) integrate meaningfully with teachers who bring valuable contextual insights to classroom implementation. This project is supported by two NSF programs: the Discovery Research preK-12 (DRK-12) program, which aims to significantly improve the learning and teaching of science, technology, engineering, and mathematics (STEM) for preK-12 students and teachers through research and development of innovative resources, models, and tools; and the Innovative Technology Experiences for Students and Teachers (ITEST) program, which supports projects that advance understanding of the practices, program elements, contexts, and processes that foster students' knowledge of and interest in STEM disciplines and information and communication technology (ICT) careers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.