Collaborative Research: CISE Core Small: HCC: Towards Accessible and Safe Human-VPA Interaction in Smart Home Environments

About This Grant

Virtual Personal Assistants (VPAs) such as Amazon Alexa, Google Assistant, and Apple’s Siri are essential in modern smart homes due to their ease of use and ability to perform a variety of tasks. Some challenges of VPA use include 1) difficulty for deaf or hard-of-hearing users who cannot rely on auditory commands; 2) difficulty in understanding speech patterns that VPAs do not recognize; and 3) safety risks associated with children potentially accessing inappropriate content. This project will develop novel technologies and mechanisms to enhance the usability and safety of human-VPA interaction in smart home environments. This research also supports broader social goals such as engaging STEM students in the development of advanced algorithms, improving computer science curricula at participating institutions, and strengthening awareness of accessibility and safety issues in technologies. This research project proposes a suite of innovations to improve usability and safety in human-VPA interactions. First, this project develops an end-to-end system that enables users to interact with VPAs using sign language as input, effectively bypassing voice-based interaction barriers. Second, this project proposes a novel acoustic testing tool to identify usability issues caused by VPA’s that are not trained on certain speech patterns. Finally, this project develops a situational multi-user access control mechanism that allows primary users (e.g., parents or homeowners) to specify access control policies for non-primary users (e.g., children or guests) for safe human-to-VPA interactions in multi-user smart homes. The work will provide insights on design principles for usable and safe human-VPA interactions. The research is integrated with education and outreach efforts including the development of curriculum modules, research experiences for undergraduate and graduate students, and public engagement activities to disseminate findings and raise awareness about usable technology design. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.