CRCNS US-Israel Research Proposal: NSF-BSF: The Perception of Time and Memorability in the Visual Hierarchy

About This Grant

This project aims to explore how the brain processes time when looking at images, with important implications for understanding memory, learning, and visual perception. The investigators have discovered that memorable images - those more likely to be remembered later - are also perceived as lasting longer in time than forgettable ones. This finding suggests that the perception of time relates to the gathering and remembering of visual information. The research team will conduct experiments using brain imaging, eye tracking, and computer modeling to understand how this process works across different parts of the visual system. This work could lead to better treatments for conditions like Alzheimer's disease, autism, and schizophrenia, where both time perception and visual memory are disrupted. The project will also advance artificial intelligence by creating new computer models that process images more like humans do, incorporating time as a feature that current AI systems ignore. The researchers intend to develop a public database of images with time perception information that can train AI systems to predict how long visual scenes will be remembered by humans, potentially improving a wide range of AI tools including educational software that adapts to how students process visual information. Technically, the project is designed to investigate whether memorable images undergo prioritized processing that dynamically extends temporal windows to maximize information extraction. Using behavioral experiments, electroencephalography (EEG), functional magnetic resonance imaging (fMRI), eye tracking, and recurrent convolutional neural network models, the project intends to pursue three main objectives: determining the extent of memorability's influence on time perception across different image features, mapping the neural mechanisms underlying this interaction throughout the visual hierarchy, and testing whether manipulating time perception can causally alter memorability. The experimental approach includes temporal bisection and reproduction tasks using images from established memorability databases, combined with multimodal neuroimaging to track spatiotemporal dynamics from early visual areas to higher-level timing regions. The project aims to use using recurrent neural networks to predict both memorability and perceived duration, testing the hypothesis that faster processing speeds in neural representations correspond to time dilation effects. The findings informs the development of next-generation AI systems that incorporate temporal processing as a core feature for processing visual information, addressing a critical gap in current computer vision models. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.