Innovating Measurement of Youth Social Media Activity: An Approach for Using a Large Language-Based Computer Vision Model to Study What Youth Are Doing on Social Media and Links to Mental Health

About This Grant

Project Summary/Abstract A rise in rates of depressive symptoms and self-injurious thoughts and behaviors (STBs) and youth using social media over the past decade have raised alarms about the potential role of social media (SM) in youth mental health. However, SM platforms include opportunities for a wide range of SM activity, such as creating content, viewing content, and direct messaging, etc., and youth are able to transition rapidly between potentially detrimental and beneficial SM activity over the course of minutes. Theoretical and empirical work suggest that engaging in some types of SM activity, such as viewing self- or peer-related posts, may contribute to negative SM experiences such as heightened emotional responses to SM, digital feedback seeking, and social comparison and increase risk for depressive symptoms and STBs. On the other hand, some types of SM activity, such as direct messaging, may be associated with positive SM experiences such as social connectedness and decreased risk for depressive symptoms and STBs. Yet, prior research on youth SM use is characterized by methodological limitations such as retrospective, self-reported, and general measures of time that are weakly associated with objective measures and do not capture the range of potentially negative and positive types of SM activity. One recent methodological approach, referred to as screenomics, addresses some limitations of prior research by capturing high frequency screen images to obtain an objective, comprehensive record of youth SM activity in real time. However, a screenomics approach is not yet available at a large scale, given that it yields up to thousands of screen images per participant, and studies have relied on humans to manually code each image. The overall goal of this project is to address methodological gaps in the field of SM and youth mental health research by developing and validating a novel approach for using artificial intelligence computer vision models that can be used to advance objective, ecologically valid, and nuanced measurement of detrimental and beneficial youth SM use in real time as it naturally occurs in daily life. Specifically we aim to (1) develop an approach that uses a multimodal large language model–based computer vision model to accurately identify types of SM activity from youth SM screen images and (2) investigate the validity of objective measures of SM activity obtained from screen images by showing that these measures uniquely predict, over and above daily self-report measures of SM activity, negative and positive SM experiences, depressive symptoms, and STBs (i.e., investigate incremental and predictive validity). To accomplish project aims, we will focus on a popular SM platform (Instagram). We will (1) consult a youth research advisory board to help inform study procedures and (2) recruit 50 youth (ages 14–16) with a range of depressive symptoms to submit daily screen recordings of Instagram sessions and surveys measuring self-reported SM activity, SM experiences, depressive symptoms, and STBs over 21 days total (three 7-day periods spaced 1 month apart).