Statistical Methods for Integrating Irregularly Collected Longitudinal Multi-Modal data into Prediction Models

About This Grant

PROJECT SUMMARY For individuals with chronic illnesses such as diabetes, heart failure, cancer, or obesity, early intervention prevents symptom escalation, acute care use, and mortality. The growing availability of longitudinal electronic health record (EHR), patient-reported outcome (PRO), and mobile health (mHealth) data, including passively collected accelerometer, smartphone, and sensor data, offers new opportunities for proactive intervention. However, the rapid expansion of routinely collected mHealth data has outpaced the research community's ability to interpret it effectively. In particular, the high dimensionality and irregular collection of longitudinal EHR, PRO, and mHealth data introduces key challenges for predictive modelling due to planned sparsity or unplanned missingness. Current methods fall short in three key areas: 1. Informative missingness: Data gaps often carry predictive signal, but are typically treated as nuisance, obscuring meaningful patterns in their timing and duration. 2. Loss of intra-day detail: Fine-grained mHealth data are often reduced to pre-specified daily or weekly summaries, discarding rich intra-day information with potential predictive value. 3. Population heterogeneity: Models trained on populations often perform poorly for underrepresented groups and fail to generalize to individuals, especially when only limited data are available per person. To address these gaps, we propose developing a robust methodological framework for predictive modelling using irregularly collected EHR, PRO, and mHealth data that improves upon imputation-based standard of care methods. In Aim 1, we will develop univariate and multivariate longitudinal models that account for delays between predictors and outcomes and incorporate detailed intra-day mHealth patterns using distributional learning with low-dimensional, near-lossless embeddings. In Aim 2, we address population heterogeneity by personalizing prediction through an embedding-based approach using landmark multidimensional scaling (MDS) and transfer learning, with reweighting of MDS landmarks to improve performance for underrepresented subgroups. In Aim 3, we validate these methods across diverse mHealth and EHR datasets, including NIH All of Us, UK Biobank, and other disease-agnostic and -specific retrospective and prospective datasets, using mixed-methods studies among clinicians to operationalize model outputs for clinical decision support. Though broadly applicable to multi-modal longitudinal data of all types and a range of disease settings, we focus on four chronic conditions with high clinical impact: cancer, congestive heart failure, diabetes, and obesity. The success of this project and its open-source tools will help close a critical methodological gap and enable effective use of multi-modal longitudinal data to improve clinical decision-making for chronic disease management.