Collaborative Research: Sharp Identification and Specification Testing in Potential Outcome Models: A Computational Approach

About This Grant

Research using observational data and natural experiments relies on statistical analysis to provide reliable results. This project develops new methods to help data analysts test hypotheses about the causes of observed outcomes. The team improves statistical methods in a practical way that can be widely adopted by researchers, business analysts, policy analysts, and others who want to isolate the effects of changes in business and/or government methods, policies, and regulations. This award funds development of (a) computationally simple methods for sharp identification of causal parameters, (b) good estimators for the bounds on partially identified parameters, (c) computationally reliable methods to derive identifying restrictions, and (d) translational research through a publicly available code library that implements the methods and makes these advances available to the broad community that uses statistical tools to conduct program evaluation. The research advances knowledge by developing a unified framework for identification, counterfactual prediction, and specification analyses for potential outcome models through two subprojects. The first subproject uses a new approach, based on random set theory, to bound counterfactuals of interest in a class of potential outcome models. Crucially, this approach avoids computing the sharp identified set for the joint distribution of potential quantities, which is often intractable. The team obtains simple closed-form solutions in several well-studied settings where the bounds have previously been expressed through high dimensional linear programs or intractable optimization problems. The second subproject derives sharp testable implications of the modeling assumptions in a class of potential outcome models. So far, such testable implications have been studied case-by-case in a limited set of models. Using a novel graph-based representation of the model, the team provides a systematic way of deriving sharp testable implications of commonly used identifying assumptions. The research achieves broader impacts through those who conduct empirical research and program evaluation via a translational research component. The team provides practitioners with an accessible “guided tour” of the existing results, focusing on implementation. The guide discusses which of the available approaches (moment inequalities, support functions, linear programs) leads to the most tractable description of the identified set and provide guidance on estimation and inference procedures. Furthermore, the PIs develop a Python library associated with the guided-tour paper and the subprojects described above. The library is accompanied by “hands-on” tutorials hosted on a GitHub repository. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.