Combining expert judgment by hierarchical modeling: An application to physician staffing

Expert panels are playing an increasingly important role in U.S. health policy decision making. A fundamental issue in these applications is how to synthesize the judgments of individual experts into a group judgment. In this paper we propose an approach to synthesis based on Bayesian hierarchical models, and apply it to the problem of determining physician staffing at medical centers operated by the U.S. Department of Veteran Affairs (VA). Our starting point is the supra-Bayesian approach to synthesis, whose principal motivation in the present context is to generate an estimate of the uncertainty associated with a panel's evaluation of the number of physicians required under specified conditions. Hierarchical models are particularly natural in this context since variability in the experts' judgments results in part from heterogeneity in their baseline experiences at different VA medical centers. We derive alternative hierarchical Bayes synthesis distributions for the number of physicians required to handle the (service-mix specific) daily workload in internal medicine at a given VA medical center. The analysis appears to be the first to provide a statistical treatment of expert judgment processes for evaluating the appropriate use of resources in health care. Also, while hierarchical models are well established, their application to the synthesis of expert judgment is novel.