Modeling muddles: Validation beyond the numbers

A nonnumerical methodology for estimating the validity of complex models is presented. The validity of such models was estimated qualitatively by identifying the arguments used by the interested parties associated with the model. The patterns of reasoning about the foundations of the model were idenitfied through an analysis of the arguments defending and denying the validity of the model. The approach is illustrated by using the expected utility (EU) model as an example. The analysis revealed contrasting sets of pro-EU and con-EU model arguments. A fundamental difference in the forms of arguments was observed: The pro-EU model used broad arguments covering a variety of issues (top-down) and the con-EU model used specific empirical results to question the fundamental grounding of EU model theory (bottom-up). The approach holds promise for large complex models that are difficult, it not impossible, to validate quantitatively.