Comparing diagnoses from expert systems and human experts

This paper discusses a comparison of one heuristic and two Bayesian belief network based expert systems used to aid veterinarians in the process of differential diagnoses of equine diseases where coughing is the presenting clinical sign. Each implementation infers the likelihood of the presence of a number of diseases based on information on the presence or absence of certain clinical signs. The Bayesian belief network approaches are similar except that one includes the use of prior information in the form of disease prevalence estimates. Both are implemented using the Hugin software package. The three approaches were compared using test cases and the lists of resulting diagnoses were examined for agreement using a measure of concordance. The results indicated a difference between the heuristic approach which used the rule-based scoring mechanism and the Bayesian systems. There was, however, little difference between the diagnoses produced by the two Bayesian implementations, indicating that the incorporation of prevalence data makes little difference in diagnostic systems of this type. The findings were also compared with those of clinical experts. The analysis indicated that clinicians were not always in agreement. Moreover, using the same set of test cases the experts were more in agreement with the Bayesian approaches than with the heuristic approach.