How likely is it that a run of poor outcomes is unlikely?

Clinical audit of surgical outcomes often involves examination of the number of poor outcomes from a relatively short sequence of the most recent operations performed. This is potentially misleading since, given a long sequence of operations over a number of years, it may be likely that a short term run of poor outcomes would occur by chance. Calculating the probability of this is not straightforward particularly if the risk of poor outcome is different for each operation in the sequence. A method is described for estimating the probability of occurrence of a run of poor surgical outcomes within some larger sequence of operations. Naturally, the longer the sequence of operations, the higher the probability of encountering such a run of poor outcomes. In addition to providing a useful audit tool, this raises an issue with substantial policy implications. Given the desire to promote quality standards, surgeons and surgical units are increasingly subject to audit of their outcomes. The question arises of how best to judge that outcomes have been so poor that remedial action is necessary. Given the large number of operations that are performed, there is a relatively high probability that apparently poor runs of outcome will be encountered purely as a chance occurrence. There is a need to balance the rate at which such false alarms occur against the need to detect poor performance. The results described present a method for estimating false alarm rates if audit is based on examining relatively short sequences of outcome.