Medicine & Math
Question: If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease, assuming you know nothing about the person’s symptoms or signs?
This seemingly simple problem, when posed to attending physicians, house officers and medical students, yielded concerning results in two studies conducted 35 years apart. In both the original paper by Casscells et al. published in the New England Journal of Medicine in 1978 and a follow-up study published in 2014 in JAMA Internal Medicine by Manrai et al., the answer most commonly given by respondents was 95%.
The correct answer is, however, less than 2% (1.96% to be precise).
While getting to the right positive predictive value can be calculated using Bayes’ theorem, applicable when there is knowledge of prior facts (in this case, the prevalence of the disease in the population) that might be related to the event in question (here, the positive result), it is worth noting the common sense approach to the correct solution laid out in the original paper:
"Only one of 1000 people studied will … have the disease, and 5% of the others (0.05 x 999), or roughly 50 persons (from the 1000 tested), will yield (falsely) positive results. Thus only one of 51 positive results will be truly positive, and the chance that any one positive result represents a person with the disease is one in 51, or less than 2 per cent."
The fact that the broad population can’t calculate or has poor instinct for statistics such as positive predictive value is one thing, but more serious concerns about public health and safety are raised when the main clinical actors in the medical profession are less than proficient in understanding basic statistical tools.