Counterintuitivity

Counterintuitive Predictions

1. True or False. What occurs in the world is not always reasonable, logical, or right. Even so, it might be true. You’ll decide whether the Premises below are True or False.

2. Reasonable or Unreasonable. People act for reasons other than logic; among them sympathy, loyalty, hope, fear, vested interest, greed, and ineptitude. You’ll decide whether the Premises below are Reasonable or Unreasonable.

3. Right or Wrong. Decisions based on logic or reason can be ethical and moral, unethical or immoral. You’ll decide whether the Premises below are Good or Bad, Ethical or Unethical, Moral or Immoral.

Your Predictions

Before hearing about the Mammogram team at Kaiser Permanente Hospital or reading the associated article, respond in three ways to the Premises below.

First: declare whether the statements made appear to be True or False (you could also answer Likely or Unlikely).

Second: declare whether the statements appear to be Reasonable or Unreasonable (or if you prefer: Batshit Crazy, or Not Insane).

Third: Declare the statements’ moral or ethical position to be Good or Bad. (If the statement doesn’t permit a moral judgment, you could still pronounce it a Good thing, or fundamentally Just Wrong.)

Respond in three ways for each Premise.

1. Likely / Batshit Crazy / Bad
2. False / Reasonable / Good
3. Unlikely / Unreasonable / Wrong
4. True / Not Crazy / Right

Of course, in paradise, the Reasonable would always be True and Good, and the Crazy would always be Untrue, and universally recognized as Bad. But we know better, don’t we? At the end of class, return to your predictions. How many of your expectations were met?

The Article

Mammogram Team Learns from its Errors

The Premises

1. Women who find out how many cancers their doctors miss in routine mammograms stop getting mammograms.
2. Radiologists who perform mammograms are held accountable for the accuracy of their readings.
3. A doctor who finds hundreds of tumors in a year and a half, but who misses 10, is almost always fired.
4. Doctors who read only a few mammograms a month are removed from film-reading teams so that they read none at all.
5. Publishing the failure rates of radiologists improves their accuracy to the best the discipline can achieve.
6. The best technique for improving diagnosis accuracy has been adopted by almost no radiology departments.
7. Congress demands that radiologists be held accountable for their accuracy at detecting tumors in mammogram films.
8. The 20,000 US doctors who read breast X-rays are trained to do so; their accuracy is known and tested.
9. The medical profession accepts that, to varying degrees, all doctors make the same mistakes.
10. Doctors who do mammographies follow up with those patients to discover whether their diagnoses were correct.
11. Doctors appreciate knowing whether they missed actual tumors or misread the “shadows and swirls” of a mammogram as a tumor.
12. The “shame” of confronting an incorrect diagnosis is a valuable teaching tool for doctors who diagnose cancers from mammograms.
13. An accuracy rate of 80% in detecting cancers from mammograms is something to brag about.
14. The best doctor to head a radiology department is a squeamish physician who trained as a lawyer and prefers not to deal with patients “and their blood.”
15. Radiology can be tracked well statistically because patients either have tumors or they don’t.
16. When the director of the radiology department discovers a way to improve the accuracy of cancer diagnoses, his method is immediately embraced by hospital administrators.
17. When New York hospitals began to publish their surgeons’ heart surgery successes and failures, the death rate fell by 40%.
18. The falling death rate meant that heart surgeons were doing more careful work.
19. Hospitals that reduce their false diagnoses proudly advertise that they “make 20% fewer errors” than their competitors.
20. Publishing the error rates of mammography radiologists results in an uncertain but significant number of cancer deaths in women who avoid testing.
21. A radiologist who is known to have missed a tumor is likely to have missed a dozen out of 3000 he declared to be tumor-free.
22. Out of those 3000, when 250 were scanned again, and 30 were biopsied, 10 were found to have cancers he had missed.
23. Finding those 10 cancers was reported as a front-page medical scandal instead of a triumph of an enlightened new technique for avoiding missed diagnoses.
24. Many of the 250 women who were told they needed followup were angry.
25. Of the ten whose cancers were missed by the first doctor but discovered in followup screenings, most sued the hospital for malpractice.
26. The doctor who missed the 10 tumors felt he had been treated unfairly, that only 3 of the cancers could be blamed on him, and that his error rate was acceptable.
27. After being fired, he was hired as a fill-in radiologist in five states bordering North Carolina.
28. The radiologists on the terminated doctor’s team supported him, not the hospital, and resent having their work scrutinized and their failure rates published.
29. While some doctors read 14,000 films a year, and others fewer than 500, failure rates are very similar.
30. Doctors who read just 500 films a year get re-assigned to other work since their sample size is too small to determine their accuracy.
31. Doctors who are “fired” from film reading based on low volume are relieved to have the diagnostic responsibility taken from them.
32. Doctors would rather bring a patient back for a second look or a biopsy than miss a tumor.
33. Doctors are much happier to find evidence on the film of a cancer that has “been around for awhile.”
34. Routinely experiencing the shame of missed diagnoses in tests every four months builds confidence in radiologists.
35. Most hospitals send out lists of actual missed tumors or “false negatives” to their radiologists every year so they can study the films they misinterpreted.
36. The Kaiser Permanente department has learned to detect various “presentations” of tumors on film by studying films of actual missed tumors after the fact.
37. In North Carolina, for every two cancers radiologists find, they miss one.
38. If the results at Kaiser Permanente were replicated nationwide, better than 80% of cancers would be found and 10,000 more cancers would be correctly detected each year.
39. False positives are easy to track, but almost nobody tracks false negatives (missed tumors that show up in later mammograms).
40. There is no routine followup for women who, on the basis of their mammograms, are determined to be tumor free.
41. Holding radiologists to a higher standard of competency results in reduced access to quality care.
42. Making failure rates public increases the likelihood of malpractice claims, which in turn drives up insurance rates, which in turn drives good doctors from the field.
43. Having two doctors instead of one review every film improves accuracy and drives down costs.
44. A nationwide 70% effectiveness rate is considered the best that can be achieved practically and politically.
45. Government oversight of physician performance to standardize techniques nationally has actually reduced accuracy.
46. Dr. Adcock, who improved effectiveness in his radiology department by 25%, took himself off the team when his volume dropped.
47. The most conscientious doctors, who agonize over the presence or absence of tumors on every film, are by far the most effective.
48. When they have a choice, women are best served by the doctors who send the largest percentage of women for biopsies because they miss the fewest cancers.
49. The best indicator of whether a doctor is competent to read mammograms is the number of times she’s been sued.
50. A good day for mammograms is Mother’s Day, when many clinics offer free or discounted exams.

The Blind Summary.

When you finish classifying the claims for Truth/Reasonableness/Goodness, in one paragraph, try to summarize the article you have not read. Use the heading “BLIND SUMMARY.”