PsychologyOrbit
Sign in Join

#11 ·

Spot the flaw #11: Ranking two depression clinics

A past puzzle, with the answer. Try today's puzzle.

What is the flaw in this study?

Simpson's paradox. Clinic A does better than Clinic B for severe cases and for mild cases, yet worse overall. This reversal happens because A treats far more severe patients, who recover less often wherever they are treated. The overall rates mostly reflect the case mix, not…

A fictional study written for practice (Spot the flaw #11).

Fictional study Clinical · Dhaka, Bangladesh

The study

A health ministry review in Bangladesh compared two district clinics that each treated 1,000 adults for depression in a year. At Clinic A, 600 patients (60%) recovered; at Clinic B, 700 (70%). The appendix gives more detail. Clinic A, which takes referrals from across the division, saw 700 severe cases and 300 mild ones; Clinic B saw 200 severe and 800 mild. Among severe cases, 50% recovered at A and 40% at B. Among mild cases, 83% recovered at A and 78% at B. The ministry ranked Clinic B the better clinic and plans to train Clinic A's staff in Clinic B's methods.

What is the flaw in this study?

  • A Ceiling effect
  • B Simpson's paradox The flaw
  • C Demand characteristics
  • D Order effects
Simpson's paradox. Clinic A does better than Clinic B for severe cases and for mild cases, yet worse overall. This reversal happens because A treats far more severe patients, who recover less often wherever they are treated. The overall rates mostly reflect the case mix, not the quality of care. When groups differ on a factor that strongly affects the outcome, comparisons should be made within levels of that factor, or adjusted for it, here by severity. On the figures given, Clinic A's methods look at least as good, so the ministry's plan has it backwards.

The concept: Aggregated data can reverse a trend

This study is fictional, written for practice. No real study, institution or researcher is described.