PsychologyOrbit
Sign in Join

#8 ·

Spot the flaw #8: Memory training for older adults

A past puzzle, with the answer. Try today's puzzle.

What is the flaw in this study?

Ceiling effect. Almost everyone scored 19 or 20 out of 20 before training even started, so there was no room left to improve. A test this easy cannot show a benefit even if one exists, and the null result says more about the measure than the training. The study needed a…

A fictional study written for practice (Spot the flaw #8).

Fictional study Cognitive · Chennai, India

The study

At community centres in Chennai, 140 adults aged 65 to 80 were randomly allocated to eight weeks of computerised memory training or to eight weeks of crossword puzzles. Before and after, everyone took a memory test: they saw 20 everyday words, and a minute later picked each one out from a pair. Both groups averaged between 19.0 and 19.4 out of 20 on both occasions, and the groups did not differ after training. Attendance was above 90% in both groups. The authors conclude that the training gives older adults no memory benefit and that centres should not spend money on it.

What is the flaw in this study?

  • A Multiple comparisons without correction
  • B Survivorship bias
  • C No random allocation
  • D Ceiling effect The flaw
Ceiling effect. Almost everyone scored 19 or 20 out of 20 before training even started, so there was no room left to improve. A test this easy cannot show a benefit even if one exists, and the null result says more about the measure than the training. The study needed a harder outcome with room at the top, such as delayed free recall of a longer list, ideally piloted first so scores spread out. The design itself was sound: allocation was random and attendance high, so those are not the problem.

The concept: Measures too easy to detect change

This study is fictional, written for practice. No real study, institution or researcher is described.