PsychologyOrbit
Sign in Join

#1 ·

Spot the flaw #1: A mindfulness app before exams

A past puzzle, with the answer. Try today's puzzle.

What is the flaw in this study?

No control group. Stress was measured once before exams and once after they were over. Most students feel calmer when exams finish, app or no app, so the fall could be nothing more than the natural course of the term. Without a comparison group of similar students who did not…

A fictional study written for practice (Spot the flaw #1).

Fictional study Clinical · Lahore, Pakistan

The study

Researchers at a university in Lahore invited 120 second-year students to use a free mindfulness app for ten minutes a day during the four weeks before their final examinations. Students completed the Perceived Stress Scale at the start of the four weeks and again in the week after their last paper. All 120 students finished both questionnaires. Average stress fell from 24.1 to 18.6, a drop that was highly significant on a paired t-test (p < .001), and 81% of students said they found the app easy to use. The authors conclude that the app reduces exam stress and recommend that the university provide it to every student from next term.

What is the flaw in this study?

  • A No control group The flaw
  • B Ceiling effect
  • C Simpson's paradox
  • D Survivorship bias
No control group. Stress was measured once before exams and once after they were over. Most students feel calmer when exams finish, app or no app, so the fall could be nothing more than the natural course of the term. Without a comparison group of similar students who did not use the app, measured at the same two points, there is no way to tell how much of the change the app caused. A randomised design, with some students given the app and others a waiting list, would answer the question. The analysis itself (a paired t-test) was appropriate.

The concept: Why a study needs a control group

This study is fictional, written for practice. No real study, institution or researcher is described.