PsychologyOrbit
Sign in Join

#16 ·

Spot the flaw #16: A classroom behaviour programme

A past puzzle, with the answer. Try today's puzzle.

What is the flaw in this study?

Raters not blind to condition. The people rating behaviour were the same teachers who delivered the programme and knew which children had taken part. They had invested ten weeks in it and may, without meaning to, notice good behaviour in "their" group and overlook incidents, or interpret…

A fictional study written for practice (Spot the flaw #16).

Fictional study Educational · Chicago, US

The study

In 24 elementary classrooms in Chicago, children were randomly allocated within each class either to a ten-week behaviour programme, delivered by their class teacher in weekly small-group sessions, or to usual classroom activities. In total 480 children took part and the two groups were similar at the start. At the end of the ten weeks, each class teacher completed a behaviour checklist for every child in the class, recording disruptive incidents over the final fortnight. Children in the programme had 30% fewer recorded incidents than their classmates. The authors conclude that the programme reduces disruptive behaviour and recommend it for all elementary schools.

What is the flaw in this study?

  • A No random allocation
  • B Survivorship bias
  • C Base-rate neglect
  • D Raters not blind to condition The flaw
Raters not blind to condition. The people rating behaviour were the same teachers who delivered the programme and knew which children had taken part. They had invested ten weeks in it and may, without meaning to, notice good behaviour in "their" group and overlook incidents, or interpret the same behaviour differently. Expectations like these can create a difference on their own. A stronger design would use observers who do not know each child's group, or objective records such as office referrals. Allocation was random and the groups were similar at baseline, so randomisation is not the weakness.

The concept: Blinding outcome assessors

This study is fictional, written for practice. No real study, institution or researcher is described.