Don’t assume a single study is enough.
Don’t jump to conclusions based on just one study about the effects of a health action (something people do for their health). One study alone may not give us the full picture, and we usually need more than one study of a health action to be sure about its effects.
Explanation
Considering the results of all the fair comparisons of a health action can help us be surer about the effects of the health action. When we look at many similar comparisons together, for example, in a systematic review or meta-analysis, we might find results that are different to what a single study showed.
There are several reasons why single studies can be misleading. Single studies are often too small or contain too many errors to provide reliable results. Furthermore, the results of studies that address the same question often have inconsistent results and studies that show benefits, especially large benefits, are more likely to be noticed.
Even when we consider many similar comparisons, it’s important to remember that we might not have access to all the relevant studies. Some studies might not have been published in research journals, or they may be incomplete or hard to find. So, we need to be cautious when interpreting the results and keep in mind that there could be missing information.
Example
Diarrhoea is a leading cause of childhood death in middle- and low-income countries. Children with diarrhoea lose body water and sometimes become dehydrated. A solution of sugar and salt dissolved in water (called oral rehydration solution or ORS) is widely used to treat dehydration caused by diarrhoea. For more than 20 years, the World Health Organization (WHO) recommended a standard ORS with a large amount of sugar and salt mixed in water. However, some researchers believed that it might be better to use a smaller amount of sugar and salt and they found that the results from comparisons varied. Only by considering the results of 11 fair comparisons of ORS with larger and smaller amounts of sugar and salt, did researchers find that a solution containing smaller amounts of sugar and salt was better for treating dehydration in children. Based on the combined results of all 11 comparisons, the WHO changed its recommendation.
Remember: The results of a single study of a health action can be misleading. Consider all the relevant fair comparisons when deciding what to believe about the effects of a health action.
Primary school
- Lesson 7. Fair comparisons with many people. In: The Health Choices Book: Learning to think carefully about treatments. A health science book for primary school children.
- Animation: 7: Fair comparisons with many people – The Health Choices Book
Other
- Podcast: Ep. 8 of The Health Choices Programme: “Single comparisons”
- Podcast: Episode 3: Single Comparisons. In: Parents Making Informed Health Choices.
- Book: Assessing all the relevant, reliable evidence. In: Testing Treatments.
- Book: Summaries of all the best quality randomized trials. In: Smart Health Choices.
- Video: Not all scientific studies are created equal This is a TEDEd video lasting 4:27 minutes.
- Blog: Consider all the relevant fair comparisons. Students 4 Best Evidence.