Consider the relevance of the health actions that were compared.
When deciding about health actions, the findings from studies may not apply because the health actions compared in the studies differ so much from the available options that the findings from the studies are not relevant.
Explanation
Characteristics of health actions, including for example the length of treatment, the dose, how the treatment is given (e.g., by mouth or through a vein), and skill of the person delivering the treatment can influence the effectiveness of a treatment. Unfortunately, characteristics such as these are often poorly described, especially when the treatment is a health action that involves more than taking a drug. This can make it difficult to judge the relevance of health actions compared in studies to the actual health actions that might be taken. For example, a study of the effects of a surgical procedure done in a specialised hospital or delivered by an experienced practitioner may not provide a reliable estimate of its effects and safety in other settings, or in the hands of less experienced practitioners.
Similarly, comparing a new drug to a placebo or another drug that is not commonly used would not provide a relevant estimate of how the new drug compares to commonly used drugs. Many drug trials are funded by pharmaceutical companies who typically choose to compare their drugs to a placebo rather than to another drug. When there is more than one effective treatment available, people often need to decide which treatment to use, not whether to use a particular treatment or a placebo. In addition, studies of new drugs often measure substitute outcomes rather than outcomes that are important to people.
When direct comparisons of treatments are not available, indirect comparisons (across studies), or findings from studies of similar treatments from the same class of drugs are sometimes used. In both these situations, the findings can be misleading.
Example
Results from a large randomized trial showed that a surgical procedure, called endarterectomy, to treat narrowing of the large arteries in the neck reduced the chance of having a stroke. However, the trial only accepted surgeons with a good safety record, rejecting 40% of applicants. The results of the comparison suggested a much lower number of deaths and strokes than when the same surgical procedure was carried out by surgeons outside the trial.
Remember: The treatments (health actions) that are available may differ so much from those in the research studies that the findings from the studies may not be relevant.
- Blog: Are the treatments practical in your setting? Students 4 Best Evidence.