Could you help us review these resources?

Posted on 23rd December 2016 by Selena Ryan-Vig

Confusing claims about the effects of treatments – from drugs to diets – are everywhere. People need to know how to assess these claims, using fair comparisons to inform their treatment choices. Below is a list of 34 key concepts for assessing treatment claims. By clicking on one of the links below, you will be taken to a list of learning resources taken from The Critical thinking and Appraisal Resource library (CARL). (Big thanks to one of our partners, Testing Treatments interactive for these!) These learning resources help to explain and illustrate the key concept.

We need your help evaluating these resources! If you’d like to get involved, we suggest that you pick a particular concept first and then work your way through the related resources. You need not have ever blogged before and you can use the questions at the bottom of this page as a guide.

1. Claims: are they justified?

1-1 Treatments can harm

1-2 Anecdotes are unreliable evidence

1-3 Association is not the same as causation

1-4 Common practice is not always evidence-based

1-5 Newer is not necessarily better

1-6 Expert opinion is not always right

1-7 Beware of conflicting interests

1-8 More is not necessarily better

1-9 Earlier is not necessarily better

1-10 Hope may lead to unrealistic expectations

1-11 Explanations about how treatments work can be wrong

1-12 Dramatic treatment effects are rare

2. Comparisons: are they fair and reliable?

2-1 Treatments should be compared fairly

2-2 Comparison groups should be similar

2-3 Peoples’ outcomes should be analyzed in their original groups

2-4 Comparison groups should be treated equally

2-5 People should not know which treatment they get

2-6 Peoples’ outcomes should be assessed similarly

2-7 All should be followed up

2-8 Consider all of the relevant fair comparisons

2-9 Reviews of fair comparisons should be systematic

2-10 All fair comparisons and outcomes should be reported

2-11 Subgroup analyses may be misleading

2-12 Relative measures of effects can be misleading

2-13 Average measures of effects can be misleading

2-14 Fair comparisons with few people or outcome events can be misleading

2-15 Confidence intervals should be reported

2-16 Don’t confuse “statistical significance” with “importance”

2-17 Don’t confuse “no evidence” with “no effect”

3. Choices: making informed choices

3-1 Do the outcomes measured matter to you?

3-2 Are you very different from the people studied?

3-3 Are the treatments practical in your setting?

3-4 How certain is the evidence?

3-5 Do the advantages outweigh the disadvantages?

Questions to think about when reviewing these resources…

Roughly how long did it take you to read/view/complete the resource?

What did you think of the resource? (e.g. was it clear or difficult to understand?)

Do you feel it improved your understanding?

What problems did you find with it (if any) / how do you think it could be improved?

Overall, what would you score the resource out of 5? (and why?)

If you’ve looked at all of the resources for a particular concept, which is your favourite  resource, and why?

Tags: