Don’t confuse “statistical significance” with “importance”

This is the twenty-ninth blog in a series of 36 blogs explaining 36 key concepts we need to be able to understand to think critically about treatment claims.

“Statistical significance” is often confused with “importance”. The cut-off for considering a result as statistically significant is arbitrary, and statistically non-significant results can be either informative or inconclusive.

Práticas comuns nem sempre são baseadas em evidências

This is a Portuguese translation of the fourth in a series of 36 blogs explaining 36 Key Concepts we need to be able to understand to think critically about treatment claims. With thanks to Guilherme de Mattos Queiroz and Cochrane Brazil for the translation.

This blog explains that treatments that have not been properly evaluated but are widely used or have been used for a long time are often assumed to work. Sometimes, however, they may be unsafe or of doubtful benefit.

Confidence intervals should be reported

This is the twenty-eighth blog in a series of 36 blogs explaining 36 key concepts we need to be able to understand to think critically about treatment claims.

The confidence interval is the range within which the ‘true difference’ is likely to lie, after taking into account the play of chance. Whenever possible, consider confidence intervals when assessing estimates of treatment effects. Do not be misled by p-values.

Fair comparisons with few people or outcome events can be misleading

This is the twenty-seventh blog in a series of 36 blogs explaining 36 key concepts we need to be able to understand to think critically about treatment claims.

When there are only few outcome events, differences in outcome frequencies between the treatment comparison groups may easily have occurred by chance and may mistakenly be attributed to differences between the treatments.

Blinding: taking a better look at the blind side

Blinding is a common element used in rigorously designed trials. Most people are familiar with the general concept but what is its purpose and what is the best way to perform it? This blog by Neelam Khan explores both of these questions and discusses ways to tackle situations where blinding cannot be done.

Average measures of effects can be misleading

This is the twenty-sixth blog in a series of 36 blogs explaining 36 key concepts we need to be able to understand to think critically about treatment claims.

This blog explains that for outcomes measured on a scale (e.g. weight, or pain) the difference between the average in one treatment group and the average in a comparison group may not make it clear how many people experienced a big enough change (e.g. in weight or pain) for them to notice it, or that they would regard as important.