Sample size: A practical introduction

Posted on 27th September 2017 by Saul Crandon

Overview

The sample size is the proportion of the general population that are taking part in the study. In most cases, it’s important that the sample chosen is representative of the wider population, so that any conclusions drawn from the study can be reasonably extrapolated to individuals who did not directly take part.

Sample size

There are a number of different methods for calculating sample size, with many alternate factors that influence these. Nevertheless, the common features include:

• The significance level
• The power of the study
• The predicted effect size

Significance level

A significance level of P < 0.05 is universally accepted in most research fields. The American Statistical Association define the p-value as “the probability under a specified statistical model that a statistical summary of the data (for example, the sample mean difference between two compared groups) would be equal to or more extreme than its observed value”.

Power

Statistical power is the likelihood that a study will detect an effect when there is an effect there to be detected. It is calculated as:

Power = 1 – β error (type 2 error)

A type 2 error is failing to detect a significant difference when there actually is one. This usually occurs when the sample size is too small.

In other words, power is the ability to avoid a false negative result, i.e. correctly rejecting the null hypothesis when it is actually false. If statistical power is high, the probability of making a Type II error, or concluding there is no effect when, in fact, there is one, goes down. So, the higher the power level the better. However, a power level of 80% is generally acceptable in most clinical research studies. Other areas of research will have different standards of power levels.

Predicted effect size

The statistics involved in calculating a specific sample size can become quite complex. Part of this calculation involves inputting a ‘predicted effect size’. The smaller the predicted effect size you wish to obtain, the larger the sample must be. It is vital that the predicted effect size used in the calculation is properly justified. As this figure is user-generated, it can be manipulated by researchers to achieve a sample size figure that they desire. To combat this, predicted effect sizes should be based on a multitude of sources, including previous high quality evidence and clinical experience.

Additional considerations

It is also important to consider the expected drop-out or death rates in the relevant research field. Therefore, the calculation will produce a buffer of extra participants to account for any inevitable loss of numbers. Again, the exact drop-out and/or death rates should be based on previous studies.

Sample size calculations must always be performed before the start of the study to eliminate any bias or deviation from study protocols. Another factor to consider is also practicality. In the real world sample sizes are, more often than not, influenced by administration limitations, costs as well as available resources.

Why should I bother calculating my sample size?

One of the main reasons that studies should be powered, and sample sizes should be calculated is ethics. It is unethical to subject patients to a particular experimental intervention if the study was not adequately powered to be able to detect any significant difference in the first place. Not only does this expose the participants to potential unnecessary harm, it is a significant detriment to both costs and resources.

Summary and links to power calculators

Sample size calculations are generally required for any well designed clinical study. The implementation of these calculations should be based on acceptable p-values, power and a justified effect size. Calculating this figure by hand can be quite arduous for the average researcher. As a result, there are a number of online calculators that can be useful. Below are some links:

http://powerandsamplesize.com

http://www.raosoft.com/samplesize.html

http://www.danielsoper.com/statcalc/category.aspx?id=19

https://www.surveymonkey.co.uk/mp/sample-size-calculator/

http://www.nss.gov.au/nss/home.nsf/pages/Sample+size+calculator

References

1. Bhalerao S, Kadam P. Sample size calculation. International Journal of Ayurveda Research. 2010;1(1):55. Available from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2876926/
2. Nickson, C. Power and Sample Size Calculation [Internet]. LITFL. Life In The Fast Lane Medical Blog. 2017. Accessed 24/09/2017. Available from https://lifeinthefastlane.com/ccc/power-and-sample-size-calculation/

Tags: