Internal and external validity: what are they and how do they differ?
Posted on 4th May 2023 by Gabriela Negrete-Tobar
Is this study valid? This is a question that should be asked when looking for reliable evidence. For you to answer it correctly, you must understand how to find the elements of validity in medical research. Additionally, these elements change according to the type of study that is being analysed.
Let’s use an example to bring this concept to life
Your aunt has mild bilateral knee osteoarthritis (OA). She complains of pain and can’t run due to it. According to her doctor, her main treatment plan consists of physical therapy and pain management. Aquatic cycling was offered to her at the gym as a method of pain control for patients with OA so she wants to know if she should sign up for it.
You start your search and find a single-blinded parallel group randomized control trial. It was independently funded and set at a University Medical Centre in a European country. It aims to compare an Aquatic cycling (AC) supervised 12-week plan of 45-minute sessions twice a week vs a Usual Care (UC) plan consisting of individualized physical therapy and mobility aids.
Now let’s look at the trial’s design in more detail
The study’s participants consisted of patients diagnosed with knee OA with mild-moderate pain and indication for conservative treatment. Participants were excluded if they had received corticosteroid or hyaluronic acid injections 3-6 months prior or had pain or OA in any other joints. Researchers used a randomizing software to allocate participants into either Aquatic Cycling (AC) or Usual Care (UC). Originally, 111 patients were randomized and 55 were assigned to AC compared to 56 into UC. However, out of the UC group 9 patients dropped out after baseline as they didn’t want to undergo physical therapy. It is important to note that the researchers that conducted allocation and professional assessments were blinded. In contrast, participants were not blinded and received information on their treatment plan before starting the trial.
During the trial the follow-up time was 24 weeks. Primary outcomes of the amount of pain and knee mobility were measured using the self-reporting Knee Injury and Osteoarthritis Outcome Score (KOOS) questionnaire. Equally important, physical therapists applied the lower extremity functional scale, as well as evaluated pain after 6-minute walk and asked for fear of injury/re-injury, as secondary outcomes. Significantly, analysis of all the outcomes was made using the intention-to-treat principle.
What were the authors’ conclusions?
They reported information from 90 participants. These participants had a mean age of 59 years, were predominantly overweight women, and in general both groups had similar pre-trial symptom severity. They reported that in the AC group 80% attended all 24 sessions. During the trial’s follow-up time they lost 51% of participants in the AC group due to comorbidities and other unknown reasons. Additionally, 18% presented with exacerbations of pain after the first AC session as an adverse effect. Despite these reports, it was not clear in the analysis if these adverse effects altered the reported outcomes.
In comparison the UC group participants were free to consult a physical therapist according to their symptom severity, but only 32% did. Compared to the AC group, the UC participants didn’t report adverse effects. Lastly, as it was promised by the researchers, after the follow-up period ended the UC participants received complementary AC therapy.
Finally, they concluded that a 12-week training program of AC improved knee pain and physical functioning in patients with osteoarthritis.
What is Validity?
Validity looks at how accurate the measurement made in a study was. It assesses if the information reported reflects the effects of an intervention on the sample group, and it evaluates if it is useful for people of similar characteristics. It is divided into internal and external validity.
So, is this trial valid? To answer you can follow the mnemonic: BOAS bite CALF PAIRS
Internal validity
Internal validity shows if the study truthfully reports the intervention’s effect on the selected group of participants. It is influenced by the methods used to reduce the risk of bias and chance.
External validity
External validity looks at a study’s clinical usefulness and applicability on a group of patients. In other words, can we generalise the findings from this study to other contexts?
Hopefully, boas don’t bite calf pairs, however this mnemonic can assist you in remembering the elements of validity. Do you think that this study is valid? Also, is Aquatic Cycling beneficial for your aunt? You can add your comments and thoughts in the comments below!
Acknowledgments to Dr. Claudia Granados-Rugeles for her contributions on this tutorial.
References (pdf)
You may also be interested in the following blogs to help you understand some of the concepts mentioned in the blog:
A beginner’s guide to confounding
Blinding: taking a better look at the blind side
Allocation concealment: the key to effective randomisation
Sample size: a practical introduction
Many of the blogs from our ‘Key Concepts‘ series will be useful to you as well. These help explain concepts that people may need to understand to assess treatment claims.
