Carryover effects: What are they, why are they problematic, and what can you do about them?

Posted on 23rd August 2018 by Ludvig Daae Bjorndal

Carryover effects are challenging for within-subjects research designs, that is, when the same participants are exposed to all experimental treatments and results are compared across different treatments. These designs are useful, among other reasons, because they allow a near perfect match of subject characteristics as the researcher compares measurements of the same participant. However, as the same subjects are used in all experimental treatments, there is a possibility that a previous treatment can alter behaviour in a subsequent experimental treatment. This is known as a carryover effect.

Why are carryover effects problematic?

Carryover effects will confound the effect of an independent variable and destroy the near perfect matching that is so valuable in within-subjects designs. The internal validity of the experiment is greatly reduced. In other words, because a previous treatment has altered observed behaviour in a later treatment, the extent to which the research design tests the hypothesis it is designed to test is reduced.

Imagine a study where you examine the effect of a noisy environment on the ability to solve mathematical equations. Subjects first solve mathematical equations in a noisy environment and then in a quiet environment. If you observe an improvement in performance it is possible that this is due to the lack of noise. It is, however, also possible that the improvement is due to practice; perhaps the subjects in the second treatment have “warmed up” or are more focused. Put simply, it is possible that the earlier experimental treatment (noisy environment) affects performance in the second treatment (quiet environment) through practice; and not due to lack of noise per sé.  This muddies the evaluation of how a noisy environment affects mathematical ability, as, at the very least, some of the improvement is due to carryover effects.

What are sources of carryover effects?

There are lots of possible sources for carryover effects – relevant or irrelevant depending on the particular research setting and design. Learning is a common source of carryover effects (as mentioned in the previous example). If treatments lead to fatigue, this can also create carryover effects. Other possible sources are (among others) habituation, sensitization, and adaptation.

What can you do about them?

There are a few strategies for dealing with carryover effects:

• Minimize and eliminate: Obvious, but important nonetheless. Pre-training is one example, if you’re not interested in the actual effects of learning.
• Counterbalancing: This is an important method for reducing carryover effects. The researcher assigns the experimental treatments in different orders for different subjects. You can use either complete counterbalancing (every possible treatment order is assigned to at least one subject) or partial counterbalancing (includes some of the possible treatment orders). The goal is to spread the carryover effects across the different experimental treatments, so that the average difference between the groups is unaffected by them. The idea is that the carryover effects within the different treatments will cancel each other out. In order to achieve complete counterbalancing you’ll usually need lots of subjects for statistical power, so partial counterbalancing is often your best bet.
• Making treatment order an independent variable: This is another option, where again, different subjects are exposed to different orders of treatments. However, enough subjects are allowed in each group so that a statistical analysis of the treatment order as a separate, independent variable, can be done. This allows for a measurement of the size of the carryover effects.

Carryover effects can affect outcomes and results of research, and are important to consider, particularly in the design phase of a study. They can pose a threat to the validity of results and therefore have to be minimized.

References

Bordens, K. S. & Abbott, B. B.: Research Design and Methods A process approach, 2011. McGraw-Hill. ISBN: 0072887648.

Tags: