The Delphi Technique

Posted on 15th November 2017 by Kristen Dufresne

What is it?

The Delphi technique (also referred to as Delphi procedure or process), is a method of congregating expert opinion through a series of iterative questionnaires, with a goal of coming to a group consensus. In fact, in 150 studies that used the Delphi technique, there was no universally agreed upon working definition of the technique. There are many variants used, some of which have departed widely from the original Delphi technique.

Since its development in the 1950’s by the RAND Corporation, several refinements and modifications have been made, such as specific strategies for different fields, including business, government, and healthcare.

There are four characteristic features of the Delphi technique that distinguish it from other group decision making processes. These are: anonymity, iteration with controlled feedback, statistical group response, and expert input.

When is it used?

The Delphi Technique can be an especially useful research methodology when there is no true or knowable answer, such as decision-making, policy, or long-range forecasting. A wide range of opinions can be included, which can be useful in cases where relying on a single expert would lead to bias.

The Delphi technique was recommended as the method of choice when:

• subjective statements made on a collective basis are desired;
• face-to-face interaction is difficult due to large sample sizes;
• anonymity is preferred, such as when issues are intractable or political; and
• there is a chance of domination of a group discuss by one person.

How is it done?

1. Survey Development
   • Define the research problem/questions and develop the first-round survey
   • Pilot the survey with a small group to ensure the responses will elicit appropriate answers to the research question
   • Round one is like a ‘brainstorming’ round, and allows participants to provide their own responses to the question. The responses are categorized by the researchers to provide the response options in future rounds. In the first round, participants may be asked to limit themselves to one response, or answer as many times as they would like depending on the research question and number of participants. Alternatively, pre-existing options could be provided for ranking or response, however, this approach could bias the responses or limit the available options
2. Participant Recruitment
   • Research has indicated that participant subject matter knowledge (i.e. being an “expert”) may not have a substantial impact on study results, so it might be best to choose participants who have some understanding of the topic and an interest in the outcome of the study to limit attrition and encourage thoughtful responses to the surveys
   • Often, participants are selected via non-probability sampling techniques (either purposive sampling or criterion sampling), to save resources and ensure appropriate participants are selected
3. Data Analysis
   • The first round of the Delphi technique involves participants providing answers to the research question, which will then be ranked in future rounds. In a classic Delphi, no items should be added or removed, and the wording used by participants should be kept for round two. However, this may be difficult or not feasible depending on the number and types of responses provided, and content analysis can be used to group similar themes prior to the second round. An informal literature review can also be used to identify further items depending on the research question
   • In subsequent rounds, participants are asked to rank/respond to the analysed options from round one. Between rounds the group’s responses are analysed, summarised, and communicated back to the participants, a process called controlled feedback. This is repeated until consensus is reached, or for a planned number of rounds depending on the research question
   • Subsequent rounds are analysed to identify convergence of participant responses, and to provide controlled feedback. Central tendencies (mean, median, and mode) and levels of dispersion (standard deviation and the inter-quartile range) are often used. These results are fed back to participants in the next round, although no consistent method for reporting exists
4. Ending the Delphi Process
   • The process typically ends once acceptable level of consensus has been reached, however, there is no universally agreed cut-off. The level of agreement reached depends on sample numbers (i.e. high attrition or participant burnout), aim of the research (i.e. if complete consensus is required), and available resources
   • Two to four rounds will typically be conducted to ensure study goals are met but to avoid sample fatigue and unnecessarily use of resources

Pros and Cons of the Delphi Technique:

Pros:

• Allows use of a “committee” with fewer drawbacks (scheduling, travel/space requirements, lengthy discussions)
• Anonymity reduces impact of dominant individuals and helps reduce peer pressure to conform, and allows opinions to be considered in a non-adversarial manner
• Responses are weighted equally so no one person can shift the opinions of the group
• Providing controlled feedback on the group opinion reduces noise and allows participants to reconsider based on others’ rankings

Cons:

• There are a lack of clear methodological guidelines
• Continued commitment is required from participants who are being asked a similar question multiple times
• There is no evidence of reliability (i.e. if two panels received the same question they may not come to the same consensus)
• Does not allow participant discussion and there is no opportunity for participants to elaborate on their views
• The existence of a consensus does not necessarily mean that the correct answer, opinion, or judgement has been found, it merely helps to identify areas that one group of participants or experts consider important in relation to that topic

Similar/Alternative Methodologies:

Brainstorming and nominal group technique are similar techniques that allow incorporation of many individual perspectives.

References

Goodman CM. The Delphi technique: a critique. Journal of Advanced Nursing. 1987 Nov;12(6):729–734. doi:10.1111/j.1365-2648.1987.tb01376.x. Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2648.1987.tb01376.x/full

Sackman H. (1974). Delphi critique; expert opinion, forecasting, and group process. Lexington, Mass: Lexington Books.

Hasson F, Keeney S, and McKenna H. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing. 2000 Oct;32(4):1008–1015. doi:10.1046/j.1365-2648.2000.t01-1-01567.x. Available from: http://onlinelibrary.wiley.com/doi/10.1046/j.1365-2648.2000.t01-1-01567.x/full

Hsu C & Sandford BA. The Delphi Technique: Making Sense Of Consensus. Practical Assessment, Research & Evaluation. 2007 Aug;12(10). Available from: http://pareonline.net/getvn.asp?v=12&n=10

McKenna HP. The Delphi technique: a worthwhile research approach for nursing? Journal of Advanced Nursing. 1994 June;19(6):1221–1225. doi:10.1111/j.1365-2648.1994.tb01207.x. Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2648.1994.tb01207.x/full

Linstone HA, Turoff M. (1975). The Delphi Method: Techniques and Applications. Addison-Wesley Pub. Co., Advanced Book Program.

Tags: