Futures Workshop

Turku

7th June 2006

Futures ToolsPresenting the Delphi Method

Definition of the Delphi Method

The word “Delphi” refers to the hallowed site of the most revered oracle in ancient Greece. Forecasts and advices from gods were sought through intermediaries at this oracle.

Delphi may be characterised as a method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem.

Linstone and Turoff (1974)

The Delphi method is an exercise in-group communication among a panel of geographically dispersed experts

Adler and Ziglio, (1996)

The essence of the technique

It comprises a series of questionnaires sent either by mail or via computerised systems, to a pre-selected group of experts. These

questionnaires are designed to elicit and develop individual responses to the problems posed and to enable the experts to refine their views as the group’s work progresses in accordance with the assigned task.

The main point behind the Delphi method is to overcome the disadvantages of conventional committee action.

Anonymity, controlled feedback, and statistical responsecharacterise Delphi. The group interaction is anonymous, in the sense that comments, forecasts, and the like are not identified as to their originator, but are presented to the group in such a way as to suppress any identification.

Fowles (1978)

Main types of the technique

The most common is the paper-and-pencil version, the "Delphi Exercise." In this situation a small monitor team designs a questionnaire, which is sent to a larger

respondent group, after the questionnaire is returned the monitor team summarizes the results and, based upon the results, develops a new questionnaire for the

respondent group. The respondent group is usually given at least one opportunity to re-evaluate its original answers based upon examination of the group response. This

form can be labelled as conventional Delphi.

A newer form, sometimes called a "Delphi Conference", replaces the monitor team to a large degree by a computer, which has been programmed to carry out the compilation of the group results. This approach has the advantage of eliminating the delay caused in summarising each round of Delphi. However, it does require that the characteristics of the communication be well defined before Delphi is undertaken, whereas in a paper-

and-pencil Delphi exercise the monitor team can adjust these characteristics as a function of the group responses. This latter form can be labelled real-lucre Delphi.

Phases

The first phase is characterised by exploration of the subject under discussion, wherein each individual contributes additional information he feels is pertinent to the issue.

The second phase involves the process of reaching an understanding of how the group views the issue (i.e., where the members agree or disagree and what they mean by relative terms such as importance, desirability, or feasibility).

If there is significant disagreement, then that disagreement is explored in the third phase to bring out the underlying reasons for the differences and possibly to evaluate them.

The last phase, a final evaluation, occurs when all previously gathered information has been initially analysed and the evaluations have been fed back for consideration

Usefulness of the Delphi Method

The Delphi Method is based on a “structured process for collecting and distilling knowledge from a group of experts by means of a series of questionnaires interspersed with controlled opinion feedback” (Adler and Ziglio, 1996).

According to Helmer (1977) “Delphi represents a useful communication device among a group of experts and thus facilitates the formation of a group judgement”.

Linstone and Turoff (1974) supported that “When viewed as communication processes, there are few areas of human endeavour, which are not candidates for application of Delphi”.

Applications and main uses

Delphi is mainly known as a forecasting procedure because of its significant use in that area. However, there is a surprising variety of other application areas:

• Gathering current and historical data not accurately known or

available• Examining the significance of

historical events• Evaluating possible budget

allocations • Exploring urban and regional

planning options• Planning university campus and

curriculum development

• Putting together the structure of a model

• Delineating the pros and cons associated with potential policy

options

• Developing causal relationships in complex economic or social

Phenomena

• Distinguishing and clarifying real and perceived human motivations

• Exposing priorities of personal values, social goal

Other Modern Applications

• Advanced materials • Nanotechnology • Microelectronics • Photonics Software and simulation • Molecular electronics or bioelectronics • Cellular biotechnology • Information, production and management engineering

Results expected already in the fields of: • manufacturing • food production • medicine • energy• environment • transportation

Time Horizon and length

The Delphi applications are practical when accurate information is unavailable or expensive to obtain, or evaluation models require subjective inputs to the point where they become the dominating parameters. Thus, the method is quite time consuming.

The time horizon of Delphi is the long run (more than 20 years), when it proves to be a really useful method. In the late 50’s, ‘long range’ was defined as the span of ten to fifty years. There are also cases, where the method was used for short range forecasting (2-3 years).

Does the method produce an accurate view of the future? It is no more accurate, probably, than any expert, single or composite. Or suppose we wanted to explore the range of future events that could affect population growth or weaponry or war. No better way exists to collect and synthesise opinions than Delphi.

The Group Judgement

The data from a Delphi can be displayed in several ways. The group judgment should be based on the median rather than the mean, since single extreme answers can "pull" the mean unrealistically.

Furthermore, it is incumbent on the analyst to show the spread of opinion, which can be done by showing the interquartile range (the range that contains the answers of 50 %of the respondents). An example is shown below:

Strengths and other points

A great deal of attention must be given to the choice ofparticipants; the questionnaires must be meticulouslyprepared and tested to avoid ambiguity. Multi-round studies require a great deal of time; inevitably, some participants will drop out during the process.

The primary strength of Delphi is its ability to explore, coolly and objectively, issues that require judgment.

Delphi is a powerful technique when used to seek answers to appropriate questions.

Weaknesses and concerns

• Discounting the future: Future (and past) happenings are not as important as the current ones; therefore one may have a tendency to discount the future events. • The simplification urge: Experts tend to judge the future of events in isolation from other developments. A holistic view of future events, where change has had a pervasive influence, cannot be visualized easily. At this point cross-impact analysis is of some help. • Illusory expertise: some of the experts may be poor forecasters. The expert tends to be a specialist and thus views the forecast in a setting which is not the most appropriate one. • Sloppy execution: there are many ways to do a poor job. Execution of the Delphi process may loose the required attention easily. • Format bias: it should be recognised that the format of the questionnaire may be unsuitable to some potential societal participants. • Manipulation of Delphi: The responses can be altered by the monitors in the hope of moving the next round responses in a desired direction. • A weakness is the ease with which questions can be asked for which bettertechniques exist.

Common reasons for the failure of a Delphi application

• Imposing monitor view's and preconceptions of a problem upon the respondent group by over specifying the structure of the Delphi and not allowing for the contribution of other perspectives related to the problem.

• Assuming that Delphi can be a surrogate for all other human communications in a given situation.

• Poor techniques of summarizing and presenting the group response and ensuring common interpretations of the evaluation scales utilized in the exercise

• Ignoring and not exploring disagreements, so that discouraged dissenters drop out and an artificial consensus is generated

• Underestimating the demanding nature of a Delphi and the fact that tire respondents should he recognized as consultants and properly compensated for their time if the Delphi is not an integral part of their job function

Complementarity/synergy with other tools

The term “Technology Foresight” (TF), describes a systematic effort to look ahead in science and technology with the aim of identifying the areas of strategic research and the emerging generic technologies likely to yield the greatest economic and social benefits. The distinctive characteristic of technology foresight is the Delphi Method.

Other forms that work together with Delphi: I) The questions relate to the value of independent variables that are used in quantitative simulation models. II) The in-depth interviews with experts have been used at The Futures Group and elsewhere with great success as an alternative to questionnaires. III) For some applications, group meetings among experts have now become practical. Delphi had its birth in concern about spurious factors that intrude in face-to-face meetings among experts

Case Studies: Japan

Japan has been conducting Technological Foresights for a long period

and in a variety of public and private organisations and contexts. Since

1971, the Science and Technology Agency has been conducting Delphi

surveys at regular intervals, about every 5 years, to provide information

for science and technology strategies and planning. First, major

technological categories were set up, covering the fields of materials and

processing, electronics, information, life sciences, space, marine and

earth science, resources and energy, environment, agriculture,

production and machinery, urbanisation and construction,

communication, transportation, health and medical care.

A French case study and the implementation procedure (1)

The French survey on critical technologies (Les 100 technologies clés) that had a narrower scope, but combined Delphi expert interviews withbibliometric studies and patent analysis. The initial questions were very pragmatic and concerned the more important technologies for the French industry, the European leadership in these fields, and the actions to be taken.

The first step involved the definition of technology selection criteria. A steering committee outlined nine selection criteria: actual and potential markets, impact on foreign trade, social and cultural acceptability, vulnerability, contribution to national needs, connection with the national industry, diffusion capacity, and assessment of competitiveness.

A French case study and the implementation procedure (2)

The next step used these criteria to identify technologies; large groups were defined, which were further analysed in 136 technology topics.

In the third step, each technology topic was assessed on the basis of bibliometric studies, patent analysis, and interviews with experts. Additional information was collected on markets, players, companies, R&D programmes.

In the final step, 105 technologies were identified as ‘critical’, accompanied with a short description, a ranking of the degree ofdevelopment, and the relative scientific leadership in Europe. A good information base was produced, and the positive impact of this TF exercise helped to reorient industrial research subsidies of the Ministry of Industry in relation to the 100 critical technologies.

Case study: The Region of Central Macedonia (1)

Foresight in Central Macedonia was an action of the project ‘Excellence in Central Macedonia’ under the Innovative Actions 2000-2006 Programme. A total of 280 persons were directly involved with the various stages of the action's development., approximately sixty high-level experts were selected to form a balanced mix of the elements of the regional innovation system (academia, industry, research, regional administration, NGOs, technology transfer organisations) and elaborate eight thematic areas of major strategic importance to the Region:

(a) Information and Communication Technologies(b) Agro- and Bio- technologies(c) Industrial processes and Materials (d) Environment(e) Transport networks(f) Energy(g) Human Resources (h) the south-eastern European economic area

Case study: The Region of Central Macedonia (2)

The Delphi paradigm was used to evaluate and fine-tune forward predictions. 1240 selected individuals were asked to evaluate 215 Delphi statements classified in eight on-line questionnaires. 200 individuals actually participated in the process. Two iterations of the Delphi questionnaires took place between October 2003 and February 2004. Following the convergence of the Delphi process, the teams of experts’elaborated one final report per thematic area. All eightreports together with a Methodological Guide and a Summary of Findings were published.

This action's primary objective was to support strategic decision-making for all the elements of the regional innovation system towards identifying and correlating the “correct mix of technological developments” to “exploitable opportunities”; to provide hints on re-orientating the regional innovation system towards exploiting these opportunities; and influencing the strategy development process of all regional actors towards supporting the transition of the Region of Central Macedonia to global-grade competitiveness so that the opportunities will be effectively exploited.

Thank you for your attention!

Gountaras Nikolas

Researcher at URENIO Research and Innovation Unit in Aristotle University of Thessaloniki

Website: www.urenio.orgE-mail: ng@urenio.org