1.3hickey

Requirements ElicitationTechniquesAnalyzing the Gap between TechnologyAvailability and Technology Use

ANN M. HICKEY, ALAN M. DAVIS, and DENALI KAISER

279

INTRODUCTION

The current state of building information systems is terrible. At least$185 billion is wasted on development projects that fail, often because thesoftware does not satisfy users’ needs (Standish Group, 1995). Althoughthere are many possible reasons for these failures, problems related to

ABSTRACT Many software development projects failbecause the resulting software does not satisfy user needs. The pro-cess of determining user needs is generally termed requirementselicitation. Although there are many possible reasons for softwarefailures, if analysts practiced more effective requirements elicita-tion, fewer projects would fail. Although hundreds of requirementselicitation techniques have been developed by researchers to aidanalysts in effectively determining user needs, few have ever beenused by practitioners. This paper reports on research to study thenature of the gap between requirements elicitation technique avail-ability and use, identifies the major factors that impact the transferof elicitation techniques to practice, and explores how to improvethe transfer of elicitation techniques from research to practice.

Comparative Technology Transfer and Society, volume 1, number 3 (December 2003):279–304© 2003 by Colorado Institute for Technology Transfer and Implementation

Mary Botto

understanding users’ needs are consistently identified as among the mostimportant (Standish Group, 1999). Requirements elicitation techniquesare the means by which systems analysts determine the problems, oppor-tunities, and needs of the customers, so that systems development person-nel can construct systems that actually resolve those problems, leveragethose opportunities, and/or address customers’ needs. In response to aless-than-acceptable rate of failure of systems, hundreds of elicitation tech-niques have been created by researchers. But the majority of these tech-niques are rarely, if ever, used by practitioners. Solutions appear to beavailable, yet we continuously fail to make use of them. Given this gap be-tween elicitation technique technology availability and use, barriers to thesuccessful transfer of requirements elicitation techniques from researchersto practitioners appear to exist.

Problems with the transfer of new requirements elicitation techniquesto practice impact more than the software development industry. First,information technology, and more specifically the information systemssoftware that supports an organization’s operations, is critical to the suc-cess of the vast majority of organizations today. Therefore, any problemthat impacts the successful development of those systems impacts allorganizations. Second, information technology (e.g., knowledge manage-ment systems) is viewed as a key enabler of technology transfer and infor-mation dissemination. Therefore, the successful development of informa-tion systems potentially impacts the successful transfer of many othertechnologies. Third, continuous process improvement, and the transfer ofnew processes and knowledge required to support that improvement, is anecessity in today’s rapidly changing and competitive environment. There-fore, problems with the transfer of processes such as new requirementselicitation techniques in an industry that has traditionally been viewed asan early adopter of new technology may provide important, early insightsinto the transfer of new processes and knowledge in other, less pro-inno-vation industries.

The research reported in this paper seeks to investigate the nature ofthe gap between requirements elicitation technique availability and use;identify the major factors that impact the transfer of new processes andknowledge, such as requirements elicitation techniques, to practice; andexplore how the transfer of requirements elicitation techniques and othernew process technologies may be improved. This paper describes ouroverall research plan, provides details of the exploratory research we haveconducted using surveys and in-depth interviews with requirements ex-perts, and summarizes our results.

Comparative Technology Transfer and Society, December 2003, vol. 1, no. 3

280

BACKGROUND

Requirements elicitation is the process of learning and understandingthe problems, opportunities, and desires of customers. It is part of the setof activities usually termed requirements management, whose goal is tounderstand the customers’ requirements for, and document the desiredexternal behavior of, a new or revised system. Although much of the re-quirements literature describes elicitation as an early activity of the re-quirements phase, it is actually conducted throughout the requirementsphase in conjunction with the other requirements activities. But even moreimportantly, because customers’ problems constantly change, the entire setof requirements activities is performed continuously throughout the sys-tem development process. Therefore, requirements elicitation should beconsidered an ongoing process throughout the life of a system.

Requirements elicitation is performed by analysts (also known as sys-tems analysts, requirements engineers, and requirements analysts) using elic-itation techniques. In 1996, Capers Jones estimated that there would beapproximately 12 million software developers worldwide by 1998. Assum-ing 5% growth per year, and assuming that 1 out of every 15 developers isan analyst, there were approximately 1 million practicing analysts in 2003;so it is clear that there are many potential users of requirements elicitationtechniques. To better understand elicitation techniques, and why they areso important to product success, Table 1 provides a brief overview (ex-tracted from Davis, Hickey, & Zweig [2003]) of several major classes ofelicitation techniques including interviewing, brainstorming, collaborativeworkshops, prototyping, questionnaires, observation, and modeling.

When faced with a new requirements elicitation situation, analysts se-lect from the following types of elicitation techniques using a variety of ap-proaches:

1. They select an elicitation technique because it is the only one that they know.

2. They select their favorite elicitation technique.

3. They select an elicitation technique because they understand intuitively that the technique will be effective in the current circumstance.

Using either of the first two approaches yields the potential for disas-trous consequences: alienated customers, incorrect requirements, and sys-tems that fail to meet the customers’ needs. Both researchers and practi-tioners seem to recognize that poor requirements elicitation, and by exten-

Hickey, Davis, and Kaiser | REQUIREMENT ELICITATION TECHNIQUES

281


282

Table 1

REQUIREMENTS ELICITATION TECHNIQUES

General Classes of Elicitation Techniques History of Techniques in Survey

Interviewing is the process of analysts asking Interviewing has been performed since thequestions and prompting one or more stake- dawn of computers in the 1950s. Goal-Ori-holders to verbalize their thoughts, opinions, ented Elicitation during interviews or work-concerns, and needs. Gause and Weinberg shops was first referenced in 1993 (1989) provide a wealth of ideas on how to (Dardenne, van Lamsweerde, & Fickas,interview. 1993).

Brainstorming is the process of gathering mul- Brainstorming was invented by Osborne intiple stakeholders in a room, posing an issue 1939 (Osborne, 1963), and has been usedor question, encouraging the stakeholders to throughout the history of software express their ideas aloud, and having those development.ideas recorded.

Workshops involve gathering multiple stake- The earliest documented cases of using facili-holders together in structured, facilitated tated workshops for elicitation date back to workshops to define the requirements for a the Joint Application Development (JAD) ap-system. Facilitators lead stakeholders through proach developed by IBM in the late 1970s a series of preplanned activities designed to (Wood & Silver, 1995). Role Playing in work-produce the requirements deliverables needed. shops was first referenced as a means to elicitGottesdiener (2002) provides an outstanding requirements in 2000 (Leffingwell & Widrig,compendium of ideas on how to use work- 2000). shops for requirements elicitation.

Prototyping is the process of creating a partial Prototyping has been around for as long as implementation of a system, demonstrating it humans have manufactured products. Its use to stakeholders, and perhaps allowing them to gather requirements feedback from poten-to play with it. Davis (1995) provides an ex- tial users and customers seems to go back as cellent overall summary of prototyping tech- far as 1977 (Bally, Brittan, & Wagner, 1977).niques and effects.

Questionnaires are collections of closed- and Questionnaires are one of the oldest tech-open-ended questions that are distributed to niques for gathering data from large popula-many stakeholders. Responses are analyzed to tions, and have been used since the dawn of understand general trends and stakeholders’ computers in the 1950s.opinions.

Observation is an ethnographic technique Anthropologists have been using ethno-whereby the analyst observes the users and graphic observation of people since at least customers performing their regular activities. the late 1880s (Stocking, 1996). Its use to A good survey of techniques involving obser- elicit requirements goes back to the early vation can be found in Goguen & Linde 1980s (Parkin, 1980).(1993).

Modeling involves the creation of graphical or Modeling of systems has been performed textual representations of the problem or its since the dawn of computers with the earliestsolutions to increase communication and pro- record of flow charts and decision trees datingvide fresh insights into the problem or solu- back to the 1950s (Couger, 1973); data flow tion. A wide range of modeling approaches diagrams to 1977 (Ross, 1977); scenarios to

continued

sion, poor use of requirements elicitation techniques or selection of inap-propriate techniques, is almost guaranteed to result in a poor product(Hickey & Davis, 2002). One would think that the plethora of availableelicitation techniques would make it easy for analysts to find one with ahigh probability of working effectively, yet few are used in practice.

From a researcher’s perspective, the intended transfer of elicitationtechnique technology from research to practice is as follows: Researcherscreate new technology, which, along with existing technology, may beadopted by consultants and practitioners. The technology could movedirectly from researchers to practicing analysts, or from researchers to con-sultants (who in theory should be early adopters [Moore, 1991]), and thentransferred to practicing analysts via the consultative activity. Unfortun-ately, given the gap between the number of elicitation techniques availableand those that are actually used in practice, the transfer of this technolo-gy does not appear to be taking place as anticipated. The purpose of thispaper is to explore the nature of the gap between the availability and useof requirements elicitation technique technology.

TECHNOLOGY TRANSFER THEORIES

Many theories have been posed by the information systems communityconcerning why elicitation techniques are created but not used in practice(Hickey & Davis, 2002; Jones, 1995; Zmud, 1983). The following theoriesseem the most plausible (these are not necessarily mutually exclusive).

More time is needed for transfer to occur (Redwine & Riddle, 1985).Stated another way, perhaps the techniques are too immature for transi-


283

Table 1

REQUIREMENTS ELICITATION TECHNIQUES (continued)

General Classes of Elicitation Techniques History of Techniques in Survey

exists, including: object diagrams, data flow the 1970s (Alford & Burns, 1975; Davis, diagrams (DFD), the Unified Modeling 1982); Quality Function Deployment (QFD) Language (UML), Z, finite state machines appearing in 1986 (Haag, Raja, & Schkade, (FSM), Petri nets, the System Description 1996); statecharts to their invention in 1987 Language (SDL), statecharts, flow charts, (Harel, 1987); use cases to 1993 (Jacobson, use cases, scenarios, decision tables and Christerson, Jonsson, & Overgaard, 1992) trees, and so on. See Davis (1993a), Kowal and the Unified Modeling Language (UML) (1992), and Wieringa (1996) for descrip- to 1995 (Rational Software Corporation, tions of most of these notations. 1997).

tioning (Heslop, McGregor, & Griffith, 2001). As shown in Figure 1, anynew elicitation technique must undergo a series of maturity levels before itis ripe for widespread use by the community of practitioners. Specifically,

• Practicing analysts do not know of the existence of techniques.Technology transfer does not occur because path A2 in Figure 1 is not traversed due to the analysts and consultants not knowingabout the availability of techniques. This could be caused by eitherresearchers insufficiently marketing their new technology or practi-tioners insufficiently paying attention.

• Practicing analysts know of techniques’ existence, but do not knowhow to apply them. Thus, technology transfer does not occur be-cause path A3 in Figure 1 is not traversed. This could be caused byresearchers or educators insufficiently teaching practitioners aboutthe new technology or consultants and analysts failing to take theappropriate courses.

• Practicing analysts know of techniques’ existence, and know how to apply them, but do not know when to apply them. In this case,path A4 in Figure 1 is not traversed, again negatively impactingtechnology transfer of elicitation techniques. Preliminary ideas concerning when to apply specific techniques can be found in Davis (1993a), Hudlicka (1996), Kotonya & Sommerville (1998),Lauesen (2002), Leffingwell & Widrig (2000), Macaulay (1996),Maiden & Rugg (1996), and Wiegers (1999), but more comprehen-sive, situation-specific guidance is not available (Hickey & Davis,2003).


284

Figure 1 Elicitation technique maturity levels.

In conclusion, technology transfer becomes effective once enough timepasses, allowing the technique to move at a natural pace through paths A2,A3, and A4.

Many other theories relate to the readiness of the technology and theorganizations/individuals for the transfer and the communication requiredto facilitate that transfer. For example, the available techniques may not bepractical or sufficiently demonstrated to be so (Davis & Hickey, 2002). Or,practicing analysts may be perfectly content with what they are doing (in-cluding, for example, using ad hoc techniques), so do not care about alter-native techniques and have no motivation to absorb any techniques thatare “ready” for adoption. Or, analysts may not be doing elicitation at all!

More generally, Rogers (1995) identifies four main elements in the dif-fusion of technological innovations: time, the innovation, the social sys-tem, and communication channels. By definition, technological innovationsmay include hardware, software (including computer software as well asother information or process knowledge), or both. However, Rogers(1995) observes that most research has been conducted on hardware orhardware/software combinations with virtually none on the diffusion ofsoftware-only (e.g., information or process) innovations. Regardless, theproposed elicitation technique transfer theories seem to fall within thesecategories. Therefore, the current research will use Rogers’ framework todetermine which conditions or sets of conditions are the most prevalentfactors impacting the transfer of requirements elicitation techniques fromresearch to practice.

RESEARCH APPROACH

Given the multitude of theories regarding technology transfer in gen-eral, but lack of specific research on the transfer of requirements elicita-tion techniques, we have chosen a multi-method exploratory research ap-proach. Specifically, we have conducted qualitative in-depth interviews(Seidman, 1998) with selected requirements experts, and quantitative sur-veys (Fowler, 1993) of practicing analysts to explore the research goalsposed in the introduction. Specifically, we will

1. Determine whether or not a gap really exists between the avail-ability and use of new requirements elicitation techniques and their transfer from research to practice;

2. Identify the major factors that impact the transfer of elicitationtechniques to practice;


285

3. Explore how the transfer of elicitation techniques may beimproved.

In-depth interviews with 11 of the world’s experts at elicitation wereconducted to explore all three research questions. Key to this step was thepurposeful selection of the interviewees (Seidman, 1998). Our current cri-teria include (1) practicing with both a breadth and depth of elicitationexperience, (2) self-aware, (3) considered by many to be experts, (4) arelooking for ways to improve, and (5) demonstrated a desire to share theirknowledge with others via publication. The experts interviewed wereGrady Booch, Larry Constantine, Tom DeMarco, Don Gause, Ellen Gottes-diener, Tim Lister, Lucy Lockwood, Don Reifer, Suzanne Robertson, KarlWiegers, and Ed Yourdon. Combined, these experts have over 285 yearsexperience analyzing requirements on more than 700 projects around theworld.

A survey of practicing analysts was conducted to investigate the firsttwo research questions. Based on an extensive literature search to identifyexisting requirements elicitation techniques, we prepared our initial sur-vey instrument (Appendix A). This survey captures the state of therespondents’ knowledge, use, and assessment of selected elicitation tech-niques. Because of the vast numbers of techniques identified, we includedonly 17 techniques in our initial survey. We selected this subset out of themore extensive list based on the following criteria: (1) have made the ini-tial technology transfer to at least some innovators (Moore, 1991), (2) arenot proprietary, (3) have been successfully used in practice, (4) providecategory breadth (i.e., coverage of all the categories described in the Back-ground section of this paper), (5) provide age breadth (i.e., coverage oftechniques introduced from 40 years ago to recently), (6) provide applica-tion applicability breadth (i.e., coverage of techniques applicable to a widerange of applications such as engineering, real-time, management infor-mation systems, and so on), and (7) provide generality breadth (i.e., in-clude techniques that are generally applicable as well as ones that are de-signed for specific purposes). We have collected responses to this instru-ment from 46 individuals; their educational background, number of years’experience in requirements engineering, and number of requirements pro-jects worked on are depicted in Figures 2, 3, and 4, respectively. Althoughthe respondents represent a convenience sample of attendees at two inter-national conferences, and practicing analysts in several large softwaredevelopment organizations, the breadth of their education spanning tech-nical, business, and other fields (see Figure 2) and experience rangingfrom novice to expert (shown in Figures 3 and 4) indicate a reasonable


286


287

Figure 2 Survey respondents: educational background.

Figure 3 Survey respondents: years of requirements experience.

Figure 4 Survey respondents: number of requirements projects.

representation of the target population. We also interviewed eight of thepracticing analysts to explore their responses in more detail.

We have organized our research results into three sections correspon-ding to our research questions: (1) interview and survey results about theexistence of a gap between requirements elicitation technique availabilityand use, (2) interview and survey results about the factors impactingtransfer of elicitation techniques to practice, and (3) interview resultsabout ideas for improving that transfer. Whenever counts are provided(e.g., 5 of 11) in the interview results, the reader should assume the otherexperts did not mention that issue, not that they necessarily disagreedwith the others.

RESULTS: EXISTENCE OF A GAPBETWEEN AVAILABILITY AND USE

The interviews with the 11 experts strongly support our hypothesisconcerning the existence of a gap between the availability and use of newrequirements elicitation techniques. In fact, there was no dissention. Gen-erally, the state of practice (per experts) is that analysts either interviewstakeholders or conduct group discussions. A few are considerably better,but most have their heads in the sand. Some use modeling.

The survey provides two key indicators of the existence of a gap im-pacting the use of new requirements elicitation techniques in practice.First, the survey asks respondents if they know of the existence of selectedelicitation techniques, which is a necessary precursor to use (see Figure 1).Results were somewhat equivocal on this indicator. On average, res-pondents knew of the existence of 14 of the 17 techniques included in thesurvey. However, although the majority of techniques were familiar to vir-tually all respondents, one-third (6) of the techniques were familiar tofewer than 80% of the respondents, with 2 techniques familiar to 50% orless. Although these results show a generally positive transfer of knowledgeof the selected techniques’ existence, they do show the possible existenceof knowledge gaps for some of the techniques. These gaps were somewhatunexpected given the experience level of the survey respondents (seeFigure 3) and the nature of the techniques included in the survey.

The second indicator asks respondents if they have ever used theselected techniques. Results on this indicator provided stronger supportfor the existence of a gap between technology availability and use. Onaverage, respondents had used approximately half of the techniques in-cluded in the survey. Whereas some techniques such as interviewing and


288

brainstorming were widely used (93% and 89%, respectively), about halfof the techniques have been used by between 40% and 66% of the respon-dents, and 4 have been used by fewer than one third of the respondents.These data are consistent with the experts’ perception that interviews andbrainstorming are widely used in practice and that many, even well-known, techniques are less used. Therefore, we believe these results gen-erally support our hypothesis that a gap does exist between the availabili-ty and use of new requirements elicitation techniques.

RESULTS: FACTORS IMPACTINGTRANSFER TO PRACTICE

The theories concerning the factors that impact the transfer of require-ments elicitation techniques to practice were listed in an earlier section.This section presents our results concerning these factors, grouped byRogers’ (1995) four main elements in the diffusion of innovations.

time

To assess the impact of time on the adoption of requirements elicita-tion techniques, we need to correlate the degree of awareness of the tech-niques to the age of the techniques. Table 1 summarizes what we learnedconcerning the documented dates of availability of each of the techniquesincluded in the survey instrument. The survey results show the degree ofknowledge and use that each of these techniques exhibit. When we plotthe introduction date against the degree of knowledge by the respondents,we arrive at Figure 5. When we plot the introduction date against thedegree of use by the respondents, we arrive at Figure 6.

Although these charts do not show a clear trend, grouping the tech-niques by decade does (Table 2). In particular, these preliminary resultsshow a classic correlation between age of the technique and both theknowledge and the use of techniques within the user community. Thus,Table 2 shows a general increase in knowledge of techniques as they getolder (53%, 88%, and 97% as we move from 10 to 20 to 40 years old) aswell as a general increase in their use (28%, 62%, and 72%). In both cases,we see a spike in popularity among the most recently introduced tech-niques due to the lemmingineering phenomenon (Davis, 1993b). See theclassic age-related technology absorption graph with the lemmingineeringspike in Figure 7.

More specifically, to assess the impact of the maturity of the tech-


289

niques, we explored each of the maturity levels shown in Figure 1. Onaverage, survey respondents knew of the existence of 84% of the 17 elicita-tion techniques included in the survey (see Table 2). If we believe that oursurvey respondents are representative of the population at large, then there


290

Figure 5 Timeline of elicitation technique introduction versus knowledge.

Figure 6 Timeline of elicitation technique introduction versus use.


291

appears to be no problem here except with a few techniques as discussedin the previous section. If we believe that our survey respondents are earlyadopters (Moore, 1991), then this number seems low, indicating thatknowledge of existence is a potential problem. In addition, if we believethat the 17 elicitation techniques represent some of the most commonlyknown techniques, then knowledge of these techniques may not be gen-eralizable to knowledge of the more than 200 techniques identified inpreparation for this research, and knowledge of the existence of tech-niques may still be a potential problem. Many of the experts did agree thatknowledge is part of the problem for practitioners. In fact, when describ-ing how they learn about the existence of techniques, all the experts iden-tified the large number of industry and academic publications they read tostay current—an option simply not practical for the majority of practicinganalysts.

Figure 7 Absorption curve.

Table 2

TECHNIQUE KNOWLEDGE AND USE AS FUNCTION OF AGE

Average percentage Average percentageAge (years) of techniques known of techniques used

≥ 40 97 72

20–30 88 62

10–20 53 28

< 10 77 39

All 84 56

The degree of respondents’ use of various techniques is shown in Table2; on average, the survey respondents have used half of the elicitation tech-niques included in the survey. Technique usage ranged from 93% for inter-views to less than one third for 4 of the 17 techniques. Although this doesnot indicate whether the techniques were used correctly or effectively, itdoes seem to imply at least some knowledge of how to use at least half of thetechniques respondents have used. Techniques with lower usage levels(e.g., Joint Application Development [JAD], quality function deployment[QFD]) generally require more specific knowledge to use, so it may be thattheir low usage was caused by lack of knowledge of how to use.

We have no data from our survey results to support whether or notanalysts know when to use techniques, but several of our experts felt thatanalysts seemed overwhelmed by the number of available elicitation tech-niques and that guidance on when to use techniques could help to reducethe information overload. Hickey and Davis (2003) report on efforts tobetter understand when various techniques are applicable.

the innovation

Eight of the experts highlighted concerns relating to the readiness ofelicitation techniques for transfer to practice. The interview results exhib-ited quite a bit of comment on whether techniques created by researcherswere useful or practical. In particular, six felt strongly that most of thetechniques created by researchers are not useful or practical. For example,one interviewee stated that “90% of academic research is crap.” A secondstated that only about 25% to 33% of new techniques are even seen bypractitioners, and of these, only 25% to 33% ever get into practice. If thisis the case, then only 6% to 11% of new elicitation techniques are everused by practitioners. Another was even more pessimistic, estimating thatonly 1 in 100 will make it. Still another stated that 65% to 70% of elicita-tion research represents “useless academic results,” 5% is “stimulating,”and the rest “may have some value.” More specifically, one intervieweestated, “For each book [that I read on elicitation, I] may find 1 to 3 ideasthat [could] change practice. From papers, [the most I can expect to getis] possibly a great quote/question/table.” Two others implied that toomany academics do not have real-world experience, and that as a result,much of what is developed is totally disconnected from the real world.

On a similar note, two of the experts challenged whether new tech-niques are actually being developed by researchers, or whether old tech-niques are just being renamed or repackaged. Specifically, one stated thatthere has been nothing new in the past 30 years. The other was concerned


292


293

that similar attitudes would cause practitioners to miss new techniques.The ability to clearly define what is new or different about a technique isimportant to successful transfer, as Greiner and Franza (2003) foundwhen they observed that “technology which is more understandable, de-monstrable, and unambiguous is easier to transfer.”

Experts were also concerned about the ability to make a business casefor a specific elicitation technique. One expert observed that more casestudies are needed where “real” practitioners report successful use of thetechnique. Similarly, another emphasized that organizations/analysts needto experience (or observe) positive results from a technique before theywill adopt it. A third highlighted that there is not even any agreement onthe “best practices” for elicitation. At the opposite end of the spectrum,one expert stated that a major problem is solution vendors who overselltheir technique/tool as a “silver bullet” and preach that one size fits all,that is, that their solution works all the time. According to Heslop, Mc-Gregor, and Griffith (2001), the ability to offer “significant identifiable andquantifiable benefits” is a primary indicator of technology readiness, so wemust do a better job of documenting the benefits of elicitation techniquesif we hope to successfully transfer them to practice.

the social system

Seven of the experts we interviewed laid part of the blame for poortechnology transfer on the practitioners and their organizations. One ex-pert emphasized that organizations do not understand the risk of notdoing requirements, or, as another stated, the resources it takes to dorequirements correctly. Another expert felt that organizations often do notrecognize the need to improve: “We’re doing fine. Our products fail in themarketplace for reasons other than poor elicitation.” A third expert ob-served that organizations have been burned before by oversold techniques,and are now risk averse. Not surprisingly, therefore, as mentioned by fourof those interviewed, few companies or analysts dedicate the resources tofinding, let alone learning or implementing, new elicitation techniques.Another said that there are too many political and cultural barriers inindustry to be able to effectively assimilate new elicitation techniques.Furthermore, they are mostly looking for “silver bullets that require lesseffort, not more effort. Developing on time is far more important to mostcompanies than doing it right.” Another stated that corporate egos andlimited budgets can interfere with acceptance of a new technique. Finally,one of those interviewed summarized his feelings by saying that industryhas too much “apathy, ignorance, and inertia.”

communication channels

The problem of communicating research results to practitioners, essen-tial to successful technology transfer, became evident when we asked theexperts how they learn about the existence of new elicitation techniques.Seven of the experts read a very wide range of academic and practitionerjournals and books on a regular basis, with two emphasizing the need toread outside the field (e.g., other design disciplines). Four mentioned re-viewing conference or journal papers, and five stated they regularly at-tended conferences. Four of the experts felt that networking with associ-ates, students with work experience, and/or researchers at universitieshelped them to stay current. However, several observed that the academicliterature is extremely inaccessible to practitioners. One specifically statedthat academics tend to speak in a language quite different from that ofpractitioners, and that there is a need for translation as well as closer inter-action between academics and practitioners. The sheer volume and inac-cessibility of these information resources make them impractical for thevast majority of practitioners. Because effective communication is criticalfor successful technology transfer (Kremic, 2003), it is clear that commu-nication mechanisms for disseminating information about requirementselicitation techniques must be improved.

RESULTS: IDEAS FOR IMPROVINGTRANSFER

The interviews provided us with numerous, consistent ideas abouthow to improve the transfer of requirements elicitation techniques to prac-tice. However, one expert felt that was no problem to be solved; very sim-ply, researchers are not producing useful results, so why should we try toincrease the transfer of such [useless] technology to practice? This is con-sistent with Rogers’ (1995) observation that an innovation must providean advantage before it will be used.

Many of the experts’ ideas related to the social system. Most important-ly, the software industry needs to increase the percent of organizationsdoing requirements. Capers Jones (1991, p. 228) discovered that more than75% of organizations do not pay adequate attention to requirements activ-ities. To overcome this, researchers need to create a better business case forperforming requirements. Much of this business case should emphasize thecosts and risks of not doing requirements. This business case also needs toreport the necessary costs, hard work, and dedication by all stakeholders


294


295

that are associated with doing good requirements. One expert emphasizedthe need to blame poor requirements more often during litigation; this typeof negative press is bound to make more companies pay more attention torequirements. Two experts expressed the key role that the technologychampion plays in making new techniques work in practice.

Other ideas for improvement relating to the innovation itself includethe need for a clearer definition of requirements elicitation techniques.Most of the experts interviewed complained that when a new technique isunveiled, they are unable to understand what the technique is or whetherit is any different from techniques they already know. If this is a problemfor the experts, it must be overwhelming for the typical practitioner. Re-searchers, working with industry, also need to determine and documentwhich techniques have value in which situations; they must identify bestpractices. Only after this is done, can we begin to build business cases forindividual elicitation techniques. In our research, we found just one (Got-tesdiener, 2002) documentation of a template for making the business casefor a specific technique; in this case, collaborative workshops. Most ofthose interviewed had not themselves even thought about how they selecta technique. To make matters worse, almost all the experts reported theirfrustration with vendors who oversell their techniques as “silver bullets.”Another expert suggested that techniques tend to be more easily adoptedwhen packaged with some type of automated tool. Two of the experts em-phasized the critical role played by business cases, case studies, or projectretrospectives to help future projects understand how well or poorly cer-tain techniques worked in the past.

The experts and our own observations indicate a critical need for im-provements in the communication channels used to transfer requirementselicitation techniques from research to practice. First, we must eliminatethe mismatch between where researchers publish and what practitionersread. Faculty members are rewarded for publishing in the most research-oriented journals, yet practitioners have time to read only the most practi-cal of publications. We also must improve education. Unfortunately, mostacademic institutions teach students prescriptive approaches, rather thanthe more important skill of critical thinking, which is essential for effectiverequirements elicitation. In addition, mechanisms must be established toencourage practitioners to become academic instructors. One expert saidthat educational institutions do not give their students enough experienceworking in teams. To support continuing education, we need to increasethe number of practitioner-oriented conferences. One expert interviewedstated that enlightened organizations allow and in fact encourage theiremployees to attend conferences so they can improve their understanding

of the field. However, the same expert also reported that few conferencesare designed for the practitioner. Many claim to be for the practitioner, butare organized by academics and attract mostly fellow academics.

Three final areas address both the social system and communicationchannels. Many studies (Potts, 1993) have shown that successful technol-ogy transfer can be enhanced via close collaboration between industry,consultants, and academe. The result of such collaboration is that re-searchers better understand real problems in industry and are able todevelop techniques to solve them, and industry better understands hownew technologies apply to their situation. However, only two experts sug-gested closer collaboration in response to our open-ended question, “Howwould you fix the technology transfer problem?” One expert did suggestchanging the consultant role so that consultants do less of the actual workwhile undertaking an assignment. Instead, they should spend more timefacilitating work so the organization becomes trained, leaving nuggets ofnew ideas. Consultants need to keep up to date and translate academicresearch into terms that practitioners understand. Finally, although notmentioned directly in the interviews, analysis of many of the experts’ rec-ommendations made it clear that the inconsistency between the academicreward structure and technology transfer must be resolved. Academicrewards at most research institutions are based on successful publicationof research results in journals of little interest to practitioners. Moreover,once published, little or no incentive exists to see the results transferred topractice. Some possibilities for addressing this issue are profit sharing frompatents and encouraging faculty to take sabbaticals in industry.

LIMITATIONS

The primary limitations of our research results relate to the survey. Thesurvey respondents may not be representative of our desired practitionerpopulation from two perspectives: (1) 29 of the respondents were atten-dees at requirements engineering and information systems conferences,which attract a high percentage of attendees from academia versus indus-try, and (2) the 17 practicing analysts who responded may be more expe-rienced than average practitioners. Limiting the survey to 17 techniques,while providing interesting results regarding those techniques, impacts thegeneralizability of our results to all elicitation techniques. We also need tofine tune the measures to gain a better understanding of the extent of theknowledge and use of those techniques.

In addition, although expert interviews provided a rich source of


296

information, they also have their limitations. Alternative methods may beneeded to collect more detailed information on the specific factors impact-ing technology transfer so that we more adequately assess the validity ofour proposed theories and better identify mechanisms for improving thetransfer of elicitation techniques to practice.

FUTURE RESEARCH OPPORTUNITIES

Our research goals, survey and interview results, and the limitations ofour research provide the following opportunities for future research on thetransfer of requirements elicitation techniques from research to practice.

A more comprehensive survey instrument could be developed based onthese results to evaluate a greater number of elicitation techniques usingmore exact measures of knowledge that could then be distributed to a larg-er, more representative sample of practicing analysts. This survey couldalso explore respondents’ assessments of the factors impacting technologytransfer of requirements elicitation techniques identified in this paper.

The recommendations presented in this paper could be extended inseveral ways. For general factors relating to Rogers’ four elements, a liter-ature search should identify how other fields are addressing these issues.For factors unique to requirements elicitation, ideas could be identifiedthrough further discussions with experts, working groups at requirementsconferences, and additional targeted research. Specifically, anticipatingthat one of the factors is analysts’ lack of understanding of when to applyelicitation techniques, we have already begun complementary researchinto how experts select elicitation techniques so that their expertise can beshared with practicing analysts (Hickey & Davis, 2002, 2003).

The graph depicted in Figure 7 seems logical to us, but represents amodel of technology adoption that we have not seen in the technologytransfer literature. More research could be done to validate the rate of ab-sorption, and understand the conditions under which adoption demon-strates this phenomenon rather than following more traditional S-curves.

SUMMARY AND CONCLUSIONS

This paper reports results of our research into the factors impacting thetransfer requirements elicitation techniques from research to practice.Results of our interviews with 11 experts and a survey of 46 individualssupport our hypothesis that a gap between the availability and use of


297

requirements elicitation techniques does exist, and identified several crit-ical factors that impact the transfer of elicitation techniques to practice.The interview results also provided interesting ideas about how to improvethat transfer. Our goal is to share these recommendations to improve elic-itation technique transfer and the state of requirements elicitation prac-tice, and, as a result, improve future software so that it will more fully sat-isfy users’ needs. Better software will positively impact all organizationsusing that software for operations, and enable improved information dis-semination and transfer of other technological innovations.

This research also highlighted academia’s pro-innovation bias. Re-searchers often refer to technology transfer barriers, assuming that theirnew innovations should be adopted, and tend to blame industry for notaccepting their research results. That is as silly as a company with a prod-uct blaming the customer for not buying it. The onus is always on the sup-plier to construct a useful product. The most successful mass marketers ofproducts understand the needs of their intended customers before theybuild their product! Similarly, researchers who want to transfer their re-sults to industry need to understand the desires of industry before theycommence their research and clearly demonstrate the value of their newinnovations to potential customers. This research has shown that this hasnot been done for requirements elicitation techniques.

More generally, these results provide insight into the transfer of otherprocess innovations to practice. First, the research demonstrated that evenindustries known for the early adoption of hardware and computer soft-ware innovations can still have problems with the transfer of process inno-vations. Second, many of the factors impacting the transfer of elicitationtechniques also impact the transfer of other process innovations. Third,the recommendations for improving elicitation technique transfer aretherefore also applicable to the transfer of other process innovations. Fi-nally, the results demonstrate that Rogers’ framework for the diffusion oftechnological innovations aided understanding of the transfer of require-ments elicitation techniques and therefore also applies to software-onlyand process innovations such as this. These results and Rogers’ (1995)analyses of the impact of the characteristics of the innovation, time, socialsystem, and communication channels on the diffusion of technological in-novations can therefore aid the practitioner in the diffusion of many otherprocess innovations.


298

ACKNOWLEDGMENTS

The authors would like to thank Daedra Studniarz, the Colorado Insti-tute for Technology Transfer and Implementation (CITTI), and El PomarFoundation for their financial support and encouragement to pursue thisresearch. We would also like to thank Dean Gary Klein and the entireCollege of Business at the University of Colorado at Colorado Springs forestablishing an environment where research is rewarding, rewarded, andenjoyable.

REFERENCES

Alford, M., & Burns, I. (1975). An approach to stating real-time processingrequirements. Conference on Petri Nets and Related Methods. Cambridge, MA: MIT Press.

Bally, L., Brittan, J., & Wagner, K. (1977). A prototype approach to informationsystem design and development. Information & Management, 1(1), 21–26.

Couger, D. (1973). Evolution of business system analysis techniques. ACMComputing Surveys, 5(3), 167–98.

Dardenne, A., van Lamsweerde, A., & Fickas, S. (1993). Goal-directed require-ments acquisition. Science of Computer Programming, 20, 3–50.

Davis, A. (1982). The design of a family of applications-oriented requirementslanguages. IEEE Computer, 15(5), 21–28.

Davis, A. (1993a). Software Requirements: Objects, Functions and States. UpperSaddle River, NJ: Prentice Hall.

Davis, A. (1993b). Software lemmingineering. IEEE Software, 10(5), 79–81, 84.Davis, A. (1995). Software prototyping. Advances in Computers, 40 (pp. 39–63).

New York: Academic Press.Davis, A., & Hickey, A. (2002). Requirements researchers: Do we practice what

we preach? Requirements Engineering Journal, 7(2), 107–11.Davis, A., Hickey, A., & Zweig, A. (2003). Requirements management in a

project management context. In Morris, P. & Pinto, J. (Eds.). The WileyManaging Projects Resource Book. New York: Wiley.

Fowler, F., Jr. (1993). Survey Research Methods (2nd ed.). Newbury Park, CA:Sage.

Gause, D., & Weinberg, G. (1989). Exploring Requirements: Quality Before Design.New York: Dorset House.

Goguen, J., & Linde, C. (1993). Software requirements analysis and specifica-tion in Europe: An overview. First International Symposium on RequirementsEngineering (pp. 152–64). Los Alamitos, CA: IEEE Computer Society Press.

Gottesdiener, E. (2002). Requirements by Collaboration. Reading, MA: Addison-Wesley.


299

Greiner, M., & Franza, R. (2003). Barriers and bridges for successful environ-mental technology transfer. Journal of Technology Transfer, 28(2), 167–77.

Haag, S., Raja, M., & Schkade, L. (1996). Quality function deployment—Usagein software development. Communications of the ACM, 39(1), 41–49.

Harel, D. (1987). Statecharts: A visual formalism for complex systems. Science of Computer Programming, 8, 231–74.

Heslop, L., McGregor, E., & Griffith, M. (2001). Development of a technologyreadiness assessment measure: The cloverleaf model of technology transfer.Journal of Technology Transfer, 26(4), 369–84.

Hickey, A., & Davis, A. (2002). The role of requirements elicitation techniquesin achieving software quality. Requirements Engineering Workshop: Founda-tions for Software Quality (REFSQ ’02) (pp. 165–71). Essen, Germany:Essener Informatik Beiträge.

Hickey, A., & Davis, A. (2003). Elicitation technique selection: How do expertsdo it? 11th IEEE International Requirements Engineering Conference. Los Alamitos, CA: IEEE Computer Society Press.

Hudlicka, E. (1996). Requirements elicitation with indirect knowledge elicita-tion techniques: Comparison of three methods. 2nd International Conferenceon Requirements Engineering (ICRE ’96) (pp. 4–11). Los Alamitos, CA: IEEE Computer Society Press.

Jacobson, I., Christerson, M., Jonsson, P., & Overgaard, G. (1992). Object-Oriented Software Engineering: A Use Case Driven Approach. Reading, MA:Addison-Wesley.

Jones, C. (1991). Applied Software Measurement: Assuring Productivity andQuality. New York: McGraw Hill.

Jones, C. (1995). Why is technology transfer so hard? IEEE Computer, 28(6),86–87.

Jones, C. (1996). Patterns of Software Systems Failure and Success (p. 55). Boston, MA: Thomson Computer Press.

Kotonya, G., & Sommerville, I. (1998). Requirements Engineering. New York:Wiley.

Kowal, J. (1992). Behavior Models. Upper Saddle River, NJ: Prentice Hall.Kremic, T. (2003). Technology transfer: A contextual approach. Journal of

Technology Transfer 28, 149–58.Lauesen, S. (2002). Software Requirements: Styles and Techniques. London:

Addison-Wesley.Leffingwell, D., & Widrig, D. (2000). Managing Software Requirements.

Reading, MA: Addison-Wesley.Macaulay, L. (1996). Requirements Engineering. London: Springer.Maiden, N., & Rugg, G. (1996). ACRE: Selecting methods for requirements

acquisition. Software Engineering Journal, 11(5), 183–92.Moore, G. (1991). Crossing the Chasm. New York: HarperCollins.Osborne, A. (1963). Applied Imagination. New York: Scribner.Parkin, A. (1980). Systems Analysis. Cambridge, MA: Winthrop.Potts, C. (1993). Software engineering research revisited. IEEE Software, 10(5),

19–28.


300

Rational Software Corporation (1997). UML Summary, January 1997.Redwine, S., & Riddle, W. (1985). Software technology maturation. Eighth IEEE

International Conference on Software Engineering (ICSE) (pp. 189–200). Los Alamitos, CA: IEEE Computer Society Press.

Rogers, E. (1995). Diffusion of Innovations (4th ed.). New York: Free Press.Ross, D. (1977). Structured analysis: A language for communicating ideas.

IEEE Transactions on Software Engineering, 3(1), 16–34.Seidman, I. (1998). Interviewing as Qualitative Research: A Guide for Researchers

in Education and the Social Sciences (2nd ed.). New York: Teachers CollegePress.

Standish Group (1995). The chaos report. Available: www.standishgroup.com.Standish Group (1999). Chaos: A recipe for success. Available: www.standish-

group.com.Stocking, G. (1996). After Tylor: British Social Anthropology 1888–1951.

Madison: University of Wisconsin Press.Wiegers, K. (1999). Software Requirements. Redmond, WA: Microsoft Press.Wieringa, R. (1996). Requirements Engineering. Chichester, UK: Wiley.Wood, J., & Silver, D. (1995). Joint Application Development (2nd ed.).

New York: Wiley.Zmud, R. (1983). The effectiveness of information channels in facilitating inno-

vation within software development groups. MIS Quarterly, 7(2), 43–58.

APPENDIX ASURVEY INSTRUMENT

1. Name: (optional) ___________________________________________________________

2. Field of study during education: ___________________________________________________________

3. Number of years involved in requirements: ___________________________________________________________

4. Approximate number of projects in which you played a requirements role: ______________________________________________________

5. Please complete the following table to indicate your knowledge and experience with a variety of techniques. We understand that this list of elicitation techniques is but a small fraction of those that are avail-able, and that some of the techniques listed may overlap with others.

NOTES from the FIELD

301

Thank you for your time!


302

Value (check one column for each technique)

I know of I have It has It has noits existence used it value for value for I don’t

Technique (Y or N) (Y or N) elicitation elicitation know

Brainstorming

Interviewing

Joint Application Development (JAD)

Quality Function Deployment (QFD)

Questionnaires

Prototyping

Facilitated Workshop

Scenarios

Use Cases

Modeling

Unified Modeling Language (UML)

Data Flow Diagrams

Observation

Goal-Oriented Elicitation

Role Playing

Decision Trees

Statecharts

Other (be specific)

____________________

Other (be specific)

____________________

Other (be specific)

____________________

1.3hickey

Documents

customers requirements

requirements phase

requirements experts

requirements literature

transfer of processes

transfer of new processes

termed requirements

information technology