on the distinction of functional and quality requirements in practice

Technische Universität München

On the Distinction of Functional and Quality Requirements in Practice

Daniel Méndez

Technical University of MunichGermany

PROFES 2016Trondheim, Norway

@mendezfeJoint work withJonas Eckhardt (Technical University of Munich)Andreas Vogelsang (Technical University of Berlin)

Objective

— RQ 1

— RQ 2

— RQ 3

We want to better understand

• whether practitioners handle Functional Requirements (FR) differently from Quality Requirements (QR),

• why they do so, and

• what the consequences are.

MethodologyOverview

Vehicle: Survey Research

Purpose: Exploratory / Curiosity-driven

Target Population: Practitioners handling requirements (directly or indirectly)

Data Collection: Feb 4th - Feb 22nd

Data Analysis: Descriptive statistics and manual coding of free-text answers

Details: goo.gl/EppYXr– Instrument*– Raw data – Coding results

Demographics

* Instrument (overview)

Documentation Practice

No documentation of QR

Documentation of QR

No distinction(from FR)

Distinction (from FR)

RQ 1

ReasoningRQ 2

Consequences (problems and benefits)RQ 3

http://goo.gl/EppYXr

Study resultsDemographics / Sample Characteristics

• 103 responses considered for data analysis (283 reached via different channels)

• 93% of respondents with more than 3 years experience

??

0 0,25 0,5 0,75 1

0,0137%21%41%rather agilerather plan-drivenmixeddid not know...

Software processes mix of agile and plan-driven

0 0,25 0,5 0,75 1

0,0534%37%24%embedded systemsbusiness inf. systemshybrid systemsconsumer SW

Balanced families of systems

Diverse project team sizes

0 0,25 0,5 0,75 1

0,0624%46%24%< 11 team members11-50 team members> 50 team membersdid not know...

Requirements specification has an important role

0 0,25 0,5 0,75 1

19%23%57%for in-house developmentfor external developmentsubcontractors using SRS

Study resultsRQ 1 - Do practitioners handle QRs differently?

Do practitioners document QRs (at all)?

0 0,25 0,5 0,75 1

12%88% DocumentationNo documentation

0%

25%

50%

75%

100%

Agile Mixed Plan-driven

Document QRs No QRs

blocking


Do practitioners differentiate between QRs and FRs in the documentation?

0 0,25 0,5 0,75 1

15%85%DistinctionNo distinction

0%

25%

50%

75%

100%

Agile Mixed Plan-driven

Distinction No distinction

blocking


To what extent do development activities for QRs differ from activities for FRs?

Distinction No distinctionRE

Arch/DesignImpl

Testing

0% 25% 50% 75% 100% 0% 25% 50% 75% 100%

Differs strongly Differs slightly Does not differ at all Don't Know

52% 31%12%48% 31% 8%13%

27% 43% 13% 17%23% 47% 26%

57% 21%21%21% 43% 21% 14%

36% 43% 14%7%14% 36% 43% 7%

Distinction No distinctionRE

Arch/DesignImpl

Testing

0% 25% 50% 75% 100% 0% 25% 50% 75% 100%

Differs strongly Differs slightly Does not differ at all Don't Know

52% 31%12%48% 31% 8%13%

27% 43% 13% 17%23% 47% 26%

57% 21%21%21% 43% 21% 14%

36% 43% 14%7%14% 36% 43% 7%

“NFRs are drivers for architectural decisions. Tests depend strongly on NFRs.”

“It does not make any difference. Both [QR and FR] are important.”

versus

Study resultsRQ 2 & 3 - Reasons, Problems, and Benefits

Distinction / No Distinction

Reason (#Occurence)

Reason (#Occurence)

Project phase

Project phase

Reason (#Occurence)

Reason (#Occurence)

Consequence (#Occurence)


Project phase

Project phase



RQ 2 What are the reasons for distinguishing (or not) between QRs and FRs in the documentation?

RQ 3 What are (positive / negative) consequences of distinguishing (or not) QRsand FRs in the documentation?


General&ProjectOrganiza2on

RE

Design&Implementa2on

Valida2on&Verifica2on

InfluencetheArchitecture(6)

QRsrequiredifferentverifica9onmethods(9)QRscanonlybereviewed(2)


RE



Focusedimplementa9on(1)


RE



FocustoomuchonFR(2)

Latearchitecturalchanges(2)

Uncleartes9ngprocessforQRs(2)

Distinction btw. QRs and FRs

Compliancetocustomerconstraints(2)

CompanyPrac9ce(10)

QRsarecross-func9onal(8)Improvedreuse(3)

Differentresponsibility(3)Differentexper9se(2)

Findinforma9oninoneplace(2)

QRshavedifferentnature(10)

QRscannotbecompletedbeforeendoftheproj.(2)

CompletenessofRequirements(3)

ReduceAmbiguity(2)

Differentstakeholders(4)

QRdefectmayleadtoweakuseracceptance(2)QRsareforgoUen(2)

Dis9nc9onbetweenQRandFRunclear(2)QRsareneglected(2)

Traceabilitybecomesexpensive(4)


Separa9onofconcerns(3)IncreasedawarenessofQRs(4)

StructuredProcess(4)ReuseofQRs(4)

IncreasedProductQuality(2)BeUerdomain/systemunderstanding(2)


ClearerresponsibilityforQRs(2)

IncreasedawarenessofQRs(2)


FocusedDesign(2)

FocusedTests(4)ExplicitQRTests(3)

ReduceAmbiguity(2)Differentpriority(2)

Distinguishing QRs and FRs


RE



InfluencetheArchitecture(6)

QRsrequiredifferentverifica9onmethods(9)QRscanonlybereviewed(2)


RE



Focusedimplementa9on(1)


RE



FocustoomuchonFR(2)

Latearchitecturalchanges(2)

Uncleartes9ngprocessforQRs(2)

Distinction btw. QRs and FRs

Compliancetocustomerconstraints(2)

CompanyPrac9ce(10)

QRsarecross-func9onal(8)Improvedreuse(3)

Differentresponsibility(3)Differentexper9se(2)


QRshavedifferentnature(10)

QRscannotbecompletedbeforeendoftheproj.(2)


ReduceAmbiguity(2)

Differentstakeholders(4)

QRdefectmayleadtoweakuseracceptance(2)QRsareforgoUen(2)

Dis9nc9onbetweenQRandFRunclear(2)QRsareneglected(2)

Traceabilitybecomesexpensive(4)


Separa9onofconcerns(3)IncreasedawarenessofQRs(4)

StructuredProcess(4)ReuseofQRs(4)

IncreasedProductQuality(2)BeUerdomain/systemunderstanding(2)


ClearerresponsibilityforQRs(2)



FocusedDesign(2)

FocusedTests(4)ExplicitQRTests(3)

ReduceAmbiguity(2)Differentpriority(2)


General&ProjectOrganiza2onRE



QRsarealreadytranslatedintoFRs(1)

Tolimitresources(1)Missingguideline(1)

Toolinglimita9ons(1)Compliancetocustomerconstraints(1)

Thereisnodifference(2)

DifferentPriority(1)

Separa9onresultsinredesignsinlateprojectphases(1)


RE




Cohesivedocuments(1)Noaddi9onaltrainingrequired(1)

Avoidslateredesign(1)Morefreedomintheimply.ofQRs(1)

Morecomprehensivetests(2)


RE



Decreasedproductquality(1)Tracingbecomesexpensive(1)

FocustoomuchonFRs(1)

CustomerspecifiesunfeasibleQRs(1)

QRsareunfeasible(1)

MissingV&VforQRs(1)

No distinction btw QRs and FRs

Reqs.alreadyonadetailedlevel(1)

Noaddi9onaltrainingrequired(1)

No distinction between QRs and FRs

General&ProjectOrganiza2onRE



QRsarealreadytranslatedintoFRs(1)

Tolimitresources(1)Missingguideline(1)

Toolinglimita9ons(1)Compliancetocustomerconstraints(1)

Thereisnodifference(2)

DifferentPriority(1)

Separa9onresultsinredesignsinlateprojectphases(1)


RE




Cohesivedocuments(1)Noaddi9onaltrainingrequired(1)

Avoidslateredesign(1)Morefreedomintheimply.ofQRs(1)

Morecomprehensivetests(2)


RE



Decreasedproductquality(1)Tracingbecomesexpensive(1)

FocustoomuchonFRs(1)

CustomerspecifiesunfeasibleQRs(1)

QRsareunfeasible(1)

MissingV&VforQRs(1)

No distinction btw QRs and FRs

Reqs.alreadyonadetailedlevel(1)

Noaddi9onaltrainingrequired(1)

Some observationsReasoning not clear

» Contrary arguments “Both are requirements” versus “[they] are different”

» Same line of reasoning“increase awareness” by distinguishing and by not distinguishing QRs and FRs

Heterogenous, extrinsic decision drivers

» Lack of guidance“There is no real guidelines how to do it”

» Enforced guidance“[…] as requested by customer”, “Our specification template prescribes […]”

» Unclear difference in practice “Most people have problems to distinguish them, so they mix”

Unclear decision implications

» Impact on testingDistinguishing leads to more specialised tests, but also to not testing them at all

» (potential) Conventional wisdom Arguments from literature not underpinned by consequences

Take-AwaysReasoning not clear

» Contrary arguments “Both are requirements” versus “[they] are different”

» Same line of reasoning“increase awareness” by distinguishing and by not distinguishing QRs and FRs

What we haveconventional wisdomreasoning based on norms

Heterogenous, extrinsic

decision drivers

» Lack of guidance

“There is no real guidelines how to do it”

» Enforced guidance

“[…] as requested by customer”, “O

ur

specification template prescribes […

]”

» Unclear difference in practice

“Most people have problems to

distinguish them, so they mix”

Unclear decision implications» Impact on testingDistinguishing leads to more specialised tests, but also to not testing them at all» (potential) Conventional wisdom

Arguments from literature not underpinned by consequences

What we lackevidence

Takk!Daniel Méndez

[email protected]

@mendezfe

Approach me if you need material (study details & raw data)

Future workIncrease understanding about• implications of (not) distinguishing QRs from FRs• where preconceptions emerge from

➡ Contribute to testable theories in RE to debunk “(RE) Leprechauns”

on the distinction of functional and quality requirements in practice

Engineering