who cares about software process modelling? a first investigation about the perceived value of...

Technische Universität München

Who cares about Software Process Modelling? A First Investigation about the Perceived Value of Process

Engineering and Process Consumption

Joint work withMarco Kuhrmann, TUMAlexander Knapp, University of Augsburg

Daniel Méndez

Technische Universität MünchenGermany

PROFES 2013Paphos, Cyprus

13.06.2013

@mendezfe

Software Processes• Blueprint of all relevant artefacts, activities, and roles

➡ Have underlying paradigm:

Context: Software Process Modelling

Activity Orientation

A

Artefact Orientation

B

SW Process

Analysis

Conceptualisation

Construction

Evaluation

Software Process Modelling• Systematic design and implementation of a software process

➡ Usually conducted as part of an SPI initiative

A. Activity orientation (e.g., RUP)

B. Artefact orientation (e.g., V-Modell XT)

Software Process Paradigms Current State of (reported) Evidence

Activity Orientation

Artefact Orientation

• Current view based on mapping study (EASE’ 13)

➡Only few reports on evaluation papers

• Current view based on own experiences & case studies

➡ Indicate to benefits regarding quality in artefacts and flexible process

Software Process Paradigms The truth remains...

Current studies focus on• Requirements engineering (having its own particularities) and/or

• Socio-economic contexts with given experiences, expectations and desires about particular paradigms (based on particular goals)

Why Experimentation? Taking some steps back

➡ Need for experimentation• What implications have the paradigms in “nearly context-free” situations:

– No expectations and limited experiences– Without particular pre-defined improvement goals

Experimental Set-Up Goals and Coarse Setting

Research Objectives Analyse the perceived value of a chosen paradigm from the perspective of process engineers and process consumers in context of process life cycle

Working hypothesis The selection of a paradigm for establishing a process management does not affect its actual consumption.

Research questions (condensed)1. How suitable is a paradigm to cover the needs of process engineers?2. To what extent does a paradigm matter to process consumers?

• Controlled environment / setting

• Pre-defined treatments & assessment criteria

• Randomisation

Still no controlled experiment with

statistical hypothesis testing

Process Framework

• Process Frameworks with underlying paradigm-associated meta model and tool support

• Activity orientation: Eclipse Process Framework (Composer)

• Artefact orientation: V-Modell XT (Editor)

Experimental Set-Up Cases and Subjects

Process

• Workshop organisation process of a German interest group on “Software Development Processes” / German Computer Society

Subjects

• Two groups covering each both roles (process engineers and process consumers)

• 8 Students from the course “Software Engineering Processes”

Experimental Set-Up Data Collection Procedure

Analysis Conceptualisation Construction Evaluation

Workshop 1Workshop 2

Workshop 3Audit

Analysis Concept

ImplementationConcept

Consolidation

Implementation(Software Process)

Interviews

Process Engineers

Process Consumers

Context

• Overview as details of phases

• Covered in lecture

• Conducted as own workshop

Procedure

• Assignment of subjects into two groups /paradigms

• Consolidation by lecturers

• Cross-examination at the end (audit)

Workshops Goals and Results




Goals1. Understand process2. Elicit and sort process elements

input



Goals1. First sketch of process elements2. Clustering and dependencies

Artefacts

Process structure



Goals1. Tool-supported implementation2. Export process documentation



Goals1. Evaluation (engineers)2. Evaluation/Audit (consumers)

5

5,5

6

6,5

7

Q1-8: Overall completeness of

artefacts

Q1-9: Completeness roles

Q1-10: Completeness

artefacts

Q1-11: Completeness relationships

Q1-12: Completeness

activities

Q1-13: Completeness overall process

EPF

V-Modell XT

4,67 5,00

6,00

6,33

7,00

7,00 6,67

5,67

6,33

6,67

6,00

4,20

3,80

3,00

4,00

5,20

6,00 6,20

4,40

3,80

4,75

4,67

Q2-1: HTML export completeness

Q2-2: HTML export accessibility

Q2-3: Overall process presentation

Q2-4: Process verifiability

Q2-5: Implementation completeness

Q2-6: Completeness roles Q2-7: Completeness artefacts


Q2-9: Completeness activities

Q2-10: Implementation adequateness

Q2-11: Process consistency

EPF VMXT

• Artefact-oriented framework supports– Completeness in artefacts and responsibilities (roles)– Completeness of relationshps

• Activity-oriented framework supports– Completeness in activities, but also...– Overall completeness of artefacts

5

5,5

6

6,5

7

Q1-8: Overall completeness of

artefacts

Q1-9: Completeness roles

Q1-10: Completeness

artefacts


Q1-12: Completeness

activities

Q1-13: Completeness overall process

EPF

V-Modell XT

Evaluation from Process Engineers

4,67 5,00

6,00

6,33

7,00

7,00 6,67

5,67

6,33

6,67

6,00

4,20

3,80

3,00

4,00

5,20

6,00 6,20

4,40

3,80

4,75

4,67

Q2-1: HTML export completeness

Q2-2: HTML export accessibility

Q2-3: Overall process presentation

Q2-4: Process verifiability

Q2-5: Implementation completeness

Q2-6: Completeness roles Q2-7: Completeness artefacts


Q2-9: Completeness activities

Q2-10: Implementation adequateness

Q2-11: Process consistency

EPF VMXT

Evaluation from Process Consumers

• Activity-oriented process export overall better rated than artefact-oriented export, e.g.:– Process consistency– Completeness relationships

• Not expected: Activity-oriented process export rated as better regarding– Completeness artefacts – Completeness roles

?

Summary

• Artefact orientation seems to be perceived of higher value by process engineers

• No similar effects for value perceived by process consumers– Activity-oriented export rated overall better– Most surprising: artefact completeness rater better in activity-oriented export

Threats to validity?• Construct: Completeness of criteria?• Internal validity: Mistakes during export?• External: Barely given, but necessary first step!

Our impression: We are still here...

➡Future work:• Further experimentation

(starter kit available soon!)• More differentiated view

on paradigms

Thank you!