experience report: scrum, automotive spice® and functional...

Experience Report: SCRUM, Automotive SPICE® and Functional Safety

Achim HoenowMay 2012

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

Answers we hear much too often

We use SCRUM.

Our Software development

improved a lot because

of SCRUM.

ReleaseAchim Hoenow

© Continental AG copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012


Statements about

delivery content are not

possible. We use SCRUM

please ask again

3 weeks before

the delivery.

ReleaseAchim Hoenow


right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012


Architecture and

Design Documents

are not necessary!

Our use case descriptions

are well enough for

coding.

ReleaseAchim Hoenow


right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

Agenda

1. Continental’s Software SQM Department

2. SCRUM within automotive and automotive tools projects

3. Observed strength and weaknesses of assessed SCRUM projects

4. Fulfilling functional safety process requirements

5. Tailoring, Tailoring, Tailoring

6. Summary and Conclusion

ReleaseAchim Hoenow

© Continental AG


copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

1. Continental’s Software SQM DepartmentResponsiblity of Continental‘s Software Supplier Quality Management

Performing Automotive SPICE Assessments of Continental’s Supplier which deliver

ECU related software and embedded software

IT Software

Supporting ISO 26262 tool classification with Automotive SPICE assessments including safety requirements of ISO 26262 Part 6 and Part 8

Performing ISO 26262 safety audits at software suppliers

Monitoring and assessing suppliers improvement programs

Mandatory Software Quality Contracts with ECU related software suppliers and IT

ReleaseAchim Hoenow

© Continental AG

Mandatory Software Quality Contracts with ECU related software suppliers and IT suppliers

Supporting Continental’s IT and business units in case of software supplier problems

Supporting System Development Assessment with strategic semiconductor suppliers

System Development Assessment strategy presented in 2011’s VDA SYS Conference in Berlin

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

2. SCRUM within automotive and automotive tools projects

Software development projects in automotive industry have a sequential character with a high number of functional and non-functional requirements and strict delivery dates.

Initial Requirements

Collection

Release Planning

Software Realization

Release Validation

Maintenance

Does SCRUM methodology completely matches to the project needs of automotive and high robust software projects?

ReleaseAchim Hoenow

© Continental AG

Product Backlog

Sprint Backlogs

Sprints

Does SCRUM methodology completely matches to the project needs of automotive and high robust software projects?

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

3. Observed strength and weaknesses of assessed SCRUM projectsProject Management (MAN.3)

Observed Strength:

Well established tool support for Backlogs and Sprint monitoring

Detailed effort estimation

Strong monitoring and control of software realization phase

Systematic planned and actual comparison during project

Period project retrospectives (lessons learned) with continues improvements

ReleaseAchim Hoenow

© Continental AG

Observed Weaknesses:

Poor liability of overall planning and committed delivery dates

Process tailoring is not documented reference to SCRUM description not enough

Used process doesn‘t fit to project requirements

Missing systematic risk management

No project management plan in place and project management documents are highly distributed

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

3. Observed strength and weaknesses of assessed SCRUM projectsQuality Assurance (SUP.1)

Observed Strength:

100% of the software code and code changes are reviewed before check-in


ReleaseAchim Hoenow

© Continental AG


Rules for reviewing specification and plans are not defined and supervised

Review methods (e.g. walk-through, inspections) are not specified

No evidence that quality department supports in process tailoring

Process compliance is not checked systematically

Project quality assurance plan not in place

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

3. Observed strength and weaknesses of assessed SCRUM projectsCM, PR and CR-Management (SUP.8, SUP.9, SUP.10)

Observed Strength:

Well established Configuration Management System

Problem and Change Management Systems proceeding with full tool support

Release criteria and requirements for release notes are in place


ReleaseAchim Hoenow

© Continental AG

Hot Fix process is not established

Problem trend statistics are not reported periodically

Baselines cover not all project relevant documents

Configuration Management plan with gaps

No Configuration Management Audit established

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

3. Observed strength and weaknesses of assessed SCRUM projectsRequirements Management (ENG.4)

Observed Strength:

Systematic break down of requirements to user stories

Detailed analysis and prioritization of requirements

Tooling for managing requirements and user stories


ReleaseAchim Hoenow

© Continental AG

Non functional requirements are not covered in requirements specification completely

No evidence of robustness and reliable requirements

Functionally for tracing requirements vertically and horizontally is partly not in place

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

3. Observed strength and weaknesses of assessed SCRUM projectsSoftware Design (ENG.5)

Observed Strength:

Tool support for creating software design

Usage of UML


Phase for creating an architectural design is not in place

ReleaseAchim Hoenow

© Continental AG

Incomplete software architecture including static and dynamic views

No systematic robustness analysis established

Software design is partly covered in user stories but no design rules in place where to document what

Poor description of overall system (architecture and design) which is a high risk for the maintenance of software

Traceability between requirements, architecture and design is partly not established

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

3. Observed strength and weaknesses of assessed SCRUM projectsSoftware Construction (ENG.6)

Observed Strength:

Well established automated unit tests basis on use-cases

Targets for code coverage (line and branch coverage)

Strict rules for check-in software code including code reviews

Monitoring of software KPIs (e.g. static code check, delta lines of code)


ReleaseAchim Hoenow

© Continental AG

Manual tests are partly not documented

Traceability between software design and code partly not established

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

3. Observed strength and weaknesses of assessed SCRUM projectsSoftware Integration and Software Testing (ENG.7, ENG.8)

Observed Strength:

Continues integration with nightly builds

Nightly automated integration tests established

Detailed test specification for project features


ReleaseAchim Hoenow

© Continental AG

No strategy established how to manage regression, stress and feature tests in synchronization with Sprint and release planning

No evidence that stress tests are complete and identified systematically

No evidence that regression tests are complete due to missing requirements of legacy code

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

4. Fulfilling functional safety process requirements

Identified weaknesses in assessed projects seems to be fixable by tailoring the used process according the project needs

No ISO26262 requirement visible which cannot be fulfilled in a SCRUM project but

Software architecture phase has to be established with robustness analysis

Software (black-box) testing has to ensure full and consistent verification of the software product

Specification Reviews have to fulfill safety requirements

ReleaseAchim Hoenow

© Continental AG

SCRUM projects often use development techniques which are state of the art and cover important safety requirements

usage of semi-formal methods for the software design

100% static code check

High branch coverage established with automated unit tests

Code reviews as walk-through reviews

Independent software test (black box)

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

5. Tailoring, Tailoring, Tailoring

SCRUM based projects needs to be tailored to the project needs

Mixing traditional and agile methods should be combined on basis of SCRUM’s strong project management methodology.

Product Backlog

Sprint Backlogs

Sprints

ReleaseAchim Hoenow

© Continental AG

Product Backlog

Sprint Backlogs

Sprints

Initial Requirements

Collection

Release Planning

SW Architecture & Robustness

Analysis

Software Realization

Release Verification and Validation

Maintenance

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012


SCRUM is a useful methodology which reflects needs of today’s software development

Clear strength in project management, requirements management and software coding

Weaknesses in software architecture, software testing and quality assurance can be closed by tailoring the process to the project needs

SCRUM should be integrated in companies’ process kits with clear tailoring rules

ReleaseAchim Hoenow

© Continental AG

The project shall be in focus and not the agile method itself

copy

right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012

Answers we like to hear in the future

We tailor our SCRUM

process according the

project needs and

now we even fulfill

ISO 26262 part 6

requirements.

ReleaseAchim Hoenow


right

VD

A A

utom

otiv

e S

YS

Con

fere

nce

| M

ay 2

012