kits to find the bits that fits

1

Kits to Find

the Bits that Fits

[email protected]

SAM’15, ICSE, Florence Italy, May 2015

Learning adaptive architectural

principles in the post-agile world

SOUNDS BITES

• Agile (is old)

• There are only three numbers

• Zero, one or many

• Not architecture, but architectureS,

• Architectures will change

• Rework cheaper than we thought

• Programmers (soon) will be much cheaper

• Architectural incubators

• The age of the app

• Variability modeling (feature maps)

2

Before we begin

FYI: I really like

architecture(s)

ARCHITECTURES = FENCES

Something that lets you work alone …

• … half the time

Team members

• can be productive in isolation

• But know when they need to talk to others

Good fences make good neighbors

4

LARGE SCALE ARCHITECTURES:

BOEING 787

5

Agile (is old)

ARCHITECTURES IN

THE POST-AGILE WORLD

• The following may be old news to some of you

• But for the rest:

• Agile is old hat.

• Welcome to DevOps

• And after DevOps?

• Continuous Deployment

• standard practice by 2020

• tools like MEAN

8

9

Source: D. Reimer, A. Cho, Moving to DevOps and Beyond: https://goo.gl/25PxmH

10


11


12


QUESTION

How to make architectural reasoning flexible enough to

handle this pace of change?

13

Adaptive conclusions

and the PROMISE

Project

THE PROMISE REPO

OPENSCIENCE.US/REPO#storingYourResearchData

URL

• openscience.us/repo

• Data from 100s of projects

• E.g. EUSE:

• 250,000K+ spreadsheets

Oldest continuous repository of SE data

• For other repos, seeTable 1 of goo.gl/UFZgnd

15

Serve all our data, on-line

The PROMISE Project: 2005… 2015

TRANSFERRING LESSONS LEARNED:

TURKISH TOASTERS TO

NASA SPACE SHIPS

16

B.Turhan,

T.Menzies, A.

Bener, J. Di

Stefano.

2009. On the

relative value

of cross-

company and

within-

company

data for

defect

prediction.

Empirical

Softw. Eng.

14(5) 2009,

17

Perspective on

Data Science

for Software

Engineering

Tim Menzies

Laurie Williams

Thomas

Zimmermann

2014 2015 2016

Lessons Learned

Our summary. And other related

books The MSR

community and others

Initial, naïve, view:

• Collect enough data …

• … and the truth will emerge

Reality:

• The more data we collected …

• … the more variance we observed

• Its like the microscope zoomed in

• to smash the slide

So now we routinely slice the data

• Find local lessons in local regions.

18Lessons Learned:

Locality, Locality, Locality

CONUNDRUM

Q: How can I believe in architectural principles?

• That transcend individual projects

• Yet doubt the general value of any particular architecture?

• General, that is, to any other project that this current

project at this current time.

A1: Not general models

• But cost effective ways to find best local models

A2: Architectural incubators (see below)

20

Architectures:

Adaptive?

“THE” ARCHITECTURE = THE

DECISIONS YOU CANNOT CHANGE

• The constants

• The base assumptions,

• made at the start of the

project.

• The decisions that you

• bless, or curse,

• every day of the project.

22

THE THING YOU’D BETTER

GET RIGHT

Since…

24

THE THING YOU’D BETTER

GET RIGHT

Since…

25

Or, if Doug Schmidt, just a few

AND IF YOU GET THE

ARCHITECTURE WRONG

Early mistakes are

most expensive

error to fix (Boehm

1981)

26

27

Architect-

ural

Evolution

of DoD

Combat

Systems.

Doug

Schmidt,

SEI blog,

Nov 15,

2013

28

Architect-

ural

Evolution

of DoD

Combat

Systems.

Doug

Schmidt,

SEI blog,

Nov 15,

2013

29

• Doug Schmidt:

• “The Department of Defense (DoD) must

move away from stove-piped solutions “

• “… towards a limited number of

technical reference frameworks … “

• “based on reusable hardware and

software components and services”

Architect-

ural

Evolution

of DoD

Combat

Systems.

Doug

Schmidt,

SEI blog,

Nov 15,

2013

30

• Tim Menzies

• “a limited number???”

• Only three numbers in the universe

• Zero

• One

• Many

• And if > 1, then expect many many more

• Need software architects: to tame the herd of architectures

Architect-

ural

Evolution

of DoD

Combat

Systems.

Doug

Schmidt,

SEI blog,

Nov 15,

2013

Some good news

1) Rework not so costly

2) Reworkers: cheaper, more plentiful

32

Boehm, 1981

SEI TSP

DATA

33

171 projects

2005 to 2014.

Duration: (46 ,90) days (median, max)

Team size: (7,40) people (median,max)

William Curtis, Forest Shull,

Tim Menzies, Lucas Layton

FSE’15 (submitted)

SEI TSP

DATA

34

171 projects

2005 to 2014.

Duration: (46 ,90) days (median, max)

Team size: (7,40) people (median,max)

COMMENT

it is not crazy to explore extensive architectural evolution

• Continuously, during continuous deployment

Since:

• Cheaper to change code than we thought

• Programming costs are about to plummet (see next slide)

35

PROGRAMMER SALARIES:

ABOUT TO COLLAPSE

• Automatic programming

• Devanbu, ICSE NIER 2015:

• Most code repeats other code

• Large scale auto-coding?

• “The next billion”

• All of India + Africa on-line

• Many programmers,

competing for work

• Better distributed development

• Bird et al ICSE 2009,

Kocaganuli et al ICSE SEIP 2013:

• better ways to divide distributed teams

• “Micro-tasking” : LaToza and van der Hoek

ICSE NIER’15.

• Divide development into tiny (say) 5 minute

steps

• Next-gen distributed development

• Github DevOps, continuous integration

• Yang et al. ICSE NIER’13:

• Crowd sourced development 5 times cheaper

36

Architectural

Incubators

ARCHITECTURES =

PRODUCTIVITY

38

LARGE ASSEMBLES NEED

LARGE ARCHITECTURES

39

• Life expectancy:

• 50 years

• A backbone on which 1000s of sub-systems can be

• born,

• evolved

• replaced

LARGE ASSEMBLES NEED

LARGE ARCHITECTURES

40

• Life expectancy:

• 50 years

• A backbone on which 1000s of sub-systems can be

• born,

• evolved

• replaced Architectural

incubator

ANOTHER LARGE SOFTWARE

ARCHITECTURE (DECADES OLD)

41

Any geeks in this audience?

Recognize this diagram?

Do you know the author?



42




Distributed packet-switching networks (On Distribute

Communications): Rand memorandum RM-3103-PR, August 1964



43




Distributed packet-switching networks (On Distribute

Communications): Rand memorandum RM-3103-PR, August 1964

Architectural

incubator

ANOTHER OLD ARCHITECTURE

(NOT QUITE AS WIDELY-USED)

The Blackboard Model of Problem Solving and the Evolution of

Blackboard Architectures, H. Penny Ni, AI Magazie, 7(2), 1986.

44

ANOTHER SOFTWARE ARCHITECTURE

(NOT QUITE AS WIDELY-USED)

The Blackboard Model of Problem Solving and the Evolution of

Blackboard Architectures, H. Penny Ni, AI Magazie, 7(2), 1986.

45

How AI handled software

engineering in the 1970s

SOME ARCHITECTURES ARE

MORE CHANGE TOLERANT

THAN OTHERS

46

More

brittle

Less

brittle

SOME ARCHITECTURES ARE

MORE CHANGE TOLERANT

THAN OTHERS

47

More

brittle

Less

brittleArchitectural

incubator

PHASING OUT INFLEXIBLE

ARCHITECTURES

48

AFTER LAMP, COMES “MEAN”

• M = MongoDB: a nonSQL DB(nested key-value pairs) ( death to SQL). No data traps

• E = Express.js : controller layer, directing application flow and marshaling data

• A = AngularJS : handles data presentation.

• N = Node.js: an extensive javascript library (look ma, no operating system)

• MEAN: one language up and down the stack (javascript).

• Faster integrated testing.

• Faster invention of new patterns

49

AFTER LAMP, COMES “MEAN”

• M = MongoDB: a nonSQL DB(nested key-value pairs) ( death to SQL). No data traps

• E = Express.js : controller layer, directing application flow and marshaling data

• A = AngularJS : handles data presentation.

• N = Node.js: an extensive javascript library (look ma, no operating system)

• MEAN: one language up and down the stack (javascript).

• Faster integrated testing.

• Faster invention of new patterns

50

Architectural

incubator

FUNCTIONAL PROGRAMMERS EAT

PATTERNS, FOR BREAKFAST 51

def visit(thing):

if isinstance(thing,list):

for x in thing:

for y in visit(x):

yield y

else:

yield thing

def all(thing):

return [x for x in visit(thing)]

map(lambda x: x**0.5, all(x)) # sqrt everything

reduce(lambda x,y: x*y, all(thing)) # mult everything

Architectural

incubatorBTW: MEAN? All Jscript,

all functional

OO PATTERNS : INCUBATOR

OR FREEZE BOX

52

Traditional view: Much like Schmidt’s limited number of technical reference frameworks.

First listed in 1992. Surprisingly few updates since.

Adaptive architecture

research

(at NCSU)

IF EVERYTHING MATTERS,

THEN CAREFULLY DESIGN

EVERYTHING

But what if only little bits matter…

Chris Theisen, Ncstate

• 10,000,000 stack dumpsMs Windows

• So little of the systemimplicated in those errors

• Reaction of MS engineers?

• Lets redesign the dependenciesaround those crash traces

54

THE KEY TO CHANGE: YAGNI

• You ain’t gonna need it (to be architected)

• Curiosuly: most software is never exercised enough to be buggy

• 80% bugs in 20% of the code

• Ostrand & Weyyeuker, AT&T, 2004

• Koru, IEEE Software 2009, Ericsson, IBM

• Tosun et al, IAAI’10: Turkish Software

• Hammill & Goseva IEEE TSE’09, NASA systems

55

WELCOME TO THE

AGE OF THE APP

Old school:

• MsOffice: a locked in world that users never leave

• All software from one vendor

• Large, slow to change, hard to compete

New school:

• The app

• All software from N vendors

• Small, very fast to change

56

FEATURE-ORIENTED, OR

VARIABILITY-ORIENTED, PROGRAMMING

Feature models = a

lightweight method for

defining a space of options

De facto standard for

modeling variability

57

Cross-Tree Constraints

Cross-Tree Constraints

Size ? 10 Features, 8 Rules

LARGE FEATURE MODELS =

ARCHITECTURES

58

Size: 6888 items

Features: 344,000 rules

NO SINGLE

“OPTIMUM” SOLUTION

59

Higher-level

Decision

MakingThe Pareto Front

The Chosen Solution

LEARNING DESIGN CHANGE

(A “LESS” APPROACH)

Multi-objective optimization = navigating competing concerns

• Success criteria = choose features that achieve the user

preferences!

60

Suppose each feature had the following metrics:1. Boolean USED_BEFORE?

2. Integer DEFECTS

3. Real COST

Show me the space of “best options” according to the objectives:1. That satisfies most domain constraints (0 ≤ #violations ≤ 100%)

2. That offers most features

3. Using features with least known defects

4. Using features with least cost

5. That we have used most before

Architectural

incubator

STATE OF THE ART

61

Features

9

290

544

6888

SP

LO

TL

inu

x (

LV

AT

)

Pohl

‘11Lopez-

Herrejo

n ‘11

Henard

‘12

Velazc

o ‘13

Sayyad &

Menzies’13bJohanse

n ‘11

Benavide

s ‘05 Fixing model inconsistencis

27 min. for 94 features

Multi-obj configuration, IBEA

>100 hours for E-Shop, 80% HV

Single-obj, CSP

Up to 25 features

Exponential time

White ‘07, ‘08, 09a, 09b,

Shi ‘10, Guo ‘11

Test covering arrays, timed out on all

ops for Linux 6888 features (and

some ops on smaller FMs)

Multi-obj configuration, IBEA

30 configs in 30 minutes for Linux

6888 features

Objectives

Multi-objSingle-obj

Sayyad&

Menzies

’13a

Career advice

(for a plural world)

63

• Doug Schmidt: “… a limited number of technical reference frameworks …

• Tim Menzies:

• Architectures: zero or one or many

• But perhaps just a few architectural incubators?

• Supported by

• Cheaper programmers (more to them) writing more code, faster, cheaper

• Next gen inference, functional programming, feature-oriented code

Architect-

ural

Evolution

of DoD

Combat

Systems.

Doug

Schmidt,

SEI blog,

Nov 15,

2013

EXPLORE

“ARCHITECTURAL INCUBATORS”

64

Functional

programming

Feature

Models

??

66

CODERS

67

CODERS

You want

to be

the architect

that knows how

to co-ordinate

teams that

commission new

architectureS

while rapidly

retiring

old ones

Cause the

softworld

will be

architectureS in

constant flux as

teams patch

and port old

functionality

to new

architectureS

68

CODERS

You want

to be

the architect

that knows how

to co-ordinate

teams that

commission new

architectureS

while rapidly

retiring

old ones

Cause the

softworld

will be

architectureS in

constant flux as

teams patch

and port old

functionality

to new

architectureS

plural

plural

plural

SOUNDS BITES

• Agile (is old)










69

End of my tale

SOUNDS BITES

• Agile (is old)










70

End of my tale

Back up slides

ARCHITECTURAL ASSESSMENT

FOR SOFTWARE SECURITY

Gary McGraw, Cigital:

72

“50% OF SECURITY FLAWS ARE

ARCHITECTURAL BUGS” -- G. MCGRAW

IMPLEMENTATION BUGS

• Buffer overflow

• String format

• One-stage attacks

• Race conditions

• TOCTOU (time of check to time of use)

• Unsafe environment variables

• Unsafe system calls

• System()

• Untrusted input problems

ARCHITECTURAL BUGS

• Method over-riding problems

(subclass issues)

• Misuse of cryptography

• Compartmentalization problems in

design

• Privileged block protection failure

(DoPrivilege())

• Catastrophic security failure (fragility)

• Type safety confusion error

• Insecure auditing

• Broken or illogical access control (RBAC

over tiers)

• Signing too much code

73

SO YOU’D BETTER GET THE

ARCHITECTURE RIGHT

Architectural assessment methods (e.g. Mylopoulos, Soft

goals

74

ARCHITECTURAL

ASSESSMENT

Assessing options of criteria

• predictability (1), security (2), adaptability (3),

coordinability (4), cooperativity(5), availability (6), integrity

(7), modularity (8), or aggregability (9)

• Which is best? Ask the user for their value propositions

75

INDUSTRIAL RELEVANCE OF

SOFTWARE ARCHITECTURES

Source: AADL, CMU

76

kits to find the bits that fits

Education

company data

current project

data science

datafind local lessons

mean89 source

gl25pxmh questionhow

general value

manynot architecture