Artificial software diversity: automatic synthesis of program sosies Benoit Baudry

Joint work with Simon Allier, Ioannis Kavvouras, Julien Langlois and Martin Monperrus

Diversity to handle / increase uncertainty

l  Navigate through the program space l  brittleness versus plasticity of software

l  Failure detection l  Moving target l  Self-repair of software

Program sosie

• Given a specification

3

Program sosie

4

• Given a specification S • Given a program P that conforms to S

correct implementation

Program sosie

5

• Given a specification S • Given a program P that conforms to S • A sosie of P is a variant that also conforms to S

a sosie

Program sosie

6

potential failures or breaches

failure diversity

• Given a specification S • Given a program P that conforms to S • A sosie of P is a variant that also conforms to S

Specification: data and properties

l  The test input data specifies the input domain l  The assertions specify the level of abstraction

fun : Function assert abs(fun(.5) - 0.25) < 0.05 assert abs(fun(.4) - 0.16) < 0.05 assert abs(fun(.3) - 0.09) < 0.05

Sosies and Diversity

l  There is a diversity of sosies l  There may be a diversity of output outside the

specified domain l  The specified input domain and the

associated level of abstraction allows more or less diversity

The diversity is a dependent variable of the input domain and level of abstraction

Sosies are not

l  The identity / the clone l  Program equivalence (the same output for all

possible inputs) l  The same output

l  on the specified input domain

l  at a given level of abstraction

l  Could be called "phenotypic equivalence"

Research questions

Do sosies exist? Can we automatically synthesize them? What are effective transformations?

10

Automatic Synthesis of Sosies

l  We replace a given piece of code by another one and see whether all assertions remain satisfied

l  Pieces of code: l  Method calls

l  Methods

l  Expressions

l  ...

Example of sosie

12

@Override public void report(SortedMap<String, Gauge> gauges, SortedMap<String, Counter> counters, SortedMap<String, Histogram> histograms, SortedMap<String, Meter> meters, SortedMap<String, Timer> timers) { final long timestamp = TimeUnit.MILLISECONDS.toSeconds(clock.getTime()); for (Map.Entry<String, Gauge> entry : gauges.entrySet()) { reportGauge(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Counter> entry : counters.entrySet()) { reportCounter(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Histogram> entry : histograms.entrySet()) { reportHistogram(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Meter> entry : meters.entrySet()) { reportMeter(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Timer> entry : timers.entrySet()) { reportTimer(timestamp, entry.getKey(), entry.getValue()); } }

Example of sosie

13

@Override public void report(SortedMap<String, Gauge> gauges, SortedMap<String, Counter> counters, SortedMap<String, Histogram> histograms, SortedMap<String, Meter> meters, SortedMap<String, Timer> timers) { final long timestamp = TimeUnit.MILLISECONDS.toSeconds(clock.getTime()); for (Map.Entry<String, Gauge> entry : gauges.entrySet()) { reportGauge(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Counter> entry : counters.entrySet()) { reportCounter(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Histogram> entry : histograms.entrySet()) { reportHistogram(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Meter> entry : meters.entrySet()) { reportMeter(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Timer> entry : timers.entrySet()) { reportTimer(timestamp, entry.getKey(), entry.getValue()); } }

InputContext:  [long]  OutputContext: void codeFragment: if (least >= bound) throw new java.lang.IllegalArgumentException(); InputContext: [long] OutputContext:  void codeFragment: if (n <= 0) throw new java.lang.IllegalArgumentException("n must be positive");

Example of sosie

14

@Override public void report(SortedMap<String, Gauge> gauges, SortedMap<String, Counter> counters, SortedMap<String, Histogram> histograms, SortedMap<String, Meter> meters, SortedMap<String, Timer> timers) { final long timestamp = TimeUnit.MILLISECONDS.toSeconds(clock.getTime()); for (Map.Entry<String, Gauge> entry : gauges.entrySet()) { reportGauge(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Counter> entry : counters.entrySet()) { if (timestamp <= 0) throw new java.lang.IllegalArgumentException("n must be positive"); } for (Map.Entry<String, Histogram> entry : histograms.entrySet()) { reportHistogram(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Meter> entry : meters.entrySet()) { reportMeter(timestamp, entry.getKey(), entry.getValue()); } for (Map.Entry<String, Timer> entry : timers.entrySet()) { reportTimer(timestamp, entry.getKey(), entry.getValue()); } }

variable mapping: {n=timestamp} InputContext: [long] OutputContext:  void codeFragment: if (n <= 0) throw new java.lang.IllegalArgumentException("n must be positive");

What is effective for sosiefication?

• Evaluate the efficiency of 9 transformations: •  Replace/Add/Delete

•  CMNVM Replace/Add: context mapping but not variable mapping

•  NCMVMN Replace/Add: not context mapping but mapping on variable name

•  NCM Replace/Add: no context mapping

15

Preliminary result

NCM Replace

NCM/VNM

Replace

CM/NVM

Replace Normal Replace

NCM Add

NCM/VNM Add

CM/NVM Add

Add Delete

junit #trial

#variant #sosie

500 36 2

500 80 27

500 177 32

500 310 43

500 38 33

500 61 42

500 140 70

500 195 79

500 253 25

metrics #trial

#variant #sosie

"

1960 116 12

1960 282 69

1960 693 86

1960 1262 174

1960 157 108

1960 269 142

1960 700 352

1960 908 347

1960 977 110

clojure #trial

#variant #sosie

680 30 0

680 123 3

680 154 7

680 342 21

680 30 7

680 90 3

680 124 35

680 152 28

680 391 29

Sosiefication with reactions #variants #incorrect-

variants #sosies % sosies

JUnit 5265 4377 888 16.86% Metrics 4699 4299 400 8.51% Codec 14435 11080 3355 23.24% Math 45517 40500 5017 11.02%

Clojure 32335 20706 11629 35.96% bubble-sort 23 18 4 17.39% insert-sort 18 17 1 1.26% quick-sort 553 525 7 0.42% merge-sort 4759 4739 20

What to do with sosies? l  Demonstrate plastic properties of software l  Functional equivalence and repair

l  Functional resillience; replacement in case of bugs (ICSE'13 Gorla et al.)

l  Use as gene pool l  Randomize execution to create a moving target l  Functional sense of self ("detect-fast")

The diversity of functionally equivalent code improves the robustness and resilience

of software