neutral networks of real-world programs and their application to...
TRANSCRIPT
Neutral Networks of Real-World Programs
and their Application to
Automated Software Evolution
Eric Schulte
Department of Computer ScienceUniversity of New Mexico
Albuquerque, NM
July 2014
Neutral Networks and Automated Software Evolution Introduction 0
Software EngineeringMotivation
Bugs cost estimated $312 billion per year[Britton, University of Cambridge 2013]
Security millions of dollars per exploit[Van, OECD Publishing 2008]
Resources 1% of global energy to data centers in 2010[Koomey, Analytics Press 2011]
Developers projected 1.5 million in US by 2018[U.S Department of Labor, 2011]
Neutral Networks and Automated Software Evolution Introduction 1
Software EngineeringMotivation
Bugs cost estimated $312 billion per year[Britton, University of Cambridge 2013]
Security millions of dollars per exploit[Van, OECD Publishing 2008]
Resources 1% of global energy to data centers in 2010[Koomey, Analytics Press 2011]
Developers projected 1.5 million in US by 2018[U.S Department of Labor, 2011]
Neutral Networks and Automated Software Evolution Introduction 1
Software EngineeringMotivation
Bugs cost estimated $312 billion per year[Britton, University of Cambridge 2013]
Security millions of dollars per exploit[Van, OECD Publishing 2008]
Resources 1% of global energy to data centers in 2010[Koomey, Analytics Press 2011]
Developers projected 1.5 million in US by 2018[U.S Department of Labor, 2011]
Neutral Networks and Automated Software Evolution Introduction 1
Software EngineeringMotivation
Bugs cost estimated $312 billion per year[Britton, University of Cambridge 2013]
Security millions of dollars per exploit[Van, OECD Publishing 2008]
Resources 1% of global energy to data centers in 2010[Koomey, Analytics Press 2011]
Developers projected 1.5 million in US by 2018[U.S Department of Labor, 2011]
Neutral Networks and Automated Software Evolution Introduction 1
Software EngineeringEvolution and Automation
Evolved Ecosystem
I applications
I libraries
I compilers
I operating systems
I architectures
Evolvable Software
I automated diversity
I assembly and binary repair
I patch closed source executable
I optimization
Neutral Networks and Automated Software Evolution Introduction 2
Software EngineeringEvolution and Automation
Evolved Ecosystem
I applications
I libraries
I compilers
I operating systems
I architectures
Evolvable Software
I automated diversity
I assembly and binary repair
I patch closed source executable
I optimization
Neutral Networks and Automated Software Evolution Introduction 2
GenprogAutomatically Fix Bugs in C Software
Neutral Networks and Automated Software Evolution Introduction 3
GenprogAutomatically Fix Bugs in C Software
Collaboration between UNM and UVA
Dr. Stephanie Forrest Dr. Westley Weimer
Neutral Networks and Automated Software Evolution Introduction 3
GenprogAutomatically Fix Bugs in C Software
Input
XXXXX
EvolveFitness Evaluation
Mutants
Output
XXXXXDiscard
Accept
Minimize
[Le Goues, 2013, Figure 3.2]
Neutral Networks and Automated Software Evolution Introduction 3
GenprogAutomatically Fix Bugs in C Software
Strengths
Effective Repaired 55/105 bugs for $8 each
General Multiple classes of bugs and security defects
Best Papers ICSE 2009, GECCO 2009, SBST 2009
Humies Gold 2009, Bronze 2012
Neutral Networks and Automated Software Evolution Introduction 3
GenprogAutomatically Fix Bugs in C Software
ButWhy doesn’t this break my software?
Neutral Networks and Automated Software Evolution Introduction 3
Outline
Introduction
Program Representation and Transformation
Mutational Robustness and Neutral Networks
Application: Program Diversity
Application: Assembler- and Binary-Level Program Repair
Application: Patching Closed Source Executables
Application: Optimizing nonfunctional Program Properties
Future Work
Conclusion
Neutral Networks and Automated Software Evolution Program Representation and Transformation 4
Program Representation
Sourceif(a==0){
printf("%g\n", b); }
else {
while(b!=0){
if(a>b){ a=a-b; }
else { b=b-a; }}}
printf("%g\n", a);
Tree
: Cil, CLang
if(a==0)
printf("%g\n",b); while(b!=0)
if(a>b)
a=a-b; else
b=b-a;
printf("%g\n",a);
Vector
: ASM, LLVM, ELF
main:
.cfi startproc
pushq %rbp
.cfi def cfa offset 16
.cfi offset 6, -16
movq %rsp, %rbp
.cfi def cfa register 6
subq $48, %rsp
Neutral Networks and Automated Software Evolution Program Representation and Transformation 5
Program Representation
Sourceif(a==0){
printf("%g\n", b); }
else {
while(b!=0){
if(a>b){ a=a-b; }
else { b=b-a; }}}
printf("%g\n", a);
Tree
: Cil, CLang
if(a==0)
printf("%g\n",b); while(b!=0)
if(a>b)
a=a-b; else
b=b-a;
printf("%g\n",a);
Vector
: ASM, LLVM, ELF
main:
.cfi startproc
pushq %rbp
.cfi def cfa offset 16
.cfi offset 6, -16
movq %rsp, %rbp
.cfi def cfa register 6
subq $48, %rsp
Neutral Networks and Automated Software Evolution Program Representation and Transformation 5
Program Representation
Sourceif(a==0){
printf("%g\n", b); }
else {
while(b!=0){
if(a>b){ a=a-b; }
else { b=b-a; }}}
printf("%g\n", a);
Tree
: Cil, CLang
if(a==0)
printf("%g\n",b); while(b!=0)
if(a>b)
a=a-b; else
b=b-a;
printf("%g\n",a);
Vector
: ASM, LLVM, ELF
main:
.cfi startproc
pushq %rbp
.cfi def cfa offset 16
.cfi offset 6, -16
movq %rsp, %rbp
.cfi def cfa register 6
subq $48, %rsp
Neutral Networks and Automated Software Evolution Program Representation and Transformation 5
Program Representation
Representations
Rep. Type Languages
CLang tree C-family sourceCil tree C Int. Lang.
LLVM vector LLVM SSA IRASM vector x86 & ARMELF vector x86 & MIPS
Tree: Cil, CLangif(a==0)
printf("%g\n",b); while(b!=0)
if(a>b)
a=a-b; else
b=b-a;
printf("%g\n",a);
Vector: ASM, LLVM, ELFmain:
.cfi startproc
pushq %rbp
.cfi def cfa offset 16
.cfi offset 6, -16
movq %rsp, %rbp
.cfi def cfa register 6
subq $48, %rsp
Neutral Networks and Automated Software Evolution Program Representation and Transformation 5
Program Mutation Operations
Tree Delete
→
Tree Insert
→
Tree Swap
→
Vector Delete Vector Insert Vector Swap
Neutral Networks and Automated Software Evolution Program Representation and Transformation 6
Program Mutation Operations
Tree Delete
→
Tree Insert
→
Tree Swap
→
ELF Delete
0x0
Vector Insert Vector Swap
Neutral Networks and Automated Software Evolution Program Representation and Transformation 6
Program Mutation Operations
Tree Delete
→
Tree Insert
→
Tree Swap
→
ELF Delete
0x0
ELF Replace Vector Swap
Neutral Networks and Automated Software Evolution Program Representation and Transformation 6
LLVM Mutation Illustration
C sourceint main(int argc , char *argv []){
int x=2;
x+=3;
x=x*x;
printf("%d\n", x);
return 0;
}
Neutral Networks and Automated Software Evolution Program Representation and Transformation 7
LLVM Mutation Illustration
LLVM
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %0, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
%mul
x
%3 printf
Neutral Networks and Automated Software Evolution Program Representation and Transformation 7
LLVM Mutation Illustration
LLVM: Delete 4
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %0, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
%mul
x
%3 printf
Neutral Networks and Automated Software Evolution Program Representation and Transformation 7
LLVM Mutation Illustration
LLVM: Delete 4
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %0, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
%mul
x
%3 printf
Neutral Networks and Automated Software Evolution Program Representation and Transformation 7
LLVM Mutation Illustration
LLVM: Insert 9 4
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
%mul
x
%3 printf
Neutral Networks and Automated Software Evolution Program Representation and Transformation 7
LLVM Mutation Illustration
LLVM: Insert 9 4
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
%add.insert = add nsw i32 %0, 3
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
%mul
x
%3 printf
%add2
i32 3
Neutral Networks and Automated Software Evolution Program Representation and Transformation 7
LLVM Mutation Illustration
LLVM: Insert 9 4
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
%add.insert = add nsw i32 %0, 3
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
%mul
x
%3 printf
%add2
i32 3
Neutral Networks and Automated Software Evolution Program Representation and Transformation 7
Conclusion
Representations
Rep. Type Languages
CLang tree C-family sourceCil tree C Int. Lang.
LLVM vector LLVM SSA IRASM vector x86 & ARMELF vector x86 & MIPS
Differences
I Mutation operations andSearch Space
I Languages
I Requirements
I Expression as executables
I Communication ofmutations
Neutral Networks and Automated Software Evolution Program Representation and Transformation 8
Outline
Introduction
Program Representation and Transformation
Mutational Robustness and Neutral Networks
Application: Program Diversity
Application: Assembler- and Binary-Level Program Repair
Application: Patching Closed Source Executables
Application: Optimizing nonfunctional Program Properties
Future Work
Conclusion
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 9
Software Mutational Robustness[Schulte et al., GPEM 2013]
percentage of mutants which are functional
Program Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 10
Software Mutational Robustness
[Schulte et al., GPEM 2013]
percentage of mutants which are functional
Program Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 10
Software Mutational Robustness
[Schulte et al., GPEM 2013]
percentage of mutants which are functional
Program Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 10
Software Mutational Robustness
[Schulte et al., GPEM 2013]
percentage of mutants which are functional
Program Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 10
Definition
MutRB(P ,T ,M)P programT test suiteM mutation operators
MutRB(P ,T ,M) =|{P ′ | m ∈ M . P ′ ← m(P) ∧ T (P ′)}|
|{P ′ | m ∈ M . P ′ ← m(P)}|
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 11
Mutational Robustness by Test SuiteSorters
%M
utR
B
0
10
20
30
40
50
bubble-sort
insertion-sort
merge-sort
quick-sort
Siemen’s
%M
utR
B
0
10
20
30
40
50
printtokens
schedulesed space
tcas
Systems Programs
%M
utR
B
0
20
40
60
80
100
bzip2 1.0.2
– ccrypt 1.2
– imagemagick 6.5.2
jansson 1.3
leukocyte
lighttpd 1.4.15
nullhttpd 0.5.0
oggenc 1.0.1
– potion 40b5f03
redis 1.3.4
tiff3.8.2
vyquon 335426d
grep
Cil ASM
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 12
Fitness Distribution by Representation
0
500
1000
1500
2000
2500
3000
3500
0 1 2 3 4 5 6 7 8 9 10
Num
ber
ofV
aria
nts
Fitness
CLANGCIL
LLVMASMELF
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 13
Fitness Landscapes and Neutral NetworksProgram Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Program Space
Test SuiteTest Suite
Specification
Original Program
Neutral Mutant
Killed Mutant
Original Program
Neutral MutantFitness
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 14
Fitness Landscapes and Neutral Networks
Program Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Program Space
Test Suite
Test Suite
Specification
Original Program
Neutral Mutant
Killed Mutant
Original Program
Neutral Mutant
Fitness
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 14
Fitness Landscapes and Neutral Networks
Program Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Program Space
Test Suite
Test Suite
Specification
Original Program
Neutral Mutant
Killed Mutant
Original Program
Neutral MutantFitness
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 14
Fitness Landscapes and Neutral Networks
Program Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Program Space
Test Suite
Test Suite
Specification
Original Program
Neutral Mutant
Killed Mutant
Original Program
Neutral Mutant
Fitness
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 14
Fitness Landscapes and Neutral Networks
Program Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Program Space
Test Suite
Test Suite
Specification
Original Program
Neutral Mutant
Killed Mutant
Original Program
Neutral Mutant
Fitness
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 14
Fitness Landscapes and Neutral Networks
Program Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Program Space
Test Suite
Test Suite
Specification
Original Program
Neutral Mutant
Killed Mutant
Original Program
Neutral Mutant
Fitness
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 14
Fitness Landscapes and Neutral Networks
Program Space
Test SuiteSpecification
Original Program
Neutral Mutant
Killed Mutant
Program Space
Test Suite
Test Suite
Specification
Original Program
Neutral Mutant
Killed Mutant
Original Program
Neutral Mutant
Fitness
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 14
Span of Neutral Networks
Insertion Sort Neutral Variants
170
180
190
200
210
220
230
0 50 100 150 200 25012
13
14
15
16
17
18
19
20
21
Avg
.L
OC
%N
eutr
alV
aria
nts
Number of Applied Mutations
Avg. LOC% Neutral Variants
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 15
Span of Neutral Networks
Insertion Sort Neutral Variants: size controlled
172
173
174
175
0 50 100 150 200 2504
6
8
10
12
14
16
18
20
Avg
.L
OC
%N
eutr
alV
aria
nts
Number of Applied Mutations
Avg. LOC% Neutral Variants
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 15
Random Walks in Program Space
Quick Sort Random Variants
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 1 2 3 4
Fra
ctio
nof
Var
iant
sw
hich
are
Neu
tral
Number of Applied Mutations
0.35831x
Fraction Found Neutral Experimentally
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 16
Conclusion
Software Neutral Networks are
wide span large distances in program space
thick consistent mutational robustness
accessible automatically explorable
Program Space
Test Suite
Fitness
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 17
Conclusion
Software Neutral Networks are
wide span large distances in program space
thick consistent mutational robustness
accessible automatically explorable
Neutral Networks and Automated Software Evolution Mutational Robustness and Neutral Networks 17
Outline
Introduction
Program Representation and Transformation
Mutational Robustness and Neutral Networks
Application: Program Diversity
Application: Assembler- and Binary-Level Program Repair
Application: Patching Closed Source Executables
Application: Optimizing nonfunctional Program Properties
Future Work
Conclusion
Neutral Networks and Automated Software Evolution Application: Program Diversity 18
Automated Diversity
Proactive Bug Repair
Program Space
Test SuiteSpecification
Original Program
Mutant
Bug
Neutral Networks and Automated Software Evolution Application: Program Diversity 19
Automated Diversity
Proactive Bug Repair
Program Space
Test SuiteSpecification
Original Program
Mutant
Bug
Neutral Networks and Automated Software Evolution Application: Program Diversity 19
Automated Diversity
Proactive Bug Repair
Program Space
Test SuiteSpecification
Original Program
Mutant
Bug
Neutral Networks and Automated Software Evolution Application: Program Diversity 19
Automated Diversity
Proactive Bug Repair
Program Space
Test SuiteSpecification
Original Program
Mutant
Bug
Neutral Networks and Automated Software Evolution Application: Program Diversity 19
Automated DiversityProactive Bug Repair
Program Space
Test SuiteSpecification
Original Program
Mutant
Bug
Neutral Networks and Automated Software Evolution Application: Program Diversity 19
Contribution
Automated Diversity
I Neutral networks expose meaningful diversity[Baudry, ISSTA 2014]
I Diverse variants proactively fix defects
I Select small mutually diverse populations
Neutral Networks and Automated Software Evolution Application: Program Diversity 20
Outline
Introduction
Program Representation and Transformation
Mutational Robustness and Neutral Networks
Application: Program Diversity
Application: Assembler- and Binary-Level Program Repair
Application: Patching Closed Source Executables
Application: Optimizing nonfunctional Program Properties
Future Work
Conclusion
Neutral Networks and Automated Software Evolution Application: Assembler- and Binary-Level Program Repair 21
Program Repair in Embedded Devices[Schulte et al., ASPLOS 2013]
Resource Constraints
I Small disks
I Less memory
I Slow processors
I Slow, costly comm. mai
argc
,ch
ar*a
rgv[
])
mai
n(in
tar
gc,
[])in
ta;
int
b;if(
a!Neutral Networks and Automated Software Evolution Application: Assembler- and Binary-Level Program Repair 22
Genprog
: Assembler- and Binary-Level
Automatically Fix Bugs in C Software
Strengths
Effective Repaired 55/105 bugs for $8 each
General Multiple classes of bugs and security defects
Best Papers ICSE 2009, GECCO 2009, SBST 2009
Humies Gold 2009, Bronze 2012
Neutral Networks and Automated Software Evolution Application: Assembler- and Binary-Level Program Repair 23
Genprog
: Assembler- and Binary-Level
Automatically Fix Bugs in C Software
Strengths
Effective Repaired 55/105 bugs for $8 each
General Multiple classes of bugs and security defects
Best Papers ICSE 2009, GECCO 2009, SBST 2009
Humies Gold 2009, Bronze 2012
Requirements
I Source code
I Build tool chain
I Expensive fault localization
I Expensive fitness function (compilation, test execution)
Neutral Networks and Automated Software Evolution Application: Assembler- and Binary-Level Program Repair 23
Genprog: Assembler- and Binary-LevelAutomatically Fix Bugs in C Software
Strengths
Effective Repaired 55/105 bugs for $8 each
General Multiple classes of bugs and security defects
Best Papers ICSE 2009, GECCO 2009, SBST 2009
Humies Gold 2009, Bronze 2012
Requirements
I Source code
I Build tool chain
I Expensive fault localization
I Expensive fitness function (compilation, test execution)
Neutral Networks and Automated Software Evolution Application: Assembler- and Binary-Level Program Repair 23
Embedded Repair Benchmark Programs
Program Program Description Bugatris graphical tetris game stack buffer exploit
ccrypt encryption utility segfaultderoff document processing segfault
flex lexical analyzer generator segfaultindent source code processing infinite loop
look svr4 dictionary lookup infinite looplook ultrix dictionary lookup infinite loop
merge merge sort duplicate inputsmerge-cpp merge sort (in C++) duplicate inputs
s3 sendmail utility buffer overflowuniq duplicate text processing segfault
units metric conversion segfaultzune embedded media player infinite loop
Neutral Networks and Automated Software Evolution Application: Assembler- and Binary-Level Program Repair 24
Contribution
ASM and ELF representation
I Reduce requirements of automated repairI Effective: fixing 12 (ASM) and 11 (ELF) of 13I 62% faster runtimeI 95% smaller disk footprintI 86% less memory
I Reduce resources needed to perform repair
I New program repair search spaces
Neutral Networks and Automated Software Evolution Application: Assembler- and Binary-Level Program Repair 25
Outline
Introduction
Program Representation and Transformation
Mutational Robustness and Neutral Networks
Application: Program Diversity
Application: Assembler- and Binary-Level Program Repair
Application: Patching Closed Source Executables
Application: Optimizing nonfunctional Program Properties
Future Work
Conclusion
Neutral Networks and Automated Software Evolution Application: Patching Closed Source Executables 26
NETGEAR Exploit[Schulte et al., unpublished]
1. URL starting with BRS bypasses authentication
2. URL including unauth.cgi or securityquestions.cgibypass authentication
3. unprotected page removes authentication for every pagehttp://router/BRS_02_genieHelp.html
Neutral Networks and Automated Software Evolution Application: Patching Closed Source Executables 27
Common Vulnerability
NETGEAR WNDR4700
D-Link DIR-100
Linksys WAG200G
Neutral Networks and Automated Software Evolution Application: Patching Closed Source Executables 28
NETGEAR Repair Overview
Mutate
QEMU
Select Insert
Evaluate
QEMU
QEMU
QEMU
QEMU
Virtual Machines
Threads
Population
Firmware binwalk
Environment
Binary
Minimize
Neutral Networks and Automated Software Evolution Application: Patching Closed Source Executables 29
NETGEAR Repair Overview
Mutate
QEMU
Select Insert
Evaluate
QEMU
QEMU
QEMU
QEMU
Virtual Machines
Threads
Population
Firmware binwalk
Environment
Binary
Minimize
Repair Runtime
evaluations ∼ 36, 000
runtime 86.6 min.
threads 32
Neutral Networks and Automated Software Evolution Application: Patching Closed Source Executables 29
Contribution
Genprog Requirements
I Non-stripped ELF file
I Regression test suite
I Fault localization
Enables
I repair of black box executables
I repair in data sections
I patching proprietary software
Neutral Networks and Automated Software Evolution Application: Patching Closed Source Executables 30
Contribution
Genprog Requirements
I Non-stripped ELF file
I Regression test suite
I Fault localization
Enables
I repair of black box executables
I repair in data sections
I patching proprietary software
Neutral Networks and Automated Software Evolution Application: Patching Closed Source Executables 30
Contribution
Genprog Requirements
I Non-stripped ELF file
I Regression test suite
I Fault localization
Enables
I repair of black box executables
I repair in data sections
I patching proprietary software
Neutral Networks and Automated Software Evolution Application: Patching Closed Source Executables 30
Outline
Introduction
Program Representation and Transformation
Mutational Robustness and Neutral Networks
Application: Program Diversity
Application: Assembler- and Binary-Level Program Repair
Application: Patching Closed Source Executables
Application: Optimizing nonfunctional Program Properties
Future Work
Conclusion
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 31
Optimization Problem[Schulte, Dorn et al., ASPLOS 2014]
Optimizing complex non-functional properties
properties × hardware × environment
properties memory, network, energy, etc. . .
hardware architectures, processors, memory stack, etc. . .
environment variables, load, etc. . .
Every program transformation requires
a-priori reasoning
manual implementation
guaranteed correctness
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 32
Genetic Optimization Algorithm (GOA)Post-compiler, test-driven, Genetic Optimization Algorithm
Post-compiler
source .s GOA .s .exe
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 33
Genetic Optimization Algorithm (GOA)Post-compiler, test-driven, Genetic Optimization Algorithm
Test driven
Use test cases to exercise program
I evaluate functionality
I measure runtime properties
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 33
Genetic Optimization Algorithm (GOA)Post-compiler, test-driven, Genetic Optimization Algorithm
Algorithm
Assembler Fitness Function Workload
Population
Mutate
Profile
Fitness
Minimize
Executable
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 33
Genetic Optimization Algorithm (GOA)Post-compiler, test-driven, Genetic Optimization Algorithm
Algorithm
Assembler Fitness Function Workload
Population
Mutate
Profile
Fitness
Minimize
Executable
pop 210
evals 218
time ∼ 16h
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 33
Benchmark Applications
C/C++ ASMProgram Lines of Code Description
blackscholes 510 7,932 Finance modelingbodytrack 14,513 955,888 Human video trackingfacesim no alternate inputsferret 15,188 288,981 Image search enginefluidanimate 11,424 44,681 Fluid dynamics animationfreqmine 2,710 104,722 Frequent itemset miningraytrace no testable outputswaptions 1,649 61,134 Portfolio pricingvips 142,019 132,012 Image transformationx264 37,454 111,718 MPEG-4 video encoder
total 225,467 1,707,068
Table: PARSEC benchmark applications.
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 34
Hardware Platforms
AMD Server Intel Desktop
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 35
Results: Energy Reduction
0%
20%
40%
60%
80%
100%
blackscholes
bodytrack
ferret
fluidanimate
freqmine
swaptions
vipsx264
Average
Energy Reduction
AMDIntel
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 36
Functionality on Withheld Tests
Program AMD Intel
blackscholes 100% 100%bodytrack 92% 100%ferret 100% 100%fluidanimate 6% 31%freqmine 100% 100%swaptions 100% 100%vips 100% 100%x264 27% 100%
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 37
AnecdotesBlackscholes
I 90% less energy
I removed redundant outer loop
I modified semantics
Swaptions
I 42% less energy
I improved branch prediction
I hardware specific
Vips
I 20% less energy
I substitution of memory access for calculation
I resource trade-off
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 38
AnecdotesBlackscholes
I 90% less energy
I removed redundant outer loop
I modified semantics
Swaptions
I 42% less energy
I improved branch prediction
I hardware specific
Vips
I 20% less energy
I substitution of memory access for calculation
I resource trade-off
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 38
AnecdotesBlackscholes
I 90% less energy
I removed redundant outer loop
I modified semantics
Swaptions
I 42% less energy
I improved branch prediction
I hardware specific
Vips
I 20% less energy
I substitution of memory access for calculation
I resource trade-off
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 38
Contribution
Genetic Optimization Algorithm
1. optimize complex runtime properties (energy)
2. leverages particulars of hardware, and environment
3. reveal compiler inefficiencies
4. find efficiencies, e.g., loop elimination
5. transformations presented as ASM diff
Neutral Networks and Automated Software Evolution Application: Optimizing nonfunctional Program Properties 39
Outline
Introduction
Program Representation and Transformation
Mutational Robustness and Neutral Networks
Application: Program Diversity
Application: Assembler- and Binary-Level Program Repair
Application: Patching Closed Source Executables
Application: Optimizing nonfunctional Program Properties
Future Work
Conclusion
Neutral Networks and Automated Software Evolution Future Work 40
Challenges
Interface
I Guarantee semantics preservation
I Communicate goals (fitness functions)
I Incorporate into development cycle
Power
I Novel functionality
I Combine components
Neutral Networks and Automated Software Evolution Future Work 41
Future Work
Verification, Mutation, Crossover, Hardening, Fitness-Distance Corr.
I Assembler diff chunks formally equivalent
I Static analysis to,I find bugsI measure code quality
I Invariant preservatione.g., Daikon
Neutral Networks and Automated Software Evolution Future Work 42
Future Work
Verification, Mutation, Crossover, Hardening, Fitness-Distance Corr.
“Smarter” mutation operations
I Repair templates and external code libraries
I Type-aware operations
I Learned mutation heuristics
Neutral Networks and Automated Software Evolution Future Work 42
Future Work
Verification, Mutation, Crossover, Hardening, Fitness-Distance Corr.
Heterologous Crossoverbetween different
I compilation flags
I versions
I implementations
I programs
Homologous Crossover
Heterologous Crossover
∼
Neutral Networks and Automated Software Evolution Future Work 42
Future Work
Verification, Mutation, Crossover, Hardening, Fitness-Distance Corr.
Evolution ThreadsFuzz Testere.g., Klee, S2E
Population ofProgram Variants
Test SuiteI regression tests
I fuzz test 1
I fuzz test 2
I etc.
TournamentSelection
FitnessEvaluation
Program pPassing allTests
Fuzz TestFailingProgram p
Neutral Networks and Automated Software Evolution Future Work 42
Future Work
Verification, Mutation, Crossover, Hardening, Fitness-Distance Corr.
Augment tests to increase granularity
I Count unique values of program counter
I Real-valued oracle comparison (e.g., numdiff)
Neutral Networks and Automated Software Evolution Future Work 42
Outline
Introduction
Program Representation and Transformation
Mutational Robustness and Neutral Networks
Application: Program Diversity
Application: Assembler- and Binary-Level Program Repair
Application: Patching Closed Source Executables
Application: Optimizing nonfunctional Program Properties
Future Work
Conclusion
Neutral Networks and Automated Software Evolution Conclusion 43
Conclusion
Extant software is
1. Mutationally robust with large neutral networks
2. A product of evolution
3. Amenable to automated evolutionary improvement
Software evolution mightfacilitate the automation of software engineering
Neutral Networks and Automated Software Evolution Conclusion 44
Conclusion
Extant software is
1. Mutationally robust with large neutral networks
2. A product of evolution
3. Amenable to automated evolutionary improvement
Software evolution mightfacilitate the automation of software engineering
Neutral Networks and Automated Software Evolution Conclusion 44
Conclusion
Extant software is
1. Mutationally robust with large neutral networks
2. A product of evolution
3. Amenable to automated evolutionary improvement
Software evolution mightfacilitate the automation of software engineering
Neutral Networks and Automated Software Evolution Conclusion 44
Conclusion
Extant software is
1. Mutationally robust with large neutral networks
2. A product of evolution
3. Amenable to automated evolutionary improvement
Software evolution mightfacilitate the automation of software engineering
Neutral Networks and Automated Software Evolution Conclusion 44
Thank You
Eric Schulteemail [email protected]
homepage https://cs.unm.edu/~eschulte
dissertation https://cs.unm.edu/~eschulte/dissertation
Neutral Networks and Automated Software Evolution Conclusion 45
References
Forrest, GECCO 2009 Stephanie Forrest, Westley Weimer, ThanhVu Nguyen and Claire Le Goues. A GeneticProgramming Approach to Automatic Program Repair, in GECCO, 2009
Baudry, ISSTA 2014 Benoit Baudry, Simon Allier, and Martin Monperrus. Tailored source code transformationsto synthesize computationally diverse program variants. In I SSTA, 2014.
Draghi, Nature 2010 J.A. Draghi, T.L. Parsons, G.P. Wagner, and J.B. Plotkin. Mutational robustness canfacilitate adaptation. Nature, 463(7279):353–355, 2010.
Le Goues, ICSE 2012 Claire Le Goues, Michael Dewey-Vogt, Stephanie Forrest, and Westley Weimer. Asystematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each. InI CSE, 2012.
Nguyen, SBST 2009 ThanhVu Nguyen, Westley Weimer, Claire Le Goues and Stephanie Forrest. ExtendedAbstract: Using Execution Paths to Evolve Software Patches, in SBST, 2009.
Schulte, GPEM 2013 Eric Schulte, Zachary. Fry, Ethan Fast, Westley Weimer, and Stephanie Forrest. Softwaremutational robustness. Genetic Programming and Evolvable Machines, pages 1–32, 2013.
Schulte, ASPLOS 2013 Eric Schulte, Jonathan DiLorenzo, Westley Weimer, and Stephanie Forrest. Automatedrepair of binary and assembly programs for cooperating embedded devices. In ASPLOS,2013.
Schulte, ASPLOS 2014 Eric Schulte, Jonathan Dorn, Stephen Harding, Stephanie Forrest, and Westley Weimer.Post-compiler software optimization for reducing energy. In ASPLOS, 2014.
Weimer, ICSE 2009 Westley Weimer, ThanhVu Nguyen, Claire Le Goues and Stephanie Forrest. AutomaticallyFinding Patches Using Genetic Programming, in I CSE, 2009
Neutral Networks and Automated Software Evolution References 46
Backup Slides
I Program Space
I Program Expression
I Many Bugs
I LLVM SSA
I Robustness and AdaptationTime
I Proactive Bug Repair
I Fault Localization
I Embedded Repair Results
I Distributed Repair
I Benefits of ASM
I Embedded Repair Space
I Profiling (GOA)
I Minimization (GOA)
I Energy Model (GOA)
I Runtime and EnergyReduction (GOA)
I Physical EvolutionaryComputation
Neutral Networks and Automated Software Evolution Backup Slides 47
Program SpaceBack
Total
ni+1 =ni (d − i)(e − 1)
(i + 1)
Unique
nz,i+1 =nz−1,i (d − i)1 + nz,i (d − i)(e − 2)
i + 1
Ai =∑z≤i
nz,i
(l
z
)−1
Neutral
R =∑i≤d
r i∑z≤i
nz,i
(l
z
)−1
Constants: d=3, e=5
Step Total0 11
12
2
48
3
64
Neutral Networks and Automated Software Evolution Backup Slides 48
Program SpaceBack
Total
ni+1 =ni (d − i)(e − 1)
(i + 1)
Unique
nz,i+1 =nz−1,i (d − i)1 + nz,i (d − i)(e − 2)
i + 1
Ai =∑z≤i
nz,i
(l
z
)−1
Neutral
R =∑i≤d
r i∑z≤i
nz,i
(l
z
)−1
Constants: d=3, e=5
Step Total0 11 122
48
3
64
Neutral Networks and Automated Software Evolution Backup Slides 48
Program SpaceBack
Total
ni+1 =ni (d − i)(e − 1)
(i + 1)
Unique
nz,i+1 =nz−1,i (d − i)1 + nz,i (d − i)(e − 2)
i + 1
Ai =∑z≤i
nz,i
(l
z
)−1
Neutral
R =∑i≤d
r i∑z≤i
nz,i
(l
z
)−1
Constants: d=3, e=5
Step Total0 11 122 483
64
Neutral Networks and Automated Software Evolution Backup Slides 48
Program SpaceBack
Total
ni+1 =ni (d − i)(e − 1)
(i + 1)
Unique
nz,i+1 =nz−1,i (d − i)1 + nz,i (d − i)(e − 2)
i + 1
Ai =∑z≤i
nz,i
(l
z
)−1
Neutral
R =∑i≤d
r i∑z≤i
nz,i
(l
z
)−1
Constants: d=3, e=5
Step Total0 11 122 483 64
Neutral Networks and Automated Software Evolution Backup Slides 48
Program ExpressionBack
Cil Source
CLang
LLVM
ASM
ELF
Executable
Serialize
Serialize Compile
Compile
Link
Serialize
Neutral Networks and Automated Software Evolution Backup Slides 49
Systematic Evaluation[Le Goues et al., ICSE 12] Back
Amazon EC2 Servers
Version Control Repositories
fbc
gmp
gzip
libtiff
. . .
GenProg
Neutral Networks and Automated Software Evolution Backup Slides 50
Fixing Bugs for $8 a BugBack
Avg. Cost Avg. CostDefects per Non-Repair Per Repair
Program Repaired Hours US$ Hours US$
fbc 1 / 3 8.52 5.56 6.52 4.08gmp 1 / 2 9.93 6.61 1.60 0.44gzip 1 / 5 5.11 3.04 1.41 0.30libtiff 17 / 24 7.81 5.04 1.05 0.04lighttpd 5 / 9 10.79 7.25 1.34 0.25php 28 / 44 13.00 8.80 1.84 0.62python 1 / 11 13.00 8.80 1.22 0.16wireshark 1 / 7 13.00 8.80 1.23 0.17
total 55 / 105 11.22h 1.60h
Neutral Networks and Automated Software Evolution Backup Slides 51
LLVM SSABack
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
Neutral Networks and Automated Software Evolution Backup Slides 52
LLVM SSABack
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0
Neutral Networks and Automated Software Evolution Backup Slides 52
LLVM SSABack
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
Neutral Networks and Automated Software Evolution Backup Slides 52
LLVM SSABack
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
Neutral Networks and Automated Software Evolution Backup Slides 52
LLVM SSABack
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
Neutral Networks and Automated Software Evolution Backup Slides 52
LLVM SSABack
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
%mul
Neutral Networks and Automated Software Evolution Backup Slides 52
LLVM SSABack
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
%mul
x
Neutral Networks and Automated Software Evolution Backup Slides 52
LLVM SSABack
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
%mul
x
%3
Neutral Networks and Automated Software Evolution Backup Slides 52
LLVM SSABack
%x = alloca i32, align 4
store i32 2, i32* %x, align 4
%0 = load i32* %x, align 4
%add = add nsw i32 %0, 3
store i32 %add, i32* %x, align 4
%1 = load i32* %x, align 4
%2 = load i32* %x, align 4
%mul = mul nsw i32 %1, %2
store i32 %mul, i32* %x, align 4
%3 = load i32* %x, align 4
%call = call ... @printf(@.str %3)
i32 2
x
%0i32 3
%add
x
%2%1
%mul
x
%3 printf
Neutral Networks and Automated Software Evolution Backup Slides 52
Robustness and Adaptation TimeBack
[Draghi et al., Nature 2010]
Neutral Networks and Automated Software Evolution Backup Slides 53
Proactive Bug RepairBack
0
1
2
3
4
5
6
7
8
0 2 4 6 8 10 12 14 161
10
100
1000
10000N
umb
erof
Uni
que
Bug
sF
ixed
Num
ber
ofS
elec
ted
Var
iant
s
Number of Bugs Seeded
Number of Selected VariantsNumber of Unique Bugs fixed
Figure: Bugs Fixed by Number of Variants in Potion
Neutral Networks and Automated Software Evolution Backup Slides 54
Fault LocalizationBack
I Focus MutationI Fault Localization
1. instrument2. compile3. evaluate
if(a==0)
puts(b); while(b!=0)
if(a>b)
a=a-b; else
b=b-a;
puts(a);
Neutral Networks and Automated Software Evolution Backup Slides 55
Fault LocalizationBack
I Focus MutationI Fault Localization
1. instrument2. compile3. evaluate
if(a==0)
puts(b); while(b!=0)
if(a>b)
a=a-b; else
b=b-a;
puts(a);
Neutral Networks and Automated Software Evolution Backup Slides 55
Light Weight Fault Localization
1. Sample program counter.
2. Translate memory addresses to program offsets.
3. Smooth sample with Gaussian convolution.
movq 8(%rdx), %rdi
xorl %eax, %eax
movl %eax, (%r15)
addl $1, %r14d
call atoi
movq -80(%rbp), %rdx
movq %rdx, -80(%rbp)
addq $4, %r15
movq 8(%rdx), %rdi
xorl %eax, %eax
movl %eax, (%r15)CPU
Machine-codeInstructions
Neutral Networks and Automated Software Evolution Backup Slides 56
Light Weight Fault Localization
1. Sample program counter.
2. Translate memory addresses to program offsets.
3. Smooth sample with Gaussian convolution.
movq 8(%rdx), %rdi
xorl %eax, %eax
movl %eax, (%r15)
addl $1, %r14d
call atoi
movq -80(%rbp), %rdx
movq %rdx, -80(%rbp)
addq $4, %r15
movq 8(%rdx), %rdi
xorl %eax, %eax
movl %eax, (%r15)CPU
Machine-codeInstructions
Neutral Networks and Automated Software Evolution Backup Slides 56
Light Weight Fault Localization
1. Sample program counter.
2. Translate memory addresses to program offsets.
3. Smooth sample with Gaussian convolution.
memory addr.
to instruction
movq 8(%rdx), %rdi
xorl %eax, %eax
movl %eax, (%r15)
addl $1, %r14d
call atoi
movq -80(%rbp), %rdx
movq %rdx, -80(%rbp)
addq $4, %r15
movq 8(%rdx), %rdi
xorl %eax, %eax
movl %eax, (%r15)CPU
Machine-codeInstructions
Neutral Networks and Automated Software Evolution Backup Slides 56
Light Weight Fault Localization
1. Sample program counter.
2. Translate memory addresses to program offsets.
3. Smooth sample with Gaussian convolution.
memory addr.
to instruction
movq 8(%rdx), %rdi
xorl %eax, %eax
movl %eax, (%r15)
addl $1, %r14d
call atoi
movq -80(%rbp), %rdx
movq %rdx, -80(%rbp)
addq $4, %r15
movq 8(%rdx), %rdi
xorl %eax, %eax
movl %eax, (%r15)CPU
Machine-codeInstructions
Neutral Networks and Automated Software Evolution Backup Slides 56
Embedded Repair ResultsBack
I Effective
I 62% faster runtime
I 95% smaller disk footprint
I 86% less memory
Total bugs repairedRep. Num. BugsAST 13ASM 12ELF 11
Average success rate
100 runs per bug
Rep. Success RateAST 78.17%ASM 70.75%ELF 65.83%
Neutral Networks and Automated Software Evolution Backup Slides 57
Embedded Repair ResultsBack
I Effective
I 62% faster runtime
I 95% smaller disk footprint
I 86% less memory
Expected fitnessevaluations
Rep. EvaluationsAST 583.98ASM 188.38ELF 207.15
Total runtimeRep. Sec.AST 229.50ASM 278.30ELF 74.20
Neutral Networks and Automated Software Evolution Backup Slides 57
Embedded Repair ResultsBack
I Effective
I 62% faster runtime
I 95% smaller disk footprint
I 86% less memory
Example: Merge SortRepair by representation
I AST, 2 of 4900 Swaps
I ASM, 1 of 280 Deletes
merge.c
if(left[l-mid -1]<= right [0]){
result=list; }
else{/* fix:swap branches */
result=merge(left ,l-mid ,
right ,mid); }
merge.s
cmpl %eax , %edx ; fix: del.
jg .L12
movq -72(%rbp), %rax
Neutral Networks and Automated Software Evolution Backup Slides 57
Embedded Repair ResultsBack
I Effective
I 62% faster runtime
I 95% smaller disk footprint
I 86% less memory
Disk size
Rep. RequirementsAST Source code
& build toolchainASM Assembly code
& linkerELF Compiled executable
Neutral Networks and Automated Software Evolution Backup Slides 57
Embedded Repair ResultsBack
I Effective
I 62% faster runtime
I 95% smaller disk footprint
I 86% less memory
Working memory
Rep. MBAST 1402ASM 756ELF 200
Neutral Networks and Automated Software Evolution Backup Slides 57
Distributed Embedded RepairBack
1
2
3
4
Neutral Networks and Automated Software Evolution Backup Slides 58
Distributed Embedded RepairBack
1
2
3
4
Neutral Networks and Automated Software Evolution Backup Slides 58
Distributed Embedded RepairBack
1
2
3
4
delete(45)
swap(23,86)
delete(4)
insert(4,23)
Neutral Networks and Automated Software Evolution Backup Slides 58
Distributed Embedded RepairBack
1
2
3
4
delete(45)
swap(23,86)
delete(4)
insert(4,23)
fault loc.
active
fault loc.
active
fault loc.
active
fault loc.
active active
Neutral Networks and Automated Software Evolution Backup Slides 58
Distributed Genetic Repair Evaluation
Relative performance of Distributed Algorithm
Expected Wall Clock# Nodes Fitness Evaluations Seconds w/SMS
1 1 1 12 0.94 0.89 1.073 0.84 0.67 0.814 0.80 0.55 0.63
Neutral Networks and Automated Software Evolution Backup Slides 59
Distributed Genetic Repair Evaluation
Relative performance of Distributed Algorithm
Expected Wall Clock# Nodes Fitness Evaluations Seconds w/SMS
1 1 1 12 0.94 0.89 1.073 0.84 0.67 0.814 0.80 0.55 0.63
Neutral Networks and Automated Software Evolution Backup Slides 59
Distributed Genetic Repair Evaluation
Relative performance of Distributed Algorithm
Expected Wall Clock# Nodes Fitness Evaluations Seconds w/SMS
1 1 1 12 0.94 0.89 1.073 0.84 0.67 0.814 0.80 0.55 0.63
Neutral Networks and Automated Software Evolution Backup Slides 59
Benefits of ASMBack
The ASM representation performs well.
Avoids
I direct addresses
I argumented instructions
Similarities to DNA
I Linear vector genome
I read sequentially
I reading frames
I start and stop codes
I padding
I bootstrapped compilers
Neutral Networks and Automated Software Evolution Backup Slides 60
Embedded Repair Search SpaceBack
Program size≈ 3× more assembly instructions than Cstatements
Search space size= |alphabet|program size
Possible program coverage
Possible ProgramsReachable Programs
Original Program
Neutral Networks and Automated Software Evolution Backup Slides 61
ProfilingBack
Assembler Fitness Function Workload
Population
Mutate
Profile
Fitness
Minimize
Executable
HardwarePerformanceCounters
Neutral Networks and Automated Software Evolution Backup Slides 62
ProfilingBack
Assembler Fitness Function Workload
Population
Mutate
Profile
Fitness
Minimize
Executable
$ perf stat -- ./blackscholes 1 input /tmp/output
6,864,315,342 cycles
5,062,293,918 instructions
2,944,060,039 r533f00
1,113,084,780 cache-references
1,122,960 cache-misses
3.227585368 seconds time elapsed
Neutral Networks and Automated Software Evolution Backup Slides 62
MinimizationBack
Assembler Fitness Function Workload
Population
Mutate
Profile
Fitness
Minimize
ExecutableDelta Debugging
5358 c5358
< .L808:
---
> addl %ebx , %ecx
5416 c5416
< addl %ebx , %ecx
---
> .L808:
5463 c5463
< .L970:
---
> .byte 0x33
5651 d5650
< .loc 1 457 0 is_stmt 0 discriminator 2
5841 d5839
< addq %rdx , %r14
6309 c6307
< xorpd %xmm1 , %xmm7
---
> cmpq %r13 , %rdi
6413 a6412
> cmpl %ecx , %esi
Neutral Networks and Automated Software Evolution Backup Slides 63
MinimizationBack
Assembler Fitness Function Workload
Population
Mutate
Profile
Fitness
Minimize
ExecutableDelta Debugging
5358 c5358
< .L808:
---
> addl %ebx , %ecx
5416 c5416
< addl %ebx , %ecx
---
> .L808:
5463 c5463
< .L970:
---
> .byte 0x33
5651 d5650
< .loc 1 457 0 is_stmt 0 discriminator 2
5841 d5839
< addq %rdx , %r14
6309 c6307
< xorpd %xmm1 , %xmm7
---
> cmpq %r13 , %rdi
6413 a6412
> cmpl %ecx , %esi
Neutral Networks and Automated Software Evolution Backup Slides 63
MinimizationBack
Assembler Fitness Function Workload
Population
Mutate
Profile
Fitness
Minimize
ExecutableDelta Debugging
5358 c5358
< .L808:
---
> addl %ebx , %ecx
5416 c5416
< addl %ebx , %ecx
---
> .L808:
5463 c5463
< .L970:
---
> .byte 0x33
5651 d5650
< .loc 1 457 0 is_stmt 0 discriminator 2
5841 d5839
< addq %rdx , %r14
6309 c6307
< xorpd %xmm1 , %xmm7
---
> cmpq %r13 , %rdi
6413 a6412
> cmpl %ecx , %esi
Neutral Networks and Automated Software Evolution Backup Slides 63
MinimizationBack
Assembler Fitness Function Workload
Population
Mutate
Profile
Fitness
Minimize
ExecutableDelta Debugging
5358 c5358
< .L808:
---
> addl %ebx , %ecx
5416 c5416
< addl %ebx , %ecx
---
> .L808:
5463 c5463
< .L970:
---
> .byte 0x33
5651 d5650
< .loc 1 457 0 is_stmt 0 discriminator 2
5841 d5839
< addq %rdx , %r14
6309 c6307
< xorpd %xmm1 , %xmm7
---
> cmpq %r13 , %rdi
6413 a6412
> cmpl %ecx , %esi
Neutral Networks and Automated Software Evolution Backup Slides 63
Energy ModelBack
energy
time= Cconst+Cins
ins
cycle+Cflops
flops
cycle+Ctca
tca
cycle+Cmem
mem
cycle
Intel AMDCoefficient Description (4-core) (48-core)
Cconst constant power draw 31.530 394.74Cins instructions 20.490 -83.68Cflops floating point ops. 9.838 60.23Ctca cache accesses -4.102 -16.38Cmem cache misses 2962.678 -4209.09
Table: Energy model coefficients.
Neutral Networks and Automated Software Evolution Backup Slides 64
GOA: Runtime and Energy ReductionBack
-10%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
blackscholes
bodytrack
ferret
fluidanimate
freqmine
swaptions
vipsx264
average
AMD Energy and Runtime Reduction
EnergyRuntime
Neutral Networks and Automated Software Evolution Backup Slides 65
Physical Evolutionary ComputationBack
Subpopulation(64 individuals)
...79965+*
...99965+*
...9804++*
...-38*8*+
...x7558**
...8679++*
...y894**5
RPN Evaluation
9728***
34
12
stack
Tournament
*6*y-8*
Crossovery+x674*6*y-8*1/9-
429965+**
y+x6749965+**
1/9-42*6*y-8*
Mutation0-51x7y58***
0-51x7558***
Hardware TileI 70MHz ARM7TDMI MCUI 32KB RAM, 16KB EEPROMI 128KB problem data storageI hardware timersI power sharing, 4 serial ports
Software Process∀ evolutionary tile
randomly create an initial populationfor ever, concurrently
with frequency fmselect one individual and mutate
with frequency fcselect two individuals and crossover
with frequency fsselect one individual and share
with frequency fd if collecting datareport statistics to central collector
Real Time GeneticOperator Timelines
mutation(fm = 20Hz)
crossover(fc = 10Hz)
sharing(fs = 0.1Hz)
Computational SpaceSample geometry: Sixteen 4.75cm × 4.75cm tilesarranged in a 4× 4 square.
Problem Space
Sample fitness function: Gravity data from thestate of Montana. Each hardware tile covers25′ latitude × 25′ longitude mapped using the ‘localdisjoint’ spatial mapping. The size and shape ofthe total covered area is determined by the numberand configuration of the hardware tiles, and by thescale and scheme used to map the hardware spaceto the problem space.
Neutral Networks and Automated Software Evolution Backup Slides 66
The EndBack
Eric Schulteemail [email protected]
homepage https://cs.unm.edu/~eschulte
dissertation https://cs.unm.edu/~eschulte/dissertation
Neutral Networks and Automated Software Evolution Backup Slides 67