the future of correct software george necula. 2 software correctness is important ► where there is...
Post on 19-Dec-2015
221 views
TRANSCRIPT
![Page 1: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/1.jpg)
The Future of The Future of Correct SoftwareCorrect Software
George NeculaGeorge Necula
![Page 2: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/2.jpg)
2
Software Correctness is Software Correctness is ImportantImportant
►Where there is software, there are bugsWhere there is software, there are bugs
► It is estimated that software bugs cost the It is estimated that software bugs cost the economy over $60B a year (1% of GDP)economy over $60B a year (1% of GDP) Average cost of downtime can be $1M/hourAverage cost of downtime can be $1M/hour
►Software bugs are responsible for over Software bugs are responsible for over 50% of known security vulnerabilities50% of known security vulnerabilities
![Page 3: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/3.jpg)
3
Software Correctness is HardSoftware Correctness is Hard►Social challengesSocial challenges
Customers still favor features and performanceCustomers still favor features and performance Programmers notoriously overconfidentProgrammers notoriously overconfident
►Economic challengesEconomic challenges Correctness costs more than extra featuresCorrectness costs more than extra features
►Technical challengesTechnical challenges Impossible to build perfect software quality tools Impossible to build perfect software quality tools False alarms and missed errors are facts of lifeFalse alarms and missed errors are facts of life
![Page 4: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/4.jpg)
4
The Open Source Quality The Open Source Quality GroupGroup
►MembersMembers Rastislav Bodik, George Necula, Sanjit SeshiaRastislav Bodik, George Necula, Sanjit Seshia Collaborators at Stanford, Microsoft, IBM, IntelCollaborators at Stanford, Microsoft, IBM, Intel And 15 graduate studentsAnd 15 graduate students
► Develop techniques and tools for Develop techniques and tools for buildingbuilding, , deployingdeploying and and monitoringmonitoring quality software quality software
►Use Open Source software as a test bedUse Open Source software as a test bed
![Page 5: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/5.jpg)
5
1. Building Correct Software1. Building Correct Software
►Tools can help only if we bring more Tools can help only if we bring more information in the software process information in the software process
►Find unobtrusive ways to get programmer Find unobtrusive ways to get programmer assistance with correctness reasoningassistance with correctness reasoning Programs express “how” things must be donePrograms express “how” things must be done Programmers know the “what” and “why” ! Programmers know the “what” and “why” !
►Example: Programming by sketchingExample: Programming by sketching by Rastislav Bodik, Sanjit Seshiaby Rastislav Bodik, Sanjit Seshia
![Page 6: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/6.jpg)
6
The Sketching ExperienceThe Sketching Experience
sketch implementation (completed sketch)
spec
specification
+
![Page 7: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/7.jpg)
7
Promising PropertiesPromising Properties
Sketched programs are developed …Sketched programs are developed …
rapidly:rapidly: the low-level details are the low-level details are synthesized automaticallysynthesized automatically
correctly: correctly: implementation guaranteed to implementation guaranteed to behave like the specificationbehave like the specification
![Page 8: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/8.jpg)
8
Example: Sorting by handExample: Sorting by hand
int[] merge (int[] a, int b[], int n) {int[] merge (int[] a, int b[], int n) {for (int i = 0; i < n; i++)for (int i = 0; i < n; i++)
if ( if ( j<n && ( !(k<n) || a[j] < b[k])j<n && ( !(k<n) || a[j] < b[k]) ) { ) { result[i] = a[j]; j++; result[i] = a[j]; j++;
} else { } else { result[i] = b[k]; k++; result[i] = b[k]; k++;
}}}}return result;return result;
}}
► The devil is in the detailsThe devil is in the details
![Page 9: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/9.jpg)
9
int[] merge (int[] a, int b[], int n) {int[] merge (int[] a, int b[], int n) {for (int i = 0; i < n; i++)for (int i = 0; i < n; i++)
if ( if ( synthesize( ||, &&, <, !, [] )synthesize( ||, &&, <, !, [] ) ) { ) { result[i] = a[j]; j++; result[i] = a[j]; j++;
} else { } else { result[i] = b[k]; k++; result[i] = b[k]; k++;
}}}}return result;return result;
}}► Sketch compiler fills in the details correctlySketch compiler fills in the details correctly
► Sketches are programs with Sketches are programs with missing detailsmissing details► Specifications can be slow/simple programsSpecifications can be slow/simple programs
Sorting Sorting sketchedsketched
hole
![Page 10: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/10.jpg)
10
Experience with Sketching Experience with Sketching CiphersCiphers
User experiment: User experiment: goal:goal: implement a mini-cipher implement a mini-cipher how:how: C programmer C programmer vs.vs. sketching programmer sketching programmer
Results:Results: sketching programmer was twice as fastsketching programmer was twice as fast sketched cipher ran 50% fastersketched cipher ran 50% faster
Next: sketching for general purpose Next: sketching for general purpose programsprograms
![Page 11: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/11.jpg)
11
2. Deploying Correct Software2. Deploying Correct Software
►Today’s view of software: Today’s view of software: Software is executableSoftware is executable
►Future view of software: Future view of software: Software is checkable and executableSoftware is checkable and executable
►We need to We need to redefine what software isredefine what software is:: Software = Executable content Software = Executable content
+ Assurance support+ Assurance support
![Page 12: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/12.jpg)
12
Today: Digital SignaturesToday: Digital Signatures
CPU
Code
Consumer
Signature
SignatureCheckingTrust the code producer
• Not a behavioral assurance
• Dangerous !• Does not scale well
Good but not enough
![Page 13: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/13.jpg)
13
Future: Semantic AssuranceFuture: Semantic Assurance
CPU
Code
Consumer
Safety Proof
ProofChecking
• Proof-carrying code• Provides semantic
assurance• Producer does the hard
work
Code producer “helps” the consumer to check the code
![Page 14: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/14.jpg)
14
ChallengesChallenges
►How small can you make the proofs? How small can you make the proofs? Today about 25% of the code and shrinkingToday about 25% of the code and shrinking
►How do you generate proofs ?How do you generate proofs ? Certifying software synthesis tools (compilers)Certifying software synthesis tools (compilers) Automatic today for memory safety, resource Automatic today for memory safety, resource
usage constraintsusage constraints
►Next: make more software tools certifyingNext: make more software tools certifying
![Page 15: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/15.jpg)
15
3. When Everything Else Fails3. When Everything Else Fails
► The future of correct software must The future of correct software must include incorrect softwareinclude incorrect software
► We must deal with execution errorsWe must deal with execution errors Monitoring, recovery, restarting, …Monitoring, recovery, restarting, …
► Example: Cooperative Bug IsolationExample: Cooperative Bug Isolation
![Page 16: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/16.jpg)
16
Post-Deployment MonitoringPost-Deployment Monitoring
Cooperative Bug Isolation
![Page 17: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/17.jpg)
17
Idea: Measure RealityIdea: Measure Reality
►Go beyond measuring crashesGo beyond measuring crashes►Monitor good and bad executionsMonitor good and bad executions
Spread cost of monitoring over many usersSpread cost of monitoring over many users Collect feedback data & mine for bug Collect feedback data & mine for bug
causescauses
►Actual user runs are a vast resourceActual user runs are a vast resource Number of real runs >> number of testing Number of real runs >> number of testing
runsruns Real-world executions are most importantReal-world executions are most important
![Page 18: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/18.jpg)
18
Bug Isolation ArchitectureBug Isolation Architecture
ProgramSource Compiler
SamplerShipping
Application
Pro le/
StatisticalDebugging
Top bugs withlikely causes
![Page 19: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/19.jpg)
19
Public Deployment in Public Deployment in ProgressProgress
0%
2%
4%
6%
8%
10%
Evolu
tion
Gaim
The GIM
P
Gnum
eric
Nautil
us
Rhythm
box
SPIM
success runs
failure runs
► Applications do have bugsApplications do have bugs► Attract more users for statistical analysisAttract more users for statistical analysis
![Page 20: The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software](https://reader035.vdocuments.net/reader035/viewer/2022062714/56649d2b5503460f94a00d84/html5/thumbnails/20.jpg)
20
ConclusionConclusion► Social factors will work in favor of Social factors will work in favor of
software correctness software correctness
► Technology must provide affordable Technology must provide affordable solutions for correctnesssolutions for correctness Bring more information into software processBring more information into software process Software synthesis from high-level Software synthesis from high-level
specificationsspecifications Software distributions with assurance supportSoftware distributions with assurance support
► Good error handling always important Good error handling always important