software complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · why technologists oppose golden keys...
TRANSCRIPT
![Page 1: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/1.jpg)
SoftwareComplexityStevenM.Bellovin
https://www.cs.columbia.edu/~smb
1
![Page 2: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/2.jpg)
WhatHappened?
• Whyisatrainarrivingin-2minutes?• Isthe10:26runningaheadofthe9:38?• (We’llignorethefactthatthey’rebothquitelate.)
2
![Page 3: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/3.jpg)
ATrainStatusDisplay
3
![Page 4: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/4.jpg)
AndinWashington…
4
![Page 5: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/5.jpg)
Optimism
“Theprogrammer,likethepoet,worksonlyslightlyremovedfrompurethought-stuff.Hebuildshiscastlesintheair,fromair,creatingbyexertionoftheimagination.Fewmediaofcreationaresoflexible,soeasytopolishandrework,soreadilycapableofrealizinggrandconceptualstructures.”
FredBrooks,TheMythicalMan-Month
5
![Page 6: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/6.jpg)
RealityCheck
“[O]nemustperformperfectly.Thecomputerresemblesthemagicoflegendinthisrespect,too.Ifonecharacter,onepause,oftheincantationisnotstrictlyinproperform,themagicdoesn’twork.Humanbeingsarenotaccustomedtobeingperfect,andfewareasofhumanactivitydemandit.”
FredBrooks,TheMythicalMan-Month
6
![Page 7: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/7.jpg)
RealSoftware
• Softwareisbuggy• Softwareisalwaysbuggy• Thebiggertheprogram,thebuggierthesoftware—always
7
![Page 8: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/8.jpg)
Why?Interactions
8
![Page 9: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/9.jpg)
ComplexityKills…
Withncomponents,thereareroughlyn2interactions
9
![Page 10: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/10.jpg)
So?
• Therearelimitstohowgoodoursoftwarecanbe• Therearethereforethingswecan’tdo• Moreprecisely,whenweincreasecomplexitywe
a) Increasethebugratedramaticallyb) Increasethedevelopmentcostdramaticallyc) Both!
10
![Page 11: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/11.jpg)
ComplexityandCurrentEvents
• Webankonline• Webuythingsonline• Wecommunicateonline• Whycan’twevoteonline?
11
![Page 12: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/12.jpg)
ProbablyJustaBug
(PhotobyEdFelten) 12
Avotingmachinetapefromthe2008presidentialprimaryinaNewJerseyprecinct.
![Page 13: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/13.jpg)
EntertheAdversary
13NYTimes
![Page 14: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/14.jpg)
GoingDark
“Asaresult,althoughthegovernmentmayobtainacourtorderauthorizingthecollectionofcertaincommunications,itoftenservesthatorderonaproviderwhodoesnothaveanobligationunderCALEAtobepreparedtoexecuteit.”
ValerieCaproni,GeneralCounseloftheFBI
14
![Page 15: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/15.jpg)
TheFBI’sSolution
• Allcommunicationsystemsneedsomeformofaccessforlawenforcement• Allencryptionsystemsneeda“backdoor”(whichtheycalla“goldenkey”)• Canwedoit?
15
![Page 16: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/16.jpg)
WiretapInterfacesareHard
• Someyearsago,theNSAevaluatedthestandardizedwiretapinterfaceon26differentphoneswitches• Allhadsecurityflaws
• Someone(probablyanintelligenceagency)hackedacellphoneswitchinAthensandabusedthewiretapinterface• About100phoneswereillegallytapped,includingthePrimeMinister’s
16
![Page 17: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/17.jpg)
CryptographyisHard
“Finally,protocolssuchasthosedevelopedherearepronetoextremelysubtleerrorsthatareunlikelytobedetectedinnormaloperation.”
RogerNeedhamandMichaelSchroeder,“UsingEncryptionforAuthenticationinLargeNetworksofComputers”
17
![Page 18: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/18.jpg)
From“KeysUnderDoormats”
“Wehavefoundthatthedamagethatcouldbecausedbylawenforcementexceptionalaccessrequirementswouldbeevengreatertodaythanitwouldhavebeen20yearsago.Inthewakeofthegrowingeconomicandsocialcostofthefundamentalinsecurityoftoday’sInternetenvironment,anyproposalsthatalterthesecuritydynamicsonlineshouldbeapproachedwithcaution...Thecomplexityoftoday’sInternetenvironment,withmillionsofappsandgloballyconnectedservices,meansthatnewlawenforcementrequirementsarelikelytointroduceunanticipated,hardtodetectsecurityflaws.”
Abelsonetal.
18
![Page 19: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/19.jpg)
WhyTechnologistsOpposeGoldenKeys
• IthasnothingtodowithdislikeoftheFBIortheNSA• Technologistscanbevictimsofcriminalsandterrorists,too• Rather,it’saquestionofcrimeprevention—thesoftwarenecessarytopermitlawenforcementaccesshasahighprobabilityofopeningupnewsecurityholes• Therootcauseisthecomplexityofsoftware
19
![Page 20: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/20.jpg)
TheInternetofThings
• We’reconnectingmoreandmore“things”totheInternet• Theserunonsoftware;thissoftwareisoftenpoorlywrittenandneverpatched
20
![Page 21: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/21.jpg)
Self-DrivingCars
• Almostcertainly,wewillseesomecrashesduetobuggycode• Possibly(thoughnotcertainly),therewillbecrashesduetohacking
• Eventoday’s“dumb”carscontain50-75networkedcomputers• Amoderncarisactuallyamobiledatacenter!
• But—self-drivingcars,flawsandall,willalmostcertainlybesaferthanhuman-drivencars• Carsdon’tgetdrunk,sleepy,distracted,etc.
21
![Page 22: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/22.jpg)
UsersDon’tSeeMostoftheComplexity
• Goodsoftwareoftenhideshowcomplexitis• But—thecomplexityisstillthere• Often,it’sthepartsyoudon’tknowaboutthatcancausethemosttrouble
22
![Page 23: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/23.jpg)
SoWhatDoWeDo?
• Giveup?
23
![Page 24: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/24.jpg)
SoWhatDoWeDo?
• Giveup?• No;thatsacrificesthebenefitsofcomputers.Therearereasons(andgenerallygoodones)whywerelyonsoftware
24
![Page 25: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/25.jpg)
SoWhatDoWeDo?
• Giveup?• No;thatsacrificesthebenefitsofcomputers.Therearereasons(andgenerallygoodones)whywerelyonsoftware• Often,somesmallrateoffailureisquiteacceptable—nothingelseisperfect,either
25
![Page 26: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/26.jpg)
SoWhatDoWeDo?
• Giveup?• No;thatsacrificesthebenefitsofcomputers.Therearereasons(andgenerallygoodones)whywerelyonsoftware• Often,somesmallrateoffailureisquiteacceptable—nothingelseisperfect,either• Thetrickisknowinghowtodecide.Wewantmajorbenefits,comparativelylowrisks,andacceptableconsequencesifthereisafailure
26
![Page 27: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/27.jpg)
“Thecompetentprogrammerisfullyawareofthestrictlylimitedsizeofhisownskull;thereforeheapproachestheprogrammingtaskinfullhumility…”
EdsgerDijkstra,“TheHumbleProgrammer”
27
![Page 28: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/28.jpg)
SomeSuggestions
Good
• Self-drivingcars• Communicationsapps
• Thesmartgrid?
Bad
• (Residential)lightbulbs• Bikelocks• Anti-missilesystems• Votingmachines
• Networkedsextoys?
28
![Page 29: Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys • It has nothing to do with dislike of the FBI or the NSA • Technologists can](https://reader030.vdocuments.net/reader030/viewer/2022040103/5e8144277a84bb56bb7e0c53/html5/thumbnails/29.jpg)
Questions?
29