evaluang secure programming knowledgenob.cs.ucdavis.edu/bishop/talks/2017-wise-1/seceval.pdf ·...

Evalua&ngSecureProgrammingKnowledgeMa6Bishop,UCDavis

JunDai,CalStateSacramentoMelissaDark,PurdueUniversityIdaNgambeki,PurdueUniversity

PhillipNico,CalPolySanLuisObispoMinghuaZhu,UCDavis

Specialthanksto:Somdu6aBose,UCDavis;StevenBelcher,NSA

May30,2017 WISE10 1

“Secure”Programming

•  Properdefini&on:programmingdesignedtosa&sfyasecuritypolicy– Butitisrarelyusedtomeanthis…

•  Usualusage:programmingdesignedtopreventproblemsthatmightcausesecuritybreaches– Hence“defensiveprogramming”or“robustprogramming”

May30,2017 WISE10 2

TheProblem

•  SoYwarequalityispoor…verypoor•  Classroomteaching:–  Inregularclasses,willcrowdoutexis&ngcontent– Also,manyfacultydon’tknow(ordon’tcare)aboutthis;theyfocusonclasscontent

–  Inaclassfocusingonthis,can’trequireallstudentstotakeitasschedulesfull

– Alsothatwon’thelpnon-CSstudents!

May30,2017 WISE10 3

Hey,howdoIgettoCarnegieHall?

Prac&ce,Madam,prac&ce!

May30,2017 WISE10 4

LookatHumani&es,SocialSciences

•  Thinkofawri&ngclinic!– English(andother)departments,lawschools– Focusongrammar,wri&ngstyle,organiza&on– Andnotcontent!

•  Analogueforcomputerscience:– Focusonrobustprogrammingprac&ce,codingstyle,organiza&on

– Andnotcorrectnesswithrespecttotheassignment!

May30,2017 WISE10 5

Benefits

•  Studentslearnrobustprogrammingtechniquesthroughanalysisoftheirownprograms– Toolsaregood;studentslearnhowtousethem,howtointerpretresults

•  Studentslearnrobustprogrammingtechniquesapplytoallprograms,notonlytoaspecificclassorassignment

May30,2017 WISE10 6

HowWeDoThis

•  Understandhowstudentsthinkaboutrobustprogramming

•  Assesswhethertheclinicishavingdesiredeffectonstudentunderstandingofrobustprogramming– Pre-clinicassessmenttest– Post-clinicassessmenttest

May30,2017 WISE10 7

ConceptInventory

SecureProgrammer

CodeDesign

AlgorithmsSWAToolsAssumptions ProgrammingDevelopmentEnvironment

Inputs BadCode

MemoryManagement

InputValidation

AuthoritativeCryptography

1

23 4 5 6

7 8

9 10

11

12

AB

C

D

E

F

G H

IJ

K

L

MN i ii

iii

iv

v

May30,2017 WISE10 8

ConceptInventoryKeyVeryImportant

1. Assumewhatevercangowrongwill2. Assumeanyinputisgoingtobemalformedornotwhatyouexpect3. Donotmakeasecuritydecisionbasedonun-trustedinputs4. Checkthatallargumentsareofthecorrecttypeandwillnotoverflowanyarrays5. Usedataabstractiontoenablethecompilertoperformrigoroustypecheckingandto

enforceconstraintsonvaluesandlengths6. Understandthecontextinwhichtheprogramwillexecute7. Validateyourinputstreamtoensurethatthecommandsinvokedareexpectedandno

othercommandsareinjected8. Whenperforminginputvalidationtakeintoaccounthowprogramsinvokedwiththose

argumentscouldinterpretthem9. Avoidhardcodedpasswordsandsecretsinyourprogram10. Usewellknownandacceptedcryptographicalgorithmsand.Don'tuseobsoleteor

deprecatedcryptographicalgorithmsorcreateyourownalgorithms11. Usewellknownandacceptedcryptographicrandomnumbergeneration.Don'tuse

obsoleteordeprecatedcryptographicalgorithmsorcreateyourownalgorithms12. Manytoolshelpyoucreateasecureprogram,pleasetakeadvantageofthem

SomewhatImportant

i. Hidedetailsthatusersdon'tneedtoknowaboutii. Avoidsideeffectsinargumentstounsafemacros.Ifadeveloperisusingamacrothat

usesitsargumentsmorethanonce,thenthedevelopermustavoidpassinganyargumentswithsideeffectstothatmacro

iii. Useparenthesesaroundmacroreplacementlists.Otherwiseoperatorprecedencemaycausetheexpressiontobecomputedinunexpectedways

iv. Minimizethescopeofvariablesandfunctions.Thispreventsmanyunexpectedchangestothevariablesduetoprogrammingerror

v. Whenthememoryapointerpointstoisfreed,setthepointertoNULL.Otherwise,thesedanglingpointerscouldcausewritingtofreedmemory,andcreateadoublefreevulnerability.

Important

A. Ifyouhavenoreasontotrustit,don'ttrustit.Takegreatercarewithanyinputyouhavenotgenerated

B. Ifitcannothappen,checkforit.Someonemaymodifytheprograminsuchawaythatitcanhappen...oryoumaybewrong

C. Donotuseinputorconstructorstringfunctionsthatdonotperformanyboundchecking

D. DonotuseinputorconstructorfunctionsthatcannotcheckthelengthoftheinputE. CandC++compilersgenerallydonotchecktypesrigorously.Adevelopercanincrease

thislevelofcheckingbyturningoncompilerwarnings,whichwilloftencatchmoretypeerrorsthaniftheyarenotused

F. Avoidcallstomalloc()withtheparameter(numberofbytestobeallocated)setto0.EitherthefunctionreturnsNULL,oritreturnsapointertospacethatcannotbeusedwithoutoverwritingunallocatedmemory

G. ControltheinputvalueswhenpossiblebylimitingthemtoafinitesetH. Callingfunctionswithnullparametersforinputshouldbecheckedforanddefended

againstI. Typeconversionissuesespeciallyforcasesthatmayresultinintegerwraparoundand

overflowsJ. Rulesforpointerarithmeticasvulnerabilitiescanarisewhenadditionorsizechecks

involvetwopointertypesK. Whenperforminginputvalidationmakesurethatanyvalidatedpathdoesnotallow

escapingfromarestricteddirectoryL. Beforecreatingadirectoryorfile,makesureyouhavesetthecorrectdefault

permissionspecificationM. BewaryofoffbyoneerrorsN. Whenusingformatstringfunctions,makesurethattheformatstringcanbe

authenticated/trusted

May30,2017 WISE10 9

ExampleQues&on:HandlingUserInput

Concept:“Ifyouhavenoreasontotrustit,don'ttrustit.Takegreatercarewithanyinputyouhavenotgenerated.”Ques&on:Userinputcanbeunpredictable.Whichofthefollowingisthebestwaytoavoidproblemsprocessingthatinput?a)  Elevateprivilegeswhenprocessinguser-providedinput,toensure

thecomputa&oncanbedone.b)  Dropunnecessaryprivilegeswhenprocessinguser-providedinput,

tolimittheeffectsofbaduserinput.c)  Keepprivilegesconstantwheneverpossible,formorereadable

codethatiseasiertomaintainwithoutintroducingerror.d)  Assignelevatedprivilegestoanewprocessorthreadthatreadsthe

inputanddoesthecomputa&on,sothatanymaliciousside-effectsdonotaffecttheprimaryprocessorthread.

e)  Keepprivilegesthesamebutconstraintheprocessexecu&oninasandboxsothatanymaliciousside-effectsarecontained.

May30,2017 WISE10 10

ExampleQues&on:HandlingUserInput

Ques&on:Userinputcanbeunpredictable.Whichofthefollowingisthebestwaytoavoidproblemsprocessingthatinput?a)  Elevateprivilegeswhenprocessinguser-providedinput,toensure

thecomputa&oncanbedone.5%chosethisb)  Dropunnecessaryprivilegeswhenprocessinguser-providedinput,

tolimittheeffectsofbaduserinput.18%chosethisc)  Keepprivilegesconstantwheneverpossible,formorereadable

codethatiseasiertomaintainwithoutintroducingerror.9%chosethis

d)  Assignelevatedprivilegestoanewprocessorthreadthatreadstheinputanddoesthecomputa&on,sothatanymaliciousside-effectsdonotaffecttheprimaryprocessorthread.43%chosethis

e)  Keepprivilegesthesamebutconstraintheprocessexecu&oninasandboxsothatanymaliciousside-effectsarecontained.25%chosethis

May30,2017 WISE10 11

ExampleQues&on:IndexingIntoanArray

Concept:“Checkparameterstoensurethatallargumentsareofthecorrecttypeandwillnotoverflowanyarrays.”Ques&on:Yourprogramacceptsparametersx,y,andztocalculatetheposi&onofaniteminanarrayrela&vetothecurrentitemindexedbyptr.101 newOffset = (x*colSize)+(y*rowSize)–z;102 ptr = ptr + newOffset;103 newObject = objectArray[ptr];Whichofthefollowingistrue?a)  Ishouldcheckthattheresultinline101isnotnega&ve.b)  Ishouldcheckthattheresultinline101isnotnull.c)  Ishouldcheckthattheresultinline102isnotnega&ve.d)  Ishouldcheckthattheresultinline102isnotnull.

May30,2017 WISE10 12

ExampleQues&on:IndexingIntoanArray

Ques&on:Yourprogramacceptsparametersx,y,andztocalculatetheposi&onofaniteminanarrayrela&vetothecurrentitemindexedbyptr.101 newOffset = (x*colSize)+(y*rowSize)–z;102 ptr = ptr + newOffset;103 newObject = objectArray[ptr];Whichofthefollowingistrue?a)  Ishouldcheckthattheresultinline101isnotnega&ve.28%chose

thisb)  Ishouldcheckthattheresultinline101isnotnull.17%chosethisc)  Ishouldcheckthattheresultinline102isnotnega&ve.38%chose

thisd)  Ishouldcheckthattheresultinline102isnotnull.17%chosethis

May30,2017 WISE10 13

ExampleQues&on:HandlingMissingData

Concept:“Ifyouhavenoreasontotrustit,don'ttrustit.Takegreatercarewithanyinputyouhavenotgenerated.”Ques&on:Youmustreadalistofusernamesandstar&ngdate:day,month,year.Thenyourprogrammustsorttheminascendingordertocreatealistofusersbyseniority.Somestartdatesaremissingthedayormonthofthestartdate.Thislist-sor&ngfunc&onmaybeusedelsewhere,ortweakedinthefuture.Whichstatementbelowisthemostrobustwaytohandlethemissingdata?a)  Ini&alizethevariablesformissinginforma&onwitharandom

plausiblevalue.b)  Leavethevariablesformissinginforma&onunini&alized.c)  Ini&alizethevariablesformissinginforma&onwith0.d)  Ini&alizethevariablesformissinginforma&onwiththemaximum

plausiblevalue.

May30,2017 WISE10 14

ExampleQues&on:HandlingMissingData

Concept:“Ifyouhavenoreasontotrustit,don'ttrustit.Takegreatercarewithanyinputyouhavenotgenerated.”Ques&on:Youmustreadalistofusernamesandstar&ngdate:day,month,year.Thenyourprogrammustsorttheminascendingordertocreatealistofusersbyseniority.Somestartdatesaremissingthedayormonthofthestartdate.Thislist-sor&ngfunc&onmaybeusedelsewhere,ortweakedinthefuture.Whichstatementbelowisthemostrobustwaytohandlethemissingdata?a)  Ini&alizethevariablesformissinginforma&onwitharandomplausible

value.7%chosethisb)  Leavethevariablesformissinginforma&onunini&alized.13%chosethisc)  Ini&alizethevariablesformissinginforma&onwith0.57%chosethisd)  Ini&alizethevariablesformissinginforma&onwiththemaximumplausible

value.22%chosethis

May30,2017 WISE10 15

ExampleQues&on:PointerValida&on

Concepts:“Followtherulesforpointerarithme&casvulnerabili&escanarisewhenaddi&onorsizechecksinvolvetwopointertypes”and“Bewaryofoffbyoneerrors.”Ques&on:ForaCprogramyoumustcreateanarrayofsizeintegers.Youwrite:1 unsigned long ∗start, ∗end; 2 start = malloc(size∗sizeof (unsigned long)); Assumingmallocsucceeds,thecorrectvalueforendcanbecomputedby:a) end = start + size * sizeof(unsigned long);b) end = start + size * (sizeof(unsigned long) - 1);c) end = start + (size - 1) * sizeof(unsigned long);d) end = start + size – 1;e) end = start + sizeof(unsigned long) – 1;

May30,2017 WISE10 16

ExampleQues&on:PointerValida&on

Concepts:“Followtherulesforpointerarithme&casvulnerabili&escanarisewhenaddi&onorsizechecksinvolvetwopointertypes”and“Bewaryofoffbyoneerrors.”Ques&on:ForaCprogramyoumustcreateanarrayofsizeintegers.Youwrite:1 unsigned long ∗start, ∗end; 2 start = malloc(size∗sizeof (unsigned long)); Assumingmallocsucceeds,thecorrectvalueforendcanbecomputedby:a) end = start + size * sizeof(unsigned long); 10% chose thisb) end = start + size * (sizeof(unsigned long) - 1); 16%

chose thisc) end = start + (size - 1) * sizeof(unsigned long); 31%

chose thisd) end = start + size – 1; 20% chose thise) end = start + sizeof(unsigned long) – 1; 16% chose this

May30,2017 WISE10 17

ExampleQues&on:InputValida&on

Concepts:“Assumeanyinputisgoingtobemalformedornotwhatyouexpect.”Ques&on:Youmustwriteafunc&onthatstoresanintegerinthedes&na&onpointedtobyvalue,andreturnsanintegerindica&ngsuccessorfailure.Youstartwiththisfunc&onprototype:

int getSeconds (int * secondsParameter ) Whichofthefollowingmustyoudobeforeorinsteadofanyoftheothers?a)  Imustdereferencethepointertogetthememoryloca&on.b)  Imustfindthevaluethatthepointerrefersto.c)  Imustcheckthatthepointerpassedindoesnotalreadyhavea

value.d)  ImustcheckthatthepointerpassedinisnotNULL.

May30,2017 WISE10 18

ExampleQues&on:InputValida&on

Concepts:“Assumeanyinputisgoingtobemalformedornotwhatyouexpect.”Ques&on:Youmustwriteafunc&onthatstoresanintegerinthedes&na&onpointedtobyvalue,andreturnsanintegerindica&ngsuccessorfailure.Youstartwiththisfunc&onprototype:

int getSeconds (int * secondsParameter ) Whichofthefollowingmustyoudobeforeorinsteadofanyoftheothers?a)  Imustdereferencethepointertogetthememoryloca&on.10%chosethisb)  Imustfindthevaluethatthepointerrefersto.8%chosethisc)  Imustcheckthatthepointerpassedindoesnotalreadyhaveavalue.13%

chosethisd)  ImustcheckthatthepointerpassedinisnotNULL.69%chosethis

May30,2017 WISE10 19

Analysis

•  Analyzetestques&on,distractors–  Itemeffect:whichstudentswithahighoverallscoregotapar&cularques&oncorrect•  –1.00to1.00

–  Iden&fiesques&onsthatarenotfunc&oning,ie.lowornega&vecorrela&onwithoverallscore

– Thisimpliesdistractorsconfusestudentswhoknowthematerial

May30,2017 WISE10 20

Analysis

May30,2017 WISE10 21

Conclusion

•  Evalua&onofdistractorsimportanttobeabletomeasureeffec&venessofsecureprogrammingclinic

•  Sofar,3schoolsinvolvedintheclinic,andithasbeenrunfor3differentclasses

•  Thusfar,clinicseemstobeeffec&ve– Butwe’res&llgatheringdata…

May30,2017 WISE10 22

Weinberg’sSecondLaw

Ifbuildersbuiltbuildingsthewayprogrammerswroteprograms...

thefirstwoodpeckertocomealongwoulddestroyciviliza&on

May30,2017 WISE10 23

ThanksTo

FundedbyNSFunderGrantNo.DGE-1303048toPurdue University and Grant No. DGE-1303211totheUniversityofCaliforniaatDavisAny opinions, findings, and conclusions orrecommenda&onsexpressedinthismaterialarethose of the author(s) and do not necessarilyreflect the views of the Na&onal ScienceFounda&on, theuniversi&es involved,oranyofourpets

May30,2017 WISE10 24

PresenterMa6Bishop

DepartmentofComputerScienceUniversityofCaliforniaatDavis

1ShieldsAve.Davis,CA95616-8562

USA

email:mabishop@ucdavis.eduphone+1(530)752-8060web:h6p://seclab.cs.ucdavis.edu/~bishop

May30,2017 WISE10 25