AIT 681 Secure Software Engineering

Topic #5. Seven Software Security Touchpoints (II)

Instructor: Dr. Kun Sun

Reading

• This lecture– McGraw: Ch.6

2

Seven Touchpoints

1. Code review2. Architectural risk analysis3. Penetration testing4. Risk-based security testing5. Abuse cases6. Security requirements7. Security operations

Plus external analysis

3

3. Penetration Testing

4

Penetration Testing

• Artifact: system running in its environment• Understanding fielded software in its environment• Information supplied by architectural risk analysis• Who does it?• White/Black Hat

Requirement and Use cases

Architecture and Design Test Plans Code Tests and

Test ResultsFeedback from

the Field

5. Abuse cases

6. Security Requirements

2. Risk Analysis

External Review

4. Risk-Based Security Tests

1. Code Review(Tools)

2. Risk Analysis

3. Penetration Testing

7. Security Operations

5

Software Testing

• Application fulfills functional requirements• Dynamic, functional tests late in the SDLC• Contextual information

6

Security Testing

• Look for unexpected but intentional misuses of the system

• Must test for all potential misuse types using– Architectural risk analysis results– Abuse cases

• Verify that – All intended security features work (white hat)– Intentional attacks cannot compromise the system

(black hat)

7

Penetration Testing

• Testing for negative – what must not exist in the system

• Difficult – how to prove “non-existence”?• If penetration testing does not find errors then – Can conclude that under the given circumstances

no security faults occurred – Little assurance that application is immune to

attacks

• Feel-good exercise

8

What is Pen Testing?

• Penetration testing assesses security by actively trying to find exploitable vulnerabilities– Black hat activity (for a good purpose). Practitioners

variously called red teams, tiger teams, etc.

• Can be applied at different levels of granularity– program (single process)– complete application (communicating processes)– network of many applications– generally not libraries or incomplete pieces of code

9

Pen Testing is Both Art and Science

• Humans probe and interact with a system, looking for different weaknesses or attack vectors– Employing cleverness, adaptation, ingenuity

• Once patterns of exploration and exploitation emerge, you write computer programs (tools) to do the work– Ingenuity automated

Science is what we understand well enough to explain to a computer. Art is everything else we do. —Donald Knuth

10

Who and How?

• Pen testers employ ingenuity and automated tools– To rapidly explore a system’s attack surface, looking

for weaknesses to exploit • Typically carried out by a separate group within

or outside an organization, separated from developers– Avoids tunnel vision: Same reason doctors tend to not

treat themselves or their own families• Given varied access to system internals (threat

model)– From no access, like outside attacker, to full access,

like a knowledgeable insider

11

History of Pen Testing

• 1967 Ware Report– Task force of experts headed by Willis Ware of RAND

Corp. formally assessed the security problem for time-sharing computer systems. Used term “penetration”• http://www.rand.org/pubs/reports/R609-1/index2.html

• 1970s: DOD penetration testing teams emerge to assess “real” security of government computer systems

• Today: Penetration testing is expanding– Popular with students, e.g., “CTF” competitions– Many companies can be contracted to do it– IACRB Certified Penetration Tester (CPT)

• http://www.iacertification.org/cpt_certified_penetration_tester.html

12

Success of Penetration Testing

• Depends on skill, knowledge, and experience of the tester

• Important! Result interpretation• Disadvantages of penetration testing: – Often used as an excuse to declare victory and go

home– Everyone looks good after negative testing results

13

Testing Process

• External Testing: across the internet.– Simulate attacker’s environment– Gathering information related to remote access, IP

addresses, open ports, allowed services, etc.– Tools to support

• Internal Testing: onsite. – View of the system behind the external perimeters– Software penetration testing tools– Attempt to exploit vulnerabilities

14

Testing Activities

• Scoping: assessing target system• Discovery: building information about the

system– Offline and online activities

• Vulnerability scanning: testing system components

• Target penetration: within testing parameters• Analysis: of results of previous stages• Reporting: detailed findings and

recommendations

15

Pen Tester’s Bag of Tricks

• A pen tester approaches a target knowing– the workings of the target domain (e.g., the web)– how systems are built in that domain• Protocols (e.g., HTTP, TCP, …)• Languages (e.g., PHP, Java, Ruby, …)• Frameworks (e.g., Rails, Dream Weaver, Drupal)

– common weaknesses in the software/system• Bugs (e.g., SQL injections, XSS, CSRF, …)• Misconfigurations, bad design (e.g., default passwords,

“hidden” files, …)

16

Web Hacking: A Professional’s View

70% messing with parametersIf the URL is http://tgt.com/buy?item=1&price=5.00

Then change it to: • /buy?item=1&price=0.01• /buy?item=10&price=5.00• /buy?item=1&price=5.00<script>alert(“test”);<

/script>

• /buy?item=1&price=5.00’

Client parameters (unwisely) trusted?

Susceptible to XSS?

Susceptible to other injection?

17

Web Hacking: A Professional’s View

10% default passwords– Always research the default password and try it • Works way more often than you’d think

10% hidden files and directories– Look through the manuals for clues– Directory brute forcing

10% other– Authentication problems (bypass, replay, …)– Insecure web services– Configuration page gives away your root password

18

Benefits

• Penetrations are certain and reproducible,demonstrated by tests– Not hypothetical– Applied to a whole component• not (code) fragments

– No false alarms

• “Feel good” factor– Produces evidence of real vulnerabilities that

would otherwise have gone unfixed– Thus results in a clear improvement to security

Beware of bugs in the above code; I have only proved it correct, not tried it —Donald Knuth, 1977

19

Drawbacks

• Absence of penetrations is not evidence of security– After fixing any issues there may be others still lurking

• Changes to the system necessitate a retest– Security is not compositional: a change to one

component may render another component insecure• So must retest the entire system

– But changes are common!• Can be expensive to retest too frequently

• Nevertheless, penetration testing worth doing

20

Pen Testing Tools

• Pen testers use tools to– Probe a target– Gather information and test hypotheses about it– Exploit a vulnerability (or attempt to)

• Which tool depends on the goal, and the target– If an enterprise network, want to find, probe, and

exploit machines, routers, topology, etc.– If a single machine, want to consider installed

software, running programs, interesting files– if a single program, want to explore and exploit

possible inputs and interactions

21

Sample Tools

• Science is captured in tools. • We’ll briefly consider

• Nmap for network scanning• Zap web proxy for probing, exploitation• Metasploit for general-purpose exploitation• … and provide pointers to more tools

22

A comprehensive list is at http://sectools.org/

Nmap for Network Probing

• Nmap stands for “network mapper”. Figures out– what hosts are available on the network, – what services (application name and version) those

hosts are offering, – what operating systems (and OS versions) they are

running, – what type of packet filters/firewalls are in use– … and more

• Works by sending raw IP packets into the network and observing the effects

• Free, open source (commercial tools too)http://nmap.org/

23

Finding Hosts, Services

• Nmap will ping a specified range of IP addresses– ICMP Echo Request and/or Timestamp request

• Standard “ping” protocol– TCP SYN to port 443, TCP SYN/ACK to port 80

• Looking for running HTTPS or HTTP servers– Other things, as determined by the operator

• Protocol-specific UDP packets to particular ports• Probes to other TCP ports• Probes that elicit different responses on different OSes

(“fingerprinting”)

• Be stealthy– A flurry of scanning activity may be detected– Control the rate of scanning to “work under the radar”

24

Web Proxies

• Web applications are common pen testing targets

• Web proxies sit between the browser and server– Displaying exchanged packets– Modifying them as directed by the tester

Proxy

Server

Some proxies have additional features for vulnerability scanning/exploitation, site probing, etc.

25

Zap

• OWASP Zed Attack Proxy (Zap)– GUI-based inspection/modification of captured packets – Can set “breakpoints” to allow packets through until a

certain condition is met• Additional features– Active scanning: attempts XSS, SQL injection, etc.– Fuzzing: context-specific payloads– Spider: explores a site to construct a model of its structure

• Free, open-source https://github.com/zaproxy/zaproxy

• See also the Burp suitehttp://portswigger.net/burp/

26

Metasploit

• Metasploit advanced open-source platform for developing, testing, and using exploit code. – Boasts an extensible model through which payloads,

encoders, no-op generators, and exploits can be integrated

• Scripting attacks– Probe remote site looking for vulnerable services– Construct payload based on versions, other features– Encode payload to avoid detection– Inject payload– Wait for shellcode to connect back; command

prompt!

27

Metasploit

1. Probe vulnerable target

28

Metasploit

2. Generate attack payload

3. Send payload

4. Trigger vulnerability and compromise target

29

Metasploit UI

• msfconsole — interactive console for executing metasploit commands– Also web-based frontend and command-line interface– Supports probing and communications commands,

payload construction (and encoding)– Supports active (go get `em) and passive (wait til they

come to us) attacks• Meterpreter - command processor injected into

the target, e.g., in the memory of a compromised process– Permits the pen tester to probe more stealthily

• msfpayload, msfencode — generate (stealthy) shellcode

30

100’s of Modules, Scripts

• Exploits against particular vulnerabilities– Along with stagers and other modifiers to generalize

these exploits to different platforms

• Password sniffing – Reading unprotected passwords off of the network

• Privilege escalation– After penetrating, try to get SYSTEM privileges

• Keylogging and backdoors– For persistent presence

• … and much morehttp://www.offensive-security.com/metasploit-unleashed/Main_Page

31

Kali

• Kali is a Linux distribution with many open-source pen testing tools installed and configured

• The ones we have already mentioned– Nmap, Zap, Metasploit, Burp Suite

• and dozens more– John the Ripper for password cracking– Valgrind for dynamic binary analysis– Reaver for Wifi password cracking– peepdf for scanning PDF files for attack vectors– … and more

http://www.kali.org/32

Ethical Hacking

• Penetration testing tools are meant to reveal security vulnerabilities– So they can be fixed– Not so they can be exploited in the wild

• But people use tools for nefarious purposes– Don’t be one of them!

Beware the dark side!

33

Fuzzing

34

What is Fuzzing?

• A kind of random testing– Find improperly handled inputs, where failure implies

a good chance for exploitation• Goal: make sure certain bad things don’t happen,

no matter what– Crashes, thrown exceptions, non-termination– All of these things can be the foundation of security

vulnerabilities• Complements functional testing– Test features (and lack of misfeatures) directly– Normal tests can be starting points for fuzz tests

35

Kinds of Fuzzing

• Black box– The tool knows nothing about the program or its input– Easy to use and get started, but will explore only

shallow states unless it gets lucky• Grammar based – The tool generates input informed by a grammar– More work to use, to produce the grammar, but can

go deeper in the state space• White box– The tool generates new inputs at least partially

informed by the code of the program being fuzzed– Often easy to use, but computationally expensive

36

Fuzzing Inputs

• Mutation– Take a legal input and mutate it, using that as input– Legal input might be human-produced, or automated,

e.g., from a grammar or SMT solver query• Mutation might also be forced to adhere to grammar

• Generational– Generate input from scratch, e.g., from a grammar

• Combinations– Generate initial input, mutate, generate new inputs, …– Generate mutations according to grammar

37

File-based Fuzzing

• Mutate or generate inputs• Run the target program with them• See what happens

38

Examples: Radamsa and Blab

• Radamsa is a mutation-based, black box fuzzer– Mutates inputs that are given, passing them along

• Blab generates inputs according to a grammar (grammar-based), specified as regexps and CFGs

% echo "1 + (2 + (3 + 4))" | radamsa --seed 12 -n 45!++�� (3 + -5))1 + (3 + 41907596644)1 + (-4 + (3 + 4))1 + (2 +�� (3 + 4% echo … | radamsa --seed 12 -n 4 | bc -l

https://code.google.com/p/ouspg/wiki/Radamsa https://code.google.com/p/ouspg/wiki/Blab

% blab -e '(([wrstp][aeiouy]{1,2}){1,4} 32){5} 10’soty wypisi tisyro to patu

39

Example: American Fuzzy Lop

• It is a mutation-based, white-box fuzzer. Process: – Instrument target to gather run-time information

• Tuple of <ID of current code location, ID last code location>

– Run a test. Mutate test input to create a new one if unseen tuple generated; otherwise discard the test• Mutations include bit flips, arithmetic, other standard stuff

– Periodically cull gathered tests, to avoid local minima% afl-gcc -c … -o target% afl-fuzz -i inputs -o outputs targetafl-fuzz 0.23b (Sep 28 2014 19:39:32) by <lcamtuf@google.com>[*] Verifying test case 'inputs/sample.txt'...[+] Done: 0 bits set, 32768 remaining in the bitmap. …———————Queue cycle: 1n time : 0 days, 0 hrs, 0 min, 0.53 sec …

http://lcamtuf.coredump.cx/afl/

40

Network-based fuzzing

• Act as 1/2 of a communicating pair– Inputs could be produced by replaying previously

recorded interaction, and altering it, or producing it from scratch (e.g., from a protocol grammar)

41

Network-based fuzzing

• Act as a “man in the middle”– mutating inputs exchanged between parties

(perhaps informed by a grammar)

42

Example: SPIKE

• SPIKE is fuzzer creation kit, providing a C-language API for writing fuzzers for network based protocols

s_size_string(“post”,5);s_block_start(“post”);s_string_variable(“user=bob”);s_block_end(“post”);spike_tcp_connect(host,port);spike_send();spike_close_tcp();

backpatch for length field… for this block

prefix of string to fuzzend of block; include length

connect to serversend this buffer

close connection

http://www.immunitysec.com/downloads/SPIKE2.9.tgzhttp://resources.infosecinstitute.com/intro-to-fuzzing/

43

Example: Burp Intruder

• Burp automates customized attacks against web applications

• Similar to SPIKE in allowing the user to craft the template of a request, but leave “holes” (called payloads) for fuzzing– Nice GUI front end

• Integrates with the rest of the Burp Suite, which includes a proxy, scanner, spider, and more

http://portswigger.net/burp/intruder.html44

Dealing with Crashes

• You fuzz. A crash occurs. Questions:• What is the root cause (so it can be fixed)?– Is there a way to make the input smaller, so it is

more understandable?– Are two or more crashes signaling the same bug?• Yes, if they “minimize” to the same input

• Does the crash signal an exploitable vulnerability?– Dereferencing NULL is rarely exploitable– Buffer overruns often are

45

Finding Memory Errors

• Compile the program with Address Sanitizer (ASAN)– Instruments accesses to arrays to check for overflows,

and use-after-free errors– https://code.google.com/p/address-sanitizer/

• Fuzz it• Did the program crash with an ASAN-signaled

error? Then worry about exploitability• Similarly, you can compile with other sorts of

error checkers for the purposes of testing– E.g., valgrind memcheck http://valgrind.org/

46

A few Other Fuzzers

• CERT Basic Fuzzing Framework (BFF)– Based in part on Zzuf (http://caca.zoy.org/wiki/zzuf)– Found bugs in Adobe Reader and Flash Player, Apple

Preview and QuickTime, and others– https://resources.sei.cmu.edu/library/asset-

view.cfm?assetID=507974

• Sulley ( https://github.com/OpenRCE/sulley) provides lots of extras to manage the testing process:– watches the network and methodically maintains records – instruments and monitors the health of the target, capable

of reverting to a known good state– detects, tracks and categorizes detected faults– fuzzes in parallel, if desired

There are many more …47

Summary

• Penetration testers simulate real attackers– Try to find exploitable vulnerabilities in complete

systems

• Penetrations signal real problems– Lack of penetrations is not proof of impossibility

• Pen testers employ a variety of tools– Scanners, proxies, exploit injectors, fuzzers

• And require ingenuity and guile

48