performance programming ipeisert/teaching/ecs142-sp09/lecture_slides/... · fnms...

1

Performance Programming& Static Analysis for Security

Lecture 28

(partially taken from slides given at Lawrence Livermore National Labs by Larry Carter and

Sean Peisert)

Dr. Sean Peisert – ECS 142 – Spring 2009

Wednesday, June 3, 2009

Status

• Project 3 back sometime in the middle of this week.

• Project 4 due this Friday, 11:55pm

2


3

Approach

• Engineer’s method:– DO UNTIL (exhausted)– tweak something– IF (better) THEN accept_change

• Scientific method:– DO UNTIL (enlightened)– make hypothesis– experiment– revise hypothesis


4

Power3’s power … and limits

• Eight pipelined functional units

• 2 floating point

• 2 load/store

• 2 single-cycle integer

• 1 multi-cycle integer

• 1 branch

• Powerful operations

• Fused multiply-add (FMA)

• Load (or Store) update

• Launch 4 ops per cycle• Can’t launch 2 stores/cyc

• FMA pipe 3-4 cycles long

• Memory hierarchy (Tues)


5

Can its power be harnessed?

for (j=0; j<n; j+=4){ p00 += a[j+0]*a[j+2]; m00 -= a[j+0]*a[j+2]; p01 += a[j+1]*a[j+3]; m01 -= a[j+1]*a[j+3]; p10 += a[j+0]*a[j+3]; m10 -= a[j+0]*a[j+3]; p11 += a[j+1]*a[j+2]; m11 -= a[j+1]*a[j+2];

}

8 FMAʼs

4 Loads

Runs at 4.6 cycles/iteration (= 772 MFLOP/S)

CL.6:

FMA fp31=fp31,fp2,fp0,fcrLFL fp1=(*)double(gr3,16)FNMS fp30=fp30,fp2,fp0,fcr

LFDU fp3,gr3=(*)double(gr3,32)FMA fp24=fp24,fp0,fp1,fcrFNMS fp25=fp25,fp0,fp1,fcrLFL fp0=(*)double(gr3,24)FMA fp27=fp27,fp2,fp3,fcrFNMS fp26=fp26,fp2,fp3,fcrLFL fp2=(*)double(gr3,8)FMA fp29=fp29,fp1,fp3,fcrFNMS fp28=fp28,fp1,fp3,fcr

BCT ctr=CL.6,


6

Can its power be harnessed (part II)

• 8 FMA, 4 Load - 1.15 cycle/load (previous slide)

• 8 FMA, 6 Load - 1.3 cycle/load

• 8 FMA, 8 Load - 1.2 cycle/load

• 4 Add, 4 Load - 1.1 cycle/load

• Shift, Add, Load, Store - 1.15 cycle/MemOp

• Load, Store - 1.1 cycle/MemOp

• I haven’t broken the 1 cycle/MemOp barrier!• but I’ve only spent 2 days trying …maybe the AGEN unit is disabled ...


7

FLOP to MemOp ratio

• Most scientific programs have at most one FMA per MemOp

• Matrix-vector product: (K+1) loads, K fma’s

• FFT butterfly: 8 MemOps, 10 floats (but 5 or 6 FMA)

• DAXPY: 2 Loads, 1 Store, 1 FMA

• DDOT: 2 Loads, 1 FMA

• A few have more (use ESSL!)

• Matrix multiply (well-tuned): 2 FMA per load

• Radix-8 FFT • Performance is limited by Memory Operations!


8

The effect of pipeline latency

for (i=0; i<size; i++) { sum = a[i] + sum;

}

for (i=0; i<size; i+=4) { sum0 += a[i];

sum1 += a[i+1]; sum2 += a[i+2]; sum3 += a[i+3];

} sum = sum0+sum1+sum2+sum3;

3.86 cycles/addition

1.1 cycles/addition

Next add can’t start until previous is finished (3 to 4 cycles later)


8

The effect of pipeline latency

for (i=0; i<size; i++) { sum = a[i] + sum;

}

for (i=0; i<size; i+=4) { sum0 += a[i];

sum1 += a[i+1]; sum2 += a[i+2]; sum3 += a[i+3];

} sum = sum0+sum1+sum2+sum3;

3.86 cycles/addition

1.1 cycles/addition

Next add can’t start until previous is finished (3 to 4 cycles later)

May change answer due to different rounding.


What’s so great about Fortran??

DO I = 1, N A(I) = B(I)

ENDDO

CL.8:L4A gr0=b(gr5,4)L4A gr6=b(gr5,8)L4A gr7=b(gr5,12)L4AU gr8,gr5=b(gr5,16)ST4A a(gr4,8)=gr6ST4A a(gr4,4)=gr0ST4A a(gr4,12)=gr7ST4U gr4,a(gr4,16)=gr8BCT ctr=CL.8,

for (i=0; i<N; i++) { b[i] = a[i];

}

CL.6:ST4U gr4,(*)int(gr4,4)=gr24L4AU gr24,gr3=(*)int(gr3,4)BCT ctr=CL.6,


10

Fortran vs C - what’s going on??

• C prevents compiler from unrolling code

• A feature, not a bug!

• User may want b[0] and a[1] to be same location

• tricky way to set a[n] = ..… = a[1] = a[0]

• Most C compilers don’t try to prove non-aliasing

• a and b were malloc-ed in this example

• Fortran doesn’t allow arrays to be aliased

• Unless explicit, e.g. via EQUIVALENCE


11

Fortran vs. C - does it matter??

for (i=0; i<N; i+=4) { b0 = a[i];

b1 = a[i+1]; b2 = a[i+2]; b3 = a[i+3]; b[i] = b0;

b[i+1] = b1; b[i+2] = b2; b[i+3] = b3;

}

• Yes - Fortan code should perform better– My tests show both are about 1 cycle/MemOp– Fortran should be .5 cycle/MemOp

• No - you could get the “Fortran” object code from


12

Decreasing MemOp to FLOP Ratiofor (i=1; i<N; i++)

for (j=1; j<N; j++)

b[i,j] = 0.25 *

(a[i-1][j] + a[i+1][j] + a[i,j-1] + a[i][j-1]);

for (i=1; i<N-2; i+=3) {

for(j=1; j<N; j++) {

b[i+0][j] = ... ;

b[i+1][j] = ... ;

b[i+2][j] = ... ;

}

}

3 loads4 floats1 store

5 loads12 floats3 store


Compilers: Topics Not Covered

• Lambda Calculus

• Foundation for Programming Languages

• Instruction Scheduling

• Compiling Exceptions


A Few Thoughts on Compilers and Security


Vulnerable Program?

int main(int argc, char *argv[]) {

char buffer[500];

strcpy(buffer, argv[1]);

printf("Safe program?");

return 0;

}


Shellcode

\x31\xc0\x50\x68\x2\x2\x73\x68\x68\x2\x62\x69\x6\x89\xe3\x50\x53\x50\x54\x53\xb0\x3\x50\xcd\x80


local var

local var

local var

local var

saved frame ptr

return addr

stack

stack

a

a

a

a

a

\x31\xc0\x50\x68\x2

etc...

stack ptr


Buffer Overflows

18


Impact of Buffer Overflows

• Buffer overflows (including those on the stack, the heap, and also integer overflows) have dominated exploitable programming flaws for years. (http://www.sans.org/top20/)

• E.g., worms: Blaster, Morris, Slammer, Witty, etc....

• It would be great if programmers just wrote better code and did their own bounds checking. But in practice, even good coders make mistakes.


Programming Languages

• C/C++ allows overwriting memory arbitrarily via pointer accesses.

• C/C++ are the root of a lot of evil (but not all) in computer security


Countermeasures

• Compilers and compiler extensions can perform some countermeasures via static analysis and code modification at compile time.

• Binary rewriters can also do this statically.

• Much of what compilers do is look for certain invariants: conditions that should hold and sometimes do not.


Static Analysis

• Some things are easy at compile time:

• strcpy → strncpy

• strcat → strncat

• Some things are easy at runtime, e.g., bounds checking:

• for strcpy(dest,src), how large is the space allocated for dest vs. the length of src?

• (strncpy does this anyhow)

• Java also does this


Safe Functions/Libraries

• Some C Functions Have “Safe” Equivalents

• strcat/strncat

• strcpy/strncpy

• sprintf/snprintf

• Some functions cannot be used safely:

• gets/fgets

• (the Morris worm exploited this in 1988)


Modifying Code

void func() {char buf[100], buf2[100], buf3[200];/* do stuff */

}

void func_mod() {char buf[100], buf2[100], buf3[200];__stack_buf(buf1, sizeof(buf1));__stack_buf(buf2, sizeof(buf2));__stack_buf(buf3, sizeof(buf3));/* do stuff */

}


StackGuard and Others

• Checks the to see if the stack has been altered when a function returns.

• A “canary” value is stored where the return address usually is stored (and maybe other places too).

• Sometimes the return address it put elsewhere

• Sometimes the return address is just stored XORed

• When the function returns it checks to see if the canary has changed.

• Can’t protect against buffer overflows in the heap.


Pointer Protection

• XOR-ing pointers is used in other places too, and doesn’t have to refer only to return addresses.

• Can XOR all pointers.

• Microsoft worked on this for Windows at one point using a compiler extension.


Address Space Layout and Direction

• The stack can run up, down, or both.

• Or, address areas (stack, heap, etc...) can be placed arranged randomly in a process’s address space.

• Larger memory space = more entropy = harder to attack


Non-Executable Stack

• Some operating systems prevent execution (e.g., shellcode that says, “exec /bin/sh”) on the stack.


Bounds Checking

// C allows you do do this

int main(int argc, char *argv[]) {

char myArray[500];

printf(“ThisValue: %d”,

(int)myArray[argv[1]]);

return 0;

}


Bounds Checking

// Maybe compilers should do this?int main(int argc, char *argv[]) {

char myArray[500]; if ((int)argv[1] < 500) {

printf(“ThisValue: %d”,(int)myArray[argv[1]]);

}else { ... }return 0;

}


Numeric Bounds Checking

// C allows you do do thisint main(int argc, char *argv[]) {

short a = 32767;int i;for (i = 0; i < atoi(argv[1]); i++)

a++;if (a > 0)

...} // 32677, -32768, -32767, -32766...


TOCTTOUTime-of-check-to-time-of-use

if (access(filename, W_OK) == 0) { if ((fd = open(filename, O_WRONLY)) == NULL){

perror(filename); return(0);

} /* now write to the file */


TOCTTOU Flaw

Computing Systems

9

(2) pp. 131–152 (Spring 1996). Page 3 of 20

checked).

The scenario in Figure 1 is an example of the TOCTTOU binding flaw. Figure 1a shows the

state of the system at the time of the

access

system call; the solid arrow indicates the

access

refers

to “/tmp/X”. Both “/tmp/X” and “/etc/passwd” name distinct objects. However, before the process

makes its

open

system call, “/tmp/X” is deleted and a direct alias (hard link) for “/etc/passwd” is

created, and is named “/tmp/X”. Then the

open

accesses the data associated with “/etc/passwd”

when it opens “/tmp/X”, since “/tmp/X” and “/etc/passwd” now refer to the same file. Figure 1b

shows this, with the dashed arrow indicating which data is actually read and the solid arrow indi-

cating the name given to

open

. The unprivileged process can then write to the protected password

file. Several versions of the terminal emulation program

xterm

(1) [16] suffer from this flaw, which

arises when logging sessions to a file.

Another instance of this flaw occurs on SunOS and HP/UX systems. The program

passwd

(1)

allows the user to name the password file as a parameter. An attacker can gain access to any other

user’s accounts using a variant of the attack presented above [6]. Under normal conditions, the

passwd

program takes the following steps:

1. opens and reads the password file to get the entry for the user; then closes the password file;

2. creates and opens a temporary file called “ptmp” in the directory of the password file;

3. opens the password file again, and copies the contents to “ptmp”, updating the changed infor-

mation; and

4. closes the password file and “ptmp” and renames “ptmp” to be the password file.

Figure 1. Example of the TOCTTOU binding flaw.

/

etc

passwd

tmp

X

open(“/tmp/X”, O_WRITE)

passwd data

/

etc

passwd

tmp

X

access(“/tmp/X”, W_OK)

X datapasswd dataX data

Figure 1a. Figure 1b.


TOCTTOUint main() {

// start with root privilegesdo something with privilege();exec (“/bin/sh/”);

}

vs. int main() {

// start with root privilegesdo something with privilege();// drop privilegesexec (“/bin/sh/”);

}


Privilege EscalationWe have found previously unknown weaknesses in five

out of the eight programs that we checked. Our experience

has demonstrated that model checking security properties

in large, real programs is practical and useful. Though our

experiments focus on MOPS, MOPS is just one example

of a whole class of model checking tools, and the lessons

we learned are likely to be generally applicable to other

model checking systems, as well.

2. Overview of MOPS

MOPS is a static (compile-time) analysis tool [6].

Given a program and a security property, MOPS checks

whether the program can violate the security property.

The security properties that MOPS checks are tempo-

ral safety properties, i.e., properties requiring that pro-

grams perform certain security-relevant operations in cer-

tain orders. For example, MOPS might be used to check

the following property: a setuid-root program had bet-

ter drop root privilege before executing an untrusted pro-

gram; otherwise, the untrusted program may execute with

root privilege and therefore compromise the system (see

Section 3.1.1. for details on this property). The MOPS

user describes a security property by a Finite State Au-

tomaton (FSA). Figure 1(a) shows a simplified FSA de-

scribing the above property1 and Figure 1(b) shows a pro-

gram that violates this property.

MOPS checks if a program may violate a temporal

safety property using pushdown model checking. Model

checking technique exhaustively searches the control flow

graph of a program to check if any path may violate

a safety property. Pushdown model checking enables

searching inter-procedural paths in a context-sensitive

manner. If MOPS finds violations, it reports error traces,

program paths that cause such violations. In general,

model checking tends to be best at checking properties

that refer primarily to the control flow of the program; in

contrast, model checking is usually less well suited for

checking data-flow intensive properties.

2.1. Soundness

MOPS strives for soundness, precision, and scalabil-

ity. A sound tool will not overlook any violations of the

security property in the program2. A precise tool will

have very few false alarms. However, it is challenging

to achieve all three criteria at once. Since MOPS is de-

signed to be a practical tool for checking lots of security

1The FSA depicted here considers only one call,

setuid(getuid()), for dropping root privilege and only one

call, execl(...), for executing an untrusted program. Other calls are

omitted for clarity.2A tool that proves whether a program satisfies a property is sound

if all the programs that it can prove to satisfy the property do in deed

satisfy the property.

priv error

setuid(getuid())

unpriv

execl()

(a) An FSA describing this property.

// The program has root privilege

if ((passwd = getpwuid(getuid())) != NULL)

{fprintf(log, “drop priv for %s”, passwd->pw name);setuid(getuid()); // drop privilege

}execl(“/bin/sh”, “/bin/sh”, NULL); // risky syscall

(b) A setuid-root program that violates this property. One path of

the program satisfies the property, but the other path violates it,

giving the user a shell with full privilege.

Figure 1. An FSA describing the property

that “A setuid-root program should drop root priv-

ilege before executing an untrusted program” and

a program violating it.

properties on large programs, it tries to strike a balance

between soundness, precision, and scalability. To strive

for soundness, MOPS is path sensitive, i.e., it follows ev-

ery path in the program (including arbitrary number of

iterations of loops) except a few minor cases discussed

below. MOPS is also context sensitive, i.e., MOPS can

match each function return with its call site. To ensure

scalability, MOPS takes the approach of sacrificing on the

precision of its data-flow analysis rather than sacrificing

on scalability. Since data-flow analysis presents many dif-

ficulties for scalability, MOPS chooses to be data-flow in-

sensitive. In other words, MOPS ignores most data values

in the program and assumes that each variable may take

any value3. Therefore, MOPS assumes that both branches

of a conditional statement may be taken and that a loop

may execute anywhere from zero to infinite iterations.

As such, MOPS is mostly suitable for properties that are

control-flow centric.

MOPS is sound under the following assumptions:

• The program is single-threaded. In other words,

3MOPS implements limited data flow analysis so that it recognizes

the same variable x in different expressions such as x=open() and

close(x).

We have found previously unknown weaknesses in five

out of the eight programs that we checked. Our experience

has demonstrated that model checking security properties

in large, real programs is practical and useful. Though our

experiments focus on MOPS, MOPS is just one example

of a whole class of model checking tools, and the lessons

we learned are likely to be generally applicable to other

model checking systems, as well.

2. Overview of MOPS

MOPS is a static (compile-time) analysis tool [6].

Given a program and a security property, MOPS checks

whether the program can violate the security property.

The security properties that MOPS checks are tempo-

ral safety properties, i.e., properties requiring that pro-

grams perform certain security-relevant operations in cer-

tain orders. For example, MOPS might be used to check

the following property: a setuid-root program had bet-

ter drop root privilege before executing an untrusted pro-

gram; otherwise, the untrusted program may execute with

root privilege and therefore compromise the system (see

Section 3.1.1. for details on this property). The MOPS

user describes a security property by a Finite State Au-

tomaton (FSA). Figure 1(a) shows a simplified FSA de-

scribing the above property1 and Figure 1(b) shows a pro-

gram that violates this property.

MOPS checks if a program may violate a temporal

safety property using pushdown model checking. Model

checking technique exhaustively searches the control flow

graph of a program to check if any path may violate

a safety property. Pushdown model checking enables

searching inter-procedural paths in a context-sensitive

manner. If MOPS finds violations, it reports error traces,

program paths that cause such violations. In general,

model checking tends to be best at checking properties

that refer primarily to the control flow of the program; in

contrast, model checking is usually less well suited for

checking data-flow intensive properties.

2.1. Soundness

MOPS strives for soundness, precision, and scalabil-

ity. A sound tool will not overlook any violations of the

security property in the program2. A precise tool will

have very few false alarms. However, it is challenging

to achieve all three criteria at once. Since MOPS is de-

signed to be a practical tool for checking lots of security

1The FSA depicted here considers only one call,

setuid(getuid()), for dropping root privilege and only one

call, execl(...), for executing an untrusted program. Other calls are

omitted for clarity.2A tool that proves whether a program satisfies a property is sound

if all the programs that it can prove to satisfy the property do in deed

satisfy the property.

priv error

setuid(getuid())

unpriv

execl()

(a) An FSA describing this property.

// The program has root privilege

if ((passwd = getpwuid(getuid())) != NULL)

{fprintf(log, “drop priv for %s”, passwd->pw name);setuid(getuid()); // drop privilege

}execl(“/bin/sh”, “/bin/sh”, NULL); // risky syscall

(b) A setuid-root program that violates this property. One path of

the program satisfies the property, but the other path violates it,

giving the user a shell with full privilege.

Figure 1. An FSA describing the property

that “A setuid-root program should drop root priv-

ilege before executing an untrusted program” and

a program violating it.

properties on large programs, it tries to strike a balance

between soundness, precision, and scalability. To strive

for soundness, MOPS is path sensitive, i.e., it follows ev-

ery path in the program (including arbitrary number of

iterations of loops) except a few minor cases discussed

below. MOPS is also context sensitive, i.e., MOPS can

match each function return with its call site. To ensure

scalability, MOPS takes the approach of sacrificing on the

precision of its data-flow analysis rather than sacrificing

on scalability. Since data-flow analysis presents many dif-

ficulties for scalability, MOPS chooses to be data-flow in-

sensitive. In other words, MOPS ignores most data values

in the program and assumes that each variable may take

any value3. Therefore, MOPS assumes that both branches

of a conditional statement may be taken and that a loop

may execute anywhere from zero to infinite iterations.

As such, MOPS is mostly suitable for properties that are

control-flow centric.

MOPS is sound under the following assumptions:

• The program is single-threaded. In other words,

3MOPS implements limited data flow analysis so that it recognizes

the same variable x in different expressions such as x=open() and

close(x).


TOCTTOU FSA Property

unprivnoexec

privnoexec

seteuid(0)

seteuid(!0)

other other

unpriv exec

priv exec

execl( ) other execl( )

execl( )

sete

uid(

0)

(a) An FSA describing Property 3

// Here ruid=x (a normal user), euid=0 (root)execl(“/bin/sh”, “sh”, NULL);

(b) A segment from a setuid-root program that violatesProperty 3. The user will receive a shell with full rootaccess, which may not have been intended. Probablythe programmer should have called seteuid(x) to dropprivilege before spawning the shell.

Figure 3: An FSA illustrating Property 3 (execl()must not be called in privileged state) and a pro-gram violating it

function drop privilege drops privilege, but the path [d0d1]fails to do so. So the path [m1d0d2d3d4m2m3] satisfies Prop-erty 3, but the path [m1d0d1m2m3] violates it. These typesof path-dependent errors are common in programs, but suchinterprocedural errors are di!cult to discover with testing ormanual review, especially if the caller and callee are in dif-ferent source files. As a result, we conclude that automatedtools to help with this task are needed.

In this paper, we describe an automated approach to helpexamine security-related temporal safety properties (abbre-viated as security properties henceforth) in software. Wehave built MOPS 2, a program analysis tool that allows usto make these properties explicit and to verify whether theyare properly respected by the source code of some applica-tion.

MOPS determines at compile time whether there is anyexecution path through a program that may violate a se-curity property. Since it is infeasible to traverse every exe-cution path because there are prohibitively many paths, weuse techniques from model checking and program analysisto structure the analysis. We model the security propertyas a Finite State Automaton (FSA) and the program as aPushdown Automaton (PDA). We then use model checkingto determine whether certain states representing violation ofthe security property in the FSA are reachable in the PDA.Our approach may be viewed as an application of lightweightformal methods to an interesting class of security properties.

MOPS is distinguished from other related tools in thefollowing aspects. First, since it is based on a solid for-mal foundation, i.e., model checking, it can take advantage

2MOdel Checking Programs for Security properties

int main(int argc, char *argv[]){ // start with root privilege

m0: do something with privilege();m1: drop privilege();m2: execl(“/bin/sh”, “/bin/sh”, NULL); // risky syscallm3: }

void drop privilege(){

struct passwd *passwd;

d0: if ((passwd = getpwuid(getuid())) == NULL)d1: return; // but forget to drop privilege!d2: fprintf(log, “drop priv for %s”, passwd->pw name);d3: seteuid(getuid()); // drop privileged4: }

Figure 4: A program where the security property isviolated on one execution path but not on the otherone.

of existing algorithms and future advances from the modelchecking community. Second, because it fully supports in-terprocedural analysis and because interprocedural bugs aremore elusive than intraprocedural ones, MOPS promises tocomplement manual auditing where an automated tool isneeded the most. Third, MOPS is sound (modulo the mildassumptions to be discussed in Section 6): it reliably catchesall bugs of the specified types. This property makes MOPSuseful not only in finding security bugs but also in verify-ing security properties. Fourth, thanks to a novel techniquethat substantially reduces the size of a program without af-fecting the result of model checking [7], MOPS scales well tolarge programs in both time and space, overcoming the scal-ability problem that hinders many software model checkingsystems. Other tools have some of these properties, but tothe best of our knowledge MOPS is the only tool that hasall of these desirable properties.

This paper is organized as follows. Section 2 and 3 de-scribe the formal models that are the foundations of thisapproach. The former presents an abstract view of the mod-els and the latter describes their implementation. Section 4discusses how to derive a security model from the operat-ing system accurately. Section 5 presents our experiences inusing MOPS to examine several security-relevant software.Section 6 discusses the soundness of this approach and itslimitations. Section 7 reviews the related work and com-pares them to MOPS.

2. FORMAL MODELS

MOPS is based on a formal approach that builds a formalmodel of a program and of a security property and thenanalyzes the models. We start by describing the problem.

2.1 The Problem

Given a program and a security property, the goal is toverify whether the program satisfies the property, and ifnot, identify why. Typically, the program performs sev-eral security-relevant operations, and the security propertyspecifies certain sequences of security operations that lead


Other Race Conditionsint a; // global

main() {

spawn_threads();

}

thread_run() {

if(thread_num % 2 = 0)

a++;

...

}


Other Race Conditionsint a; // global

main() {

spawn_threads();

}

thread_run() {

if(thread_num % 2 = 0)

lock();

a++;

unlock();

...

}


There are a lot of elements to security

• Good system design (technical and procedural)

• Good coding practice

• Good tools, including compilers (static analysis and instrumentation) and OS/virtual environments (runtime)

• Good languages that have fewer tradeoffs between security vs. usability vs. performance


performance programming ipeisert/teaching/ecs142-sp09/lecture_slides/... · fnms...

Documents