defect discovery features for hpux/ipf compilers (vaccine project) suneel jain xinliang david li...

Defect Discovery Features for HPUX/IPF Compilers(Vaccine Project)

Suneel JainXinliang David LiXiaohua ZhangCarl Burch

September 20, 2005 AADEBUG 2005

Agenda

• Features Overview

• Compile time checks+wsecurity+wlint+w64bit

• Runtime checks+check=all|none|stack|malloc|uninit


Features Overview

• Compile time security vulnerability checking– Advanced analysis using interprocedural optimization

Compile time detection of potential runtime errors– Whole program lint (similar to FlexeLint)– Diagnostics about 32->64 bit porting problems

Runtime error detection via compiler instrumentation

– Out of bound accesses for stack, heap objects– Memory allocation/deallocation problems

Available with the latest compilers on IPF (AR0509)


Static Analysis of Security VulnerabilitiesBuffer overflow vulnerabilities occur with stack buffers

being overflowed by an attacker so that the return address can be overwritten, unsafe use of strcpy etc …

Format string vulnerabilities occur when untrustworthy data is used as a format string argument.

• Untrustworthy data data that could potentially be controlled by an attacker

• Simplest option is to treat all format strings not known at compile time as potentially dangerous, but that approach results in lots of warnings.

• We treat all program inputs that could be controlled by an attacker as tainted and then track the propagation of tainted values through each program operation.


Static Analysis of Security Vulnerabilities

Uses the option +wsecurity

• Simple local checks for unsafe libc calls, format string problems (similar to RATS, ITS4)– not enough context leads to many false positives

• Cross-Module analysis to detect potential use of tainted values at critical reference points– much more accurate– provides more information to the user


Cross-Module Static Analysis

• Detect if tainted data can reach critical reference

• Examples of critical references– length argument in string copy APIs– size argument in allocators– target buffer for memory copy APIs– array reference indexing– loop exit condition

• Interprocedural and context-sensitive flow analysis– vastly improves precision of the analysis– show path from the suspect definition to the reference


Example (xchat)

------------------------ xchat1.c -----------------------------

#include <sys/socket.h>extern int getlen();extern void getdata(int len, char* buf);int main(){ char buf[100]; int len = getlen(); getdata(len, buf); copydata(len,buf);}

------------------------ xchat2.c ----------------------------

#include <sys/socket.h>int getlen(){ int buf[10]; recv(1, buf, 1, 0); return buf[0] + 2;}

------------------------ xchat3.c -----------------------------

#include <sys/socket.h>void getdata(int len, char *buf){ recv(1, buf, len, 0); // Line 4}

------------------------ xchat4.c -----------------------------

#include <sys/socket.h>void copydata(int len, char *buf){ int i; for (i = 0; i < len; i++) // Line 7 { buf[i] = i; }}


Call Graph

main

getlengetdata

copydata


Data flow

main

getlengetdata

copydataTainted data detected

Critical reference

detectedCritical reference

defected

Tainted data

Flow back to

main

Tainted data flow into critical reference

Tainted data flow into critical reference


Compiler Diagnostics for xchat

"xchat4.c", line 5: warning #20114-D: (SECURITY) Tainted value may be used in loop exit condition computation (procedure "copydata")

++ 'copydata' is called by 'main' at line 9 in file xchat1.c ++++ tainted value is returned from 'getlen' called by 'main' at

line 7 in file xchat1.c

"xchat3.c", line 4: warning #20111-D: (SECURITY) Tainted data may be used in data length computation (procedure "getdata")

++ 'getdata' is called by 'main' at line 8 in file xchat1.c ++++ tainted value is returned from 'getlen' called by 'main' at

line 7 in file xchat1.c


+wlint option

• Enables several warnings in the compiler that provide lint like functionality.

• These compile time diagnostics can be very useful in detecting potential problems in the source code.

• To disable a specific warning introduced by +wlint, a +Wnnnn option can be used after the +wlint option.


+w64bit option

• Enable warnings that help detection of potential problems in converting 32 bit applications to 64 bit.

• The +w64bit option applies only to a 64 bit compile (using +DD64).


Runtime Security Checks

• Prefer preventive checking via static analysis over runtime detection– runtime checks do not find the root cause

of the problem– execution time overhead due to

instrumentation


New runtime check options

• +check=all|none|bounds|malloc|stack|uninit

• The +check=xxx options provide runtime checks to detect potential errors.

• These options introduce additional instructions and may significantly slow down the program.

• A failed check will result in the program aborting at runtime. The +check options need to be specified at both compile time and link time since they may require additional libraries to be linked into the user program.


+check=bounds

• Enables array bounds checking. • This introduces additional runtime overhead for

instrumentation of array references. • The +check=bounds option applies to local and

global array variables. • It also applies to references to array fields of

structs.• It does not apply to arrays allocated dynamically

using malloc or alloca. It also does not cover pointer dereferences.


+check=bounds

#include <stdio.h>int x[200];int main () { int inp; scanf("%d", &inp); return x[inp]; /* Bounds check fails if inp<0 or inp>199. */}


+check=bounds

• cc –v +check=bounds 1.c• ./a.out

200Runtime Error: out of bounds in file 1.c at line 6(0) 0x04001aa0 _rtc_raise_fault + 0x620 [a.out](1) 0x04002150 _rtc_check_bounds + 0x490 [a.out](2) 0x04001200 main + 0x1a0 [a.out]Bus error (core dumped)


+check=stack

• +check=stack[:frame|:variables|:none]

• This option enables runtime checks to detect illegal overwriting of the stack. This can be done for the whole stack frame or around individual stack variables.

• +check=stack:frame – The option enables runtime checks for illegal overwriting

of the stack frame.

+check=stack:variablesThis option enables runtime checks around variables on the stack.


+check=stack

• +check=stack:none – This option disables all runtime stack checking.

+check=stack

This default form of the option +check=stack is equivalent to +check=stack:variables at optimization levels 0 and 1.

It is equivalent to +check=stack:frame for optimization level 2 and above.

When a stack check fails at runtime, the stack trace is written to stderr and the program is aborted.


+check=stack:frame

$ cat stack_frame.c int main() { int a[2]; int i=0; for (;i<=12; i++) a[i] = i; return a[1]; }


+check=stack:frame

$ cc +check=stack:frame stack_frame.c $ a.out

Runtime Error: stack overrun at 0x7fffee9c(0) 0x04001340 _rtc_raise_fault + 0x120

[./a.out](1) 0x040010b0 main + 0x130 [./a.out]Abort(coredump)


+check=stack:variables

$ cat stack_variables.c int main() { int a[12]; int i=0; for (;i<=12; i++) a[i] = i; return a[1]; }



$ cc +check=stack:variables stack_variables.c$ a.out Runtime Error: storage around end of variable "a" corrupted(0) 0x040013a0 _rtc_raise_fault + 0x120 [./a.out](1) 0x04001da0 _rtc_check_guards + 0x460 [./a.out](2) 0x04001110 main + 0x130 [./a.out]Abort(coredump)



• The +check=stack:variables guards any array, struct/class/union, or object whose address is taken by putting head and foot guard values before and after these variables. If any part of the guards are changed the function will abort when it returns.

• The runtime error message for +check=stack:variables provides detailed info including the variable name, whether the begin or the end of the variable is corrupted, and the stack trace of the aborting function.

• This option also protects the stack frame, so there is no need to use both +check=stack:variables and +check=stack:frame,


Detecting use of uninitialized variables

• Programming errors result in certain stack variables being used before they are initialized.

• The detection of the use of stack variables before they have been initialized can occur both at compile time and at runtime.

• Compile time warnings may occur for code paths which may not be taken at runtime.

• It is difficult for the programmer to separate out the true issues (warnings from code paths which will actually be taken) from the large number of warnings produced.

• Having a runtime check in the compiler which will trigger a runtime assertion only when there is actually a use of an uninitialized variable in the course of program execution will help during program development and testing.

• Use the option +check=uninit for this.


+check=uninit

void Test6(int value) { int uninitialized; int var; switch (value) { case 3: uninitialized = 4; case 2: var = 5 * uninitialized; break; case 1: int *var2; var2= &uninitialized; var = 5 * uninitialized; break; } }int main(void) { Test6(1);}


+check=uninit

$ cc -v -o ./uninit +check=uninit test1.c

$ ./uninit Runtime Error: use of uninitialized variable in file uninit.c line 15 variable uninitialized(0) 0x040015c0 _rtc_raise_fault + 0x120 [./uninit](1) 0x04002520 _rtc_init_check + 0x120 [./uninit](2) 0x04001270 Test6 + 0x1f0 [./uninit](3) 0x04001340 main + 0x30 [./uninit](4) 0x6c028de0 main_opd_entry + 0x50 [/usr/lib/hpux32/dld.so]ABORT instruction

• This will result in the program getting aborted due to the use of uninitialized variable "uninitialized" in line 15 in file test1.c


+check=uninit

#include <stdio.h>void Test6(int value) { int uninitialized; int var; int var6; switch (value) { case 3: uninitialized = 4; case 2: var = 5 * uninitialized; break; case 1: break; } var6 = 3 * uninitialized; printf("%d is the value of var6\n", var6);}int main(void) { Test6(1);}


+check=uninit

• $ cc -v -o ./uninit1 +check=uninit test2.c

• $ ./uninit1

• We will get the following runtime assertion:

• Runtime Error: use of uninitialized variable in file test2.c line 15 • variable uninitialized

• (0) 0x0000000004001540 _rtc_raise_fault + 0x120 [./uninit1]• (1) 0x00000000040024a0 _rtc_init_check + 0x120 [./uninit1]• (2) 0x0000000004001200 Test6 + 0x180 [./uninit1]• (3) 0x00000000040012c0 main + 0x30 [./uninit1]• (4) 0x60000000c0028de0 main_opd_entry + 0x50

[/usr/lib/hpux32/dld.so]• ABORT instruction


+check=malloc

• Detects several kinds of memory problems:

• memory leaks• heap memory bounds overrun and underrun• freed memory read• bad free and duplicate free

• and provide memory heap info


+check=malloc requirements

• The +check=malloc option is a link time option. It doesn't require recompiling the source files with this option, but the user should use the +check=malloc cc or aCC driver option to link the program.

• It requires the latest /opt/langtools/lib/hpux[32|64]/librtc.so, with latest wdb installed.

• malloc/free/realloc should be in a shared library so they can be overridden by librtc.so.

• In order to get the line number in the error message, the program should not stripped.


+check=malloc

• Uses librtc.so to do memory management and checking, the librtc.so is linked into the program as the first shared library.

• All the references to malloc/free functions are through the functions in librtc.so.

• The malloc() function records the allocation info and does appropriate initialization specified by rtc config (either default config or config file/config env variable ).

• The free() function checks the freed pointer against the allocation info to find bad frees, and does the memory bounds violation check if specified. It may also scramble the freed memory block to catch freed memory read.

• All actual malloc and free are done by the malloc and free in libc.so, so even if the program has its own version of malloc/free, they are not used for memory allocation and deallocation.


+check=malloc

• If a memory problem (such as bounds violation, bad free, out of memory) is found when doing allocation/de-allocation, the program will print out error message and abort the execution.

• User can change the rtc config to not abort on particular events and continue execution.

• At program exit time, librtc.so will check the memory info to find memory leaks.

• The leak info and memory corruption info are dumped to log files, the log file names are written to stderr.


+check=malloc

$ cat memory_bug1.c#include <stdio.h>#include <stdlib.h>void foo() { char *p; int i;

p = (char *)malloc(10240);

p += 2; free(p); // bad free and memory leak }int main() { foo(); return 0; }


+check=malloc

$ cc +check=malloc malloc-bug1.c

$ a.out warning: Memory corruption info is written to "./a.out.21983.mem". Abort on memory event RTC_BAD_FREE, see log file ./a.out.21983.mem for detail (0) 0x6ca0e120 __rtc_free + 0x640 [/opt/langtools/lib/hpux32/librtc.so] (1) 0x04000a40 foo + 0xa0 [./a.out] (2) 0x04000a80 main + 0x20 [./a.out] (3) 0x6c13cde0 main_opd_entry + 0x50 [/usr/lib/hpux32/dld.so] Abort(coredump)


+check=malloc

$ cat ./a.out.21983.mem -----------------------------------------Attempt to free unallocated or already freed object at 0x404978a2(0) 0x6ca14860 print_stack_trace_to_log_file + 0x220 [/opt/langtools/lib/hpux32/librtc.so](1) 0x6ca15cf0 __rtc_event + 0x130 [/opt/langtools/lib/hpux32/librtc.so](2) 0x6ca1a8e0 rtc_record_free + 0xdb0 [/opt/langtools/lib/hpux32/librtc.so](3) 0x6ca0dee0 __rtc_free + 0x400 [/opt/langtools/lib/hpux32/librtc.so](4) 0x04000a40 foo + 0xa0 [./a.out](5) 0x04000a80 main + 0x20 [./a.out](6) 0x6c53cde0 main_opd_entry + 0x50 [/usr/lib/hpux32/dld.so]


+check=malloc limitations

• The runtime memory check library maintains data about each allocated memory block, and does checks when the memory is freed, so there is some memory space and runtime overhead to the program using the service.

• The optimized program may have functions inlined into other functions, so the reported stack trace may not reflect the actual function which has memory leaked or corrupted. To get an accurate stack trace, the user needs to disable inlining with the "+d" option.

• Since the runtime memory checker uses gdb to print its report at exit time, if the user interrupts the program at that time the program may stop in gdb.

• To get the log files the program can not be traced, that means it can not be run under a debugger or be traced by tusc or caliper.


For Further Information …

www.hp.com/go/c++

defect discovery features for hpux/ipf compilers (vaccine project) suneel jain xinliang david li...

Documents