securing linux the immunix way crispin cowan, ph.d chief scientist, wirex communications, inc

Securing Linuxthe Immunix Way

Crispin Cowan, Ph.DChief Scientist,

WireX Communications, Inc.

Copyright © 2002, WireX Communications, Inc.

2

Host Security

Host security is really simple:• Make sure you only run perfect

software• Configure your perfect software

perfectlyEasy, right? :-)


3

Ok, Plan B: Minimize Exposure• Software is largely crap

The larger -> the crappier

• So run as little of it as possible uninstall stuff disable stuff

• This is employed by Bastille, OpenBSD, OWL


4

Plan C:Make the Software Suck Less• Do something to reduce the plausible

vulnerability of the software• Statically

Source code audits, e.g. OpenBSD, OWL Source code auditing tools, e.g. ITS4

• Dynamically Behavior bounds to restrict exploitability Most of Immunix


5

The Immunix Family• Dynamic behavior limitations

StackGuard: compiler hack to resist buffer overflows

FormatGuard: compileresque hack to resist printf format bugs

RaceGuard: kernel hack to resist temp file race bugs

• Static analysisSardonix security auditing portal

• Minimizing exposureSubDomain access control system


6

Buffer Overflow Attacks:How They Work• Attacker must achieve two objectives:

Arrange for attack code to be in the victim program’s address space

Get the victim program to jump to the attack code

• When victim program jumps to attack code, it typically does “exec(sh)’ Giving the attacker a shell prompt with the

privilege of the victim program If the victim is a root program, attacker now

owns the machine


7

Categorizing Buffer Overflow Attacks• Classify attacks according to how they

achieve the two required objectives• Arrange for attack code to be in the

victim program’s address space: Inject it It’s already there

• Get the victim program to jump to the attack code Corrupt a code pointer


8

Delivering Attack Code:Inject It• Attacker provides attack code as input

string to program String bytes are actually native CPU instructions This is the “buffer” part: Instructions end up in

victim program’s buffer

• Buffer can be located anywhere On the stack On the heap Static data area


9

Delivering Attack Code:It’s Already There• Attack code really is

just a few bytes: set up a system call and

call it to do “exec(sh)”

• Often these bytes are already resident in the victim program’s address space Especially since these

bytes are in common libraries like libc and glibc

• Attacker need only parameterize the code Use overflow to

change state of some variables that the library uses

Point at library code instead of injecting code


10

Getting Victim to Jump to Attack Code• The “overflow” part:

Find a program with weak bounds checking on input

Provide input larger than buffer size Overflow of buffer changes adjacent

state in unconstrained ways

• Classify according to the adjacent state being corrupted


11

Kinds of State to Corrupt

Activation Records: change function call return address or frame pointer

Function Pointers: “void (* foo)()”, i.e. pointer to function of type void

Longjmp Buffers: C’s simple checkpoint & rollback system

• In all cases, attacker changes pointer When program dereferences pointer,

program jumps to attack code


12

Example Buffer Overflow Attack: The “Stack Smash”• Everything done on stack:

Buffer is a local variable Adjacent state is the return

address

• A single input string does the whole job: String over-writes return

address String injects code

• Function return jumps to injected code

bufferbufferoverflowattack

returnaddress

stack

Changes returnaddress

stackframe

Injects code attackcode


13

Example Buffer Overflow Attack: The “Stack Smash”• Attack need not be

precise Approximate location of

return address Approximate start of

attack code

• Would-be hacker need only find an unprotected buffer in trusted code There are great cook

books for thisbuffer

bufferoverflowattack

stack



NOPNOP

Stomp

Stomp

Stomp


14

Variation: Attack Code on Heap• Use two inputs

One injects attack code into the heap

• No overflow required here

Second overflows stack buffer, pointing return address into the heap

• When function returns, program jumps to attack code on heap buffer

bufferoverflowattack

returnaddress

stack


stackframe


heap


15

Defenses• Write Correct Code• Operating Systems Approach• Direct Compiler Approach

Bounds checking

• Indirect Compiler Approach Integrity checking Obfuscation


16

Defenses• Write Correct Code• Operating Systems Approach• Direct Compiler Approach

Bounds checking

• Indirect Compiler Approach Integrity checking Obfuscation


17

Indirect Compiler Approach• Instead of preventing overflow, make

the attack ineffective• Code pointer integrity checking

Detect that an overflow has corrupted a pointer before the pointer is used


18

Code Pointer Integrity Checking: StackGuard• Generalizes Snarskii’s method• Compiler emits code that does stack

inspection prior to function return• Detect buffer overflows that attack

return address in function activation records Do integrity checking by placing a canary

word next to the return address


19

• Stack smash goes through Attack code injected Return address altered

• But Stack smash also smashes the Canary Function checks for Canary

before returning If Canary smashed,

program halts instead of yielding control to the attacker

bufferbufferoverflowattack

stack

Protect returnaddress

attackcode

NOPNOP

ReturnaddressCanary

Code Pointer Integrity Checking: StackGuard

Demo


20

Code Pointer Integrity Checking: StackGuard• Important that attacker cannot forge a

canary word and embed in attack string• Several approaches to canary integrity

Terminator Canary: canary is Null, CR, LF, and EOF• These symbols terminate most C string functions

Random Canary: canary is a random number• Chosen at exec() time, read from /dev/urandom


21

Code Pointer Integrity Checking: StackGuard

XOR Random Canary: like random canary, but XOR’d with the return address value Fixes a 1999 StackGuard vulnerability “Emsi” found a way to use two buffer

overflows to change a string pointer to point directly at the return address

XOR ties return address to canary value• Attacker cannot change return address

without detection


22

Present and FutureStackGuard• StackGuard 2.0: only ships the Terminator

Canary If you have an “Emsi” or Format string attack,

then you can hit many other targets, so there’s no point in the extra weight of XOR Random Canaries

• StackGuard 3.0: Prototype working with GCC 3.0 Moves Canary below the FP to block FP attacks


23

Format Bugs:The Basic Problem• Discovered suddenly in June 2000

Remote root vulnerability in WU-FTPD Followed by dozens of similar vulnerabilities

• Basis: arcane %n printf format string directive Tells printf to treat corresponding argument as

an int * and write back number of items formatted so far

• Problem: programs that pass un-filtered user input strings direct to printf


24

Format Bug Attacks• Program normally expects

a plain text string E.g. for user-ID “fred”

User-IDfred

Server ProgramNormalnetworkinput


25



• Attacker provides a format string E.g. “fred %n”

User-IDfred %n



26




• Program printf’s it Interpreting %n writes to

some other part of the program

User-IDfred %n


0x1234 CallStack


27




• Program printf’s it Interpreting %n writes to

some other part of the program

• Taking control of the programUser-ID

fred %n


0x1234 CallStack


28

What to do?• Remove the %n feature

Can’t: real programs use it

• Permit only static format strings Can’t: real programs legitimately create

dynamic format strings, especially for internationalization

• Count the arguments to printf, look for “extra” % directives Problematic ...


29

Varargs and Counting Arguments• Printf uses C’s varargs mechanism to

permit a variable argument list• Caller: pushes arbitrary sequence of

objects onto the stack• Callee:

Assumes the type of the first (few) items on the stack and pops them off

Uses initial items to determine type and count of other arguments to be popped off the stack


30

Varargs and Counting Arguments• The varargs argument list is arbitrarily

long, and of arbitrary type Null termination won’t work: arbitrary data is a

legitimate argument Passing the count: callee would interpret count

as an argument

• Can’t do in-band signaling for argument counting without breaking the varargs protocol Which would break compatibility with all

existing static and dynamic libraries


31

FormatGuard: Counting Arguments with CPP• We use GCC/CPP macros:

GCC/CPP lets you condense & expand variable argument lists, Lisp-style

Built an argument_count macro Defined printf(args) ->

safe_printf(arg_count(args), args) safe_printf counts the number of % directives in

the format string reject mis-matched calls


32

Frantzen’s Macro#define printf mikes_print(&cnt, print0

#define print0(x, args...) x ,print1(## args)

#define print1(x, args...) x+(++cnt-cnt) ,print2(## args)


...

void mikes_print(int *args, char *format, ...);

• So expanding printf(a, b, c);


33





...


• So expanding printf(a, b, c);mikes_print(&cnt, print0 (a, b, c);


34





...



mikes_print(&cnt, a, print1(b, c ) ;


35





...




mikes_print(&cnt, a, b +(++cnt-cnt),print2(c );


36





...




mikes_print(&cnt, a, b +(++cnt-cnt),print2(c );

mikes_print(&cnt, a, b +(++cnt-cnt),c +(++cnt-cnt) );


37

Frantzen’s Macromikes_print(&cnt, a, b +(++cnt-cnt),c +(++cnt-cnt) );

• The +(++cnt-cnt) expression evaluates to zero, but increments cnt each time it appears At the end of the call, cnt = number of

arguments presented to printf

• Limitation: cnt is a stateful variable not reentrant, not thread-safe, breaks if

argument is “a ? b : c”


38

Lokier’s Refinement#define __fg_counter(y...) __fg_count1 ( , ##y)

#define __fg_count1(y...) __fg_count2 (y, 5,4,3,2,1,0)

#define __fg_count2(_,x0,x1,x2,x3,x4,n,ys...) n

#define printf(x...) \

__fg_printf (__fg_counter(x) - 1 , ## x)


39






• __fg_counter: captures the zero case, so a null-argument call to printf works

• __fg_count1: appends place holding arguments• __fg_count2: barrel shifts argument list so correct

place holder lands on n, producing argument count


40






• So expanding printf(a, b, c);__fg_printf (__fg_counter( a, b, c ) - 1,a, b, c ) ;


41







__fg_printf (__fg_count1 ( ,a, b, c ) - 1,a, b, c ) ;


42








__fg_printf (__fg_count2 ( ,a, b, c , 5,4,3,2,1,0) - 1,a, b, c ) ;


43








__fg_printf (__fg_count2 ( ,a, b, c , 5,4,3,2,1,0) - 1,a, b, c ) ;

__fg_printf (3 - 1,a, b, c ) ;


44

What’s Happening• The arguments to match the

__fg_count2 rule in the following way:

__fg_count2 ( , a, b, c, 5, 4, 3, 2, 1, 0)

^ ^ ^ ^ ^ ^ ^ ^ ^ ^

| | | | | | | | | |

_ x0 x1 x2 x3 x4 n ys...

• Thus n gets matched to 3, which is what is returned


45

FormatGuard Implementation: Patched glibc• Patched glibc to include FormatGuard

put the macros in stdio.h put the wrapped printf functions in glibc

• Caveat: even though FormatGuard comes in a library, you must recompile applications to get protection


46

FormatGuard Demo• Xlock: standard terminal locking &

screen saver has a format bug

• Attack a FormatGuard-protected xlock Syslog the event Kill the process Dem

o


47

Race Conditions• Scenario: Root

process wants to create a unique /tmp fileStep 1: choose a

nameStep 2: check to see

if it existsStep 3: if not exists,

create

Here’s the Problem:


48





create

Here’s the Problem: attacker interrupts

between steps 2 and 3


49





create



Creates a link from expected /tmp file name to a major file, I.e. /etc/passwd


50





create



Creates a link from expected /tmp file name to a major file, I.e. /etc/passwd

When root process does the create, it stomps /etc/passwd with root’s authority


51

RaceGuard Defense• Kernel enhancement to detect race

attacks mid-way through• Makes both mktemp() and vulnerable

hand-rolled code safer to run• Abstract method: detect changes

between stat() and open() accesses to the same file name


52

RaceGuard Defense• If file “foo” does not exist at stat()

time, and does exist at open() time, then someone is racing us Cache names that are probed and not found Monitor for names that are opened and

found, and hit the cache

• Detects the difference between “probe; create” and “probe; attacker meddling; create”


53

RaceGuard Design• Attach RG cache to task descriptors• If file probes get “non-existant file” then

cache the name• If creation hits an existent file, and name

matches a name in the cache, then RaceGuard Alert Deny the open: return EPERM Kill the process

• If creation succeeds without hitting a file, and the name is in the cache, then clear it from the cache


54

Process Families• Races often involve process families:

Parent does the stat(), child does the open()

• Solution: inherit cache state from parent to child


55

RaceGuard Implementation• Mediate 3 kinds of system calls:

probing: stat(), lstat(), access(), newstat(), and newlstat()

creating: open(), creat(), mkdir(), mknod(), link(), symlink(), rename(), and bind()

process creating & removal: fork() and exit()


56

RaceGuard Implementation• On probing: if file then cache name• On creating: if name matches cache

if file then clear name from cache else RaceGuard Alert

• On process creating: copy cache from parent process

• On process removal: de-allocate cache space


57

RaceGuard Cache Management• Because RG is an intrusion rejector it

is very important to not generate false positives Aggressively propagate cache clearing

from children to parents Conservatively do not propagate cache

populating between children and parents


58

RaceGuard Cache Management• Fast approximation to FIFO

Don’t want to clear most recent entry, it’s likely about to be used

Empty slots appear naturally due to cache clearing upon successful open

Can’t use LRU: “usage” induces cache clearing

Demo


59

Minimizing Exposure Through Access Controls

Notion: Program Containment Limit the files & resources that a program

can access

Chroot: basic isolation for vulnerable programs

Immunix SubDomain: flexible confinement for vulnerable programs


60

File System Defenses:Chroot• “Change root”: makes

some subdirectory appear to be the root (“/”) directory for the calling process and its children Available as both a shell

command and a system call

• Effect: chroot’d programs cannot affect anything outside the chroot “jail” Limits impact of bugs in

program, e.g. chroot BIND

Benefits:Standard: Comes with most

UNIX’sCompatible: several current

programs have been modified to work within a chroot jail

Fast: no performance degradation

Limitations:Work: must move copies of

everything a jailed program needs into the jail

Isolation: jailed program cannot interact at all with the rest of the system


61

File System Defenses:Immunix SubDomain• Immunix Kernel Extension:

Specify the list of files that a SubDomained program may access

• Effect: SubDomained programs cannot affect anything they don’t explicitly need access to Limits impact of bugs in

program, e.g. SubDomain CGI scripts

Benefits:Flexible: SubDomained

programs can have controlled interaction with the rest of the system

Compatible: SubDomain can confine binary programs without modifications

Fast: 1% or less performance overhead

Limitations:Work: must specify “shape”

of SubDomain


62

File System Defenses:SubDomaining PHF• PHF: infamous vulnerable

CGI script legitimate function: database

lookup of user information sloppy parsing of CGI input can get PHF to start an xterm

on an arbitrary display

• To SubDomain PHF: Specify all the files that PHF

needs

• Effect: access to all other files is

denied Including xterm :-)

Place this file in /etc/subdomain.conf/phf

/home/httpd/cgi-bin/phf {

/bin/sh x 67f9f26b16172ce4f9caba49a2e00e0c ,

/etc/ld.so.cache r ,

/etc/nsswitch.conf r ,

/lib/ld-linux.so.2 r ,

/lib/libc.so.6 r ,

/lib/libtermcap.so.2 r ,

/usr/local/bin/ph ix 51458bc36b8a7d4e053c9292062a8a5e ,

}Demo


63

SubDomain and Chroot• SubDomain designed as an improved

chroot: Contain by reference instead of by value Allow controlled interaction with the system

• Feasible to confine all programs on your system

• Special feature: can contain PERL & PHP scripts, even when run via Apache modules


64

SubDomain Extension: NetDomain• Network Access restrictions in

SubDomain profiles• Can control:

Protocol: TCP/UDP Local and remote IP numbers Port numbers TCP: accept/connect UDP: send/receive


65

Major Impact:Low-Effort Protection • These tools are highly transparent:

Performance overhead: under 2% across the board, usually lower

Compatibility issues: minimal• Under 5% of all Linux programs need trivial source

patches to compile with StackGuard and FormatGuard• RaceGuard works on binary code, currently breaks

nothing

Administrative overhead: nil• Well, except for SubDomain; welcome to access

controls :)


66

Community EffortsLinux Security Module (LSM):

Patching the kernel is a pain Solution: facilitate kernel loadable security

extensions

Sardonix Security Audit Portal: Source code auditing is good for the code, but

few people actually do it Solution: facilitate & encourage source code

auditing with a web portal


67

LSM: Linux Security Module

• Standard Linux kernel limited to classical UNIX security model: root is everything POSIX.1e Capabilities

• Many security enhancements available But they are all custom patches to the

Linux kernel


68

LSM: Linux Security Module• Linux Security Module

Enhance existing kernel module interface to support security modules

• Community project Major efforts coming from corporate,

academic, and Linux community sectors 450 people on the development mailing

list


69

LSM - Architecture

User-level process

KernelLSMModule

Open syscall�Std. error checks�Std. Security checks�LSM hook:�Complete request

Policy engineexamine contextdoes request pass policy?grant or deny


70

LSM - Architecture

User-level process

KernelLSMModule

Open syscallStd. error checksStd. Security checksLSM hook:Complete request



71

LSM - Architecture

User-level process

KernelLSMModule



ok with you?”


72

LSM - Architecture

User-level process

KernelLSMModule



“ok with you?”

Yes or no


73

LSM - Design Issue: Hooks

• Where to place hooks? Can’t have too many (bloat, performance) Need to unify hook placement to satisfy most modules

• Stub style: empty hooks must go very fast Use function calls to functions that immediately return Faster in modern pipelined CPUs

• Restrictive vs. Authoritative: Should the module be able to override the kernel’s

decision to deny an access? No: would be useful, but excessively confounds

existing Linux code


74

LSM - What we have now

• LSM interface implemented & stable Current Linux patches maintained for both 2.4.*

and 2.5.* kernels

• Paper: USENIX Security 2002 Collaboration with Stephen Smalley (SELinux),

Greg K-H (IBM), and James Morris

• Modules POSIX Capabilities, SELinux, DTE, LIDS OWLSM: portions of the Openwall patches:

security for hardlinks and symlinks


75

LSM - What's next

• Phase 1: Submit to Linux 2.5 kernel Pending on VFS reorganisation by Linux

people

• Phase 2: Consider extended support for Audit More permissive hooks beyond

Capabilities? See if Linus is interested


76

LSM Impact

• Security enhancing technologies currently delivered in a patched Linux kernel

• With LSM, security enhancements can be delivered as kernel modules Security features can be field-upgraded

without replacing the kernel


77

Sardonix.orgSecurity Audit Portal

• Web resource Facilitate habit of

auditing source code

• Rate auditors by: How much code they

audit How many bugs they

miss

• Rate programs by who has audited them

• Effects: Rating auditors

encourages competitive auditing

Rating programs establishes a basis for trusting code

Chronic recurring audits lets us measure development and auditing methods


78

Sardonix.orgSecurity Audit Portal

• Straw man web site up February 5• Positive press coverage:

C|Net News Securityfocus.com (Bugtraq) The Register Midrange Server

• Community interest 280 subscribers to Sardonix mailing list


79

Discussion Phase:How Should Ratings Work?What is a “vulnerability”?:

Turns out to be fuzzy Who gets to decide?

Does # of Past Bugs Predict Future?: and should it affect program rating?

Contact with Package Maintainer: Pushing fixes back can be problematic Give auditor bonus for preparing a patch?


80

Discussion Phase:How Should Ratings Work?

Locking packages: some have asked for it, but there is no apparent advantage, and several problems

Mentoring: Facilitate senior auditors training junior auditors Mentor shares the score gains and losses of the

student Mentor audit’s student’s audits? Mentor and/or student’s score go up just for

mentoring? Zero sum? If not, you can gain free points by creating

two accounts and “mentoring” yourself


81

Discussion Phase:How Should Ratings Work?Minimum score:

Should there be a minimum score for auditors? “E for effort”

Should their be a minimum score an auditor can get from a given auditing task?

Note: minimum scores are an invitation to game the system

Integrate with mentoring?


82

Discussion Phase:How Should Ratings Work?Partial Audits:an audit only for certain

classes of bugs Attractive to novices But is it useful? Allow it, but give it a (much) lower value Tie in with mentoring

Recording Raw Data: Our scoring formulas may not be right Need to record sufficient raw data to get it right

later


83

Discussion Phase:How Should Ratings Work?Popularity:

Should auditing popular packages be worth more?

What about “perceived secure” packages? Solution: tag nominated packages with “who

uses this?”

Size: Should smaller programs get a higher rating? Should larger programs be worth more to the

auditor?


84

Discussion Phase:How Should Ratings Work?Auditing Resources:

Need lots of “howto” docs and support If you have a source analysis tool, and you are

not on my list, please let me know

• The core issue: quantity (auditor-friendly rules) vs. quality (rigorous/demanding rules)

Web Site


85

SummaryImmunix *Guard Tools

StackGuard: buffer overflows, shipping in Immunix 7.0

FormatGuard: printf format bugs, shipping in Immunix 7.0

RaceGuard: temp file races, will ship in Immunix 7+

Immunix Access Control: SubDomain Will ship with Immunix 7+

Community ProjectsSardonix.org source code audit portallsm.immunix.org Linux Security Modules

securing linux the immunix way crispin cowan, ph.d chief scientist, wirex communications, inc

Documents