Slide 1

Integer Overflows James Walden Northern Kentucky University

Slide 2

CSC 666: Secure Software Engineering Topics 1.Computer Integers 2.Integers in C and Java 3.Overflow Examples 4.Checking for Overflows

Slide 3

CSC 666: Secure Software Engineering Comair Integer Overflow December 25, 2004 Flight crew scheduling software stopped. Cancelled all 1100 flights that day. What happened? Winter weather led to many crew changes. Number of changes > 32,767.

Slide 4

CSC 666: Secure Software Engineering Integers Computer integers are not the same set of numbers as mathematical integers. Finite set, not infinite. What happens when integer calculations result in a number outside that set?

Slide 5

CSC 666: Secure Software Engineering Unsigned Integers 000 001 010 011 100 101 110 111 0 1 2 3 4 5 6 7

Slide 6

CSC 666: Secure Software Engineering Twos Complement Twos complement = Ones complement + 1. Sign is represented by most significant bit. Range: -2 n-1..2 n-1 -1, only one representation of 0. +75 0 1 0 0 1 0 1 1 Comp 1 0 1 1 0 1 0 0 +1 0 0 0 0 0 0 0 1 -75 1 0 1 1 0 1 0 1

Slide 7

CSC 666: Secure Software Engineering Twos Complement 000 001 010 011 100 101 110 111 0 1 2 3 -4 -3 -2

Slide 8

CSC 666: Secure Software Engineering C Integers TypeBitsMin ValueMax Value signed char8-128127 unsigned char80255 short16-3276832767 unsigned short16065535 int32-2,147,483,6482,147,483,647 unsigned int3204,294,967,295 long32-2,147,483,6482,147,483,647 unsigned long3204,294,967,295 long 64-9.223 x 10 18 9.223 x 10 18 unsigned long long6401.844 x 10 19

Slide 9

CSC 666: Secure Software Engineering Java Integers TypeBitsMin ValueMax Value byte8-128127 short16-3276832767 char16065535 int32-2,147,483,6482,147,483,647 long64-9.223 x 10 18 9.223 x 10 18

Slide 10

CSC 666: Secure Software Engineering Java Factorial Program public static void main(String args[]) { long product = 1; for(int i = 1; i

CSC 666: Secure Software Engineering Unsigned Addition An unsigned addition unsigned int x, y, sum; sum = x + y; Precondition if( x > UINT_MAX y) /* error */ Postcondition if( (x >= 0 && sum y) ) /* error */

Slide 22

CSC 666: Secure Software Engineering Signed Addition Preconditions xyPrecondition Positive if (x > INT_MAX y) /* error */ PositiveNegativeNone NegativePositiveNone Negative if (x < INT_MIN y) /* error */

Slide 23

CSC 666: Secure Software Engineering Integer Multiplication Overflow CESA-2004-001: libpng info_ptr->row_pointers = (png_bytepp)png_malloc(png_ptr, info_ptr- >height * sizeof(png_bytep)); If height > INT_MAX / sizeof(png_bytep) Size of new buffer will be a small integer. User data in image file can be used to generate a buffer overflow attack.

Slide 24

CSC 666: Secure Software Engineering Widening Conversions A conversion from a type with a smaller range of values to type with a larger range of values. Examples: byte -> short, short -> long Sign extension Propagates signed bit from source type to all unused bits in destination type. Magnitude and sign are preserved.

Slide 25

CSC 666: Secure Software Engineering Widening Conversion Example Source type: byte Value: -7 1 1 1 1 1 0 0 1 Destination type: short Value: -7 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 1

Slide 26

CSC 666: Secure Software Engineering Narrowing Conversions Conversions from a wider type to a narrower type. Examples: long -> byte, int -> short Truncation Bits from source type that dont fit into narrower destination type are discarded. Magnitude and sign may change.

Slide 27

CSC 666: Secure Software Engineering Narrowing Conversion Example Source Type: short Value: 257 0 0 0 0 0 0 0 1 Destination Type: byte Value:1 0 0 0 0 0 0 0 1

Slide 28

CSC 666: Secure Software Engineering Sign Extension Vulnerability CERT CA-1996-22: bash yy_string_get() reads user data as chars. Each char converted to an int when parsed. A char value of 255 sign extended to int -1. Integer -1 means command separator. Example exploit bash -c 'ls\377who'

Slide 29

CSC 666: Secure Software Engineering Range Checking Check that integer ranges are valid. Be more specific than INT_MIN, INT_MAX. Liquid water temperatures range 0..100. Use type system to check. Some languages allow integer ranges. Create abstract data types in languages that dont provide integer range types.

Slide 30

CSC 666: Secure Software Engineering Proposal: Ranged Integers in C All integer types can be ranged. Static: range determined at compile time. Dynamic: range determined at run time. Semantics Saturation: values beyond range = max. Wrap: values wrap to bottom of range. Examples Saturation: int 0|..|100 temperature = 0 Wrap: long min max circular;

Slide 31

CSC 666: Secure Software Engineering Compiler Checks Microsoft VS 2005 CL Runtime integer error checks: /RTCc Use highest warning level /W4 Check for #pragma warning(disable, C####) GCC Runtime integer error checks: -ftrapv Use integer-relevant warnings: -Wconversion Wsign-compare Check for #pragma GCC diagnostic ignored option

Slide 32

CSC 666: Secure Software Engineering Secure Integer Libraries IntegerLib Designed for C, but usable in C++. Available from CERT. IntSafe C library written by Michael Howard. Uses architecture specific inline assembly. SafeInt C++ template class from David LeBlanc.

Slide 33

CSC 666: Secure Software Engineering SafeInt C++ Class int main(int argc, char *const *argv) { try { SafeInt s1(strlen(argv[1])); SafeInt s2(strlen(argv[2])); char *buff = (char *) malloc(s1 + s2 + 1); strcpy(buff, argv[1]); strcat(buff, argv[2]); } catch(SafeIntException err) { abort(); }

Slide 34

CSC 666: Secure Software Engineering When to use Secure Int Libraries? Use Secure Integer libraries when Integers come from untrusted sources. Dont use Secure Integer libraries when Integers not influenced by external sources. Tight loops: check int values before loop.

Slide 35

CSC 666: Secure Software Engineering Integer Overflow: Key Points Integer arithmetic. Twos complement format signed ints. Know your languages integer conversions. Impact of integer overflows Can be used to defeat bounds checks. Influence important data, like vote counts. Mitigating integer overflows. Precondition or postcondition testing. Use safe integer libraries where possible.

Slide 36

References 1.Brian Chess and Jacob West, Secure Programming with Static Analysis, Addison-Wesley, 2007. 2.Jeff Gennari et. al., Ranged Integers for the C Programming Language. CMU/SEI-2007-TN-027, 2007. 3.Michael Howard and David LeBlanc, Writing Secure Code, 2 nd edition, Microsoft Press, 2003. 4.Robert C. Seacord, Secure Coding in C and C++, Addison- Wesley, 2006. 5.Robert C. Seacord, CERT Secure Coding Standards: Integers, https://www.securecoding.cert.org/confluence/display/seccode/04. +Integers+(INT), 2009. https://www.securecoding.cert.org/confluence/display/seccode/04. +Integers+(INT) 6.John Viega and Gary McGraw, Building Secure Software, Addison-Wesley, 2002. 7.David Wheeler, Secure Programming for UNIX and Linux HOWTO, http://www.dwheeler.com/secure-programs/Secure- Programs-HOWTO/index.html, 2003.http://www.dwheeler.com/secure-programs/Secure- Programs-HOWTO/index.html