Compiling to Execution

C.K. Chen 2015.11.18

Compiling Flow

• What happened from compiling the source code to executing

Source Code(.c)

Header Files(.h)

Preprocessingcpp

Preprocessed(.i)

Compilationgcc

Static Library(.a)

Object File(.o)

Linking(ld)

Assembleas

Executablea.out

Assembly(.s)

Preproccessing

• Preproccessing

– Extend and remove #define

– #if, #ifdef, #elif, …..

$gcc -E hello.c -o hello.i

Compilation

• Translate high-level source code into assembly code

$gcc -S hello.i -o hello.s

Assembly

• Assemble assembly into machine code

• After assembly, we will have the ELF format file

$gcc -c hello.s -o hello.o

ELF FILE FORMAT

ELF Format

• Executable file format – Derived from COFF(Common Object File Format)

• Windows : PE (Portable Executable) • Linux: ELF (Executable Linkable Format)

– Dynamic Linking Library (DLL) • Windows (.dll); Linux (.so)

– Static Linking Library • Windows (.lib); Linux (.a)

– Object file • Windows (.obj); Linux (.o) • Like executable file format• Intermediate file between compilation and linking

File Content

• Machine code, data, symbol table, string table

• File header

– Basic file information

• File divided by sections

– Code Section (.code, .text)

– Data Section (.data)

– Special Section (.symtab, .strtab)

File Header

• File header contains following information

– Is executable

– Static Link or Dynamic Link

– Entry address

– Target hardware / OS

– Section Table

File Header Structure

• The structure of ELF header is defined as Elf_Ehdr

• e_ident

– The first 4 byte is‘\x7f’, ‘E’,’L’,’F’

– File signature

• e_type

typedef struct{

unsigned char e_ident[16]; /* ELF identification */Elf64_Half e_type; /* Object file type */Elf64_Half e_machine; /* Machine type */Elf64_Word e_version; /* Object file version */Elf64_Addr e_entry; /* Entry point address */Elf64_Off e_phoff; /* Program header offset */Elf64_Off e_shoff; /* Section header offset */Elf64_Word e_flags; /* Processor-specific flags */Elf64_Half e_ehsize; /* ELF header size */Elf64_Half e_phentsize; /* Size of program header entry */Elf64_Half e_phnum; /* Number of program header entries */Elf64_Half e_shentsize; /* Size of section header entry */Elf64_Half e_shnum; /* Number of section header entries */Elf64_Half e_shstrndx; /* Section name string table index */

} Elf64_Ehdr;

File Header Structure

• e_machine

– 62 for AMD x86-64 architecture

– 243 for RISC-V (HITCON 2015)

• e_entry

• e_shoff

– Follow this membercan find the sectiontable

typedef struct{

unsigned char e_ident[16]; /* ELF identification */Elf64_Half e_type; /* Object file type */Elf64_Half e_machine; /* Machine type */Elf64_Word e_version; /* Object file version */Elf64_Addr e_entry; /* Entry point address */Elf64_Off e_phoff; /* Program header offset */Elf64_Off e_shoff; /* Section header offset */Elf64_Word e_flags; /* Processor-specific flags */Elf64_Half e_ehsize; /* ELF header size */Elf64_Half e_phentsize; /* Size of program header entry */Elf64_Half e_phnum; /* Number of program header entries */Elf64_Half e_shentsize; /* Size of section header entry */Elf64_Half e_shnum; /* Number of section header entries */Elf64_Half e_shstrndx; /* Section name string table index */

} Elf64_Ehdr;

Sections Table

• Sections Table storethe array of section headers

– Each section is in type Elf64_Shdr

– Every element 64 bytes typedef struct{Elf64_Word sh_name; /* Section name */Elf64_Word sh_type; /* Section type */Elf64_Xword sh_flags; /* Section attributes */Elf64_Addr sh_addr; /* Virtual address in memory */Elf64_Off sh_offset; /* Offset in file */Elf64_Xword sh_size; /* Size of section */Elf64_Word sh_link; /* Link to other section */Elf64_Word sh_info; /* Miscellaneous information */Elf64_Xword sh_addralign; /* Address alignment boundary */Elf64_Xword sh_entsize; /* Size of entries, if section has table */} Elf64_Shdr;

e_shoff

.sh.strtab

.sh.strtab + 0x20

Section Table

• Traversal table to find all sections

Code Section

• Code section, most case .text, is used to save the binary code

• objdump –s

– Display the full contents of all sections

• objdump –d

– Display assembler contents of executable sections

Data Section

• There are several sections to store program’s data

– .data → Initialized global variable & static variable

– .rodata → save the constant value in the program

– .bss → save the uninitialized variables

Bss section

• BSS section is used to save uninitialized data or data filled with zero

• This section will not occupy space in the ELF file

– But have space when loading into memory

Other

• .rela.text, .symtab

– Used in symbol resolving

• .strtab

– Save strings for symbol resolution

Summary of ELF format

STATIC LINKING

Static Linking

• Static link is responsible for combining several object files into final executable

$gcc hello.o -o hello.out

Two-pass Linking

• Two-pass Linking

• Space & Address Allocation– Fetch section length, attribute and position

– Collect symbol(define, reference) and put to a global table

• Symbol Resolution & Relocation– Modify relocation entry

Space & Address Allocation

• Define Symbols

– variables

– Functions

• The virtual address is allocated after linking

Symbol Table before Linking Symbol Table after Linking

Symbol Table

typedef struct elf64_sym {Elf64_Word st_name; /* Symbol name, index in string tbl */unsigned char st_info; /* Type and binding attributes */unsigned char st_other; /* No defined meaning, 0 */Elf64_Half st_shndx; /* Associated section index */Elf64_Addr st_value; /* Value of the symbol */Elf64_Xword st_size; /* Associated symbol size */

} Elf64_Sym;

Size: 24 bytes

Fist symbol

.strtab section at 0x000005700x570+0xe(offset) = 0x57e

This symbol is named shared

Symbol Resolution & Relocation

• Resolve symbol’s address in the final executable

– Address of external symbols are unknown before linking

– Before linking, the temporary location is put into object files

– Automatic patch the address with the correct one

Relocation Table

• Relocation table records the address of symbol to

patch

typedef struct elf64_rela {Elf64_Addr r_offset;Elf64_Xword r_info;Elf64_Sxword r_addend;

} Elf64_Rela;

r_offset r_info[1] r_info[2] r_addend

shared 14 00 00 00 00 00 00 00 0a 00 00 00 09 00 00 00 00 00 00 00 00 00 00 0

swap 21 00 00 00 00 00 00 00 02 00 00 00 0a 00 00 00 fc ff ff ff ff ff ff ff

0x0a -> R_X86_64_32/R_AMD64_320X02 -> R_X86_64_PC32/R_AMD64_PC32They have different way to patch address

Relocation Type and Patch Calculation

•

A - The addend value of the relocatable field.S - The value of the symbol P - The section offset or address of the storage unit being relocated, computed using r_offset.GOT - The address of the global offset tablehttps://docs.oracle.com/cd/E23824_01/html/819-0690/chapter6-54839.html

Program Execution

• After static linking, we have the executable file

• The loader is most important to make the program run

– Loading the executable into memory

– Dynamic resolving the symbols

Creation of Process

• Create a independent virtual AS – page directory(Linux)

• Read executable file header, create mapping between virtual AS and executable file – VMA, Virtual Memory Area

• Assign entry address to program register(PC)– Switch between kernel stack

and process stack

– CPU access attribute

Section to Segment Mapping

• Several sections are merge into the segment

– Depend on it’s permission

• Read

• Write

• Execution

Load Executable into Mem

• The program header section contains the information of segments

– Program load into memory in the unit of segments

– Readelf

– Program header tablein ELF file

Disadvantage of Static Linking

• Advantage

– Independent development

– Test individual modules

• Disadvantage

– Waste memory and disk space

• Every program has a copy of runtime library(printf, scanf, strlen, ...)

• Difficulty of updating module – Need to re-link and publish to user when a module is updated

Dynamic Linking

• Delay linking until execution – Load Time Relocation

• Example: – Program1.o, Program2.o, Lib.o– Execute Program1 → Load Program1.o – Program1 uses Lib → Load Lib.o– Execute Program2 → Load Program2.o – Program2 uses Lib → Lib.o has already been loaded into

physical memory – Advantage

• Save space • Easier to update modules

Lazy Binding

• Bind when the first time use the function(relocation, symbol searching)

• More efficient

– In most executions, only small amount of function called

– Not need to resolve all the symbols

Global Offset Table

• Divide into 2 part

– .got

• Store the reference address of global variables

– .got.plt

• Store the reference address of global functions

.GOT.PLT

• The first 3 items are fixed and have special purpose– address of .dynamic section

– link_map

– dl_runtime_resolve

• The following items are functions in the share libraries

.dynamic

.got

.got.plt

.data

Addr of .dynamic

link_map

dl_resolve

print

print

.

.

.call foo@plt

…

Lazy binding

.text

jmp *(bar@GOT)push indexjmp PLT0

foo@plt

.got.plt

printf

bar@plt+6

foo

…

push *(GOT + 4)jmp *(GOT + 8)

PLT0

36

Note: GOT here means .got.plt

.

.

.call foo@plt

…

.text

jmp *(bar@GOT)push indexjmp PLT0

foo@plt

.got.plt

printf

bar@plt+6

foo

…

push *(GOT + 4)jmp *(GOT + 8)

PLT0

Lazy binding

37

.

.

.call foo@plt

…

.text

jmp *(bar@GOT)

push indexjmp PLT0

foo@plt

.got.plt

printf

bar@plt+6

foo

…

push *(GOT + 4)jmp *(GOT + 8)

PLT0

因 bar還沒 call 過所以 bar在 .got.plt 中所存的值會是.plt中的下一行指令位置所以看起來會像沒有 jmp 過

Lazy binding

38

.

.

.call foo@plt

…

.text

jmp *(bar@GOT)push index

jmp PLT0

foo@plt

.got.plt

printf

bar@plt+6

foo

…

push *(GOT + 4)jmp *(GOT + 8)

PLT0

Lazy binding

39

.

.

.call foo@plt

…

.text

jmp *(bar@GOT)push indexjmp PLT0

foo@plt

.got.plt

printf

bar@plt+6

foo

…

push *(GOT + 4)jmp *(GOT + 8)

PLT0

Lazy binding

40

.

.

.call foo@plt

…

.text

jmp *(bar@GOT)push indexjmp PLT0

foo@plt

.got.plt

printf

bar@plt+6

foo

…

push *(GOT + 4)

jmp *(GOT + 8)

PLT0

push link_map

Lazy binding

41

.

.

.call foo@plt

…

.text

jmp *(bar@GOT)push indexjmp PLT0

foo@plt

.got.plt

printf

bar@plt+6

foo

…

push *(GOT + 4)jmp *(GOT + 8)

PLT0

jmp dl_runtime_resolve

dl_runtime_resolve (link_map,index)

Lazy binding

42

.

.

.call foo@plt

…

.text

.got.plt

printf

bar@plt+6

foo

…

..

..call _fix_up

..

..ret 0xc

dl_resolve

Lazy binding

43

.

.

.call foo@plt

…

.text

.got.plt

printf

foo@plt+6

bar

…

..

..call _fix_up

..

..ret 0xc

dl_resolve

foo 找到 bar 在 library 的位置後會填回 .got.plt

Lazy binding

44

.

.

.call foo@plt

…

Lazy binding

.text

.got.plt

printf

foo@plt+6

foo

…

..

..call _fix_up

..

..ret 0xc

dl_resolve

barreturn to bar

45

Program Memory Layout

• Flat memory model

– Default regions:

• stack

• heap

• mapping of executable file

• dynamic libraries

• Stack Frame(Activate Record)

– Return address, arguments

– Temporary variables

– Context

• Frame Pointer(ebp on i386)

• Stack Pointer(esp on i386)

Calling Convention

• Consistency between caller and callee •

• Argument passing order and method

– Stack, Register(eax for return value on i386) •

• Stack maintainer

– Keep consistency before and after function call

– Responsibility of caller or callee

– Name-mangling

– Default calling convention in C language is “cdecl”

Calling Convention Example

LD_Preload

• Ordinarily the dynamic linker loads shared libs in whatever order it needs them

• $LD_PRELOAD is an environment variable containing a colon (or space) separated list of libraries that the dynamic linker loads before any others

LD_Preload

• Preloading a library means that its functions will be used before others of the same name in later libraries

• Allows functions to be overridden/replaced/ intercepted

• Program behaviour can be modified “non-invasively” – ie. no recompile/relink necessary – Especially useful for closed-source programs – And when the modifications don’t belong in the

program or the library

Example

• We want to intercept

Implement Shared Lib

• Write our own function

• Compile into share library

gcc -Wall -fpic -shared -o libmylib.so mylibc.c

Result

System Call

• User processes cannot perform privileged operations themselves

• Must request OS to do so on their behalf by issuing system calls

• System calls elevate privilege of user process

Ltrace

• Tracing system calls in Linux – strace command

• Output is printed for each system call as it is executed, including parameters and return codes

• ptrace() system call is used to implement strace – Also used by debuggers (breakpoint, singlestep, etc)– Maybe anti-debug

– How to solve?

Summary

• ELF file format

• Section

• Static Link

• Dynamic Link and Lazy Binding

• LD_Preload

• strace