computer systems principles dynamic memory management

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTS ASSACHUSETTS AAMHERST • MHERST • Department of Computer Science Department of Computer Science

Computer Systems PrinciplesDynamic Memory Management

Emery Berger and Mark CornerUniversity of Massachusetts

Amherst

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTS ASSACHUSETTS AAMHERST • MHERST • Department of Computer Science Department of Computer Science 2

Dynamic Memory Management How the heap manager is implemented

– malloc, free– new, delete

Memory Management Programs ask memory manager

– to allocate/free objects (or multiple pages) Memory manager asks OS

– to allocate/free pages (or multiple pages)

Operating System

User Program

Allocator(java, libc)

Objects (new, malloc)

Pages (mmap,brk)

Memory Management Ideal memory manager:

– Fast• Raw time, asymptotic runtime, locality

– Memory efficient• Low fragmentation

With multicore & multiprocessors:– Scalable to multiple processors

New issues:– Secure from attack– Reliable in face of errors

Memory Manager Functions Not just malloc/free

– realloc• Change size of object, copying old contents

– ptr = realloc (ptr, 10);• But: realloc(ptr, 0) = ?• How about: realloc (NULL, 16) ?

Other fun– calloc– memalign

Needs ability to locate size & object start

Fragmentation Intuitively, fragmentation stems from

“breaking” up heap into unusable spaces– More fragmentation = worse utilization

External fragmentation– Wasted space outside allocated objects

Internal fragmentation– Wasted space inside an object

Classical Algorithms First-fit

– find first chunk of desired size

Classical Algorithms Best-fit

– find chunk that fits best• Minimizes wasted space

Classical Algorithms Worst-fit

– find chunk that fits worst– name is a misnomer!– keeps large holes around

Reclaim space: coalesce free adjacent objects into one big object

Quick Activity

Program asks for: 300,25,25,100– First-fit and best-fit allocations go where?– Which ones cannot be fulfilled?

What about: 110,54,25,70,50?

Implementation Techniques Freelists

– Linked lists of objects in same size class• Range of object sizes

First-fit, best-fit in this context?– Which is faster?

Implementation Techniques Segregated size classes

– Use free lists, but never coalesce or split Choice of size classes

– Exact– Powers-of-two

Implementation Techniques Big Bag of Pages (BiBOP)

– Page or pages (multiples of 4K)– Usually segregated size classes

Header contains metadata– Locate with bitmasking

Limits external fragmentation Can be very fast

Secret Sauce for project– Use free objects to track free objects

Runtime Analysis Key components

– Cost of malloc (best, worst, average)– Cost of free– Cost of size lookup (for realloc & free)

Space Bounds Fragmentation worst-case for “optimal”:

O(log M/m)– M = largest object size– m = smallest object size

Best-fit = O(M * m) !

Performance Issues Goal: perform well for typical programs

– Considerations:• Internal fragmentation• External fragmentation• Headers (metadata)• Scalability (later)• Reliability, too

“Canned” allocator often seen as slow

“Use custom allocators”

Custom Memory Allocation Programmers replace

new/delete Reduce runtime

– Often Expand functionality

– Sometimes Reduce space

– rarely

Very common Apache, gcc, lcc, STL,

database servers…– Language-level

support in C++– Widely recommended

Drawbacks of Custom Allocators Avoiding system allocator:

– More code to maintain & debug– Can’t use memory debuggers– Not modular or robust:

• Mix memory from customand general-purpose allocators → crash!

Increased burden on programmers

Class1free list

a = new Class1;b = new Class1;c = new Class1;delete a;delete b;delete c;a = new Class1;b = new Class1;c = new Class1;

+ Fast+ Linked list

operations + Simple

+ Identical semantics

+ C++ language support

- Possibly space-inefficient

(1) Per-Class Allocators Recycle freed objects from a free list

char[MEMORY_LIMIT]

a = xalloc(8);b = xalloc(16);c = xalloc(8);xfree(b);xfree(c);d = xalloc(8);

a b cd

end_of_arrayend_of_arrayend_of_arrayend_of_arrayend_of_arrayend_of_array

+ Fast+ Pointer-bumping

allocation

- Brittle- Fixed memory size- Requires stack-like

lifetimes

(II) Custom Patterns Tailor-made to fit allocation patterns

– Example: 197.parser (natural language parser)

+ Fast+ Pointer-bumping allocation+ Deletion of chunks

+ Convenient+ One call frees all memory

regionmalloc(r, sz)regiondelete(r)

Separate areas, deletion only en masseregioncreate(r) r

- Risky- Dangling

references- Too much space

Increasingly popular custom allocator

(III) Regions

Custom Allocators Are Faster…Runtime - Custom Allocator Benchmarks

197.parserboxed-simc-breeze 175.vpr 176.gcc apachelcc

mudlle

Normalized Runtime

Custom Win32

non-regions regions

As good as and sometimes much faster than Win32

Not So Fast…Runtime - Custom Allocator Benchmarks

197.parserboxed-simc-breeze 175.vpr 176.gcc apache

lccmudlle

Normalized Runtime

Custom Win32 DLmalloc

non-regions regions

DLmalloc (Linux): as fast or faster for most benchmarks

Are custom allocators a win? Generally not worth the trouble

– Just use good general-purpose allocator• Alternative: reaps (hybrid of regions & heaps)

However…– Sometimes worth it for specialized apps

• Especially pool allocation, as in Apache

Problems w/Unsafe Languages C, C++: pervasive apps, but langs. unsafe Numerous opportunities for security

vulnerabilities, errors– Double free– Invalid free– Uninitialized reads– Dangling pointers– Buffer overflows (stack & heap)

Can memory allocator help?

Soundness for Erroneous Progs Normally: memory errors lead to crashes,

but…consider infinite-heap allocator:– All news fresh; ignore delete

• No dangling pointers, invalid frees,double frees

– Every object infinitely large• No buffer overflows, data overwrites

Transparent to correct program “Erroneous” programs sound

Probabilistic Memory Safety

Fully-randomized M-heap– Approximates with M, e.g., M=2– Increases odds of benign errors– Probabilistic memory safety

• i.e., P(no error) n– Errors independent across heaps

• E(users with no error) n * |users|

DieHard Key ideas:

– Isolate heap metadata– Randomize Allocation– Trade space for

robustness– Replication (optional)

Key influence in design of Windows 7’s Fault-Tolerant Heap

obj obj objobj

Implementation Issues Conventional, freelist-based heaps

– Hard to randomize, protect from errors• Double frees, heap corruption

What about bitmaps? (one bit per word)– Catastrophic fragmentation!

• Each small object likely to occupy one page

00000001 1010

metadata

Randomized Heap Layout

Bitmap-based, segregated size classes– Bit represents one object of given size

• i.e., one bit = 2i+3 bytes, etc.– Prevents fragmentation

00000001 1010

metadata

Randomized Allocation

malloc(8):– compute size class = ceil(log sz) – 3– randomly probe bitmap for zero-bit (free)

Fast: runtime O(1)– M=2 means E[# of probes] = 2

00010001 1010

metadata

Randomized Allocation

malloc(8):– compute size class = ceil(log sz) – 3– randomly probe bitmap for zero-bit (free)

Fast: runtime O(1)– M=2 means E[# of probes] = 2

00010001 1010

metadata

Randomized Deallocation

free(ptr):– Ensure object valid – aligned to right address– Ensure allocated – bit set– Resets bit

Prevents invalid frees, double frees

00010001 1010

metadata

00000001 1010

metadata

2 34 5 3 1 6

object size = 2i+4object size = 2i+3

11 6 3 2 5 4 …

My Mozilla: “malignant” overflow

Your Mozilla: “benign” overflow

Randomized Heaps & Reliability

Objects randomly spread across heap Different run = different heap

– Errors across heaps independent

Increasing Reliability Space Shuttle

– 3 copies of everything(hw & sw)

– Votes on every action

Failure:majority rules

broadcast vote

input output

execute replicas(separate processes)

replica3seed3

replica1seed1

replica2seed2

DieHard - Replication

Replication-based fault-tolerance– Requires randomization! Makes errors independent

DieHard Results Empirical results

– Runtime overhead– Error avoidance

• Injected faults & actual applications

Analytical results (if time, pictures!)– Buffer overflows– Uninitialized reads– Dangling pointer errors (the best)

Analytical Results: Buffer Overflows

Model overflow as random write of live data Heap half full (max occupancy)

Model overflow: random write of live data Heap half full (max occupancy)

Analytical Results: Overflows Replicas: Increase odds of avoiding overflow in

at least one replica

Analytical Results: Overflows Replicas: Increase odds of avoiding overflow in

at least one replica

Analytical Results: Overflows Replicas: Increase odds of avoiding overflow in at least one replica

P(Overflow in all replicas) = (½)3 = 1/8 P(No overflow in > 1 replica) = 1-(½)3 = 7/8

Empirical Results: Runtime

F = free space H = heap size N = # objects

worth of overflow

k = replicas

Overflow one object

Error Avoidance Injected faults:

– Dangling pointers (@50%, 10 allocations)• glibc: crashes; DieHard: 9/10 correct

– Overflows (@1%, 4 bytes over) –• glibc: crashes 9/10, inf loop; DieHard: 10/10 correct

Real faults:– Avoids Squid web cache overflow

• Crashes Boehm-Demers-Weiser(BDW) Collector & glibc– Avoids dangling pointer error in Mozilla

• DoS in glibc & Windows

The End

Backup Slides

Lea Allocator (Dlmalloc 2.7.0) Mature general-purpose allocator Optimized for common allocation patterns

– Per-size quicklists ≈ per-class allocation Deferred coalescing

– combining adjacent free objects– Highly-optimized fastpath

Space-efficient

computer systems principles dynamic memory management

Documents

dynamic memory management

4 dynamic memory allocation

1. dynamic memory allocation. dynamic memory allocation...

dynamic memory. 2 overview virtual memory what is dynamic...

2.3 dynamic memory allocation - amrita vishwa … ·...

dynamic memory allocation

optimizing dynamic memory management

dynamic memory allocation - university of north...

dynamic memory...

dynamic memory allocation(ppc)

dynamic memory allocation: basic conceptsdynamic memory...

17.dynamic memory allocation

principles of memory

memory management arrays dynamic memory dynamic arrays...

10: dynamic memory allocation process memory image dynamic

dynamic software transactional memory

dynamic objects. comp104 dynamic objects / slide 2 memory...

virtual & dynamic memory management

dynamic memory allocation ii

dynamic memory file input/output€¦ · dynamic memory:...