memory allocation & heap
TRANSCRIPT
![Page 1: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/1.jpg)
Copyright ©: University of Illinois CS 241 Staff 1
Memory Allocation & Heap
![Page 2: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/2.jpg)
Memory allocation within a process
n Stack data structure ¡ Function calls follow LIFO semantics ¡ So we can use a stack data structure to
represent the process’s stack – no fragmentation!
n Heap: malloc, free ¡ This is a much harder problem ¡ Need to deal with fragmentation
Copyright ©: University of Illinois CS 241 Staff 2
MP2!
![Page 3: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/3.jpg)
malloc Constraints
n Applications ¡ Can issue arbitrary sequence of malloc
and free requests ¡ free request must be to a malloc’d
block
Copyright ©: University of Illinois CS 241 Staff 3
![Page 4: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/4.jpg)
malloc Constraints
n Allocators ¡ Can’t control number or size of allocated blocks ¡ Must respond immediately to malloc requests
n i.e., can’t reorder or buffer requests ¡ Must allocate blocks from free memory ¡ Must align blocks so they satisfy all requirements
n 8 byte alignment for libc malloc on Linux boxes ¡ Can manipulate and modify only free memory ¡ Can’t move the allocated blocks once they are
malloc’d n i.e., compaction is not allowed (why not?)
Copyright ©: University of Illinois CS 241 Staff 4
![Page 5: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/5.jpg)
Goal 1: Speed
n Allocate fast! ¡ Minimize overhead for both allocation and
deallocation n Maximize throughput
¡ Number of completed malloc or free requests per unit time
¡ Example n 5,000 malloc calls and 5,000 free calls in 10
seconds
Copyright ©: University of Illinois CS 241 Staff 5
![Page 6: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/6.jpg)
Goal 1: Speed
n Allocate fast! ¡ Minimize overhead for both allocation and
deallocation n Maximize throughput
¡ Number of completed malloc or free requests per unit time
¡ Example n 5,000 malloc calls and 5,000 free calls in 10
seconds n Throughput is 1,000 operations/second
Copyright ©: University of Illinois CS 241 Staff 6
![Page 7: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/7.jpg)
Goal 1: Speed
n BUT ¡ A fast allocator may not be efficient in
terms of memory utilization ¡ Faster allocators tend to be “sloppier”
n Example: don’t look through every free block to find the perfect fit
Copyright ©: University of Illinois CS 241 Staff 7
![Page 8: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/8.jpg)
Goal 2: Memory Utilization
n Allocators usually waste some memory ¡ Extra metadata or
internal structures used by the allocator itself n Example: keeping track
of where free memory is located
¡ Chunks of heap memory that are unallocated (fragments)
Copyright ©: University of Illinois CS 241 Staff 8
![Page 9: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/9.jpg)
Goal 2: Memory Utilization
n Memory utilization = ¡ The total amount of
memory allocated to the application divided by the total heap size
n Ideal ¡ utilization = 100%
n In practice ¡ try to get close to 100%
Copyright ©: University of Illinois CS 241 Staff 9
![Page 10: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/10.jpg)
Fragmentation
n Poor memory utilization caused by unallocatable memory ¡ internal fragmentation ¡ external fragmentation
n malloc fragmentation ¡ When allocating memory to applications
Copyright ©: University of Illinois CS 241 Staff 10
![Page 11: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/11.jpg)
Internal fragmentation
n Payload is smaller than block size
n Caused by ¡ Overhead of maintaining heap data structures ¡ Padding for alignment purposes ¡ Explicit policy decisions
(e.g., to return a big block to satisfy a small request)
Copyright ©: University of Illinois CS 241 Staff 11
Payload Internal fragmentation
Block
Internal fragmentation
![Page 12: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/12.jpg)
Experiment
n Does libc’s malloc have internal fragmentation? How much?
n How would you test this?
Copyright ©: University of Illinois CS 241 Staff 12
Run Example
![Page 13: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/13.jpg)
fragtest
#include <stdio.h> #include <stdlib.h> int main(int argc, char** argv) { char* a = (char*) malloc(1); char* b = (char*) malloc(1); char* c = (char*) malloc(100); char* d = (char*) malloc(100);
printf("a = %p\n b = %p\n c = %p\n d = %p\n", a,b,c,d);
}
Copyright ©: University of Illinois CS 241 Staff 13
What output would you expect?
![Page 14: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/14.jpg)
fragtest - Output
Copyright ©: University of Illinois CS 241 Staff 14
a = malloc(1); b = malloc(1); c = malloc(100); d = malloc(100);
a = 0xdb64010 b = 0xdb64030 c = 0xdb64050 d = 0xdb640c0
0x20 = 32 ≠ 1 0x20 = 32 ≠ 1
0x70 = 112 ≠ 100
![Page 15: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/15.jpg)
External Fragmentation
n There is enough aggregate heap memory, but no single free block is large enough
Copyright ©: University of Illinois CS 241 Staff 15
p1 = malloc(4)
p2 = malloc(5)
p3 = malloc(6)
free(p2)
p4 = malloc(6)
Depends on the pattern of future requests Difficult to plan for
Oops! (what would happen now?)
![Page 16: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/16.jpg)
Conflicting performance goals
n Throughput vs. Utilization ¡ Difficult to achieve simultaneously ¡ Faster allocators tend to be “sloppier”
with memory usage ¡ Space-efficient allocators may not be
very fast n Tracking fragments to avoid waste generally
results in longer allocation times
Copyright ©: University of Illinois CS 241 Staff 16
![Page 17: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/17.jpg)
Implementation issues you need to solve!
n How do I know how much memory to free just given a pointer?
Copyright ©: University of Illinois CS 241 Staff 17
Keep the length of the block in the header preceding the block Requires an extra word for every allocated block
![Page 18: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/18.jpg)
Keeping Track of Free Blocks
n One of the biggest jobs of an allocator is knowing where the free memory is
n The allocator's approach to this problem affects: ¡ Throughput – time to complete a malloc() or free() ¡ Space utilization – amount of extra metadata used to track
location of free memory
n There are many approaches to free space management
Copyright ©: University of Illinois CS 241 Staff 18
![Page 19: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/19.jpg)
Implicit Free Lists
n For each block we need both size and allocation status ¡ Could store this information in
two words: wasteful!
n Standard trick ¡ If blocks are aligned, low-order
address bits are always 0 ¡ Why store an always-0 bit?
Use it as allocated/free flag! ¡ When reading size word, must
mask out this bit
Copyright ©: University of Illinois CS 241 Staff 19
Size
1 word
Payload
a = 1: Allocated block a = 0: Free block Size: block size Payload: application data (allocated blocks only)
a
Optional padding
![Page 20: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/20.jpg)
Implicit Free Lists
n No explicit structure tracking location of free/allocated blocks. ¡ Rather, the size word (and allocated bit) in each
block form an implicit “block list”
Copyright ©: University of Illinois CS 241 Staff 20
![Page 21: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/21.jpg)
Implicit Free Lists: Free Blocks
n How do we find a free block in the heap? ¡ Start scanning from the beginning of the heap. ¡ Traverse each block until (a) we find a free block
and (b) the block is large enough to handle the request.
¡ This is called the first fit strategy n Could also use best fit, etc
Copyright ©: University of Illinois CS 241 Staff 21
![Page 22: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/22.jpg)
Implicit Free Lists: Allocating Blocks
n What if the allocated space is smaller than free space?
n Split free blocks
Copyright ©: University of Illinois CS 241 Staff 22
p
![Page 23: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/23.jpg)
Implicit Free Lists: Freeing a Block
n How do you free a block? n Simplest implementation:
¡ Only need to clear allocated flag n Problem?
Copyright ©: University of Illinois CS 241 Staff 23
![Page 24: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/24.jpg)
Implicit Free Lists: Freeing a Block
n Only need to clear allocated flag n Problem?
¡ False fragmentation
Copyright ©: University of Illinois CS 241 Staff 24
free(p)
16 8 16 8
p
16
16 16 8 16 8
malloc(20) Oops! There’s enough free space but allocator won’t find it!
![Page 25: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/25.jpg)
Implicit Free Lists: Coalescing Blocks
n Join (coalesce) with next and previous block if they are free ¡ Coalescing with next block
Copyright ©: University of Illinois CS 241 Staff 25
16 8 16 8
free(p) p
16 16 8
16
24
But how do we coalesce with previous block?
![Page 26: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/26.jpg)
Implicit Free Lists: Bidirectional Coalescing
n Boundary tags [Knuth73] ¡ Replicate size/allocated word at tail end of all blocks ¡ Lets us traverse list backwards, but needs extra space ¡ General technique: doubly linked list
Copyright ©: University of Illinois CS 241 Staff 26
![Page 27: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/27.jpg)
Implicit Free Lists: Bidirectional Coalescing
n Boundary tags [Knuth73]
Copyright ©: University of Illinois CS 241 Staff 27
size
1 word
Format of allocated and free blocks
payload and padding
a = 1: Allocated block a = 0: Free block Size: block size payload: application data (allocated blocks only)
a
size a Boundary tag (footer)
16 16 16 16 24 16 24 16
Header
![Page 28: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/28.jpg)
Implicit Free Lists: Summary
n Implementation ¡ Very simple
n Allocation ¡ linear-time worst case
n Free ¡ Constant-time worst case—even with coalescing
n Memory usage ¡ Will depend on placement policy ¡ First fit, best fit,…
Copyright ©: University of Illinois CS 241 Staff 28
![Page 29: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/29.jpg)
Implicit Free Lists: Summary
n Not used in practice for malloc/free ¡ linear-time allocation is actually slow! ¡ But used in some special-purpose applications
n However, concepts of splitting and boundary tag coalescing are general to all allocators
Copyright ©: University of Illinois CS 241 Staff 29
![Page 30: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/30.jpg)
Alternative Approaches
n Explicit Free List n Segregated Free Lists
¡ Buddy allocators
Copyright ©: University of Illinois CS 241 Staff 30
![Page 31: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/31.jpg)
Explicit Free List
n Linked list among free blocks n Use data space for link pointers
¡ Typically doubly linked ¡ Still need boundary tags for coalescing ¡ Links aren’t necessarily in address order!
Copyright ©: University of Illinois CS 241 Staff 31
16 16 16 16 24 24 16 16 16 16
Forward link
Back link
A B
C
![Page 32: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/32.jpg)
Explicit Free List: Inserting Free Blocks
Copyright ©: University of Illinois CS 241 Staff 32
n Where should you put the newly freed block? ¡ LIFO (last-in-first-out) policy
n Insert freed block at beginning of free list n Pro
¡ Simple, and constant-time
n Con ¡ Studies suggest fragmentation is high
![Page 33: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/33.jpg)
Explicit Free List: Inserting Free Blocks
Copyright ©: University of Illinois CS 241 Staff 33
n Where should you put the newly freed block? ¡ Address-ordered policy
n Insert so list is always in address order ¡ i.e. addr(pred) < addr(curr) <
addr(succ)
n Con ¡ Requires search (using boundary tags); slow!
n Pro ¡ studies suggest fragmentation is better than LIFO
![Page 34: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/34.jpg)
Segregated Free Lists
n Each size class has its own collection of blocks
Copyright ©: University of Illinois CS 241 Staff 34
4-8
12
16
20-32
36-64
![Page 35: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/35.jpg)
Segregated Free Lists
n Each size class has its own collection of blocks
Copyright ©: University of Illinois CS 241 Staff 35
4-8
12
16
20-32
36-64
n Often separate size class for every small size (8, 12, 16, …)
n For larger, typically have size class for each power of 2
What is the point of having separate lists?
![Page 36: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/36.jpg)
Buddy Allocators
n Special case of segregated free lists ¡ Limit allocations to power-of-two sizes ¡ Can only coalesce with "buddy“
n Who is other half of next-higher power of two
n Clever use of low address bits to find buddies
n Problem ¡ Large powers of two result in large internal
fragmentation (e.g., what if you want to allocate 65537 bytes?)
Copyright ©: University of Illinois CS 241 Staff 36
![Page 37: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/37.jpg)
Buddy System
n Approach ¡ Minimum allocation size = smallest frame ¡ Maintain freelist for each possible frame size
n Power of 2 frame sizes from min to max ¡ Initially one block = entire buffer ¡ If two neighboring frames (“buddies”) are free, combine
them and add to next larger freelist
Copyright ©: University of Illinois CS 241 Staff 37
![Page 38: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/38.jpg)
Buddy System Example
Copyright ©: University of Illinois CS 241 Staff 38
128 Free
![Page 39: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/39.jpg)
Buddy System Example
Copyright ©: University of Illinois CS 241 Staff 39
128 Free
Request A: 16
64 Free 64 Free
32 Free 32 Free
16 A 16 Free 32 Free
64 Free
64 Free
![Page 40: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/40.jpg)
Buddy System Example
Copyright ©: University of Illinois CS 241 Staff 40
128 Free
Request B: 32
16 A 16 Free 32 Free 64 Free 32 B
![Page 41: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/41.jpg)
Buddy System Example
Copyright ©: University of Illinois CS 241 Staff 41
128 Free
Request C: 8
16 A 16 Free 32 B 64 Free
16 A 8 C 32 B 64 Free 8
![Page 42: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/42.jpg)
Buddy System Example
Copyright ©: University of Illinois CS 241 Staff 42
Request A frees
16 Free 8 C 32 B 64 Free 8
![Page 43: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/43.jpg)
Buddy System Example
n Advantage ¡ Minimizes external fragmentation
n Disadvantage ¡ Internal fragmentation when not 2^n request
Copyright ©: University of Illinois CS 241 Staff 43
Request C frees
16 Free 8 32 B 64 Free 8
16 Free 32 B 64 Free 16 Free
32 B 64 Free 32 Free
![Page 44: Memory Allocation & Heap](https://reader030.vdocuments.net/reader030/viewer/2022012211/61df14001ad158336f5b99b6/html5/thumbnails/44.jpg)
So what should I do for MP2? n Designs sketched here are all reasonable n But, there are many other possible designs n So, implement anything you want! n Suggestion: è Before you start coding, REALLY spend time thinking about 1) your mem. manag. design; 2) its correctness; 3) the assumptions your code relies on; 4) performance trade-offs
Copyright ©: University of Illinois CS 241 Staff 44 Happy coding!