Memory Management

Feng QinCSE Dept., The Ohio State University

Group Discussion

• Applications of Information Flow Checking Mechanisms?

Outline

• Background– Linux Kernel Memory Management

• ION Memory Management

Kernel Memory Management

• Physical memory

• It is not easy:– Often kernel cannot sleep– Not easy to handle memory errors

4/25/2012 CSE662: Operating Systems Laboratory 4

Pages• <linux/mm.h>struct page {

page_flags_t flags;atomic_t _count;atomic_t _mapcount;unsigned long private;struct address_space *mapping;pgoff_t index;struct list_head lru;void *virtual;

}

4/25/2012 CSE662: Operating Systems Laboratory 5

Zones

• Reasons:– Some devices are capable of performing DMA to

only certain memory addresses– Some architectures are capable of physically

addressing larger amounts of memory than they can virtually address

• Zones: ( for x86)– ZONE_DMA (< 16 MB)– ZONE_NORMAL (16-896 MB)– ZONE_HIGHMEM (> 896 MB)

4/25/2012 CSE662: Operating Systems Laboratory 6

Getting Memory (I)

• Contiguous physical memory pages:– struct page *alloc_pages (unsigned int

gfp_mask, unsigned int order)– unsigned long __get_free_pages(…)– unsigned long get_zeroed_pages(…)– void free_pages(…)– void __free_pages(…)

4/25/2012 CSE662: Operating Systems Laboratory 7

Getting Memory (II)

• Contiguous physical memory in bytes:– void *kmalloc(size_t size, int flags)– void kfree(const void *ptr)

• Contiguous virtual memory in bytes:– void *vmalloc(unsigned long size)– void vfree(void *addr)– Performance may not be good (TLB threshing)– E.g. when loading modules

4/25/2012 CSE662: Operating Systems Laboratory 8

Getting Memory (III)

• gfp_mask flags:– GFP_ATOMIC (high priority and must not

sleep)– GFP_KERNEL (normal allocation and

might sleep)– GFP_USER (allocate memory for user-space

processes)– GFP_DMA (allocate memory for device

drivers)– ….

4/25/2012 CSE662: Operating Systems Laboratory 9

High Memory Mappings

• Permanent mappings (might sleep)– void *kmap(struct page *page)– void kunmap(struct page *page)

• Temporary mappings (not sleep)– void *kmap_atomic(…)– void *kunmap_atomic(…)

4/25/2012 CSE662: Operating Systems Laboratory 10

Slab Layer (I)

• Cache objects that are frequently allocated and deallocated

4/25/2012 CSE662: Operating Systems Laboratory 11

Cache

Slab

Slab

obj

obj

obj

obj

Slab Layer (II)

struct slab {struct list_head list;unsigned long colouroff;void *s_mem;unsigned int inuse;kmem_bufctl_t free;

};

4/25/2012 CSE662: Operating Systems Laboratory 12

Statically Allocating on the Stack

• Per-process kernel stack

• Keep stack usage to a minimum– The size is really small ( 4 KB option in 2.6

kernels)

4/25/2012 CSE662: Operating Systems Laboratory 13

Per-CPU Allocations

• In modern SMP architectures• Benefits:

– Reduce locking– Avoid cache invalidation

• Usage example:unsigned long my_percpu[NR_CPUS];cpu = get_cpu();my_percpu[cpu];put_cpu();

4/25/2012 CSE662: Operating Systems Laboratory 14

ION Memory Allocator

• Manage one or more memory pools, i.e., ION heaps– Each device can have a different set of ION

heaps

• Combat fragmentation

• Serve special hardware needs:– E.g., GPU, display controller, cameras

Providing an ION Heapstruct ion_heap_ops {

int (*allocate) (struct ion_heap *heap,struct ion_buffer *buffer, unsigned long len,unsigned long align, unsigned long flags);

void (*free) (struct ion_buffer *buffer);int (*phys) (struct ion_heap *heap, struct ion_buffer *buffer,

ion_phys_addr_t *addr, size_t *len);struct scatterlist *(*map_dma) (struct ion_heap *heap,

struct ion_buffer *buffer);void (*unmap_dma) (struct ion_heap *heap,

struct ion_buffer *buffer);void * (*map_kernel) (struct ion_heap *heap,

struct ion_buffer *buffer);void (*unmap_kernel) (struct ion_heap *heap,

struct ion_buffer *buffer);int (*map_user) (struct ion_heap *heap, struct ion_buffer *buffer,

struct vm_area_struct *vma);};

Default ION driver

• ION_HEAP_TYPE_SYSTEM: memory allocated via vmalloc_user().

• ION_HEAP_TYPE_SYSTEM_CONTIG: memory allocated via kzalloc.

• ION_HEAP_TYPE_CARVEOUT: carveout memory is physically contiguous and set aside at boot.

Using ION From User Space

• Who uses ION?– User space device access libraries

• Why needs ION?– Allocate large contiguous media buffers– Sharing between two processes or processes

and kernel– E.g., still camera library, when buffer is filled

with data, it will pass the buffer to the kernel to be processed by JPEG encoder hardware

Allocate Buffer from ION

• Require Access permission to device /dev/ion• Retrieve a file descriptor (one client per

user process)client_fd = open (“/dev/ion”, O_RDONLY)

• Fill the following data structurestruct ion_allocation_data {

size_t len;size_t align;unsigned int flags; // specify which ION heap the buffer is from

(i.e., ION_HEAP_TYPE_CARVEOUT | ION_HEAP_TYPE_CONTIG)struct ion_handle *handle; // output parameter, not CPU-accessible buffer pointer

}

• Use ioctl system call to interact with IONint ioctl(int client_fd, ION_IOC_ALLOC, struct ion_allocation_data *allocation_data)

ION Buffer Sharing in User Space

• Convert ion_handle to a file descriptorint ioctl(int client_fd, ION_IOC_SHARE, struct ion_fd_data *fd_data);struct ion_fd_data {

struct ion_handle *handle;int fd;

}

• BINDER IPC mechanism can be used to send fd to another process for sharing

• To obtain the shared buffer, the second process needs to retrieve the file descriptor via

client_fd = open (“/dev/ion”, O_RDONLY)

Free ION Buffer

• Go through ioctl againint ioctl(int client_fd, ION_IOC_FREE, struct ion_handle_data *handle_data);

struct ion_handle_data {struct ion_handle *handle;

}

ION Buffers in Kernel

• ION supports multiple clients in the kernel– One for each driver that uses ION

• A kernel driver calls the following function to obtain an ION client handle

struct ion_client *ion_client_create(struct ion_device *dev, // /dev/ionunsigned int heap_mask, // selects ION heapsconst char *debug_name

)

Sharing ION Buffer between User Space and Kernel (I)

• User process– Allocates the ION buffer– Obtains a file descriptor using

ION_IOC_SHARE– Pass the file descriptor to a kernel driver

Sharing ION Buffer between User Space and Kernel (II)

• Kernel driver– Converts the file descriptor to an ion_handle

struct ion_handle *ion_import_fd(struct ion_client *client, int fd_from_user);

// based on the physical address of the buffer

– Some hardware blocks need physically-contiguous buffers with physical addresses

int ion_phys(struct ion_client *client, struct ion_handle*handle, ion_phys_addr_t *addr, size_t *len)

Source Code Cross References

• Linux kernel– http://lxr.linux.no/

• Android– http://androidxref.com/source/xref/