python objects
DESCRIPTION
Objects in memory, Objects at python level, Objects in python virtual machineTRANSCRIPT
Python Objects Mykola Kharechko, Developer
© Quintagroup, 2008
Table of contents
• Objects in memory• Objects at python level• Objects in python virtual machine
Objects in memory
• Object structure• Python memory management• Garbage collection
Object structure
Objects with same class
Python MemoryManagement
Allocation Strategy
For small requests, the allocator sub-allocates <Big> blocks of memory.Requests greater than 256 bytes are routed to the system's allocator.
Pymalloc structure
Arena256kb
Pool4kb
Fixed size block
Fixed size block
* Request in bytes Size of allocated block Size class idx* ------------------------------------------------------------------------------* 1-8 8 0* 9-16 16 1* 17-24 24 2* 25-32 32 3* 33-40 40 4* 41-48 48 5* 49-56 56 6* 57-64 64 7* 65-72 72 8* ... ... ...* 241-248 248 30* 249-256 256 31
Garbage Collection
Python performs garbage collection viareference counting and a cyclic garbagecollector that is able to detect and breakreference cycles.
Example>>> import gc>>> gc.disable() # 2.9mb>>> a = range(1000000) # 18mb>>> del a # 14mb>>> gc.enable() # 14mb>>> gc.collect() # 14mb0>>> gc.disable()>>> a = range(1000000) # 18mb>>> a.append(a) # 18mb>>> del a # 18mb>>> gc.enable() # 18mb>>> gc.collect() # 14 mb1>>>
Weak references
• weakref module• weakref.ref class• weakref.proxy class• weakref.WeakKeyDictionary
weakref example>>> from weakref import ref, proxy>>> class A(object):pass...>>> a = A()>>> ref_a = ref(a)>>> a.a = 16>>> ref_a().a16>>> proxy_a = proxy(a)>>> proxy_a.a16>>> proxy_a<weakproxy at 0x94e25f4 to A at 0x94e0fcc>>>> del a>>> ref_a()>>> ref_a().aTraceback (most recent call last):File "<stdin>", line 1, in <module>AttributeError: 'NoneType' object has no attribute 'a'>>> proxy_a<weakproxy at 0x94e23c4 to NoneType at 0x81479b8>
List object
typedef struct {PyObject_VAR_HEAD // ob_size/* Vector of pointers to list elements. list[0] is ob_item[0], etc. */PyObject **ob_item;// 0 <= ob_size <= allocated// len(list) == ob_sizePy_ssize_t allocated;} PyListObject;
List creation
PyObject *PyList_New(Py_ssize_t size){PyListObject *op;size_t nbytes;....nbytes = size * sizeof(PyObject *);.....op->ob_item = (PyObject **) PyMem_MALLOC(nbytes);memset(op->ob_item, 0, nbytes);...op->ob_size = size;op->allocated = size;...}
Integer objects
typedef struct {PyObject_HEADlong ob_ival;} PyIntObject;
Integeroptimization.
Problem>>> a = -2>>> b = -2>>> id(a) == id(b)True>>> a = 300>>> b = 300>>> id(a) == id(b)False
Integer optimization.Solution
#define NSMALLPOSINTS 257#define NSMALLNEGINTS 5/* References to small integers are saved in this arrayso that they can be shared.The integers that are saved are those in the range-NSMALLNEGINTS (inclusive) to NSMALLPOSINTS (not inclusive).*/static PyIntObject *small_ints[NSMALLNEGINTS + NSMALLPOSINTS];
Objects at pythonlevel
• Objects creation• Object attributes• Specific attributes
Objects creation
Constructor:obj = type->tp_new(type, args, kwds);type = Py_TYPE(obj);type->tp_init(obj, args, kwds);
Classes created by metaclasses
Object attributes
Attributes stored in __dict__ or in objectwhen __slots__ used
Attribute can be accessed via descriptor protocol
The most famous descriptor - property
Specific attributes
• __getattribute__ -access to all attrs• __setattr__ -opposite to __getattribute__• Descriptor protocol: __get__, __set__,
__delete__• Pickler protocol: __getstate__,
__setstate__, __reduce__
Objects in PVM
typedef struct {PyObject_HEADint co_argcount; /* #arguments, except *args */int co_nlocals; /* #local variables */int co_stacksize; /* #entries needed for evaluation stack */int co_flags;/* CO_..., see below */PyObject *co_code; /* instruction opcodes */PyObject *co_consts; /* list (constants used) */PyObject *co_names;/* list of strings (names used) */PyObject *co_varnames;/* tuple of strings (local variable names) */PyObject *co_freevars;/* tuple of strings (free variable names) */PyObject *co_cellvars; /* tuple of strings (cell variable names) *//* The rest doesn't count for hash/cmp */PyObject *co_filename;/* string (where it was loaded from) */PyObject *co_name;/* string (name, for reference) */int co_firstlineno;/* first source line number */PyObject *co_lnotab;/* string (encoding addr<->lineno mapping) */void *co_zombieframe; /* for optimization only (see frameobject.c) */} PyCodeObject;
CodeObject example
File test.py
a = 17b = a + 3print b
CodeObject Example
>>> import test34>>> pyc = open('test.pyc')>>> pyc.read(8)'\xb3\xf2\r\n>C\x8eH'>>> from marshal import load>>> co = load(pyc)>>> dir(co)[... 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']>>> co.co_consts(17, None)>>> co.co_names('a', 'b')
CodeObject example
>>> from dis import dis>>> dis(co)1 0 LOAD_CONST 0 (17)3 STORE_NAME 0 (a)
2 6 LOAD_NAME 0 (a)9 LOAD_CONST 0 (17)12 BINARY_ADD13 STORE_NAME 1 (b)
3 16 LOAD_NAME 1 (b)19 PRINT_ITEM20 PRINT_NEWLINE21 LOAD_CONST 1 (None)24 RETURN_VALUE