gil - unit foto · „cpython cannot into threads” ... goroutines. ruby python goroutines...
Post on 11-Jul-2020
17 Views
Preview:
TRANSCRIPT
GIL
„CPython cannot into threads”
1992
„CPython cannot into threads”❌
„CPython cannot into threads”„CPython can run only on single core”
❌
PyPyJythonCPythonIronPythonBrython
PyPyJythonCPythonIronPythonBrython
PyPyJythonCPythonIronPythonBrython
Python
C
Python
C
„CPython cannot into threads”„CPython can run only on single core”
❌
❌
„CPython cannot into threads”„CPython can run only on single core”"CPython process can execute Python bytecode in one thread at the time"
❌
❌
✓
kolodziejj.info@unit03
Disclaimer
So what’s the hassle?
Misconceptions
CC-SA-BY, http://dibujando.net/dib/cthulhu-96836
Evolution
Concurrencyvs.
parallelism
GILfinally
Protectorof
CPython realm
Reference counters
1. Tracing garbage collection2. Reference counting3. ...
1. Tracing garbage collection2. Reference counting
int
1. Tracing garbage collection2. Reference counting
3
1. Tracing garbage collection2. Reference counting
0
Internal global state
C-mutable data structuresdict, list, …
C-mutable data structuresdict, list, …str, tuple, int
Binary modulesand extensions
Cost?
In single-threaded code?It doesn’t matter
(duh)
On single core?It doesn’t matter
In multi-threaded I/O-bound code?
It may not matter
With binary extensions?It may not matter
>= 3.2
1
1
2
1
2
Can Ihas GIL?
Nope.
1
2switch interval
2
1
Can Ihas GIL?
2
1
Can Ihas GIL?
Yes!
2
1
Can Ihas GIL?
Nope.
2
1
gil_drop_request = 1
2
1
gil_drop_request?
2
1
2
1
switch_cond?gil_last_holder?
2
1
<= 2.7- 2010
„ticks”
>>> print(„foo”)>>> -1 in range(100000000)
>>> print(„foo”)>>> -1 in range(100000000)>>> n -= 1
„check”„check” „check”
N „ticks” N „ticks” N = 100
100s ms few ms
2
1
Can Ihas GIL?
Nope.
And now? And now?
Nope. Nope.
2
1
Can Ihas GIL?
Nope.
And now? And now?
Nope. Nope.
Aw yiss
PyPyJythonCPythonIronPythonBrython
PyPyJythonCPythonIronPythonBrython
PyPyJythonCPythonIronPythonBrython
(but: STM)
PyPyJythonCPythonIronPythonBrython
RubyPythonGo
RubyPythonGo
RubyPythonGoroutines
RubyPythonGoroutineschannels
More!
multiprocessingimport
multiprocessingbinary extensions
cdef int foo = 0
multiprocessingbinary extensions
cdef int foo = 0with nogil: ...
multiprocessingbinary extensions
Py_BEGIN_ALLOW_THREADS Py_END_ALLOW_THREADS
multiprocessingbinary extensionsasynchronous I/O
Revolution?
C
Requirements
1. Single-threaded and multi-threaded I/O-bound performance2. C extensions compatibility3. Complexity
1. Single-threaded and multi-threaded I/O-bound performance2. C extensions compatibility3. Complexity
1. Single-threaded and multi-threaded I/O-bound performance2. C extensions compatibility3. Complexity
1996
Free-threading patch
Free-threading patchsubinterpreters
Free-threading patchsubinterpreters
GILectomy
Free-threading patchsubinterpreters
GILectomybuffered
reference counting
Free-threading patchsubinterpreters
GILectomy
thread
-privat
e
locking
buffered
reference counting
Free-threading patchsubinterpreters
GILectomy
thread
-privat
e
locking
buffered
reference counting
immortal objects
--without-gilPy_LOCK(o)Py_UNLOCK(o)
Free-threading patchsubinterpreters
GILectomy
What to do?
What to do?nothing
What to do?nothing
more research
What to do?nothing
more researchmigrate to 3.5 (3.6 even!)
What to do?nothing
more researchmigrate to 3.5 (3.6 even!)
assert the baseline, then act
kolodziejj.info/talks/gil/
Thank you! :)
kolodziejj.info/talks/gil/
What to do?nothing
more researchmigrate to 3.5 (3.6 even!)
assert the baseline, then actcontribute !
top related