beating the (sh** out of the) gil - multithreading vs. multiprocessing
DESCRIPTION
Talk given at the June 2008 meeting of the New Zealand Python User Group in Auckland. Outline: An overview to approaches for parallel/concurrent programming in Python. Code demonstrated in the presentation can be found here: http://www.kloss-familie.de/moin/TalksPresentationsTRANSCRIPT
Threading Theory Multiprocessing Others Conclusion Finalise
Beating the (sh** out of the) GILMultithreading vs. Multiprocessing
Hair dryer 1920s,Dark Roasted Blend:http://www.darkroastedblend.com/2007/01/retro-technology-update.html
Guy K. Kloss | Multithreading vs. Multiprocessing 1/36
Threading Theory Multiprocessing Others Conclusion Finalise
Beating the (sh** out of the) GILMultithreading vs. Multiprocessing
Guy K. Kloss
Computer ScienceMassey University, Albany
New Zealand Python User Group MeetingAuckland, 12 June 2008
Guy K. Kloss | Multithreading vs. Multiprocessing 2/36
Threading Theory Multiprocessing Others Conclusion Finalise
Outline
1 Threading
2 Theory
3 Multiprocessing
4 Others
5 Conclusion
Guy K. Kloss | Multithreading vs. Multiprocessing 3/36
Threading Theory Multiprocessing Others Conclusion Finalise
Guy K. Kloss | Multithreading vs. Multiprocessing 4/36
Threading Theory Multiprocessing Others Conclusion Finalise
Outline
1 Threading
2 Theory
3 Multiprocessing
4 Others
5 Conclusion
Guy K. Kloss | Multithreading vs. Multiprocessing 5/36
Threading Theory Multiprocessing Others Conclusion Finalise
Source: http://blog.snaplogic.org/?cat=29Guy K. Kloss | Multithreading vs. Multiprocessing 6/36
Threading Theory Multiprocessing Others Conclusion Finalise
What People Think Now
Threading and shared memory are common(thanks to Windows and Java)Python supports threads (Yay!)Python also supports easy forking (Yay!)The GIL . . . is a problem for pure Python,non I/O bound applicationsLots of people “understand” threads . . .. . . and fail at them (to do them properly)
Guy K. Kloss | Multithreading vs. Multiprocessing 7/36
Threading Theory Multiprocessing Others Conclusion Finalise
What People Think Now
Threading and shared memory are common(thanks to Windows and Java)Python supports threads (Yay!)Python also supports easy forking (Yay!)The GIL . . . is a problem for pure Python,non I/O bound applicationsLots of people “understand” threads . . .. . . and fail at them (to do them properly)
Guy K. Kloss | Multithreading vs. Multiprocessing 7/36
Threading Theory Multiprocessing Others Conclusion Finalise
What People Think Now
Threading and shared memory are common(thanks to Windows and Java)Python supports threads (Yay!)Python also supports easy forking (Yay!)The GIL . . . is a problem for pure Python,non I/O bound applicationsLots of people “understand” threads . . .. . . and fail at them (to do them properly)
Guy K. Kloss | Multithreading vs. Multiprocessing 7/36
Threading Theory Multiprocessing Others Conclusion Finalise
What People Think Now
Threading and shared memory are common(thanks to Windows and Java)Python supports threads (Yay!)Python also supports easy forking (Yay!)The GIL . . . is a problem for pure Python,non I/O bound applicationsLots of people “understand” threads . . .. . . and fail at them (to do them properly)
Guy K. Kloss | Multithreading vs. Multiprocessing 7/36
Threading Theory Multiprocessing Others Conclusion Finalise
What People Think Now
Threading and shared memory are common(thanks to Windows and Java)Python supports threads (Yay!)Python also supports easy forking (Yay!)The GIL . . . is a problem for pure Python,non I/O bound applicationsLots of people “understand” threads . . .. . . and fail at them (to do them properly)
Guy K. Kloss | Multithreading vs. Multiprocessing 7/36
Threading Theory Multiprocessing Others Conclusion Finalise
What People Think Now
Blog post by Mark Ramm, 14 May 2008A multi threaded system is particularly important for peoplewho use Windows, which makes multi–process computingmuch more memory intensive than it needs to be. As mygrandma always said Windows can’t fork worth a damn. ;)[. . . ]So, really it’s kinda like shared–memory optimizedmicro–processes running inside larger OS level processes, andthat makes multi–threaded applications a lot morereasonable to wrap your brain around. Once you start downthe path of lock managment the non-deterministic characterof the system can quickly overwhelm your brain.
Guy K. Kloss | Multithreading vs. Multiprocessing 8/36
Threading Theory Multiprocessing Others Conclusion Finalise
Simple Threading Example
from threading import Threadfrom stuff import expensiveFunction
class MyClass(Thread):def __init__(self, argument):
self.argument = argumentThread.__init__(self) # I n i t i a l i s e the thread
def run(self):self.value = expensiveFunction(self.argument)
callObjects = []for i in range(config.segments):
callObjects.append(MyClass(i))
for item in callObjects:item.start()
# Do something e lse .time.sleep(15.0)
for item in callObjects:item.join()print item.value
Guy K. Kloss | Multithreading vs. Multiprocessing 9/36
Threading Theory Multiprocessing Others Conclusion Finalise
Our Example with Threading
Our fractal examplenow with threading.
Just a humble hair–dryer from the30s: “One of the first machines usedfor permanent wave hairstyling backin the 1920’s and 1930’s.”Dark Roasted Blend:http://www.darkroastedblend.com/2007/05/
mystery-devices-issue-2.html
Guy K. Kloss | Multithreading vs. Multiprocessing 10/36
Threading Theory Multiprocessing Others Conclusion Finalise
The GIL
Global Interpreter LockWhat is it for?
Cooperative multitaskingInterpreter knows when it’s “good to switch”Often more efficient than preemptive multi–taskingCan be released from native (C) code extensions(done for I/O intensive operations)
Is it good?Easy codingEasy modules/extensionsLarge base of available modules alredySpeed improvement by factor 2(for single–threaded applications)Keeps code safe
Guy K. Kloss | Multithreading vs. Multiprocessing 11/36
Threading Theory Multiprocessing Others Conclusion Finalise
The GIL
Global Interpreter LockWhat is it for?
Cooperative multitaskingInterpreter knows when it’s “good to switch”Often more efficient than preemptive multi–taskingCan be released from native (C) code extensions(done for I/O intensive operations)
Is it good?Easy codingEasy modules/extensionsLarge base of available modules alredySpeed improvement by factor 2(for single–threaded applications)Keeps code safe
Guy K. Kloss | Multithreading vs. Multiprocessing 11/36
Threading Theory Multiprocessing Others Conclusion Finalise
The GILAlternatives
Other implementations(C) Python uses itJython doesn’tIronPython doesn’tThey use their own/internal threading mechanisms
Is it a design flaw?Maybe . . . but . . .Fierce/intense discussions to change the code baseSolutions that pose other benefits:
Processes create fewer inherent dead lock situationsProcesses scale also to multi–host scenarios
Guy K. Kloss | Multithreading vs. Multiprocessing 12/36
Threading Theory Multiprocessing Others Conclusion Finalise
The GILAlternatives
Other implementations(C) Python uses itJython doesn’tIronPython doesn’tThey use their own/internal threading mechanisms
Is it a design flaw?Maybe . . . but . . .Fierce/intense discussions to change the code baseSolutions that pose other benefits:
Processes create fewer inherent dead lock situationsProcesses scale also to multi–host scenarios
Guy K. Kloss | Multithreading vs. Multiprocessing 12/36
Threading Theory Multiprocessing Others Conclusion Finalise
Doug Hellmann in Python Magazine 10/2007:Techniques using low–level, operating system–specific,libraries for process management are as passe as usingcompiled languages for CGI programming. I don’t have timefor this low–level stuff any more, and neither do you. Let’slook at some modern alternatives.
Guy K. Kloss | Multithreading vs. Multiprocessing 13/36
Threading Theory Multiprocessing Others Conclusion Finalise
GIL–less Python
There was an attempt/patch “way back then ...”There’s a new project now by Adam OlsenPython 3000 with “free theading” [1]Using Monitors to isolate stateDesign focus: usability(for common cases, maintainable code)Optional at compile time using --with-freethread
Sacrificed single–threaded performance(60–65% but equivalent to threaded CPython)Automatic deadlock detection(detection/breaking, giving exceptions/stack trace)Runs on Linux and OS/X
Guy K. Kloss | Multithreading vs. Multiprocessing 14/36
Threading Theory Multiprocessing Others Conclusion Finalise
Outline
1 Threading
2 Theory
3 Multiprocessing
4 Others
5 Conclusion
Guy K. Kloss | Multithreading vs. Multiprocessing 15/36
Threading Theory Multiprocessing Others Conclusion Finalise
Parallelisation in General
CPU vs. I/O bottle necksThreading: Good for I/O constrainsThis talk aims at CPU constrains
Threads vs. ProcessesThreads: Within a process on one hostProcesses: Independent on the OSProcesses are:
Heavier in memory/overheadHave their own name space and memoryInvolve less problems with competing accessto resources and their management
But:On UN*X/Linux: Process overhead is very low(C)Python is inefficient in handling threadsStackless Python is much more efficient on threading
Guy K. Kloss | Multithreading vs. Multiprocessing 16/36
Threading Theory Multiprocessing Others Conclusion Finalise
Parallelisation in General
CPU vs. I/O bottle necksThreading: Good for I/O constrainsThis talk aims at CPU constrains
Threads vs. ProcessesThreads: Within a process on one hostProcesses: Independent on the OSProcesses are:
Heavier in memory/overheadHave their own name space and memoryInvolve less problems with competing accessto resources and their management
But:On UN*X/Linux: Process overhead is very low(C)Python is inefficient in handling threadsStackless Python is much more efficient on threading
Guy K. Kloss | Multithreading vs. Multiprocessing 16/36
Threading Theory Multiprocessing Others Conclusion Finalise
Parallelisation in General
CPU vs. I/O bottle necksThreading: Good for I/O constrainsThis talk aims at CPU constrains
Threads vs. ProcessesThreads: Within a process on one hostProcesses: Independent on the OSProcesses are:
Heavier in memory/overheadHave their own name space and memoryInvolve less problems with competing accessto resources and their management
But:On UN*X/Linux: Process overhead is very low(C)Python is inefficient in handling threadsStackless Python is much more efficient on threading
Guy K. Kloss | Multithreading vs. Multiprocessing 16/36
Threading Theory Multiprocessing Others Conclusion Finalise
Parallelisation in General
CPU vs. I/O bottle necksThreading: Good for I/O constrainsThis talk aims at CPU constrains
Threads vs. ProcessesThreads: Within a process on one hostProcesses: Independent on the OSProcesses are:
Heavier in memory/overheadHave their own name space and memoryInvolve less problems with competing accessto resources and their management
But:On UN*X/Linux: Process overhead is very low(C)Python is inefficient in handling threadsStackless Python is much more efficient on threading
Guy K. Kloss | Multithreading vs. Multiprocessing 16/36
Threading Theory Multiprocessing Others Conclusion Finalise
Abstraction Level vs. Control
Abstraction levels for parallel computing models [7]
Parallelism Communication Synchronisation4 implicit3 explicit implicit2 explicit implicit1 explicit
Explicit: The programmer specifies it in the parallel programImplicit: A compiler/runtime system derives it from other information
Guy K. Kloss | Multithreading vs. Multiprocessing 17/36
Threading Theory Multiprocessing Others Conclusion Finalise
Abstraction Level vs. Control
Low level: Close to hardwareMust specify parallelism. . . communication. . . and synchronisation
→ Best means for performance tuning→ Premature optimisation?High level: Highest machine independence
More/all handled by computing modelUp to automatic parallelisation approaches
Both extremes have not been very successful to dateMost developments now:
Level 3 for specific purposesLevel 1 for general programming(esp. in the scientific community)With Python consistent level 2 possible
Guy K. Kloss | Multithreading vs. Multiprocessing 18/36
Threading Theory Multiprocessing Others Conclusion Finalise
Abstraction Level vs. Control
Low level: Close to hardwareMust specify parallelism. . . communication. . . and synchronisation
→ Best means for performance tuning→ Premature optimisation?High level: Highest machine independence
More/all handled by computing modelUp to automatic parallelisation approaches
Both extremes have not been very successful to dateMost developments now:
Level 3 for specific purposesLevel 1 for general programming(esp. in the scientific community)With Python consistent level 2 possible
Guy K. Kloss | Multithreading vs. Multiprocessing 18/36
Threading Theory Multiprocessing Others Conclusion Finalise
Abstraction Level vs. Control
Low level: Close to hardwareMust specify parallelism. . . communication. . . and synchronisation
→ Best means for performance tuning→ Premature optimisation?High level: Highest machine independence
More/all handled by computing modelUp to automatic parallelisation approaches
Both extremes have not been very successful to dateMost developments now:
Level 3 for specific purposesLevel 1 for general programming(esp. in the scientific community)With Python consistent level 2 possible
Guy K. Kloss | Multithreading vs. Multiprocessing 18/36
Threading Theory Multiprocessing Others Conclusion Finalise
Common for Parallel Computing
Message Passing Interface (MPI)for distributed memoryOpenMPshared memory multi–threadingThe two do not have to be categorised like this
Guy K. Kloss | Multithreading vs. Multiprocessing 19/36
Threading Theory Multiprocessing Others Conclusion Finalise
Art by “Teknika Molodezhi,” Russia 1966
Dark Roasted Blend: http://www.darkroastedblend.com/2008/01/retro-future-mind-boggling.htmlGuy K. Kloss | Multithreading vs. Multiprocessing 20/36
Threading Theory Multiprocessing Others Conclusion Finalise
Outline
1 Threading
2 Theory
3 Multiprocessing
4 Others
5 Conclusion
Guy K. Kloss | Multithreading vs. Multiprocessing 21/36
Threading Theory Multiprocessing Others Conclusion Finalise
Processing around the GIL
Smart multi–processingSmart task farming
Guy K. Kloss | Multithreading vs. Multiprocessing 22/36
Threading Theory Multiprocessing Others Conclusion Finalise
(py)Processing module
By R. Oudkerk [2]Written in C (really fast!)Allowes multiple cores and multiple hosts/clustersData synchronisation through managersEasy “upgrade path”
Drop in replacement (mostly) for the threading moduleTransparent to userForks processes, but uses Thread API
Supports queues, pipes, locks,managers (for sharing state), worker poolsVERY fast, see PEP-371 [3]
Jesse Noller for pyprocessing into core Pythonbenchmarks available, awesome results!PEP is officially accepted: Thanks Guido!
Guy K. Kloss | Multithreading vs. Multiprocessing 23/36
Threading Theory Multiprocessing Others Conclusion Finalise
(py)Processing module(continued)
Some detailsProducer/consumer style system– workers pull jobsHides most details of communication– usable default settingsCommunication is tweakable(to improve performance or meet certain requirements)
Guy K. Kloss | Multithreading vs. Multiprocessing 24/36
Threading Theory Multiprocessing Others Conclusion Finalise
(py)Processing module
Let’s see it!
Guy K. Kloss | Multithreading vs. Multiprocessing 25/36
Threading Theory Multiprocessing Others Conclusion Finalise
Parallel Python module
By Vitalii Vanovschi [4]Pure PythonFull “Batteries included” paradigm model:
Spawns automatically across detected cores,and can spawn to clustersUses some thread module methods under the hoodMore of a “task farming” approach(requires potentially rethinking/restructuring)Automatically deploys code and data,no difficult/multiple installsFault tolerance, secure inter–node communication,runs everywhere
Very active communigy,good documentation, good support
Guy K. Kloss | Multithreading vs. Multiprocessing 26/36
Threading Theory Multiprocessing Others Conclusion Finalise
Parallel Python module
Let’s see it!
Guy K. Kloss | Multithreading vs. Multiprocessing 27/36
Threading Theory Multiprocessing Others Conclusion Finalise
Outline
1 Threading
2 Theory
3 Multiprocessing
4 Others
5 Conclusion
Guy K. Kloss | Multithreading vs. Multiprocessing 28/36
Threading Theory Multiprocessing Others Conclusion Finalise
Honourable Mentions
pprocess [5]IPython for parallel computing [6]Bulk Synchronous Parallel (BSP) Model [7]sequence of super steps(computation, communication, barrier synch)Reactor based architectures, through Twisted [8]“Don’t call us, we call you”MPI (pyMPI, Pypar, MPI for Python, pypvm)requires constant number of processors duringcompation’s durationPyro (distributed object system)Linda (PyLinda)Scientific Python (master/slave computing model)data distribution through call parameters/replication
Guy K. Kloss | Multithreading vs. Multiprocessing 29/36
Threading Theory Multiprocessing Others Conclusion Finalise
Outline
1 Threading
2 Theory
3 Multiprocessing
4 Others
5 Conclusion
Guy K. Kloss | Multithreading vs. Multiprocessing 30/36
Threading Theory Multiprocessing Others Conclusion Finalise
Things to Note
Which approach is best?Can’t say!Many of the approaches are complimentaryNeeds to be evaluated what to use when
All, however, save you a lot of time over the alternativeof writing everything yourself with low–level libraries.What an age to be alive!Problems can arise when objects cannot be pickled
Guy K. Kloss | Multithreading vs. Multiprocessing 31/36
Threading Theory Multiprocessing Others Conclusion Finalise
Conclusion
Resolving the GIL is not necessarily the best solutionMore inefficient (single threaded) runtimeProblems with shared memory access
Various approaches to beat the GILSolutions are complimentary in many waysmany scale beyond a local machine/memory system
Guy K. Kloss | Multithreading vs. Multiprocessing 32/36
Threading Theory Multiprocessing Others Conclusion Finalise
Questions?
[email protected] and code available here:http://www.kloss-familie.de/moin/TalksPresentations
Guy K. Kloss | Multithreading vs. Multiprocessing 33/36
Threading Theory Multiprocessing Others Conclusion Finalise
References I
[1] A. Olsen,Python 3000 with Free Threading project,[Online]http://code.google.com/p/python-safethread/
[2] R. Oudkerk,Processing Package,[Online] http://pypi.python.org/pypi/processing/
[3] J. Noller,PEP-371,[Online] http://www.python.org/dev/peps/pep-0371/
Guy K. Kloss | Multithreading vs. Multiprocessing 34/36
Threading Theory Multiprocessing Others Conclusion Finalise
References II
[4] V. Vanovschi,Parallel Python,[Online] http://parallelpython.com/
[5] P. Boddie,pprocess,[Online] http://pypi.python.org/pypi/processing/
[6] Project Website,IPython,[Online] http://ipython.scipy.org/doc/ipython1/html/parallel_intro.html
[7] K. Hinsen,Parallel Scripting with PythonComputing in Science & Engineering, Nov/Dec 2007
Guy K. Kloss | Multithreading vs. Multiprocessing 35/36
Threading Theory Multiprocessing Others Conclusion Finalise
References III
[8] B. Eckel,Concurrency with Python, Twisted, and Flex,[Online] http://www.artima.com/weblogs/viewpost.jsp?thread=230001
Guy K. Kloss | Multithreading vs. Multiprocessing 36/36