20090918 agile computer control of a complex experiment
DESCRIPTION
2009 Journal ClubTRANSCRIPT
Agile Computer Controlof a Complex Experiment
by
Gaël Varoquaux
presented by Jonathan Blakes
on18th September 2009
forInter-disciplinary Optimisation Laboratory
Journal Club
2
Outline of this talk
1. Why I chose this paper2. Outline of the paper3. Section by section discussion of
underlying technologies and practical implementation
4. Some useful references
3
Why this paper?Highly practicalMirrors common experimental
workflowNice example of an appropriate
degree of programming language evangelism
Introduces TraitsUI for building GUIs◦I use Mayavi2’s mlab module for 3D
plotting in our application, maintained by Gael
4
Shameless plug for the Infobiotics Workbench
5
Outline of the paperIntroductionInterfacing with specialist
hardwareUnit testing the experimentPerforming operations in parallelEvent-driven programmingBuilding GUIsData-driven programmingPutting it all together
6
IntroductionExperimentalists use (and build) specialist
equipmentThey are are computer literate and
mathematically competent but not computer scientists
Bad software design choices can hold up research, even though it is not the 'hard' part
Need software that evolves with the physical experiment
A 'flexible and reliable code base' can help the lab reorientate with goal and resource changes
7
Interfacing with hardwareCritical code often written in C (controller API /
performance)◦ Difficult for new people to contribute with steep
learning curve◦ New code can introduce memory-management
problemsAdvocates Python as high-level language for
connecting to devices◦ Python has modules for controlling VISA
instruments◦ Wrapping API calls (using 'ctypes' module) to a
generic interface enables new devices to be incorporated without changing data-processing/visualisation routines
◦ 'Pushes' memory-management problems out of C.
8
Unit testing the experiment
As complexity of task grows the elementary operations on which relies must be robust
Automated test harness ensures bugs are picked up early
Can replace temperamental hardware with 'mock objects'◦test software without them◦forces you to understand instruments
more fully
9
Parallel operationOften have to wait for devices or
data-processing – but how long?Should run aspects concurrently
using threads:◦One thread for user interface◦another for hard/long computations◦and one for each device
Rule of thumb: no two threads to modify the same object
10
ThreadingActually very easy in Python:
Prevents blocking UI
11
Event-driven programsProblem: Events occuring out of
sequence can bring down experiment
Solution: respond to events in the order they happen rather than in the order we program them to
How?◦Listen for events (with listener
thread), accumulate callbacks in queue, execute in order.
12
Super-simple event queueInherits deque, a fast
stack/queue: O(1) pop & insert(0, x)
dispatch time-consuming function calls to queue
queue executes them in order on a separate thread
13
Building GUIsWant to setup and run experiments
interactively◦ display and (sensible) editing of parameters
But designing and building GUIs is time-consuming and difficult
Code becomes messy as model, view and controller get merged
Python library TraitsUI can help by automatically generating GUIs based on underlying datamodel (Traits)◦ Removes GUI implementation from code
14
What are Traits?Static type definitions for Python variables
(otherwise dynamically typed)Example traits:
◦ aperture = Range(0,10,1)◦ plot = Instance(MLabSceneModel)
Traits facilitate: Variable initialization – simple, complex or deferred Validation – type checking and value checking Notification – trigger events when values change
(built-in, transparent Observer pattern) Visualisation – knowing types enables appropriate
widget choiceClasses inherit from HasTraits
15
TraitsUI example
16
Other benefits of TraitsUIMultiple views per object
◦Reuse objects in different contextsCompound views
◦Reuse views in larger applicationsDon't have to embed in GUI:
◦ configure_traits(view=object.view)
runs in its own event loopSerialize state easily:
◦ configure_traits(filename=~/app/ini)
17
Data-driven programsResponding to data changes from
◦ Devices◦ User input
Traits fire events when their values change
Capture events with methods:@aperture_changed@focus_changeddef update_display(self): self.camera.take_photo()
18
Putting it all together
19
Acknowledgements (plug)Traits are an innovation of Enthought Inc.
◦ sponsor SciPy (includes NumPy)◦ run free training webinars (US time)
Part of Enthought Tool Suite (ETS), which also includes:◦ Mayavi2 (TVTK) – 3D data visualisation◦ Chaco – 2D plotting, complements Matplotlib◦ Envisage – Eclipse-like application framework
All open source, BSD License (very liberal
20
ReferencesVaroquaux G. Agile Computer Control of a
Complex Experiment. Computing in Science & Engineering (2008) 20-25
Ramachandran P. Mayavi: Making 3D Data Visualization Reusable. Proc. SciPy 2008, 51-57
Varoquax G. Writing a graphical application for scientific programming using TraitsUI. http://gael-varoquaux.info/computers/
traits_tutorial/traits_tutorial.pdfEnthought Inc.
http://code.enthought.com/
21
Other resourcesSoftware Carpentry course
◦http://software-carpentry.org/◦For non computer scientists◦Uses Python◦Explains unit-testing, version control◦Mentioned in Nature:
Wilson G. Open-source offers solutions for science software education. Nature 436: 600 (27 July 2005) doi:10.1038/nj7050-600b
22
finally:'Passing that knowledge along'Check out .coma free programming Q&A wiki