the dawnsci eclipse science usecases jun aishima software engineer diamond light source didcot, uk
TRANSCRIPT
The DAWNSci Eclipse science
usecases
Jun Aishima
Software Engineer
Diamond Light Source
Didcot, UK
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
SynchrotronAKA – cool word, but what does it
mean? syn·chro·tron/ˈsiNGkrəˌträn/
Noun:A cyclotron in which the magnetic field strength increases with the energy of the particles to keep their orbital radius constant.
“They are machines which produce very strong light used for many different type of scientific experiments and sometimes other things.”
Diamond Light Source, Didcot, Oxfordshire, UK
• (Video has been removed to allow the talk to fit into size guidelines – see http://www.diamond.ac.uk/Public/Multimedia/Animations.html to view the animation)
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
Detectors ofVarious flavors
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
…180 images
The collected data may look like this…
When what you really want is this…
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
DAWN - Acquisition and Analysis with RCP
@ Diamond Light Source
Client server technology
Communication withEPICS and hardware
Scan mechanism
www.opengda.org
Jythonand Python
Visualisation
Communicationwith external
analysis
Analysistools
Data read, write,convert
Metadata structure
Workflows
www.dawnsci.org
DAWN is a collection of generic and
bespoke ‘views’ collated into
‘perspectives’.
The perspectives and views can be
used in part or whole in either
the GDA or DAWN.
Acquisition Analysis
Collaborative Approach
Kichwa Coders Ltd.
Science Working Group
External Users
Consultants
Members
Now an Eclipse project!https://projects.eclipse.org/projects/technology.dawnsci
‘Shoulders of Giants’
AKA – didn’t think of everything• RCP many of the core features, editors, toolbars, views, projects
• Ptolemy 2 (a version known as ‘Passerelle’) workflow and pipelining
• GEF for visualization of pipeline graphs
• Draw2D for 1D and 2D plotting (SWT XY Graph)
• Pydev for python/jython scripting layer used by the scientists
• HDF5 libraries for storing large data sets
• SWT/Jface – lazy viewers being used extensively for large trees and tables
• Apache, Eclipse-WST, springsource, JDK, and many more of course...
Did you see our poster yesterday, presented by Bahá?
• The DAWNSci Eclipse Project: How Eclipse Is Used for Science (Bahá El Kassaby)– Printed copies or PDF file available! Ask us!
Data in DAWNSci
Example : Loading and Manipulating Data
This example• Columns of ASCII data• Separated by tabs• Read as 1D arrays
Other format with a loader: srs, dat, flt, gff, mca, csv, xy, xye, txt, tif, tiff, cbf, img, ciff, mccd, edf, pgm, cor, bruker, jpg, jpeg, png, f2d, msk, mib, mar3450, pck3450, raw, mrc, gz, bz2, zip, h5, hd5, hdf5, nxs, nexus, hdf, mat
Extension point to add more...
Demonstration – Visual Tools
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
The Processing pipeline
• Previous systems– Required much effort by the developer– User experience was too generic – not optimized for this
purpose
• Simplify (coding), simplify (GUI), simplify (capabilities)
• Result: wide adoption and rapid development for several experimental techniques
• Developed by Jacob Filik at Diamond• Available in DAWN
Non-crystalline diffraction
Why Processing in DAWN?
Simple? Operation
Calculation
Errorhandling
Simple? Model
Allow drag and dropProvide a button to open file dialog box
Radio box in GUI
Number box in GUI with limits
Mouseover in GUI
Simple?
File list
Input
Output
List of processing steps
Model
Demonstration of pipeline
1. What is a light source and why do science there?
2. What data do you get at a synchrotron?
3. What do you do to the data?
4. How do DAWN and DAWNSci Eclipse help with this?
5. How do we use DAWN and DAWNSci Eclipse to analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to analyze your data?
Create your own operations and pipelines!
• Create an operation – org.eclipse.dawnsci.analysis.api.operation extension point– Dataset operations – including lazy operations
• Create a model – if necessary– Any additional data required for the calculation– Reflection on data type and annotations are used to
build the GUI for users to enter parameters
A lot of existing, reusable operations
• Sector integration
– Typically, reduce a two-dimensional image into a one-dimensional dataset
• Normalization
– Remove frame-to-frame variation to put them all onto the same scale
• Subtraction
– Removing artifacts by subtracting background data from the sample data
• Averaging
– Multiple noisy points can be averaged
Future work on the Processing pipeline
• Support more sciences (XPDF)
• Add branching, merging, and other non-linear flows
• Run calculations via HPC
• Passerelle workflows
• More, faster, better processing steps
Try DAWN for yourself!
• DAWN 1.7.1, available now!
• http://www.dawnsci.org
• See directions on how to use DAWNSci – “Getting started with DAWNSci”
• Videos on how to do use slicing, data reduction, Python scripting, processing at https://www.youtube.com/user/DAWNScience
• Follow us on Twitter - https://twitter.com/dawnscience
• Read the article about us in the Eclipse newsletter - https://www.eclipse.org/community/eclipse_newsletter/2015/january/article3.php
• DAWN paper accepted for publication, doi: 10.1107/S1600577515002283
• Email us: [email protected]
+1 0 -1
Sign in: www.eclipsecon.org
Evaluate the sessions