the pymca application and toolkit - scipyconference.scipy.org/static/wiki/sole_pymca.pdf · slide:...
TRANSCRIPT
![Page 1: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/1.jpg)
Slide: 1
V.A. Solé - European Synchrotron Radiation Facility2009 Python for Scientific Computing Conference
The PyMca Application and Toolkit
![Page 2: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/2.jpg)
Slide: 2
The ESRF: Just an X-Ray Source
![Page 3: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/3.jpg)
Slide: 3
(Partial) Synchrotron Chart
![Page 4: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/4.jpg)
Slide: 4
X-Ray Fluorescence Basics
Energy of emitted X rays is element dependent
![Page 5: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/5.jpg)
Slide: 5
Visualization via Multi-Channel Analyzers (MCA)
![Page 6: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/6.jpg)
Slide: 6
PyMca?
For the end users and the specific developers
Set of programs and widgets for XRF analysis
Spectrum modeling
Quantification
ROI imaging
Fit imaging via batch processing
For the general developer
Set of python modules
Data visualization
Peak search
Function fitting
Imaging of 1D data
V.A. Solé, E. Papillon, M. Cotte, Ph. Walter, J. Susini, Spectrochimica Acta B 62 (2007) 63-68
PyMca is set of software tools on its way to become a reference in XRF
![Page 7: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/7.jpg)
Slide: 7
- First release in 2004
- Source code SVN repository available at sourceforge since 2006
- Official releases aim to provide ready-to-use binaries to end users
- External dependencies on numpy, PyQt4 and PyQwt5
- If installed, Matplotlib is used to generate high quality output
- Provides high level functional widgets that can be embedded in your own application
Generalities
![Page 8: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/8.jpg)
Slide: 8
XRF Spectrum Analysis
Typical procedure:
1. Calibration
2. Peak identification
3. Peak area extraction
Region of interest (ROI)
Deconvolution (FIT)
4. Quantification
Documentation at http://pymca.sourceforge.net/documentation.html
![Page 9: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/9.jpg)
Slide: 9
1. Calibration
Detailed documentation at http://pymca.sourceforge.net/documentation.html
![Page 10: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/10.jpg)
Slide: 10
2. IdentificationTheoretical database in ASCII format
IUPAC notation whenever possible
PyMca proposes, the user decidesPyMca proposes, the user decidesPyMca proposes, the user decidesPyMca proposes, the user decides
![Page 11: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/11.jpg)
Slide: 11
3.1 Peak area via ROI
![Page 12: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/12.jpg)
Slide: 12
3.2 Peak area via fitting
![Page 13: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/13.jpg)
Slide: 13
3.2 Peak area via fitting
![Page 14: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/14.jpg)
Slide: 14
Fit configuration Dialog (I)
![Page 15: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/15.jpg)
Slide: 15
Fit Configuration Dialog (II)
![Page 16: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/16.jpg)
Slide: 16
Emission Lines
Allowed transitions Implemented transitions
Any transition defined
KShellRates.datKShellRates.datKShellRates.datKShellRates.dat
LShellRates.datLShellRates.datLShellRates.datLShellRates.dat
MShellRates.datMShellRates.datMShellRates.datMShellRates.dat
![Page 17: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/17.jpg)
Slide: 17
4. Quantification (I)
Parallel beam approximationParallel beam approximationParallel beam approximationParallel beam approximation
![Page 18: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/18.jpg)
Slide: 18
4. Quantification (II)Nominal concentration 500 ppm
![Page 19: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/19.jpg)
Slide: 19
XRF Analysis Integration in other Applications
Integration in mxCuBE (ESRF) Integration elsewhere
![Page 20: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/20.jpg)
Slide: 20
Is it easy to embed?
from PyMca import McaAdvancedFit
fitWindow = McaAdvancedFit.McaAdvancedFit()
fitWindow.setData(x, y)
fitWindow.show()
For the previous examples, basically one just needs 4 lines of code:
It can be used from ipython just starting it as “ipython –q4thread”
![Page 21: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/21.jpg)
Slide: 21
Advanced Fit Batch processing
Select the input files
Select the fit configuration
Select the output directory
Select the output options
Start
![Page 22: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/22.jpg)
Slide: 22
Output
Images in ASCII and ESRF format
• Easy to import in other programs
Individual peak contributions in ASCII
• Use your own plotting program
Fully automated HTML report
• Browse your results!
![Page 23: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/23.jpg)
Slide: 23
+……
One spectrum for each pixel
A sum spectrum for the whole image
XRF Imaging
![Page 24: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/24.jpg)
Slide: 24
Sb
Pb
S
Ca
We wanted to know what pigments were
used in this section of the painting
Copyright C2RMF
![Page 25: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/25.jpg)
Slide: 25
Based on the batch generated element distribution maps …
![Page 26: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/26.jpg)
Slide: 26
… we were able to determine the possible presence of stibnite grains (Sb2S3) embedded in a lead containing matrix.
M. Cotte, E. Welcomme, V.A. Solé, M. Salomé, M. Menu, Ph. Walter, J. Susini, Anal. Chem. 79 (2007) 6988-6994
Sulfur and antimony correlated Lead and antimony not correlated
… and their correlations as shown by the program
![Page 27: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/27.jpg)
Slide: 27
Generic Fitting Module
Levenberg-Marquardt algorithm with constraints on fitting parameters
It accepts user defined functions
The simplest form of user function:
![Page 28: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/28.jpg)
Slide: 28
Automatic Peak Search Routines
![Page 29: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/29.jpg)
Slide: 29
In this example:
Stack = 101x200x2000 numpy array
20200 spectra of 2000 channels
Pixel[i, j] = numpy.sum(Stack[i, j, :])
Pixel[i, j] = numpy.sum(Stack[i, j, ch0:ch1])
1D Stack ROI Imaging
![Page 30: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/30.jpg)
Slide: 30
1D Stack ROI Imaging
We can generate new images by moving the cursors or defining new ROIs in the table
![Page 31: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/31.jpg)
Slide: 31
Imaging and Principal Component Analysis
Subtract the average spectrum from each spectrum of the dataset and arrange it as NpixelsxNchannels
Calculate the covariance = Data.T * Data
Get the eigenvectors of the covariance
Add the average spectrum to each row and get the projections of that dataset on each eigenvector (Npixels scalars for each eigenvector)
Reshape the projections in the original map shape to obtain the ¨Eigenimages¨
Covariance method, other ways in http:// en.wikipedia.org/wiki/Principal_components_analysis
![Page 32: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/32.jpg)
Slide: 32
Eigenimages and Eigenvectors
![Page 33: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/33.jpg)
Slide: 33
Eigenimages and Eigenvectors
![Page 34: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/34.jpg)
Slide: 34
Eigenimages and Eigenvectors
![Page 35: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/35.jpg)
Slide: 35
Eigenimages and Eigenvectors
![Page 36: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/36.jpg)
Slide: 36
Eigenimages and Eigenvectors
![Page 37: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/37.jpg)
Slide: 37
Eigenimages and Eigenvectors
![Page 38: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/38.jpg)
Slide: 38
Getting the actual information
We can select a set of pixels on any of the displayed images and display the cumulative spectrum associated to those pixels.
Here we can see the average spectrum associated to the hotter pixels of the Eigenimage 02 (in red) compared to the average spectrum of the map (in black).
![Page 39: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/39.jpg)
Slide: 39
We could have easily missed the presence of one element if we would have just analyzed the sum spectrum via ROIs.
![Page 40: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/40.jpg)
Slide: 40
What have we done?
We have used principal components analysis to know what sample regions were worth to take a closer look.
Not bad when you have a lot of data …
This data treatment is totally generic and applicable to other methods of analysis
![Page 41: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/41.jpg)
Slide: 41
Current Developments – HDF5 Support
- Collaboration with D. Dale from Cornell High Energy Synchrotron Source (CHESS)
- HDF5 expected to become the file format of most European synchrotrons sources
- Additional dependency on h5py http://h5py.alfven.org
- Partial support already available in current SVN sources
![Page 42: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/42.jpg)
Slide: 42
Current Developments – 3D and 4D
-The goal is to provide ways to visualize and interact with multiparametric data
-Already available at the ESRF through an additional PyMca set of modules
-Currently evaluating the pros and cons of using MayaVi
![Page 43: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/43.jpg)
Slide: 43
![Page 44: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/44.jpg)
Slide: 44
![Page 45: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/45.jpg)
Slide: 45
Conclusion
PyMca is a program as well as an XRF analysis toolkit
- Open source and distributed under the conditions of the GPLv2+
- Can be used as a fitting and visualization tool (for up to 4-dimensional data)
- Provides high level widgets based on PyQt that can be used independently or integrated into your application
- Allows you to specify a physically meaningful model which can quantitatively determine element concentrations from energy dispersive X-ray spectra
- Is implemented at many synchrotron facilities and labs around the world
- Active development is funded by the ESRF
![Page 46: The PyMca Application and Toolkit - SciPyconference.scipy.org/static/wiki/sole_pymca.pdf · Slide: 1 V.A. Solé - European Synchrotron Radiation Facility 2009 Python for Scientific](https://reader031.vdocuments.net/reader031/viewer/2022020413/5b90773409d3f2e6728c0a37/html5/thumbnails/46.jpg)
Slide: 46
Acknowledgements
My ESRF colleagues (mainly software group and microscopy beamlines ID21 and ID22)
Ph. Walter and M. Cotte (Centre de Recherche et de Restauration des Musées de France)
Darren Dale (Cornell High Energy Synchrotron Source)
The PyMca users, for their enthusiasm and their encouragements