update on gadgetron - ccp pet-mr · 2018-07-23 · gadgetron framework architecture, data types,...
TRANSCRIPT
Update on Gadgetron July 2018
Hui Xue1, David Hansen2
National Heart, Lung, and Blood Institute
Gradient Software
https://github.com/gadgetron/gadgetron
Gadgetron framework
Architecture, Data types,
Functionalities and toolboxes
Deployment and Integration test
Generic chain
Enable flexible triggering
ISMRMRD
Waveform, c++/python/matlab
Recent developments
Cloud computing in Gadgetron
Built-in Machine learning
Future plans
….
2
Outline
Overview
3
GADGETRON
Gadgetron
4
• Started by Michael Hansen and Thomas Sørensen in 2010
• Aim to provide flexibility, development efficiency, robust deployment and
computing power to medical imaging research
• A place to accommodate
algorithms, clinical applications, testing and deployment …
in a reusable way
Gadgetron – Developer’s view
5
Gadget 1 Gadget 2 Gadget 3
GadgetSt ream Cont roller
Reader 1
Reader 2
Reader 3
Writer 1
Writer 2
Writer 3
Message
Dispatch
Message ID?
Output
Queue
Message ID?
Socket
TCP/IP
CLIENT
APPLICATION
Shared Toolboxes
GADGETRON
Gadget can be coded by C++/Matlab/Python
Developer can write every processing unit -> Gadgets
Developer fully controls the data passing between gadgets
Gadgetron – End User’s view
6
MARS External Gadgetron HostTCP/IP
ICE
MARS
Conta inerized Gadgetr on
(chroot)
TCP/IP
ICE
MARS
Cloud Ga dgetron Host
SSH
ICE
100%
software
solution
MARS External Gadgetron HostTCP/IP
ICE
MARS
Conta inerized Gadgetr on
(chroot)
TCP/IP
ICE
MARS
Cloud Ga dgetron Host
SSH
ICE
MARS External Gadgetron HostTCP/IP
ICE
MARS
Conta inerized Gadgetr on
(chroot)
TCP/IP
ICE
MARS
Cloud Ga dgetron Host
SSH
ICE
Gadgetron processing is transparent to end user
Functionalities in toolboxes
Clinical applications (e.g. Cine: RealTime, Compressed sensing
T1/T2/T2* mapping
Fat-water separation
Cloud applications)
General algorithms
(e.g. KLT, Bspline, optimization for L1 norm, motion correction)
Basic data structures (array, matrix, image…)
Math (FFT, matrix/vector, …)
Application specific
algorithms
(e.g. epi correction, graph-cut fat water,
grappa, spirit)
Interfaces (to/from python, matlab)
Software infrastructure
Cross-platform compilation (linux, windows, mac)
Integration test, online
Docker based deployment
Live in open-source world
Data format (hdf5, Analyze …)
Commit
changes
8
Continuous development and global
deployment Every code changes has been tested before deployed in cloud
• Github, https://github.com/gadgetron/gadgetron
• Integration test, http://gtbuildbot.eastus.cloudapp.azure.com:8010/#/
Github Buildbot calls up
the integration test
Trigger
Windows
Linux
MacBook
Integration tests
on comprehensive
testing datasets
success
success
success
Docker
image
Fast global deployment of new technology
Gadgetron Deployment
map Users
Planned
London
2
1
2 2 3
5 1
22 sites
44 scanners
10
Generic recon chain in Gadgetron
Gadgetron – Generic chain
11
• What are the motivation?
Consolidate different gadgets for MR recon
Uniform “language” to specify technical “details” when we discuss MR
reconstruction
Support different clinical use cases and remains to be flexible,
configurable, high-performance
• What will this offer?
A set of common data structures
A set of gadgets following consistent convention
Use cases for different applications
Gadgetron – Generic chain
12
<!-- Data accumulation and trigger gadget --> <gadget> <name>AccTrig</name> <dll>gadgetron_mricore</dll> <classname>AcquisitionAccumulateTriggerGadget</classname> <property><name>trigger_dimension</name><value>Slice</value></property> <property><name>sorting_dimension</name><value></value></property> </gadget> <gadget> <name>BucketToBuffer</name> <dll>gadgetron_mricore</dll> <classname>BucketToBufferGadget</classname> <property><name>N_dimension</name><value>contrast</value></property> <property><name>S_dimension</name><value>average</value></property> <property><name>split_slices</name><value>false</value></property> <property><name>ignore_segment</name><value>true</value></property> <property><name>verbose</name><value>true</value></property> </gadget>
gadgetron/toolboxes/mri_core/mri_core_data.
h
gadgetron/toolboxes/mri_core/mri_core_acquisition_bucke
t.h
Gadgetron – Generic chain
13
<!-- Recon --> <gadget> <name>Recon</name> <dll>gadgetron_mricore</dll> <classname>GenericReconCartesianGrappaGadget</classname> <!-- image series --> <property><name>image_series</name><value>0</value></property> <!-- Coil map estimation, Inati or Inati_Iter --> <property><name>coil_map_algorithm</name><value>Inati</value></property> <!-- Down stream coil compression --> <property><name>downstream_coil_compression</name><value>true</value></property> <property><name>downstream_coil_compression_thres</name><value>0.01</value></property> <property><name>downstream_coil_compression_num_modesKept</name><value>0</value></property> <!-- parameters for debug and timing --> <property><name>debug_folder</name><value></value></property> <property><name>perform_timing</name><value>true</value></property> <property><name>verbose</name><value>true</value></property> <!-- whether to send out gfactor --> <property><name>send_out_gfactor</name><value>false</value></property> </gadget>
7D kspace to 7D image
Gadgetron – Generic chain
14
Other gadgets:
• SPiRIT and CS recon
• Coil compression
• Kspace filtering
• Partial fourier handling
• Image scaling and FOV
adjustment …
• Applications: cine, perfusion, t1
mapping, …
15
ISMRMRD data format
16
ISMRMRD data format is the backbone
0 5 10 15 20-500
0
500
CH
I (
RA
->LA
)
tpat3 RT Cine Bart Res160 Lighthouse FOV450 1200 (Start time: 15:49:38.8)
0 5 10 15 20-500
0
500
CH
II
(RA
->R
L)
0 5 10 15 20-500
0
500
CH
III
(LA
->R
L)
0 5 10 15 20-500
0
500
CH
IV
(LL->
RL)
MR Kspace data ECG/respiratory/gradient waveform
Image
XML meta data for
any information
(e.g. ROIs)
New
• https://github.com/ismrmrd/ismrmrd
• Support different data formats
17
ISMRMRD data format: C++/Python/Matlab
• https://github.com/ismrmrd/ismrmrd
• Stored in hdf5 file
• Scanner to ismrmrd data convertors can be downloaded at :
• Siemens: https://github.com/ismrmrd/siemens_to_ismrmrd
• GE: https://github.com/ismrmrd/ge_to_ismrmrd
• Philips: https://github.com/ismrmrd/philips_to_ismrmrd
• Bruker: https://github.com/ismrmrd/bruker_to_ismrmrd
• Load and work on ismrmrd data in:
• Matlab:
dset = ismrmrd.Dataset(filename, 'dataset’);
• Python:
import ismrmrd
f = ismrmrd.Dataset('./testdata.h5', '/dataset', True)
• C++
Gadgetron separated into toolboxes which
can be used externally.
Includes iterative and non-iterative
reconstructions.
Data preprocessing (noise adjustment, coil
compression, etc) not readily available, but
building blocks are.
18
Embedding Gadgetron in STIR
Create STIR Gadget for PET-preprocessing
Send PET data as ISMRMRD waveforms
19
Embedding STIR in Gadgetron
20
ISMRMRD Waveforms
21
ISMRMRD Waveforms
Recent development
22
GADGETRON
In cloud …
23
GADGETRON
• Unlimited computing resource
• Global availability
• Options for different
hardware/price combination
• Someone will do
maintenance/upgrade for us
Gadgetron at Microsoft Azure Motivation – Provide computing resource for advanced imaging techniques
24
Some imaging applications require a lot more computing cores and RAM
Avoid to purchase and maintain local computing devices
Money, space, staff, cooling,
gradually outdated hardware, …
Encrypted
connection
vs.
Our local rack Cloud
* East US data center, 08/06/2017
25
Gadgetron at Microsoft Azure
Traffic
monitor
Cloud
controller
Load
balancer
ssh
Data
streaming
Gadgetron
Gadgetron
Gadgetron
Gadgetron
…
Start/stop
computing
nodes
Pipe data to
nodes This is configurable,
such as:
• Number of computing nodes
• Computing node type
• Strategy to pipe data to
nodes
e.g. Cine: data from
every slice gets a node
~10x
speedup
on the
cloud
To cloud
26
Gadgetron at Cloud: Security
MARS External Gadgetron HostTCP/IP
ICE
MARS
Conta inerized Gadgetr on
(chroot)
TCP/IP
ICE
MARS
Cloud Ga dgetron Host
SSH
ICE
Passive security
• Patient identification
information stays in
scanner
• Only send kspace data
and minimal protocol
information (FOV, recon
matrix size etc.)
Active security
• SSH outbound tunnel
initiated by scanner – No
inbound connection
• RSA 4096-bit , encrypted
communication
• On demand connection –
close after each scan
*NIST recommended 3072-bit key
Machine Learning ready
27
GADGETRON
28
Data …
• Gadgetron network allows
efficient data collection
• Cloud gives us computing
power and inline training
ability
• Fast clinical deployment of
ML applications
• We know how data are
acquired
• C++ to/from python conversion
• Booth python
• Dock integration
Python interpreter
29
Develop ML applications in Gadgetron
C++
Gadgets Python
Gadget
Python
Gadget
C++
Gadgets
• Key data structures converters from/to python
• Docker images include CuDNN, tensorflow and PyTorch
• Very easy to deploy AI models on the scanner
30
Inline Cine analysis: Stop manual contouring
~200ms per image on cpu
~40ms per image on good
GPU
Contours are sent back to
scanner fully automated !
https://www.linkedin.com/pulse/gadgetron-brings-ai-mr-scanner-room-improve-cardiac-exam-hui-xue/
31
Future development
• Non-cartesian generic chain CPU/GPU regridding, kspace/image domain parallel imaging, different
spatial/temporal regularization, CS reconstruction
• Python binding for Gadgetron toolboxes Pybind11
```
import Gadgetron as gt
a = gt.wavelet(img, levels=4, wav_name=‘db2’)
```
• Enrich ISMRMRD data format to store labelled data/images
ROIs, landmarks, xml based reports etc…
• Enhanced matlab-Gadgetron integration
• and many more new toolboxes, library functions …
Imaging Computing and Recon
Analysis
32
Summary: Gadgetron, a data driven process
0 5 10 15 20-500
0
500
CH
I (
RA
->LA
)
tpat3 RT Cine Bart Res160 Lighthouse FOV450 1200 (Start time: 15:49:38.8)
0 5 10 15 20-500
0
500
CH
II
(RA
->R
L)
0 5 10 15 20-500
0
500
CH
III
(LA
->R
L)
0 5 10 15 20-500
0
500
CH
IV
(LL->
RL)
Data archive
Application
field Better images
Automation
Quantification etc…
Expert input,
Phenotype data
…
Model and
intelligence …
Raw
measurement
AI
