1

Dictionary- & Model-based methods (in quantitative MRI reconstruction)

Mariya DonevaPhilips Research, Hamburg

14.10.2019

2

Outline

• Brief introduction to MRI

• Image reconstruction as an inverse problem

• Model-based reconstruction for MR parameter mapping

• Dictionary-based methods

3

A very brief introduction to MRI

PolarizationPut a person in a strong magnetic field B0

ExcitationApply an RF pulse B1 to flip the magnetization in the transverse plane

Spatial encodingApply a magnetic field gradients

Acquisition„Listen“ to the signal using RF coils

ReconstructionDecode the measured signal to obtain an image

4

MRI data acquisition

Pulse sequence k-space

Spin echo

RF

Greadout

Gphase

Scan Time = TR × (No. phase encodes) ×(No. averages)

Data in k-space are acquired sequentially

TR

image space

Data acquisition is just a small portion of the MR scan

5

• Assumption: The measured signal is a Fourier transform of the image

Standard MRI reconstruction

• In many cases this model is too simplified

k-space image space

iFFT

6

Which factors influence the MR images?

T1 T2 PD

B0B1+/-

Diffusion

Perfusion

Chemical shift(Tissue composition)

Physiological motion

MRI signal strength is a function of the tissue parameters, system parameters, and sequence parameters

Magnetic susceptibility

TR TETI

…

Magnetization transfer

radial spiral

randomrosette lissajous

propeller

Many ways to sample the k-spacePrecise knowledge of gradient waveforms becomes important

7

Model-based Reconstruction

8

Forward model:

• 𝑦 measured data

• 𝐹 forward transform (linear or non-linear)

• 𝑥 image (T1W, T2W, parameter map,…)

• 𝑛 noise

Image reconstruction as an inverse problem

𝒚 = 𝑭𝒙 + 𝒏

Reconstruction problem: Recover 𝑥 based on the measurements 𝑦

Cause(parameter, unknown)

Effect(observation,

data)

inverse problem

forward problem

9

Well-posed/ill-posed inverse problems

Well-posed problem

• A solution exists

• The solution is unique

• The inverse mapping 𝒚 → 𝒙 is well conditioned

Image reconstruction in MRI is always an ill-posed inverse problem

• No exact solution (data inconsistencies, noise)

• Solution is not unique (incomplete data)

• Sometimes the problem is ill-conditioned (noise amplification)

10

• 𝑭𝒙 = 𝒚 has no exact solution

Least squares solution

• Solution is not unique, many choices of 𝒙 lead to the same 𝒚

Minimum norm solution

The Moore-Penrose pseudoinverse gives the minimum L2 norm solution

Solving ill-posed inverse problems

𝒙 = argmin 𝑭𝒙 − 𝒚 𝟐𝟐

minimize 𝒙subject to 𝑭𝒙 = 𝒚

11

• Ill-conditioned problem: add regularization

Regularized inverse problem

• Regularization adds prior knowledge to stabilize the solution

Regularization

𝒙 = argmin𝒙

𝑭𝒙 − 𝒚 𝟐𝟐 + 𝜆𝑅(𝒙)

data consistency regularization

Examples:

• L2 norm 𝑅 𝒙 = 𝒙 𝟐𝟐

• Total Variation 𝑅 𝒙 = Δ𝒙

• L1 norm 𝑅 𝒙 = 𝒙Propagated data error

Approximation error

𝝀

Total error

12

Modelling signal and system properties

• Ignoring many things, the measured signal in MRI is:

𝑦(𝑡) = 𝜌(𝑟)𝑒−2𝜋𝑖𝑘 𝑡 ∙𝑟 ⅆ𝑟

𝑦(𝑡) = 𝑒−𝑅2 𝑟 𝑡𝜌(𝑟) 𝑒−2𝜋𝑖𝑘 𝑡 ∙𝑟 ⅆ𝑟

𝑦(𝑡) = 𝑒𝑖Δ𝐵0 𝑟 𝑡𝜌(𝑟)𝑒−2𝜋𝑖𝑘 𝑡 ∙𝑟 ⅆ𝑟

• Include T2 relaxation

• Include off-resonance

• Include CSM 𝑦𝑗(𝑡) = 𝑐𝑗 𝑟 𝜌(𝑟)𝑒−2𝜋𝑖𝑘 𝑡 ∙𝑟 ⅆ𝑟

• Other effects: chemical shift, motion, flow, diffusion, …

• More accurate, more complete description of the measurements

• Allows estimating additional physical properties from the data

Model-based reconstruction for quantitative MR parameter mapping

14

Quantitative MRI• Improved tissue contrast

• Robust tissue segmentation

• Detection of diffuse disease

• Improved data consistency and comparability

• Synthetic MRI/ Single protocol exam

T2W FLAIR T1W

T1 map T2 map PD map

WM GM CSF

segmentation synthetic MRI

15

Generalized View on Quantitative MRI

• Collect multiple images for different acquisition parameters, each of which is “weighted” by a specific tissue property

• Use a model to extract the value of the tissue property from the weighted images

𝑠

𝑝

Parameter Map

Challenge: we need to acquire multiple images

Acquisition parameter 𝑝 𝑠 = 𝑓(𝑝, Θ) Θ

Solution: undersample the data (and use model-based reconstruction)

16

Model-based reconstruction for MR parameter mapping

Generalized signal model

𝑥 = 𝑓(𝑝, Θ)

k-p measurements

Acqusition parameters

Sampling operator

Θ = argmin1

2

𝑝

Φ𝑝𝑥𝑝(Θ) − 𝑦𝑝 2

2

Tissue parameters

Model-based reconstruction problem

Obtain the parameter maps directly from the undersampled data, solving a non-linear (and possibly non-convex) problem

Model-based reconstruction for MR parameter mapping

Θ = argmin1

2

𝑝

Φ𝑝𝑥𝑝 Θ − 𝑦𝑝 2

2+

𝑖

𝜆𝑖𝑅𝑖 𝑥𝑝, Θ

Add regularization

𝜌 𝑇2

= argmin1

2 𝑐 𝑇𝐸𝑗

𝑀𝑇𝐸𝑗ℱ𝐶𝑐𝜌 𝑟 𝑒−

𝑇𝐸𝑗

𝑇2 𝑟 − 𝑦𝑇𝐸𝑗,𝑐 2

2

+ 𝑖 𝜆𝑖𝑅𝑖 𝑥𝑝, Θ

Example: T2 mapping

Block KT et al. IEEE Trans Med Imaging 28 (2009): 1759-1769Sumpf TJ et al. JMRI 2011

Proton density R2 relaxivity

Radial 2D FSE sequence

32 x 16 echoes, ∆TE = 10 ms, TR = 7000 ms, 224 x 224 pixels (acceleration R=11)

Model based T2 mapping

Block KT et al. IEEE Trans Med Imaging 28 (2009): 1759-1769

Optimization performed with non-linear CG algorithm (CG-DESCENT)

Slide Courtesy of Dr. T Block

Proton density R2 relaxivity

Radial 2D FSE sequence

32 x 16 echoes, ∆TE = 10 ms, TR = 7000 ms, 224 x 224 pixels (acceleration R=11)

Model based T2 mapping

Block KT et al. IEEE Trans Med Imaging 28 (2009): 1759-1769

Optimization performed with non-linear CG algorithm (CG-DESCENT)

Slide Courtesy of Dr. T Block

Provides quantitative T2 & PD map from single radial FSE dataset

Radial Projections Consistent Model ResultModelMagnetization Preparation

&Look-Locker Acquisition

Termination

Criterion

n Single Projections

n Complete k-Spaces

Pixel-wise

Model Fit

Reinsert

Projections

Repeat Iteratively

Reconstruction Scheme[2]

[1] Tran-Gia, MRM 2013. [2]According to Doneva, MRM 2010. [3] Tran-Gia, PLOS ONE 2015.

20

↔ T1 map[3]

Slide Courtesy of Dr. Johannes Tran-Gia

MAP (Model-based Acceleration of Parameter mapping)

26

Model-based Regularization and Dictionaries

27

Model-based sparsity constraint in a compressed sensing reconstruction

1. Create a training data set using the model2. Learn a sparsifying transform from the training data 3. Use the constraint in the reconstruction

f(p;θ1)

p

f(p;θ2)

p

f(p;θ3)

p

f(p;θn)

p

s1 s2 s3 sn

Training dataset S = [s1, s2, s3,... sn]

Discrete vector of sampling locations p

Set of parameter values {θ1 ,...,θn}

Data modelf(p;θ1)

28

Data adapted sparse representations: Dictionaries

Dictionary Dx

s

= .

Each atom is a basic unit that is used to compose larger units atoms

0.6

0.4

29

Model-based sparsity constraint: Orthogonal transform

𝑥 = argmin1

2

𝑐

𝑝

𝑀𝑝ℱ𝐶𝑐𝑥𝑝 − 𝑦𝑝,𝑐 2

2+ 𝜆1 𝑈𝐻𝑥 1

𝑅 = 𝑆𝑆𝐻 = 𝑈Σ𝑈𝐻

𝑧 = 𝑈𝐻𝑥

• 𝑈𝐻 is a linear sparsifying transform for the measurements 𝑥

• Include the data model in the regularization term

• PCA-based constraint

30

Model-based subspace projection (REPCOM, T2 shuffling)

𝑥 = argmin1

2

𝑐

𝑝

𝑀𝑝ℱ𝐶𝑐𝑈𝑅𝑧𝑝 − 𝑦𝑇𝐸𝑗,𝑐 2

2

𝑅 = 𝑆𝑆𝐻 = 𝑈Σ𝑈𝐻

𝑧 = 𝑈𝑅𝐻𝑥

• 𝑈𝑅𝐻 projection to the subspace of the first R principal components

• Reconstruct the compressed image series

• PCA-based constraint

1) Huang et al MRM 2012 2) Tamir et al MRM 2017

31

Model-based sparsity constraint: Over-complete dictionary

• Improve the sparsity: use over-complete dictionary

Obtain overcomplete dictionary by training (K-SVD2)

basis Dx

=

s

.

Dictionary Dx

s

= .

1) Doneva et al MRM 2010 2) Aharon M et. al, IEEE Trans Signal Process 2006

Orthogonal transform(e.g. PCA)

Over-complete dictionary

minimize 𝑥 − 𝐷𝑠 2, s. t. 𝑠 0 ≤ 𝐾

32

Model-based sparsity constraint

Generate signal prototypes from

model

Dictionary training (K-SVD)

Apply dictionary in CS reconstruction

Original 2 Atoms 10 Atoms

Doneva et al MRM 2010

𝒙 = argmin𝒙

𝑭𝒙 − 𝒚 𝟐𝟐 + 𝝀 𝒙 − 𝑫𝒔 𝟐

𝟐,

s. t. 𝒔 0 ≤ 𝐾

Reconstruction

33

Undersampled T1 mapping (CS)

multi-echo IR brain data TE = 1.9ms, TR = 3.8ms, α = 10°, FOV = 250 mm, 224×224 matrix, 40 images

0.055 0.086 0.106 0.131

0.051 0.062 0.083 0.113

NRMSE

NRMSE

1x 2x 4x 6x 8x

Doneva et al MRM 2010

34

Dictionaries for higher dimensional signals

• Dictionaries for 2D image representation

– 2D image patches

Ravishankar S et al IEEE Trans Med Imag 2011; 30(5): 1028-41Caballero J et al. ISMRM 2014 #1560Katscher U et al ISMRM2017 #3641

minimize 𝑅𝑖𝑗𝑥 − 𝐷𝑠𝑖𝑗 2, s. t. 𝑠𝑖𝑗 0

≤ 𝐾

• Reconstruction

𝑅𝑖𝑗 - operator that extracts a patch centered at position i,j

Each atom in the dictionary is a 2D patch Image patch based dictionary

• Spatiotemporal dictionaries

– Spatio-temporal blocks (3D, 4D)

𝒙 = argmin𝒙

𝑭𝒙 − 𝒚 𝟐𝟐 + 𝝀

𝒊𝒋

𝑅𝑖𝑗𝑥 − 𝐷𝑠𝑖𝑗 𝟐

𝟐, s. t. 𝑠𝑖𝑗 0

≤ 𝐾

35

More Dictionary-based techniques(extremely sparse representation)

“In a dictionary with infinitely many atoms, the signal can be ultimately represented by a single atom. This is equivalent to fitting the signal to the model”

36

T2 mapping: model-based reconstruction with EMC

0

50

100

150

[ms]

22 min

Cartesian

Multi Spin-Echo Acquisitions

a e g

Single Spin-Echo

Cartesian

b f hT 2

Re

laxa

tio

n

Map

s

c

d 3:10 min

[Exponential fit] [EMC fit][Exponential fit]

Ben-Eliezer etl al, Magn Reson Med (2015) 73(2): 809-17.

Note: Parameter maps are discretized

0 2 4 6 8 10 12 140

5

10

15

20

25

30Echo train modulation

T2 = 30:2:55; 73:2:103

Echo train length = 13

29 simulations• The Echo Modulation Curve (EMC) Algorithm

1

• Compute signal evolution using Bloch simulations

Echo-modulation curves database EMC (T2, B1, …)

𝑦(𝑡) = 𝐸𝑀𝐶(𝐵1, 𝑇2)𝜌(𝑟) 𝑒−2𝜋𝑖𝑘 𝑡 ∙𝑟 ⅆ𝑟

• Fit experimental decay curve to simulated EMC database

37

Magnetic Resonance Fingerprinting

time (ms)

sign

al in

ten

sity

B0 map

M0 map

T2 map

T1 map

38

Magnetic Resonance Fingerprinting

full sampling undersamplingspiral read-out

Is the dictionary matching in MRF the same as model-based reconstruction for MR parameter mapping?

• Direct matching in MRF seems to work quite well, but it is only the first iteration of an iterative model-based reconstruction

• For long sequences and VD spiral sampling 1 iteration might be enough

• In the general case, iterative reconstruction is needed

39

Model-based reconstruction for MRF

• Davies M et al. A Compressed Sensing Framework for Magnetic Resonance Fingerprinting SIAM J. Imaging Sci., 2014, 7(4), 2623–2656.

• Pierre E et al. Multiscale Reconstruction for MR Fingerprinting MagnReson Med. 2016 Jun;75(6):2481-92

• Zhao B, et al. Maximum Likelihood Reconstruction for Magnetic resonance Fingerprinting IEEE Trans Med Imaging. 2016 Aug;35(8):1812-23. doi:10.1109/TMI.2016.2531640

• Doneva M et al. Matrix Completion-based reconstruction for undersampled magnetic resonance fingerprinting data Magn ResonImaging. 2017 Mar 3 doi:10.1016/j.mri.2017.02.007

• Assländer J et al. Low rank alternating direction method of multipliers reconstruction for MR fingerprinting Magn Reson Med. 2017 Mar 5. doi: 10.1002/mrm.26639.

direct matching iterative recon

40

MR-STATmulti-parametric model-based reconstruction (non-linear inversion)

Time-data Signal model

Slide Courtesy of Dr. Alessandro. Sbrizzi

T1

T2

True Recon

|B1+|

ΔB0

P.D.

Tx/Rxphase

True Recon

• Large scale non-linear inverse problem for all relevant tissue and system parameters

– Computationally very intensive

– Careful initialization is required

41

Summary

• Model-based reconstruction

– Generalized framework for MR reconstruction

– Modelling of physical properties, reduced artifacts, QMRI

– Insert prior knowledge as regularization

• Dictionaries

– Provide sparse(r) signal representation

– Convert a non-linear inverse problem to a linear search

– Can be learned from training data

• Examples in MR Parameter mapping

Acknowledgements

• Tobias Block

• Johannes Tran-Gia

• Noam Ben-Eliezer

• Alessandro Sbrizzi