1

Deep Learning Methods for Parallel MagneticResonance Image Reconstruction

Florian Knoll, Member, IEEE, Kerstin Hammernik, Chi Zhang, Student Member, IEEE, Steen Moeller,Thomas Pock, Daniel K. Sodickson, and Mehmet Akcakaya, Member, IEEE

Abstract—Following the success of deep learning in awide range of applications, neural network-based machinelearning techniques have received interest as a meansof accelerating magnetic resonance imaging (MRI). Anumber of ideas inspired by deep learning techniquesfrom computer vision and image processing have beensuccessfully applied to non-linear image reconstruction inthe spirit of compressed sensing for both low dose com-puted tomography and accelerated MRI. The additionalintegration of multi-coil information to recover missingk-space lines in the MRI reconstruction process, is stillstudied less frequently, even though it is the de-facto stan-dard for currently used accelerated MR acquisitions. Thismanuscript provides an overview of the recent machinelearning approaches that have been proposed specificallyfor improving parallel imaging. A general backgroundintroduction to parallel MRI is given that is structuredaround the classical view of image space and k-space basedmethods. Both linear and non-linear methods are covered,followed by a discussion of recent efforts to furtherimprove parallel imaging using machine learning, andspecifically using artificial neural networks. Image-domainbased techniques that introduce improved regularizers arecovered as well as k-space based methods, where the focusis on better interpolation strategies using neural networks.Issues and open problems are discussed as well as recentefforts for producing open datasets and benchmarks forthe community.

Index Terms—Accelerated MRI, Parallel Imaging, It-erative Image Reconstruction, Numerical Optimization,Machine Learning, Deep learning.

F. Knoll and D. K. Sodickson are with the Center for BiomedicalImaging, Department of Radiology, New York University. K.Hammernik is with the Department of Computing, Imperial CollegeLondon. T. Pock is with the Institute of Computer Graphics andVision, Graz University of Technology. C. Zhang and M. Akcakayaare with the Department of Electrical and Computer Engineering, andCenter for Magnetic Resonance Research, University of Minnesota,Minneapolis, MN. S. Moeller is with the Center for MagneticResonance Research, University of Minnesota, Minneapolis, MN.e-mails: florian.knoll@nyumc.org, k.hammernik@imperial.ac.uk,zhan4906@umn.edu, moeller@cmrr.umn.edu, pock@icg.tugraz.at,daniel.sodickson@nyumc.org, akcakaya@umn.edu. This work waspartially supported by NIH R01EB024532, NIH R00HL111410,NIH P41EB015894, NIH P41EB027061, NIH P41EB017183, NSFCAREER CCF-1651825

I. INTRODUCTION

During recent years, there has been a substantialincrease of research activity in the field of medicalimage reconstruction. One particular application area isthe acceleration of Magnetic Resonance Imaging (MRI)scans. This is an area of high impact, because MRI is theleading diagnostic modality for a wide range of exams,but the physics of its data acquisition process make itinherently slower than modalities like X-Ray, ComputedTomography or Ultrasound. Therefore, the shortening ofscan times has been a major driving factor for routineclinical application of MRI.

One of the most important and successful technicaldevelopments to decrease MRI scan time in the last 20years was parallel imaging [1–3]. All state of the artclinical MRI scanners from major vendors are equippedwith parallel imaging technology, and it is the defaultoption for a large number of scan protocols. As a conse-quence, there is a substantial benefit of using multi-coildata for machine learning based image reconstruction.Not only does it provide a complementary source ofacceleration that is unavailable when operating on singlechannel data, or on the level of image enhancement andpost-processing, it also is the scenario that ultimatelydefines the use-case for accelerated clinical MRI, whichmakes it a requirement for clinical translation of newreconstruction approaches. The drawback is that workingwith multi-coil data adds a layer of complexity thatcreates a gap between cutting edge developments in deeplearning [4] and computer vision, where the default datatype are images. The goal of this manuscript is to bridgethis gap by providing both a comprehensive review of theproperties of parallel MRI, together with an introductionhow current machine learning methods can be used forthis particular application.

A. Background on multi-coil acquisitions in MRI

The original motivation behind phased array receivecoils was to increase the SNR of MR measurements.These arrays consist of nc multiple small coil elements,where an individual coil element covers only a part ofthe imaging field of view. These individual signals are

2

Fig. 1: In k-space based parallel imaging methods, missing data is recovered first in k-space, followed by an inverse discreteFourier transform (IDFT) and combination of the individual coil elements. In image space based parallel imaging, the IDFTis performed as the first step, followed by coil sensitivity based removal of the aliasing artifacts from the reconstructed imageby solving an inverse problem.

then combined to form a single image of the completefield of view. The central idea of all parallel imagingmethods is to complement spatial signal encoding ofgradient fields with information about the spatial positionof these multiple coil elements. For multiple receivercoils, the MR signal equation can be written as follows

fj(kx, ky) =

∞∫−∞

∞∫−∞

u(x, y)cj(x, y)e−i(kxx+kyy) dx dy.

(1)In Equation (1), fj is the MR signal of coil j =1, . . . , nc, u is the target image to be reconstructed,and cj is the corresponding coil sensitivity. Parallelimaging methods use the redundancies in these multi-coilacquisitions to reconstruct undersampled k-space data.After discretization, this undersampling is described inmatrix-vector notation by

fj = FΩCju + nj , (2)

where u is the image to be reconstructed, fj is theacquired k-space data in the jth coil, Cj is a diagonalmatrix containing the sensitivity profile of the receiver

coil [2], FΩ is a partial Fourier sampling operator thatsamples locations Ω, and nj is measurement noise in thejth coil.

Historically, parallel imaging methods were put intwo categories: Approaches that operate in image do-main, inspired by the sensitivity encoding (SENSE)method [2] and approaches that operated in k-space,inspired by simultaneous acquisition of spatial harmonics(SMASH) [1] and generalized autocalibrating partialparallel acquisition (GRAPPA) [3]. This is conceptuallyillustrated in Figure 1. While these two schools ofthought are closely related, we organized this documentaccording to these classic criteria for historical reasons.

II. CLASSICAL PARALLEL IMAGING IN IMAGE SPACE

Classical parallel imaging in image space follows theSENSE method [2], which can be identified by two keyfeatures. First, the elimination of the aliasing artifactsis performed in image space after the application ofan inverse Fourier transform. Second, information aboutreceive coil sensitivities is obtained via precomputed,explicit coil sensitivity maps from either a separate

3

reference scan or from a fully sampled block of dataat the center of k-space (all didactic experiments thatare shown in this manuscript follow the latter approach).The reconstruction in image domain in Figure 1 showsthree example undersampled coil images, correspondingcoil sensitivity maps and the final reconstructed imagesfrom a brain MRI dataset. The coil sensitivities wereestimated using ESPIRiT [5].

MRI reconstruction in general and parallel imagingin particular can be formulated as an inverse problem.This provides a general framework that allows easyintegration of the concepts of regularized and constrainedimage reconstruction as well as machine learning that arediscussed in more detail in later sections. Equation (1)can be discretized and then written in matrix-vector form

f = Eu + n, (3)

where f contains all k-space measurement data pointsand E is the forward encoding operator that includesinformation about the sampling trajectory and the receivecoil sensitivities and n is measurement noise. The task ofimage reconstruction is to recover the image u. In classicparallel imaging the number of receive elements is usu-ally larger than the acceleration factor, and Equation (3)is solved in the least-squares-sense via the Pseudo-Inverse (E∗E)−1E, where E∗ denotes the conjugatetranspose of E. The reason why the acceleration factor issmaller than the number of coils is that these individualcoil elements do not measure completely independentinformation. This leads to an increase of the conditionnumber (E∗E)−1 and therefore, an ill-posed problem.This can lead to severe noise amplification in the recon-struction. In the original SENSE formulation [2], thisnoise amplification can be described exactly via the g-factor. In practice, Equation (3) is usually solved in aniterative manner, which is the topic of the followingsections.

A. Overview of conjugate gradient SENSE (CG-SENSE)

The original SENSE approach is based on equidis-tant or uniform Cartesian k-space sampling, where thealiasing pattern is defined by a point spread functionthat has a small number of sharp equidistant peaks.This property leads to a small number of pixels thatare folded on top of each other, which allows a veryefficient implementation [2]. When using alternative k-space sampling strategies like non-Cartesian acquisitionsor random undersampling, this is no longer possibleand image reconstruction requires a full inversion ofthe encoding matrix in Equation (3). This operation isdemanding both in terms of compute and memory re-quirements (the dimensions of E are the total number of

acquired k-space points times N2 where N is the size ofthe image matrix that is to be reconstructed), which leadto the development of iterative methods, in particularthe CG-SENSE method introduced by Pruessmann etal. as a follow up of the original SENSE paper [6].In iterative image reconstruction the goal is to find au that is a minimizer of the following cost function,which corresponds to the quadratic form of the systemin Equation (3)

u ∈ arg minu

1

2‖Eu− f‖22. (4)

In standard parallel imaging, E is linear and Equation (4)is a convex optimization problem that can be solved witha large number of numerical algorithms like gradientdescent, Landweber iterations or the conjugate gradientmethod. However, since MR k-space data are corruptedby noise, it is common practice to stop iterating beforetheoretical convergence is reached, which can be seenas a form of regularization. Regularization can be alsoincorporated via additional constraints in Equation (4),which will be covered in the next section.

As a didactic example for this manuscript, we willuse a single slice of a 2D coronal knee exam to illus-trate various reconstruction approaches. This data wereacquired on a clinical 3T system (Siemens Skyra) usinga 15 channel phased array knee coil. A turbo spinecho sequence was used with the following sequenceparameters: TR=2750ms, TE=27ms, echo train length=4,field of view 160mm2 in-plane resolution 0.5mm2, slicethickness 3mm. Readout oversampling with a factor of2 was used, and all images were cropped in the fre-quency encoding direction (superior-inferior) for displaypurposes. In the spirit of reproducible research, data,sampling masks and coil sensitivity estimations thatwere used for the numerical results in this manuscriptare available online1. Figure 3 shows an example of aretrospectively undersampled CG-SENSE reconstructionwith an acceleration factor of 4. The numeric tolerancewith respect to the norm of the normalized residual wasset to 5 · 10−5, which resulted in 10 CG iterations.

B. Nonlinear regularization and compressed sensing

As mentioned in Section II, the acceleration factorin classic parallel imaging is limited by the numberof independent channels of the receive coil array. Topush the boundaries of this limit, additional a-priori

1https://app.globus.org/file-manager?origin id=15c7de28-a76b-11e9-821c-02b7a92d8e58&origin path=\%2Fknee\%2F/: coronal proton density (pd) weighted data, subject 17, slice25.

4

knowledge can be introduced. This is achieved by ex-tending Equation (4) via additional penalty terms, whichresults in a constrained optimization problem defined inEquation (5), which forms the cornerstone of almost allmodern MRI reconstruction methods

u ∈ arg minu

1

2‖Eu− f‖22 +

∑i

λiΨi(u). (5)

Here, Ψi are dedicated regularization terms and λiare regularization parameters that balance the trade-offbetween data fidelity and prior. Since the introductionof compressed sensing and its adoption for MRI [7, 8],nonlinear regularization terms, in particular `1-normbased ones, are popular in image reconstruction and arecommonly used in parallel imaging [8–14]. The goalof these regularization terms is to provide a separationbetween the target image that is to be reconstructedfrom the aliasing artifacts that are introduced due toan undersampled acquisition. Therefore, they are usuallydesigned in conjunction with a particular data samplingstrategy. The classic formulation of compressed sensingin MRI [7] is based on sparsity of the images in atransform domain in combination with pseudo-randomsampling, which introduces aliasing artifacts that areincoherent in the respective domain. While waveletsare a popular choice for static imaging, sparsity in theFourier domain is commonly used for dynamic appli-cations, where periodic motion is encountered. Totalvariation based methods have been used successfullyin combination with radial and spiral acquisitions aswell as in dynamic imaging. More advanced regularizersbased on low-rank properties have also been utilized. Incontrast to linear reconstructions, where the quality ofa reconstruction can be assessed via SNR and g-factormaps, the evaluation of image quality is not trivial in thecontext of nonlinear reconstructions. Noise is suppressedat the cost of introducing a bias from the nonlinearregularizer. Therefore, it is generally not recommendedto use SNR-based metrics as a measure of image quality.Image quality is therefore usually estimated with met-rics like NRMSE, SSIM or PSNR, which compare areconstruction with a reference gold standard, ideally afully sampled reconstruction. However, this is generallyonly possible in a research setting, and image qualityevaluation of nonlinear reconstruction methods withouta reference is still an open research problem in the field.

Figure 3 shows an example of a nonlinear combinedparallel imaging and compressed sensing reconstructionwith a Total Generalized Variation [10] constraint. Theraw data was scaled such that the maximum magnitudevalue in k-space is 1 and the regularization parameterλ was set to 2.5 · 10−5 and the reconstruction was

using 1000 primal-dual iterations. While it is usuallyrecommended to use a pseudo-random acquisition whencombining parallel imaging with compressed sensing, wechose equidistant sampling for our experiments here forconsistency with classic parallel imaging reconstructionmethods. A more detailed discussion of this is providedin Section IV and in [15]. The nonlinear regularizationstill provides a superior reduction of aliasing artifactsand noise suppression in comparison to the CG-SENSEreconstruction from the last section.

III. CLASSICAL PARALLEL IMAGING IN K-SPACE

Parallel imaging reconstruction can also be formulatedin k-space as an interpolation procedure. The initial con-nections between the SENSE-type image-domain inverseproblem approach and k-space interpolation has beenmade more than a decade ago [16], where it was notedthat the forward model in Equation (3) can be restatedin terms of the Fourier transform, κ of the combinedimage, u as

f = EF∗κ , Gacqκ, (6)

where E is the forward encoding operator and F is thediscrete Fourier transform (DFT) matrix as before, fcorresponds to the acquired k-space lines across all coils,and Gacq is a linear operator. Similarly, the unacquiredk-space lines across all coils can be formulated using

funacq = Gunacqκ. (7)

Combining these two equations yield

funacq = GunacqG†acqf . (8)

Thus, the unacquired k-space lines across all coils can beinterpolated based on the acquired lines across all coils,assuming the pseudo-inverse, G†acq, of Gacq exists [16].Thus, the main difference between the k-space parallelimaging methods and the aforementioned image domainparallel imaging techniques is that the former producesk-space data across all coils at the output, whereas thelatter typically produces one image that combines theinformation from all coils.

A. Linear k-space interpolation in GRAPPA

The most clinically used k-space reconstructionmethod for parallel imaging is GRAPPA, which useslinear shift-invariant convolutional kernels to interpolatemissing k-space lines using uniformly-spaced acquiredk-space lines [3]. For the jth coil k-space data, κj , wehave

5

κj(kx, ky −m∆ky)

=

nc∑c=1

Bx∑bx=−Bx

By∑by=−By

gj,m(bx, by, c)

κc(kx − bx∆kx, ky −Rby∆ky), (9)

where R is the acceleration rate of the uniformly under-samped acquisition; m ∈ 1, . . . , R− 1; gj,m(bx, by, c)are the linear convolutional kernels for estimating themth spacing location in the jth coil; nc is the number ofcoils; and Bx, By are parameters determined from theconvolutional kernel size. A high-level overview of suchinterpolation is shown in the reconstruction in k-spacesection of Figure 1.

Similar to the coil sensitivity estimation inSENSE-type reconstruction, the convolutional kernelsgj,m(bx, by, c) are estimated for each subject, from eithera separate reference scan or from a fully-sampled blockof data at the center of k-space, called autocalibratingsignal (ACS) [3]. A sliding window approach is usedin this calibration region to identify the fully-sampledacquisition locations specified by the kernel size andthe corresponding missing entries. The former, takenacross all coils, is used as rows of a calibration matrix,A; while the latter, for a specific coil, yields a singleentry in the target vector, b. Thus for each coil j andmissing location m ∈ 1, . . . , R − 1, a set of linearequations are formed, from which the vectorized kernelweights gj,m(bx, by, c), denoted gj,m, are estimatedvia least squares, as gj,m ∈ arg ming ||b−Ag||22.GRAPPA has been shown to have several favorableproperties compared to SENSE, including lower g-factors, sometimes even less than unity in parts of theimage, and more smoothly varying g-factor maps [17].

B. Advances in k-space interpolation methods

Though GRAPPA is widely used in clinical practice, itis a linear method that suffers from noise amplificationbased on the coil geometry and acceleration rate [2].Therefore, several strategies have been proposed in theliterature to reduce the noise in reconstruction.

Iterative self-consistent parallel imaging reconstruc-tion (SPIRiT) is a strategy for enforcing self-consistencyamong the k-space data in multiple receiver coils by ex-ploiting correlations between neighboring k-space points[9]. Similar to GRAPPA, SPIRiT also estimates a linearshift-invariant convolutional kernel from ACS data. InGRAPPA, this convolutional kernel used informationfrom acquired lines in a neighborhood to estimate amissing k-space point. In SPIRiT, the kernel includes

contributions from all points, both acquired and missing,across all coils for a neighborhood around a given k-space point. The self-consistency idea suggests that thefull k-space data should remain unchanged under thisconvolution operation. The SPIRiT objective functionalso includes a term that enforces consistency with theacquired data, where the undersampling can be per-formed with arbitrary patterns, including random patternsthat are typically employed in compressed sensing [7, 8].Additionally, this formulation allows for incorporationof regularizers, for instance based on transform-domainsparsity, in the objective function to reduce reconstruc-tion noise via non-linear processing in a method called`1-SPIRiT [9]. Furthermore, SPIRiT has facilitated theconnection between coil sensitivities used in image-domain parallel imaging methods and the convolutionalkernels used in k-space methods via a subspace analysisin a method called ESPIRiT [5]. It was shown that thek-space based `1-SPIRiT and coil sensitivity-based `1-ESPIRiT perform similarly.

An alternative line of work utilizes non-linear k-spaceinterpolation for estimating missing k-space points foruniformly undersampled parallel imaging acquisitions[18]. It was noted that during GRAPPA calibration,both the regressand and the regressor have errors inthem due to measurement noise in the acquisition ofcalibration data, which leads to a non-linear relation-ship in the estimation. Thus, the reconstruction method,called non-linear GRAPPA (NL-GRAPPA), uses a ker-nel approach to map the data to a higher-dimensionalfeature space, where linear interpolation is performed,which also corresponds to a non-linear interpolation inthe original data space. The interpolation function isestimated from the ACS data, although this approachtypically required more ACS data than GRAPPA [18].This method was shown to reduce reconstruction noisecompared to GRAPPA. Note that NL-GRAPPA, throughits use of the kernel approach, is a type of machinelearning approach, though the non-linear kernel functionswere empirically fixed a-priori and not learned from data.In another line of work, GRAPPA regularization duringcalibration was explored using a sparsity-promoting [19]approach. These approaches use regularization in calibra-tion followed by a one-step reconstruction. However, wenote that this way is different than than the regulariza-tion in `1-SPIRiT, which uses the regularization duringreconstruction in an iterative manner.

C. Low-rank matrix completion for k-space reconstruc-tion

While k-space interpolation methods remain the preva-lent method for k-space parallel imaging reconstruction,

6

there has been recent efforts on recasting this type ofreconstruction as a matrix completion problem. Simul-taneous autocalibrating and k-space estimation (SAKE)is an early work in this direction, where local neigh-borhoods in k-space across all coils are restructuredinto a matrix with block Hankel form [20]. Then low-rank matrix completion is performed on this matrix,subject to consistency with acquired data, enabling k-space parallel imaging reconstruction without additionalcalibration data acquisition. Low-rank matrix modelingof local k-space neighborhoods (LORAKS) is anothermethod exploiting similar ideas, where the motivationis based on utilizing finite image support and imagephase constraints instead of correlations across multiplecoils, and which was also extended to parallel imaging tofurther include the similarities between image supportsand phase constraints across coils [21]. A further gen-eralization to LORAKS is annihilating filter-based lowrank Hankel matrix approach (ALOHA), which extendsthe finite support constraint to transform domains [22].By relating transform domain sparsity to the existenceof annihilating filters in a weighted k-space, where theweighting is determined by the choice of transformdomain, ALOHA recasts the reconstruction problem asthe low-rank recovery of the associated Hankel matrix.

IV. MACHINE LEARNING METHODS FOR PARALLEL

IMAGING IN IMAGE SPACE

The use of machine learning for image-based par-allel MR imaging evolves naturally from Equation (5)based on the following key insights. First, in classiccompressed sensing, Ψ are a general regularizers likethe image gradient or wavelet transforms, which werenot designed specifically with undersampled parallelMRI acquisitions in mind. These regularizers can begeneralized to models that have a higher computationalcomplexity. Ψ can be formulated as a convolutionalneural network (CNN), where the model parameters canbe learned from training data inspired by the conceptsof deep learning [4], as illustrated in Figure 2. This wasalready demonstrated earlier in the context of computervision with a non-convex regularizer of the followingform

Ψ(u) =

Nk∑i=1

〈ρi(Kiu),1〉. (10)

The regularizer in Equation (10) consists of Nk terms ofnon-linear potential functions ρi, and Ki are convolutionoperators. 1 indicates a vector of ones. The parameters ofthe convolution operators and the parametrization of thenon-linear potential functions form the free parametersof the model, which are learned from training data.

Fig. 2: Illustration of machine learning-based image recon-struction. The network architecture consists of S stages thatperform the equivalent of gradient descent steps in a classiciterative algorithm. Each stage consists of a regularizer anda data consistency layer. Training the network parameters Θis performed by retrospectively undersampling fully sampledmulti-coil raw k-space data and comparing the output ofthe network uS

d (Θ) to a target reference reconstruction urefd

obtained from the fully sampled data for the current trainingexample d.

The second insight is that the iterative algorithm that isused to solve Equation (5) naturally maps to the structureof a neural network, where every layer in the networkrepresents an iteration step of a classic algorithm. Thisfollows naturally from gradient descent for the leastsquares problem in Equation (4) that leads to the iterativeLandweber method. After choosing an initial u0, theiteration scheme is given by Equation (11)

us = us−1 − αsE∗(Eus−1 − f), s > 0. (11)

E∗ is the adjoint of the encoding operator and αs is thestep size of iteration s. Using this iteration scheme tosolve the reconstruction problem in Equation (5) withthe regularizer defined in Equation (10) leads to theupdate scheme defined in Equation (12), which forms thebasis of recently proposed image space based machinelearning methods for parallel MRI

us = us−1 − αs

( Nk∑i=1

(Ki)>ρ′i(Kiu

s−1)

+ λsE∗(Eus−1 − f)

). (12)

This update scheme can then be represented as a neuralnetwork with S stages corresponding to S iteration stepsin Equation (12). ρ′i are the first derivatives of the non-linear potential functions ρi, which are represented as ac-tivation functions in the neural network. The transposed

7

Fig. 3: Comparison of image-domain based parallel imaging reconstructions of a retrospectively accelerated coronal kneeacquisition. The used sampling pattern, zero-filling, CG-SENSE, combined parallel imaging and compressed sensing with aTGV-constraint, and learned reconstructions using the Variational Network and MoDL architectures are shown, along withtheir NRMSE and SSIM values to the fully sampled reference. See text in the respective sections for details on the individualexperiments.

convolution operations K>i correspond to convolutionswith filter kernels rotated by 180 degrees. The idea of thevariational network [15] follows the structure of classicvariational methods and gradient-based optimization, andthe network architecture is designed to mimic a classiciterative image reconstruction. Since this convolutionalsteps in this architecture are shallow, the network hasa small receptive field (11 × 11 convolution kernelswere used in [15, 23]) and the network cannot cap-ture global image information. The regularizer can bemodified by including elements like pooling layers, up-convolutions and skip-connections, following the popu-lar U-Net model [24]. This extends the computationalcapacity of the regularizer and given sufficient trainingdata samples, will generally improve the performance ofthe model. However, it comes at the cost of loosing thedirect connection to gradient based optimization, whichmakes the approach less interpretable. It can also lead tooverfitting in situations where only a small set of trainingdata is available. The method from Aggarwal et al. [25]is an example for such an approach. It follows a similardesign concept as the VN, but uses a deeper and morecomplex regularizer. In order to limit the total number ofmodel parameters, the same set of weights is used for allstages of the network. The other major difference is thatin contrast to the gradient in the VN (see Equation (11)),it realizes the data term via a proximal mapping that isimplemented as an unrolled conjugate-gradient scheme.

An experiment that compares the properties of imagespace based machine learning for parallel MRI to CG-

SENSE and constrained reconstructions from the previ-ous sections is shown in Figure 3. The architecture andtraining of the VN exactly follows the description in [15]and the model consists of 131,050 model parameters thatare different for all 10 stages in the network architecture.The MoDL formulation and training is a modification ofthe approach in [25]. The same learned deep learningregularizer was repeated across all 10 stages. In contrastto the approach described in the paper, the regularizerwas modified to a U-Net [24] with a total of 694,021model parameters. The publicly available multi-channelknee data that was described in Section II was used totrain the networks. The source for the approaches thatare used in these experiments is available online2,3,4. Thefigure shows a single slice of a test case that was notused during training. The normalized root mean sum ofsquares error (NRMSE) and structural similarity index(SSIM) to the fully sampled reference is shown next toeach reconstruction. The experiments illustrates the im-proved performance as the complexity of the regularizermodel is increased. Equidistant Cartesian sampling fromtraditional parallel imaging was used in all experimentsbecause this type of sampling is predominantly usedin clinical practice. For combined parallel imaging andcompressed sensing, pseudo-random sampling is gener-ally recommended to improve performance in the liter-

2https://cai2r.net/resources/software3https://github.com/VLOGroup/mri-variationalnetwork4https://github.com/hkaggarwal/modl

8

ature [7]. An analysis of the influence of the samplingscheme is not included here due to space constraints.Such a study was performed in [15] and showed thatthe improvement in image quality with the learningapproach was independent of the used sampling scheme.To determine the model parameters of the network thatwill perform the parallel imaging reconstruction task, anoptimization problem needs to be defined that minimizesa training objective. In general, this can be formulatedin a supervised or unsupervised manner. Supervisedapproaches are predominantly used while unsupervisedapproaches are still a topic of ongoing research. There-fore, we will focus on supervised approaches for theremainder of this section. We define the number ofstages, corresponding to gradient steps in the network, asS. d is the current training image out of the complete setof training data D. The variable Θ contains all trainableparameters of the reconstruction model. The trainingobjective then takes the following form

L(Θ) = minΘ

1

2D

D∑d=1

‖uSd (Θ)− uref

d ‖22. (13)

As it is common in deep learning, Equation (13) isa non-convex optimization problem that is solved withstandard numerical optimizers like stochastic gradientdescent. This requires the computation of the gradientof the training objective with respect to the modelparameters Θ. This gradient can be computed via back-propagation

∂L(Θ)

∂Θs=∂us+1

∂Θs· ∂u

s+2

∂us+1. . . · ∂uS

∂uS−1· ∂L(Θ)

∂uS. (14)

The basis of supervised approaches is the availabilityof a target reference reconstruction uref. This requires theavailability of a fully-sampled set of raw phased arraycoil k-space data. This data is then retrospectively under-sampled by removing k-space data points as defined bythe sampling trajectory in the forward operator E andserves as the input of the reconstruction network. Fortraining example d, the current output of the networkuDd (Θ) is then compared to the reference uref

d via anerror metric. The choice of this error metric has aninfluence on the properties of the trained network, whichis a topic of currently ongoing work. A popular choiceis the mean squared error (MSE), which was also usedin Equation (13). Other choices are the `1 norm of thedifference and the SSIM.

The focus of this section was machine learning basedextensions of combined parallel imaging and compressedsensing, where the machine learning was mainly usedto learn a model that serves as a more complex reg-ularizer. Another set of developments in image space

based parallel imaging is focused on the improvementof the estimation of the coil sensitivity maps via jointestimation of image content and coil sensitivities. Thefirst in a recent set developments in that directionwas proposed in [26]. This approach first performs theIDFT of the undersampled multi-channel k-space (seethe illustration in the left column of Figure 1). Theneural network is then trained to learn the mapping ofthe aliased individual coil images to the combined un-aliased image. The network thus learns how to use thesensitivity information to perform the de-aliasing withoutusing explicit coil sensitivity maps. The authors useda classic fully connected multi-layer-perceptron for thistask. The use of fully connected networks is usuallychallenging for clinically relevant image sizes due tomemory requirements, for this particular application itwas possible by performing de-aliasing separately foreach 1D line in image space in the phase encodingdirection. A more general version of this approach wasrecently proposed in [27]. The authors use a CNN, whicheliminates the memory issue and allows them to use theproposed approach for 3D time-of-flight angiography.

V. MACHINE LEARNING METHODS FOR PARALLEL

IMAGING IN K-SPACE

There has been a recent interest in using neuralnetwork to improve the k-space interpolation techniquesusing non-linear approaches in a data-driven manner.These newer approaches can be divided into two groupsbased on how the interpolation functions are trained.The first group uses scan-specific ACS lines to trainneural networks for interpolation, similar to existinginterpolation approaches, such as GRAPPA or non-linearGRAPPA. The second group uses training databases,similar to the machine learning methods discussed inimage domain parallel imaging.

Robust artificial-neural-networks for k-space interpo-lation (RAKI) is a scan-specific machine learning ap-proach for improved k-space interpolation [28]. Thisapproach trains CNNs on ACS data, and uses these forinterpolating missing k-space points from acquired ones.The interpolation function can be represented by

κj(kx, ky −m∆ky) = fj,m(κc(kx − bx∆x,

ky −Rby∆y)bx∈[−Bx,Bx],by∈[−By,By],c∈[1,nc]),(15)

where fj,m is the interpolation rule implemented viaa multi-layer CNN for outputting the k-space of themth set of uniformly spaced missing lines in the jth

coil, R is the undersampling rate, Bx, By are parametersspecified by the receptive field of the CNN, and nc is the

9

Fig. 4: A slice from a high-resolution (0.6 mm isotropic)7T brain acquisition, where all acquisitions were performedwith prospective acceleration. It is difficult to acquire fully-sampled reference datasets for training for such acquisitions,thus two scan-specific k-space methods were compared. TheCNN-based RAKI method visibly reduced noise amplificationcompared to the linear GRAPPA reconstruction. NL-GRAPPAand RAKI have similar noise properties, while RAKI producesa slightly sharper image at R = 6.

number of coils. Thus, the premise of RAKI is similarto GRAPPA, while the interpolation function is imple-mented using CNNs, whose parameters are learned fromACS data with an MSE loss function. The scan-specificnature of this method is attractive since it requiresno training databases, and can be applied to scenarioswhere a fully-sampled gold reference cannot be acquired,for instance in perfusion or real-time cardiac MRI,or high-resolution brain imaging. Example RAKI, NL-GRAPPA and GRAPPA reconstructions for such high-resolution brain imaging datasets, which were acquiredwith prospective undersampling are shown in Figure 4.These data were acquired on a 7T system (SiemensMagnex Scientific) with 0.6 mm isotropic resolution.R = 5, 6 data were acquired with two averages forimproved SNR to facilitate visualization of any residualartifacts. Other imaging parameters are available in [28].For these datasets, RAKI leads to a reduction in noiseamplification compared to GRAPPA. Note the noise re-duction here is based on exploiting properties of the coilgeometry, and not on assumptions about image structure,as in traditional regularized inverse problem approaches,

Fig. 5: Comparison of k-space parallel imaging reconstruc-tions of a retrospectively accelerated coronal knee acquisition,as in Figure 3. Due to the small size of the ACS data relativeto the acceleration rate, the methods, none of which utilizestraining databases, exhibit artifacts. GRAPPA has residualaliasing, whereas SPIRiT shows noise amplification. Theseare reduced in RAKI, though the residual artifacts remain.Respective NRMSE and SSIM values reflect these visualassessment.

as in Section II-B. However, the scan-specificity alsocomes with downsides, such as the computational burdenof training for each scan, as well as the requirement fortypically more calibration data. In this dataset, RAKI andNL-GRAPPA have similar performance for R = 4, 5.At R = 6, RAKI preserves sharper details comparedto NL-GRAPPA, although the differences are subtle.In Figure 5, reconstructions of the knee dataset fromFigure 3 are shown, where all methods, which relyon subject-specific calibration data, exhibit a degree ofartifacts, due to the small size of the ACS region, whileRAKI has the highest SSIM and lowest NRMSE.

While originally designed for uniform undersamplingpatterns, this method has been extended to arbitrarysampling, building on the self-consistency approach ofSPIRiT [29]. Additionally, recent work has also refor-mulated this interpolation procedure as a residual CNN,with residual defined based on a GRAPPA interpolationkernel [30]. Thus, in this approach called residual RAKI(rRAKI), the CNN effectively learns to remove the noiseamplification and artifacts associated with GRAPPA,giving a physical interpretation to the CNN output, whichis similar to the use of residual networks in image de-noising. An example application of the rRAKI approachin simultaneous multi-slice (SMS) imaging [31] with anSMS factor of 16, i.e. a 16-fold acceleration in coverageis shown in Figure 6, showing improvement over bothGRAPPA and NL-GRAPPA in reducing residual aliasingand noise amplification. We note that for all experimentsshown in this section, the same number of ACS lineswere used for all methods. Thus, the differences betweenmethods are not due to the size of the calibration data.

A different line of work, called DeepSPIRiT, exploresusing CNNs trained on large databases for k-spaceinterpolation with a SPIRiT-type approach [32]. Since

10

Fig. 6: Reconstruction results of simultaneous multi-sliceimaging of 16 slices in fMRI (i.e. 16-fold acceleration incoverage), where a sample of 3 slices are shown. GRAPPAmethod exhibits noise amplification at this high accelerationrate. NL-GRAPPA reduces noise amplification but suffersfrom residual aliasing and leakage. The rRAKI method, whichconsists of a linear convolutional component G, in parallelwith a non-linear CNN component F that learns the artifactsarising from G, exhibits exhibits reduced noise and reducedaliasing. Due to imperfections in the ACS data for this applica-tion, the residual component includes both noise amplificationand residual artifacts.

sensitivity profiles and number of coils vary for differentanatomies and hardware configurations, k-space data inthe database were normalized using coil compressionto yield the same number of channels [33]. Coil com-pression methods effectively capture most of the energyacross coils in a few virtual channels, with the firstvirtual channel containing most of the energy, the secondbeing the second most dominant, and so on, in a mannerreminiscent of principal component analysis. Followingthis normalization of the k-space database, CNNs aretrained for interpolating different regions of k-space. Themethod was shown to remove aliasing artifacts, thoughdifficulty with high-resolution content was noted. SinceDeepSPIRiT trains interpolation kernels on a database,

it does not require calibration data for a given scan,potentially reducing acquisition time further.

Neural networks have also been applied to the Hankelmatrix based approaches in k-space [34]. Specifically, thecompletion of the weighted k-space in ALOHA methodhas been replaced with a CNN, trained with an MSE lossfunction. The method was shown to not only improve thecomputational time, but also the reconstruction qualitycompared to original ALOHA by exploiting structuresbeyond low-rankness of Hankel matrices. In another lineof work, neural networks have been applied to a Hankelmatrix based approach that models signals as piecewisesmooth [35]. These methods are described in more detailin another article in this issue [36].

VI. DISCUSSION

A. Issues and open problems

Several advantages of neural network based machinelearning approaches over classic constrained reconstruc-tion using predefined regularizers have been proposed inthe literature. First, the regularizer is tailored to a specificimage reconstruction task, which improves the removalof residual artifacts. This becomes particularly relevantin situations where the used sampling trajectory doesnot fulfill the incoherence requirements of compressedsensing, which is often the case for clinical parallelimaging protocols. Second, machine learning approachesdecouple the compute-heavy training step from a leaninference step. In medical image reconstruction, it iscritical to have images of diagnostic quality availableimmediately after the scan so that technologists andradiologists can decide immediately whether a certainsequence needs to be repeated or acquisition parametersneed to be changed. In contrast, prolonged trainingprocedures that can be done on specialized computinghardware, are generally acceptable. For example, thetraining of the VN reconstruction in the experiment inFigure 3 took 40 hours for 150 epochs with 200 slices oftraining data on a single NVIDIA M40 GPU with 12GBof memory. Training data, model and training parametersexactly follow the training from [23]. Reconstruction ofone slice then took 200ms, which is compatible withcurrent image reconstruction times of algorithms likeSENSE and GRAPPA on clinical scanners, and thereforecompatible with clinical workflow. In comparison, thecomputation times were 10ms for zero filling, 150msfor CG-SENSE and 10000ms for the PI-CS TGV con-strained reconstructions on the same GPU hardware.

The focus in Section IV and Section V were onmethods that were developed specifically in the con-text of parallel imaging. Some architectures for image

11

domain machine learning have been designed specifi-cally towards a target application, for example dynamicimaging [37]. Another approach was recently developedthat combines k-space and image space CNN data pro-cessing [38]. In their current form, these methods werenot yet demonstrated in the context of multi-coil data.The approach recently proposed by Zhu et al. learnsthe complete mapping from k-space raw data to thereconstructed image [39]. The proposed advantage is thatsince no information about the acquisition is includedin the forward operator E, it is more robust againstsystematic calibration errors during the acquisition. Thiscomes at the price of a significantly higher number ofmodel parameters. The corresponding memory require-ments make it challenging to use this model for matrixsizes that are currently used in clinical applications.A systematic comparison of these recent approachesfrom the literature is still an open question in the field.However, a fair comparison is challenging because theirperformance also depends on the quality the trainingdata. Most approaches are also designed with a particularset of target applications in mind, and different researchgroups usually build their own data sets as part of theirdevelopments. Thus, it can be a non-trivial task to modifya particular approach so that the performance is optimalfor a new type of data. For example, in approachesthat are designed either for dynamic or static imag-ing, the regularizer models are tailored to the specificproperties of these data. We also note that there arefewer works in k-space machine learning methods forMRI reconstruction. This may be due to the differentnature of k-space signal that usually has very differentintensity characteristics in the center versus the outer k-space, which makes it difficult to generalize the plethoraof techniques developed in computer vision and imageprocessing that exploit properties of natural images.

Machine learning reconstruction approaches alsocome with a number of drawbacks when compared toclassic constrained parallel imaging. First, they requirethe availability of a curated training data set that isrepresentative so that the trained model generalizes tonew unseen test data. While recent approaches fromthe literature [15, 25, 37] have either been trained withhundreds of examples rather than millions of examplesas it is common in deep learning for computer vision,or trained on synthetic non-medical data that is publiclyavailable from existing databases. However, this is stilla challenge that will potentially limit the use of machinelearning to certain applications. Several applications inimaging of moving organs, such as the heart, or inimaging of the brain connectivity, such as diffusionMRI, cannot be acquired with fully-sampled data due to

constraints on spatio-temporal resolutions. This hindersthe use of fully-sampled training labels for such datasets,highlighting applications for scan-specific approaches orunsupervised training strategies.

These reconstruction methods also require the avail-ability of computing resources during the training stage.This is a less critical issue due to the increased avail-ability and reduced prices of GPUs. The experimentsin this paper were made with computing resources thatare available for less than 10,000 USD, which areusually available in academic institutions. In addition, theavailability of on-demand cloud-based machine learningsolutions is constantly increasing.

A more severe issue is that in contrast to conven-tional parallel imaging and compressed sensing, machinelearning models are mostly non-convex. Their properties,especially regarding their failure modes and generaliza-tion potential for daily clinical use, are understood lesswell than conventional iterative approaches based onconvex optimization. For example, it was recently shownthat while reconstructions generalize well with respectto changes in image contrast between training and testdata, they are susceptible towards systematic deviationsin SNR [23]. It is also still an open question how specifictrained models have to be. Is it sufficient to train asingle model for all types of MR exams, or are separatemodels required for scans of different anatomical areas,pulse sequences, acquisition trajectories and accelerationfactors as well as scanner manufacturers, field strengthsand receive coils? While pre-training a large number ofseparate models for different exams would be feasiblein clinical practice, if certain models do not generalizewith respect to scan parameter settings that are usuallytailored to the specific anatomy of an individual patientby the MR technologist, this will severely impact theirtranslational potential and ultimately their clinical use.More generally speaking, an improved understandingof neural network training and architecture design tooptimization theory is a very active research topic in thefield of machine learning. We expect that future researchin similar directions will further bridge the gap betweencurrent experimental results and the underlying theoryand lead to a better understanding of generalizationproperties, failure modes in worst case scenarios andarchitecture design for specific types of problems.

B. Availability of training databases and communitychallenges

As mentioned in the previous section, one open issuein the field of machine learning reconstruction for paral-lel imaging is the lack of publicly available databases of

12

multi-channel raw k-space data. This restricts the numberof researchers who can work in this field to those whoare based at large academic medical centers where thisdata is available, and for the most part excludes thecore machine learning community that has the necessarytheoretical and algorithmic background to advance thefield. In addition, since the used training data becomes anessential part of the performance of a certain model, it iscurrently almost impossible to compare new approachesthat are proposed in the literature with each other ifthe training data is not shared when publishing themanuscript. While the momentum in initiatives for publicreleases of raw k-space data is growing [40], the numberof available data sets is still on the order of hundredsand limited to very specific types of exams. Examplesof publicly available rawdata sets are mridata.org5 andthe fastMRI dataset6.

VII. CONCLUSION

Machine learning methods have recently been pro-posed to improve the reconstruction quality in parallelimaging MRI. These techniques include both imagedomain approaches for better image regularization andk-space approaches for better k-space completion. Whilethe field is still in its development, there are manyopen problems and high-impact applications, which arelikely to be of interest to the broader signal processingcommunity.

ACKNOWLEDGMENTS

The authors thank Hemant Aggarwal and MathewsJacob for providing the MODL reconstructions.

REFERENCES

[1] D. K. Sodickson and W. J. Manning, “SimultaneousAcquisition of Spatial Harmonics (SMASH): FastImaging with Radiofrequency Coil Arrays,” MagnReson Med, vol. 38, no. 4, pp. 591–603, 1997.

[2] K. P. Pruessmann, M. Weiger, M. B. Scheidegger,and P. Boesiger, “SENSE: Sensitivity encoding forfast MRI,” Magn Reson Med, vol. 42, no. 5, pp.952–962, 1999.

[3] M. A. Griswold, P. M. Jakob, et al., “General-ized autocalibrating partially parallel acquisitions(GRAPPA),” Magn Reson Med, vol. 47, no. 6, pp.1202–1210, jun 2002.

[4] Y. LeCun, Y. Bengio, and G. Hinton, “DeepLearning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.

5https://mridata.org6https://fastmri.med.nyu.edu/

[5] M. Uecker, P. Lai, et al., “ESPIRiT – An Eigen-value Approach to Autocalibrating Parallel MRI:Where SENSE meets GRAPPA,” Magn ResonMed, vol. 71, no. 3, pp. 990–1001, 2014.

[6] K. P. Pruessmann, M. Weiger, P. Boernert, andP. Boesiger, “Advances in sensitivity encoding witharbitrary k-space trajectories,” Magn Reson Med,vol. 46, no. 4, pp. 638–651, 2001.

[7] M. Lustig, D. Donoho, and J. M. Pauly, “SparseMRI: The application of compressed sensing forrapid MR imaging,” Magn Reson Med, vol. 58, no.6, pp. 1182–1195, 2007.

[8] K. T. Block, M. Uecker, and J. Frahm, “Undersam-pled radial MRI with multiple coils. Iterative imagereconstruction using a total variation constraint.,”Magn Reson Med, vol. 57, no. 6, pp. 1086–1098,2007.

[9] M. Lustig and J. M. Pauly, “SPIRiT: Iterativeself-consistent parallel imaging reconstruction fromarbitrary k-space,” Magn Reson Med, vol. 64, no.2, pp. 457–471, aug 2010.

[10] F. Knoll, K. Bredies, T. Pock, and R. Stollberger,“Second order total generalized variation (TGV) forMRI,” Magn Reson Med, vol. 65, no. 2, pp. 480–491, 2011.

[11] F. Knoll, C. Clason, K. Bredies, M. Uecker, andR. Stollberger, “Parallel imaging with nonlinearreconstruction using variational penalties,” MagnReson Med, vol. 67, no. 1, pp. 34–41, 2012.

[12] M. Akcakaya, T. A. Basha, et al., “Low-dimensional-structure self-learning and threshold-ing: Regularization beyond compressed sensing forMRI reconstruction,” Magn Reson Med, vol. 66,no. 3, pp. 756–767, Sep 2011.

[13] M. Akcakaya, T. A. Basha, R. H. Chan, W. J. Man-ning, and R. Nezafat, “Accelerated isotropic sub-millimeter whole-heart coronary MRI: compressedsensing versus parallel imaging,” Magn Reson Med,vol. 71, no. 2, pp. 815–822, Feb 2014.

[14] H. Jung, K. Sung, K. S. Nayak, E. Y. Kim, andJ. C. Ye, “k-t FOCUSS: A general compressedsensing framework for high resolution dynamicMRI,” Magn Reson Med, vol. 61, no. 1, pp. 103–116, 2009.

[15] K. Hammernik, T. Klatzer, et al., “Learning a vari-ational network for reconstruction of acceleratedMRI data,” Mag Reson Med, vol. 79, pp. 3055–71, 2018.

[16] E. G. Kholmovski and D. L. Parker, “Spatiallyvariant GRAPPA,” in Proc. 14th Scientific Meetingand Exhibition of ISMRM, Seattle, 2006, p. 285.

[17] P. M. Robson, A. K. Grant, et al., “Comprehensive

13

quantification of signal-to-noise ratio and g-factorfor image-based and k-space-based parallel imag-ing reconstructions,” Magn Reson Med, vol. 60, no.4, pp. 895–907, Oct 2008.

[18] Y. Chang, D. Liang, and L. Ying, “NonlinearGRAPPA: A kernel approach to parallel MRI re-construction,” Magn Reson Med, vol. 68, no. 3, pp.730–740, Sep 2012.

[19] D. S. Weller, J. R. Polimeni, et al., “Sparsity-promoting calibration for GRAPPA acceleratedparallel MRI reconstruction,” IEEE Trans MedImaging, vol. 32, no. 7, pp. 1325–1335, Jul 2013.

[20] P. J. Shin, P. E. Larson, et al., “Calibrationlessparallel imaging reconstruction based on structuredlow-rank matrix completion,” Magn Reson Med,vol. 72, no. 4, pp. 959–970, Oct 2014.

[21] J. P. Haldar and J. Zhuo, “P-LORAKS: Low-rank modeling of local k-space neighborhoods withparallel imaging data,” Magn Reson Med, vol. 75,no. 4, pp. 1499–1514, Apr 2016.

[22] K. H. Jin, D. Lee, and J. C. Ye, “A generalframework for compressed sensing and parallelMRI using annihilating filter based low-rank hankelmatrix,” IEEE Trans Comp Imaging, vol. 2, no. 4,pp. 480–495, Dec 2016.

[23] F. Knoll, K. Hammernik, et al., “Assessment of thegeneralization of learned image reconstruction andthe potential for transfer learning,” Magn ResonMed, 2019.

[24] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical ImageSegmentation,” Miccai, pp. 234–241, 2015.

[25] H. K. Aggarwal, M. P. Mani, and M. Jacob,“MoDL: Model-Based Deep Learning Architecturefor Inverse Problems,” IEEE Trans Med Imaging,2019.

[26] K. Kwon, D. Kim, and H. Park, “A ParallelMR Imaging Method using Multilayer Perceptron,”Medical Physics, 2017.

[27] Y. Jun, T. Eo, et al., “Parallel imaging in time-of-flight magnetic resonance angiography using deepmultistream convolutional neural networks,” MagnReson Med, 2019.

[28] M. Akcakaya, S. Moeller, S. Weingartner, andK. Ugurbil, “Scan-specific robust artificial-neural-networks for k-space interpolation (RAKI) re-construction: Database-free deep learning for fastimaging,” Magn Reson Med, vol. 81, no. 1, pp.439–453, Jan 2019.

[29] S. A. H. Hosseini, S. Moeller, S. Weingartner,K. Ugurbil, and M. Akcakaya, “Accelerated coro-nary MRI using 3D SPIRiT-RAKI with sparsity

regularization,” in Proc. IEEE International Sym-posium on Biomedical Imaging, 2019.

[30] C. Zhang, S. Moeller, S. Weingartner, K. Ugurbil,and M. Akcakaya, “Accelerated MRI using residualRAKI: Scan-specific learning of reconstruction arti-facts,” in Proc. 27th Annual Meeting of the ISMRM,Montreal, Canada, 2019.

[31] C. Zhang, S. Moeller, S. Weingartner, K. Ugur-bil, and M. Akcakaya, “Accelerated simultane-ous multi-slice MRI using subject-specific convo-lutional neural networks,” in 2018 52nd AsilomarConference on Signals, Systems, and Computers,Oct 2018, pp. 1636–1640.

[32] J. Y. Cheng, M. Mardani, M. T. Alley, J. M. Pauly,and S. S. Vasanawala, “DeepSPIRiT: Generalizedparallel imaging using deep convolutional neuralnetworks,” in Proc. 26th Annual Meeting of theISMRM, Paris, France, 2018.

[33] M. Buehrer, K. P. Pruessmann, P. Boesiger, andS. Kozerke, “Array compression for MRI with largecoil arrays,” Magn Reson Med, vol. 57, no. 6, pp.1131–1139, Jun 2007.

[34] Y. Han and J. C. Ye, “k-space deep learning foraccelerated MRI,” IEEE Trans Med Imaging, inpress, 2019.

[35] A. Pramanik, H. Aggarwal, M. P. Mani, and M. Ja-cob, “Off-the-grid model-based deep learning (O-MODL),” in Proc. IEEE International Symposiumon Biomedical Imaging, 2019.

[36] M. Jacob, M. P. Mani, and J. C. Ye, “Structuredlow-rank algorithms: Theory, MR applications, andlinks to machine learning,” IEEE Signal ProcessingMagazine, 2020.

[37] J. Schlemper, J. Caballero, J. V. Hajnal, A. N. Price,and D. Rueckert, “A deep cascade of convolutionalneural networks for dynamic MR image reconstruc-tion,” IEEE Trans Med Imaging, vol. 37, no. 2, pp.491–503, 2018.

[38] T. Eo, Y. Jun, et al., “KIKI-net: Cross-domainconvolutional neural networks for reconstructingundersampled magnetic resonance images,” MagnReson Med, vol. 80, pp. 2188–2201, 2018.

[39] B. Zhu, J. Z. Liu, S. F. Cauley, B. R. Rosen, andM. S. Rosen, “Image reconstruction by domain-transform manifold learning,” Nature, vol. 555, no.7697, pp. 487–492, 2018.

[40] F. Knoll, J. Zbontar, et al., “fastMRI: a publiclyavailable raw k-space and DICOM dataset for ac-celerated MR image reconstruction using machinelearning,” Radiology AI, in press, 2019.