code aperture optimization for spectrally agile compressive imagingarce/files/research/... ·...

Code aperture optimization for spectrallyagile compressive imaging

Henry Arguello and Gonzalo R. Arce*

Department of Electrical and Computer Engineering, University of Delaware, Newark, Delaware 19716-3130, USA*Corresponding author: [email protected]

Received March 4, 2011; revised July 22, 2011; accepted September 8, 2011;posted September 16, 2011 (Doc. ID 143197); published October 31, 2011

Coded aperture snapshot spectral imaging (CASSI) provides a mechanism for capturing a 3D spectral cube with asingle shot 2D measurement. In many applications selective spectral imaging is sought since relevant informationoften lieswithina subset of spectral bands.Capturingand reconstructingall the spectral bands in theobserved imagecube, to then throw away a large portion of this data, is inefficient. To this end, this paper extends the concept ofCASSI to a system admitting multiple shot measurements, which leads not only to higher quality of reconstructionbut also to spectrally selective imagingwhen the sequence of code aperture patterns is optimized. The aperture codeoptimization problem is shown to be analogous to the optimization of a constrained multichannel filter bank. Theoptimal code apertures allow the decomposition of the CASSI measurement into several subsets, each having in-formation from only a few selected spectral bands. The rich theory of compressive sensing is used to effectivelyreconstruct the spectral bands of interest from the measurements. A number of simulations are developed to illus-trate the spectral imaging characteristics attained by optimal aperture codes. © 2011 Optical Society of America

OCIS codes: 110.4234, 100.4145.

1. INTRODUCTIONCoded aperture snapshot spectral imaging (CASSI), first intro-duced in [1], is a remarkable imaging architecture that allowscapturing spectral imaging information of a 3D cube with justa single 2D measurement of the coded and spectrally dis-persed source field. The CASSI architecture has been studiedextensively in [2,3]. It turns out that the coded measurementsare mathematically equivalent to compressive projectionswhich are at the heart of the emerging field of compressivesensing (CS). In CS, traditional sampling is replaced by mea-surements of inner products with random vectors. In CASSI,the random projections are equivalent to spectrally dispersedcoded fields that are integrated (projected) by array detectorswith broad spectral response. The spectral images are thenreconstructed by solving an inverse problem such as a linearprogram or a greedy pursuit in a basis where the undersampled signals admit sparse representations. The key ideain CS reconstruction is the realization that most signals en-countered in practice are sparse in some domain and the the-ory of CS exploits such sparsity to dictate that far fewersampling resources than traditional approaches are needed[4–6]. Spectral image cubes are particularly well suited forsparse representation as images across different wavelengthsexhibit strong correlation [7–9]. More formally, suppose a hy-perspectral signal F ∈ RN×M×L or its vector representation f ∈RN ·M ·L is S sparse on some basis Ψ ¼ Ψ1 ⊗ Ψ2 ⊗ Ψ3, suchthat f ¼ Ψθ can be approximated by a linear combination of Svectors from Ψ with S ≪ ðN ·M · LÞ. The operator ⊗ is theKronecker product and Ψ is the Kronecker basis representa-tion of f [7]. N ×M represents the spatial dimensions and L isthe spectral depth of the image cube. The theory of compres-sive sensing shows that f can be recovered from m randomprojections with high probability when m≲ S logðN ·M ·LÞ ≪ ðN ·M · LÞ. The dimension of the Kronecker basis set

Ψ is RN ·M ·L×N·M ·L such that Ψ1 ∈ RN×N , Ψ2 ∈ RM×M , andΨ3 ∈ RL×L. The projections are given by g ¼ HΨθ, whereH is an NðM þ L − 1Þ × ðN ·M · LÞ random measurementmatrix with its rows incoherent with the columns of Ψ. Com-monly used randommeasurement matrices for CS are randomGaussian matrices (Hij ∈ fN ð0; 1=nÞg), Rademacher ma-trices (Hij ∈ f�1=n1=2g) and partial Fourier matrices. Therandom projection matrices in CASSI are realized by the codeaperture and the dispersive element. Figure 1 illustrates theprinciples behind CASSI, where a 6 × 6 × 8 spectral data cubewith 16 nonzero spectral components is depicted. The spec-tral components are coded by a code aperture where thewhite pixels block the impinging light and black pixels permitlight to pass through. A prism disperses the coded light and adetector integrates the intensity of the wave light. The pixelmeasurements at the detector are proportional to the linearcombinations of the coded spectral components.

This paper extends the CASSI spectral imaging concept to aspectral imaging architecture admitting multiple measure-ment shots. The multiple measurements are attained as sepa-rate focal plane array (FPA) measurements, each with adistinct code aperture that remains fixed during the integra-tion time of the detector. There are several advantages tomultiple shots [10]. First, the number of compressive measure-ments in CASSI may not meet the minimum needed foradequate reconstruction. Compressive sensing dictates thatthe number of measurements must be in excess ofS logðN ·M · LÞ. Failure to collect a sufficient number of mea-surements leads to a severely underdetermined inverse pro-blem and to inadequate signal reconstruction. With eachshot, CASSI invariably collects N ·M additional measure-ments regardless of the spectral depth and sparsity of thesource. For spectrally rich scenes or very detailed spatialscenes, a single shot CASSI measurement may not provide

2400 J. Opt. Soc. Am. A / Vol. 28, No. 11 / November 2011 H. Arguello and G. R. Arce

1084-7529/11/112400-14$15.00/0 © 2011 Optical Society of America

a sufficient number of compressive measurements. Increasingthe number of measurement shots will multiply the number ofmeasurements, thus rapidly overcoming such limitations.

A second advantage to multiple shots is that spectral selec-tivity can be attained by code aperture design. Notably, thecode aperture patterns can be designed so as to maximizethe information content on a prespecified subset of spectralbands of particular interest. Spectral selectivity is a character-istic of interest in many applications, including wide-area air-borne surveillance, remote sensing, and tissue spectroscopyin medicine. The optimal spectral bands in airborne surveil-lance, for instance, depend on atmospheric conditions, timeof day, the targets of interest, and the background againstwhich the targets are viewed. The development of dynami-cally programable multispectral imaging sensors are of signif-icant interest, particularly for their use in small unmannedaerial vehicles. Similarly, spectrally selective imaging isbecoming widely used in medicine. Spectral imaging of theretina, for instance, offers a route to noninvasive characteri-zation of the biochemical and metabolic processes within theretina retinopathies [11]. In the applications described above,and in many other applications, the spectral signatures of in-terest live in a spectral band subspace. Efforts placed on ac-quiring the entire spectral image cube, to then throw away alarge portion of this data is wasteful in many regards. This pa-per focuses on overcoming these inefficiencies, by developingan optimization framework for the design of code aperturesfor spectrally selective imaging.

In order to fully exploit the advantages of multishot codeapertures, this paper also develops a new model describingthe coded aperture optical system. The original CASSI modeldescribes the output of the system as a matrix operation act-ing on the input source field, where all the optical effects areimplicitly embedded in the matrix operation. The CASSImodel derived here describes the output of the system as afunction of a “code-aperture vector” that explicitly acts ona “reordered” input source. In particular, a vector representa-tion of a slice of the data cube is reordered into a matrix struc-ture, implicitly accounting for the spectral shearing operatorof the CASSI system. The reordering operator, in turn, decou-ples the code aperture effects of CASSI from the spectralshearing, thus allowing the representation of the output ofthe CASSI system as a simple matrix multiplication of the re-ordered input data with a vector whose elements are the code

aperture variables to be optimized. In this manner, the codeaperture optimization problem is shown to be analogous to amultichannel digital filter bank optimization problem wherethe filter coefficients represent the aperture codes and, con-sequently, the optimization is constrained to produce binary-valued outputs. The solution to the optimization problem athand is made feasible by the new CASSI system representa-tion. Using the resultant optimal coefficients, a filter bank de-composition of the data cube can be realized such that it ispossible to assemble a reduced set of compressive measure-ments that are sufficient to reconstruct a desired subset ofspectral bands and at the same time attain higher signal tonoise ratio. Figure 2 illustrates this concept where a set ofK measurements fgig are taken with a corresponding Kset of optimal aperture codes fwig, where i indexes the ithmeasurement. The mapping B reorders the set of measure-ments fgig into the V ≤ K sets fϱig containing only informa-tion of the spectral bands given by λi. Each subset fϱig issufficient to independently reconstruct the desired spectralband subset f~Fλig. The new code aperture agile spectral ima-ging (CAASI) system described in this paper allows multiplemeasurement shots, and, consequently, it has the aforemen-tioned advantages over CASSI. The new capabilities, however,come at the expense of more complexity in the hardware im-plementation. Multishot measurements can be attained bysuccessively shifting, along the horizontal axis, the fixed codeaperture in CASSI. A novel piezo-electrical implementation ofthis approach is described in [10,12]. An approach more sui-table for dynamic selectivity in real-time is to implement the

Fig. 1. (Color online) The principles behind CASSI imaging. A 6 × 6 × 8 spectral data cube with 16 nonzero spectral components is coded by thecode aperture and dispersed by the prism. The detector integrates the intensity of the resulting light wave. Each pixel at the detector contains acoded linear combination of the spectral information from the respective data cube slice.

Fig. 2. The CAASI system seen as a multichannel filter bank. A set ofmeasurements fgig are captured with a corresponding set of optimalaperture codes fw1;…;wKg. The mapping B reorders the K sets ofmeasurements fgig into the V sets fϱig. Each subset fϱig is sufficientto independently reconstruct the desired spectral band subset f~Fλig.

H. Arguello and G. R. Arce Vol. 28, No. 11 / November 2011 / J. Opt. Soc. Am. A 2401

time-varying aperture codes on a spatial light modulator[13–16]. Device implementation, calibration, wavelengthrange of operation, and other hardware considerations are cri-tical but fall outside the scope of this paper. These will be con-sidered in depth in a separate publication. The methodsdeveloped here are mathematical in nature and thus are ap-plicable on any hardware platform capable of acquiring multi-ple shots, each with a distinct aperture code pattern.

2. CODE APERTURE CHARACTERIZATIONIN CASSIThe coded aperture single shot spectral imaging system is de-picted in Fig. 3 [2]. The coding is realized by the coded aper-ture Tðx; yÞ as applied to the spatio-spectral density sourcef 0ðx; y; λÞ, where ðx; yÞ are the spatial coordinates and λ isthe wavelength resulting in the coded field f 1ðx; y; λÞ. Thecoded density is spectrally dispersed by a dispersive elementbefore it impinges on the FPA as f 2ðx; y; λÞ,

f 2ðx; y; λÞ ¼Z Z

Tðx0; y0Þf 0ðx0; y0; λÞhðx0 − αλ − x; y0

− yÞdx0dy0; ð1Þ

where Tðx0; y0Þ is the transmission function representing thecode aperture, hðx0 − αλ − x; y0 − yÞ is the optical impulse re-sponse of the system, and αλ is the dispersion induced bythe prism assuming a linear dispersion. Each spectral slice ofthe data cube f 0ðx; y; λÞ is thus spatially modulated by thecode aperture and dispersed by the dispersive element [2].The compressive measurements across the FPA are realizedby the integration of the field f 2ðx; y; λÞ over the detector’sspectral range sensitivity. The source f 0ðx0; y0; λÞ can be writ-ten in discrete form as ðFkÞnm, where n andm index the spatialcoordinates and k determines the kth spectral plane. Assum-ing that the bandpass filter of the instrument limits the spec-tral components between λ1 and λ2 and the side length of thesquare detector pixel is Δd, the number of resolvable bands Lis limited by L ¼ α λ2−λ1

Δd. The spectral resolution is limited by

Δdα . Additionally, it is assumed that the side length of the codeaperture square pixel Δc satisfies kΔc ¼ Δd, where k ≥ 1 is aninteger. The horizontal and vertical spatial resolutions arethus limited by Δd. The height N of the resolvable data cubeis limited by N ¼ minð Uh

kΔc; VhΔdÞ, where Uh and Vh are the ver-

tical physical dimensions of the code aperture and the detec-tor, respectively. The width M of the resolvable data cube islimited byM ¼ minðUw

kΔc; VwΔd

− Lþ 1Þ, where Uw and Vw are thehorizontal physical dimensions of the code aperture and thedetector, respectively. Following the mathematical model in[3], the coding is realized by an aperture pattern ðTÞnm.

The compressed sensing measurements at the focal planearray can be written in the following matrix form:

ðGÞnm ¼XLk¼1

ðFkÞn;mþkðTÞn;mþk þ ðωÞn;m; ð2Þ

where ω represents white noise, L is the number of spectralbands, and n, m index the pixels on the detector. A typicalexample of the measurement process using Eq. (2) is shownin Fig. 1. The expression in Eq. (2) can be expressed as

g ¼ Hf þ ω ¼ HΨθþ ω; ð3Þwhere g and f are vector representations of G and F, respec-tively [3]. These vector representations will be describedshortly. H is the projection matrix that takes into accountthe effects of the code aperture and the dispersive element.Ψ is a Kronecker basis representation and θ is a sparse coef-ficient vector representing f. If the aperture code pattern isfixed and only one snapshot is detected, the resultant spectralimager is the so-called single disperser, or single shot, CASSIarchitecture [17]. In this case, the entire 3D multispectral im-age cube is compressed into a single 2D compressive imagemeasurement at the FPA.

The spectral image cube f can be reconstructed by solvingthe optimizationproblem f¼Ψfargminθ0∥g−HΨθ0∥22þτ∥θ0∥1g,where τ > 0 is a regularization parameter that balances theconflicting tasks of minimizing the least square of the residualswhile, at the same time, yielding a sparse solution. Several al-gorithms exist in the compressive sensing literature to solvethis inverse problem [18–21].

H in Eq. (3) is anN · ðM þ L − 1Þ × N ·M · Lmatrix mappingthe vector f, which indexes the 3D data cube, into the vector g.Note from Eqs. (2) and (3) that the matrix H embeds both, thecode aperture T and the ashifting operation of the dispersiveelement. Since the aperture code is hidden in H and since it isnot directly observed in Eqs. (2) and (3), its optimization isdifficult using this CASSI formulation. Given that this analysisonly considers the dispersive and the code aperture effects,the matrix H is binary; however, due to nonlinearity in the dis-persive element, non ideal optical instruments and misalign-ments between the detector pixels and the spatial lightmodulator, these matrices have real values in practice [1].For the purposes of this paper a binaryH is used. Additionally,the model used does not consider noise. The modifications onH to address the physical limitations of the optical system canbe readily done, but omitted here in order to better illustratethe aperture code optimization problem.

Since the dispersive element shifts the propagating lightalong the horizontal dimension only, the spectral coding inCASSI is independent fromone row to another. Hence, it is pos-sible toconstruct anaperturecodemodel for just one rowof themeasurementmatrix,which is thenreplicated forall rowsof thedetector. To this end, define the vector fq that indexes one sliceof the data cube ðFkÞnm when the index n is fixed to q. Morespecifically,

ðfqÞðk−1Þ·Mþm ¼ ðFkÞqm; for m ¼ 1;…; M k ¼ 1;…; L: ð4Þ

Note that theelementsof fq are theonlyelementsof the source fhaving an impact on the qth rowofG. Let the qth rowofG be gq.The projection of fq into gq is then represented as

gq ¼ Hqfq ð5ÞFig. 3. Representation of the CASSI.


for q ¼ 1;…; N . Figure 4 illustrates thevarious vector represen-tations in Eqs. (4) and (5) of the source data cube. Since thespectral coding along rows of G are mutually independent,the vectors gq, can be arranged as in Eq. (3), where g and Hare unfolded showing the mutual independence of the rowmeasurements as

g266666664

g1...

gq...

gN

377777775

zfflfflfflffl}|fflfflfflffl{

¼

H266666664

H1 0 � � � 0

0 . .. . .

. ...

..

. . ..

Hq

. ..

0 � � � HN

377777775

zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{ f266666664

f1...

fq...

fN

377777775;

zfflfflfflffl}|fflfflfflffl{

ð6Þ

where the matrices 0 have ðM þ L − 1Þ ×M · L zero elements,the NML-long vector f ¼ ½fT1 fT2…fTN �T is the concatenation ofthe vectors fq in Eq. (4) and the vector g ¼ ½g1g2…gN �T is theconcatenation of the row vectors gq, representing the qthrow at the detector. The matrix Hq represents the effect ofthe dispersive element, the code aperture, and the integrationon the detector for the qth row of the measurement matrix G.The Hq matrix is given by

Hq ¼

266666664

diagfðTÞq;1;…; ðTÞq;Mg01×M...

..

.

01×M

��

01×MdiagfðTÞq;1;…; ðTÞq;Mg

01×M...

01×M

��

� � �� . ..

. ..

��

01×M...

..

.

01×MdiagfðTÞq;1;…; ðTÞq;Mg

377777775; ð7Þ

where diagfðTÞq;1; ðTÞq;2;…; ðTÞq;Mg is an M ×M diagonalmatrix with the elements of the qth row ofT in its diagonal. Ob-serve that the specific structure ofHq determines the structureof H in Eqs. (6) and (3). In order to separate the effects of thevarious optical elements, the representation Hq can beexpressedasHq ¼ DT ,where thematrixD represents themap-ping operation of the dispersive element. The matrix D has atmost L nonzero elements in each row. The specific structure ofthe ðM þ L − 1Þ ×ML matrix D is given by

D ¼

26664

IM 01×M � � � 01×M01×M IM � � � 01×M... ..

. . .. ..

.

01×M 01×M IM

37775; ð8Þ

where IM is anM ×M identity matrix and 01×M are zero-valuedrow vectors. The matrix T represents the effect of the codeaperture T on fq and has the following structure:

T ¼ IL ⊗ diagfðTÞq;1; ðTÞq;2;…; ðTÞq;Mg; ð9Þ

where IL is an L × L identity matrix. T is thus an M · L ×M · Ldiagonal matrix whose elements are L repetitions of the ele-ments in the qth row of T. The mapping of fq into gq inEq. (5) is then written as

gq ¼ DT fq: ð10Þ

Fig. 4. (Color online) Model equations for a row measurement. A slice of the data cube fq impinges on a row of the code aperture ti ¼ r∘wi toproduce the coded sliceWiRifq. The elements of the coded slice are reordered into the matrix Γi. The detector output giq ¼ ΓiuL sums the columnsof Γi.


The vector representation in Eq. (10), characterizing CASSI,will be critical in formulating the aperture code optimizationproblem described in the following sections.

3. CODE APERTURE AGILE SPECTRALIMAGING SYSTEM (CAASI)The spectral selectivity is determined by the aperture codesused in each shot. The mathematical model of the ith mea-surement shot of the CAASI system is identical to that shownin Eq. (2) for the CASSI system

ðGiÞn;m ¼XLk¼1

ðFkÞn;mþkðTiÞn;mþk þ ðωiÞn;m ð11Þ

for i ¼ f1;…; Kg, where K is the number of shots, except thatthe ith aperture code pattern Ti used to sense Gi is differentfrom one shot to another. Notice that the code aperture Ti canbe seen as two separate code apertures applied in tandem.The first code aperture R is a random binary code necessaryto attain randomized measurement in CASSI. The second codeaperture Wi is the ith code optimized to achieve a specificspectral band selectivity. Ti is thus represented as the Hada-mard product ðTiÞn;m ¼ ðRÞn;mðWiÞn;m, where Wi is the time-varying element in each CAASI measurement. In matrix form,the CAASI system of equations results in

gi ¼ Hif þ ωi i ∈ 1;…; K: ð12Þ

As in the single shot CASSI system, the row measurements giqare decoupled from one another such that the Hi matrices inEq. (12) are block diagonal. The qth row of Gi is then given bygiq ¼ Hi

qfq, or equivalently

giq ¼ DT ifq: ð13Þ

Note in Eq. (13) that the source vector fq does not contain theindex i since we assume that the source remains stationary

during the time interval when the K shots are measured.The only difference at this point, between the static case of theCASSI equation in Eq. (10) and the CAASI equation in Eq. (13)is that the matrix T i changes with each shot. The matrices T i

have the structure given in Eq. (9), but in this case, the qth rowof Ti can be expressed as ti ¼ r∘wi, where ∘ denotes the Ha-damard multiplication of r and wi, the qth row of the matricesR andWi, respectively. Figure 4 depicts the interaction of thevector ti with the incoming wave fq. In the static case ofEq. (9), T does not have a time-varying term and thus t ¼ r.The matrix T i can be expressed as T i ¼ IL ⊗ diagfðwiÞ1ðrÞ1;…; ðwiÞMðrÞMg. Using the product property ofthe Kronecker product, T i is written as

T i ¼ ðIL ⊗ diagfðwiÞ1;…; ðwiÞMgÞ|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}Wi

ðIL ⊗ diagfðrÞ1;…; ðrÞMgÞ|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}R

;

ð14Þ

where Wi represents the component of the code aperture wi

and Ri reflects the effect of r. While the random matrix R isnot tunable in this design, the matrix Wi contains weight ele-ments of the code aperture that will be optimized. R is at-tained via Bernoulli random realizations, however, randommatrices such as a scrambled Hadamard can be used to con-struct this random component. Using Eq. (14) in Eq. (13) weobtain giq ¼ DWiRfq. Letting ρ ¼ Rfq be the effect of thevector rq on the source fq, then giq can be written as

giq ¼ DWiρ: ð15Þ

Making some matrix index manipulations as detailed inAppendix A, Eq. (15) can be written as

giq ¼ ΓiuL; ð16Þ

where uL is the L long vector uL ¼ ½1;…; 1�T and the matrix Γiq

is given by

Γi ¼

266666664

ρ1ðwiÞ1 ρ2ðwiÞ2 … ρLðwiÞL � � � ρM ðwiÞM 0 … 00 ρMþ1ðwiÞ1 … ρMþL−1ðwiÞL−1 � � � ρ2M−1ðwiÞM−1 ρ2M ðwiÞM … 0

0 0 . ..

ρ2MþL−2ðwiÞL−2 � � � ρ3M−2ðwiÞM−2 ρ3M−1ðwiÞM−1 … 0

..

. ... ..

. � � � ... ..

. . ..

0 0 0 ρðL−1ÞMþ1ðwiÞ1 � � � ρLM−ðL−1ÞðwiÞM−Lþ1 … ρLMðwiÞM

377777775

T

; ð17Þ

where ρj is the jth element of ρ. It can be observed that theðwiÞl terms have a circular shift on l within each row of Γi.Additionally, only L pixels in the detector are affected by apoint source in the data cube. Hence, only L elements ofwi in Eq. (17) need to be optimized. Define the L-long sub-segments of wi as �wi, such that wi is formed by the modulorepetition operator

ðwiÞm ¼ ð�wiÞmodðm;LÞ for m ¼ 1;…; M and L < M: ð18Þ

Equation (18) can be expressed in matrix form as

wi ¼ uM 0 ⊗ �wi; ð19Þ

where M 0 ¼ ML is the length of the one element vector uM 0 . If

the integral part of ML is not an integer, then the first L⌊M

L ⌋

elements of wi are given by Eq. (19) and the remaining byðwiÞm ¼ ðwiÞm−L form ¼ L⌊M

L ⌋þ 1;…; M . Reordering the ele-ments of Γi in Eq. (17) such that the ðwiÞl terms appear in as-cending order and using the periodicity of wi given in Eq. (18),we obtain


�Γi ¼

266666664

ρ1ð�wiÞ1 ρMþ1ð�wiÞ1 … ρðL−1ÞMþ1ð�wiÞ1 � � � ρLM−ðL−1Þð�wiÞ1 0 … 00 ρ2ð�wiÞ2 � � � ρðL−2ÞMþ2ð�wiÞ2 � � � ρðL−1ÞM−ðL−2Þð�wiÞ2 ρLM−ðL−2Þð�wiÞ2 … 0

0 0 . ..

ρðL−3ÞMþ3ð�wiÞ3 � � � ρðL−2ÞM−ðL−3Þð�wiÞ3 ρðL−1ÞM−ðL−3Þð�wiÞ3 … 0

..

. ... ..

. � � � ... ..

. . ..

0 0 0 ρLð�wiÞL � � � ρMð�wiÞL ρ2Mð�wiÞL … ρLMð�wiÞL

377777775

T

: ð20Þ

Note that thematrix productΓiuL inEq. (16) and the product�ΓiuL of the reordered matrices in Eq. (20) are identical. HenceEq. (16) can be rewritten as giq ¼ �ΓiuL. The expression for giqcan be further simplified by factoring out the elements �wi inEq. (20) leading to

giq ¼ X�wi; ð21Þwhere

X ¼

266666664

ρ1 ρMþ1 … ρðL−1ÞMþ1 … ρLM−ðL−1Þ 0 … 00 ρ2 … ρðL−2ÞMþ2 … ρðL−1ÞM−ðL−2Þ ρLM−ðL−2Þ … 0

0 0 . ..

ρðL−3ÞMþ3 … ρðL−2ÞM−ðL−3Þ ρðL−1ÞM−ðL−3Þ … 0

..

. ... ..

.…

..

. ... . .

.

0 0 … ρL … ρM ρ2M … ρLðMÞ

377777775

T

ð22Þ

is a matrix containing the spectral source elements multipliedby the randomcomponent of the aperture code,which are thenspectrally shifted. Equation (21) is a compact expression for giq,where �wi describes the effect of the optimizable part of thecode aperture. The effect of the others optical elements onthe data cube is accounted for in the reordering of the data ma-trix X. The complete vector gi is now succinctly expressed as

2664gi1...

giN

3775 ¼

2664

X1 �wi1

..

.

XN �wiN

3775; ð23Þ

where the structure of the matrices Xj accounting for the jthdata cube slice is given by Eq. (22), and �wi

q is the designablecomponent of the qth row of the code aperture. In matrix nota-tion, Eq. (23) can be expressed as

gi ¼ χ �Wi; ð24Þ

where �Wi ¼ f�wiT1 ; �wiT

2 …�wiTN gT and X ¼ diagfX1;X2;…;XNg

is a block diagonal matrix. Equations (21) and (24) providea new representation for the measurements of CAASI whenmultiple shots are admitted. The new formulation is illustratedin Fig. 5. In the traditionalmodel [Fig. 5(a)], the input is the datacube f and theoutput gi is characterizedby thematrixHi. In thenew representation [Fig. 5(b)], the input is thematrix χ and theoutput is obtained by varying the matrix �Wi. The matrix χ re-presents a reordered version of the vector f so as to accountfor the spectral sheering and R, the random term of the code

aperture. Note that the elements of �Wi in the new representa-tion in Eq. (24) explicitly contain the components of the codeapertures thatmust be optimized. Observe that thematricesHi

inEq. (12) have a total of ðNÞðM þ L − 1Þ × ðN ·M · LÞ elementseach, compared to theNðM þ L − 1Þ × L elements of thematrixχ in Eq. (23). The newmodel is thus significantlymore compactthan that of Eq. (12) and consequently it is better suited foriterative optimization algorithms.

4. APERTURE CODE OPTIMIZATION INMULTISHOT CAASIEach measurement in CAASI uses an optimizedWi code aper-ture component. Since the spectral coding is mutually inde-pendent between the rows of the detector, it is sufficient todesign one row of Wi. The remaining rows are completedby shifted replicas of the first such that Wi ¼ uN ⊗ wi.Furthermore, since wi is a concatenation of several identicalsegments �wi, the optimization of the KWi code aperture ma-trices reduces to the optimization of K �wi vectors.

Before the optimization problem is formulated, it is useful tofirst simplify the notation by deleting the subindex q in theabove formulation, which indicates a particular slice ofthe CASSI system. It is also useful to note the similarities of theCAASI system to a multichannel filtering problem where theinput signal is the data cube f and the filter coefficientsare the elements of the code aperture �wi. The filtering problem,in this case, aims at optimizing the coefficients (code aper-tures) such that the linear combination of the outputs at eachjth position at the detector is as close as possible to a desiredresponse, one that only contains information of the specificbands of interest. The code aperture design problem is thusanalogous to finding the set of the optimal filter coefficientsin filter design. More formally, given an input signal fs and adesired output signal d, then the objective is to find the Koptimal filter weight vectors (aperture codes �w1; �w2;…; �wK )anda set of associatedweights bij such that the error vector e ¼½e1; e2;…; eMþL−1�T isminimized according to an error criterion.Figure 6 depicts the aperture code optimization problem as afiltering problem for the jth position on a detector row.


To proceed with the filter design approach, a test imagecube fs is first created as follows. Define a sequence of distinctprime numbers fs1; s2;…; sLg such that sl ≠ sj for all l ≠ j.Choosing the set of primes properly, it is possible to guaranteethat

XLj¼1j≠l

sjzj ≠ sl ð25Þ

for all l, where the coefficients zj are binary, zj ∈ f0; 1g. If acoefficient set fzjgLj¼1 is different from another fz0jgLj¼1 in atleast one element, then from the properties of prime numbersit follows that

PLj¼1 zjsj ≠

PLj¼1 z

0jsj . Given the set of prime

numbers, the test data cube to be used in the filter designis then set to

ðFskÞnm ¼ sk for all m;n: ð26Þ

Hence, Fs is a cube where each spectral slice takes on theconstant value of sk across all the spatial positions m andn. A slice of the data cube at n ¼ q in vector form is denotedby fs across all m and k. Clearly ðFs

k1Þn;m ≠ ðFs

k2Þn;m if k1 ≠ k2.

The mapping operation converting fs into X is shown in Fig. 6as the re-indexing operation.

From Eq. (25) and the CAASI Eq. (21), given the detectormeasurements sequence at the detector, gi, it is possible todetermine which spectral bands are present in the linear com-bination yielding gi. This is the case since ðFs

kÞqm and Xare known.

The desired signal d, shown in Fig. 6, is created from Fd

which contains only the desired bands and is given by

ðFdkÞnm ¼

� ðFskÞnm if ðλÞk ¼ 10 otherwise

ð27Þ

for all m, n and where the L-long binary vector λ indexesthe subset of spectral channels of interest. In particularðλÞk is defined as

ðλÞk ¼�1 if the kth spectral band is of interest0 otherwise

ð28Þ

for k ∈ f1;…; Lg. The qth slice of Fd is fd which is thus forcedto have only the spectral components of interest. The outputat the detector, when the desired data cube fd is sensed, isgiven by

di ¼ Xd ~wi; ð29Þ

where Xd is a matrix whose elements are obtained by the re-indexing of fd. Since Xd contains only information from thedesired spectral bands, then the desired signal di can be cal-culated by a code ~wi that simply adds the row elements of Xd.This operation is analogous to using an “all-pass” code ~wi thatpermits to pass all the spectral information from Xd. In thiscase, the all-pass code ~wi ¼ uL, in effect, simply sums all ele-ments of Xd. The vector di in Eq. (31) does not depend of thecode aperture ~wi and it is the desired output d. Note that thedesired signal d is unique in the sense that only the specificcombination of spectral bands given by λ can generate the out-put of CAASI equal to d. This is precisely the motivation ofusing the set of prime numbers in Eq. (25) as elements ofthe data cube fs. To further illustrate the underlying concepts,Fig. 7 shows a typical slice of the data cube fs and the corre-sponding structure of the matrix X. Observe that the elementsof the output gi are calculated by the Hadamard product of thecode aperture wi and each row of X. In Fig. 6 the operation ofextracting xj the jth row ofX is indicated with the “row-select”operation. From Fig. 6, the error at jth position in the detectoris

ej ¼ ðdÞj − ðϱiÞj j ¼ 1;…; M þ L − 1; ð30Þ

where ðϱÞj ¼P

Ki¼1 bijx

Tj �w

i. Notice that d and xj are knownterms and �wi represents the unknown aperture codes patternto be designed. The coefficients bij permit to use all the infor-mation available from the K shots to minimize the jth compo-nent at a row at the detector. The coefficients bij are restrictedto the three values f−1; 0; 1g that the value −1 indicates thesubtraction of a set of bands, 0 indicates the suppressionof a set of bands, and 1 indicates the summation of a set ofbands. Appendix B details the selection for the coefficientsbij . The errors originated from the measurements inEq. (30) are concatenated into the common array e0 ¼½e1; e2;…; eMþL−1�.

Fig. 5. (a) Traditional model where the data cube f is processed withthe highly sparse matrixHi. The code aperture pattern is hidden inHi;(b) new model of CAASI: f is first reordered and expanded into thematrix X which is then processed by the weight matrix Wi whoseelements are the code aperture patterns.

Fig. 6. Code aperture optimization as a filter design problem. Theinput is a row of the data cube fs and the desired signal is f0. The filtercoefficients (aperture codes �w1; �w2;…; �wK ) and the coefficients bij areoptimized by minimizing a cost function of the errors e1; e2;…; eMþL−1.The error at given jth position at a row at the detector is the differencebetween the linear combination of the measurements ðϱÞj and ðdÞj .


The component-wise squared error metric is first calculatedsuch as e ¼ e0∘e0. The jth element of e can be expressed as afunction of the code aperture as ðeÞj ¼ ððdÞj −

PKi¼1 bijx

Tj �w

iÞ2.Note that ðeÞj ¼ 0 occurs only when the code apertures areable to exactly extract the desired bands λ in the jth positionin the detector. Note also that when ðeÞj > 0, there are addi-tional spectral bands included or not all desired bands are pre-sent in the jth entry of the ϱ vector. Given that the spectralselectivity consists on obtaining at each position in the detec-tor exclusively the desired spectral components, it is equiva-lent to maximizing the number of elements ðeÞj equal to zero.In other words, it is equivalent to minimizing the L0 norm ofthe vector e. Hence, one seeks to obtain the vectors W ¼½�w1;…; �wK � and the coefficients B ¼ ½fb11;…; bK1gT ;…;fb1ðMþL−1Þ;…; bKðMþL−1ÞgT � such that

argminW;B

subject to

∥e∥0ðWÞki ∈ f0; 1g

ðBÞij ∈ f−1; 0; 1gk ¼ 1;…; L; i ¼ 1;…; Kj ¼ 1;…; M þ L − 1:

ð31Þ

Since the measurements gi contain complementary informa-tion between them, the functionality of the coefficients bij is toincrease the number of solutions in Eq. (31) such that an itera-tive algorithm can converge more easily. Since the elements of�wi and bij are binary and ternary, respectively, solving Eq. (31)is an non-deterministic polynomial-time (NP) hard problem.An approximate solution, however, can be sought using aheuristic approximation such as a genetic algorithm (GA)[22]. GAs are specially well suited to solve optimization pro-blems that involves binary variables. This optimization tech-nique has already been used in several optics optimizationproblems [23,24]. The GA approach to solve Eq. (31) is shown

in Table 1. The details of the algorithm are described inAppendix C. The stopping criterion to solve Eq. (31) is that∥e∥0 ¼ 0. Notice that this stopping criterion conduces to a so-lution where the optimal codes lead to measurements contain-ing exclusively the desired bands. A measurement containingthe desired components can be constructed from the resultsof the optimization as

ϱ� ¼�XKi¼1

ðB�Þi1xT1 �w�i;XKi¼1

ðB�Þi2xT2 �w�i;…;

×XKi¼1

ðB�ÞiðMþL−1ÞxTMþL−1 �w�i�; ð32Þ

where B� contains the optimal linear coefficients and �w�i arethe optimal code apertures attained in Eq. (31).

A property of the optimized aperture codes is that if thecodes �w�1;…; �w�K and their respective optimal coefficient ma-trix B� can extract the spectral bands given by λ, then thesame codes with a different coefficient matrix Bn� can extractthe spectral bands given by λn with

ðλnÞk ¼ ðλÞðhk−niLÞ; k ¼ 1;…L; ð33Þ

where hk − niL indicates the k − n modulo L operation orequivalently a n circular shift of period L. This property re-quires that λ has some circular period N and that the ratioLN is an integer. Finding the new matrix Bn� involves simplealgebraic operations. The optimized aperture codes are thuscircular shift invariant to the vector λ. The proof of this prop-erty is shown in Appendix D.

Table 1. Iterative Optimization of Code Apertures �w�1;…; �w�K and the Optimal Matrix B�

Inputs Desired spectral bands λ, Number of shots K , Prime numbers sk, Population size S, Number of iterationsN , Mutation percentage pm.Initialize Data Cube sk → fs → Xs. Desired Signal ðsk; λÞ → fd → Xd

→ d → D ¼ d ⊗ uTK. B ¼ fb1;…; bKg, where K ¼ 3K , and bm1 ≠ bm2 foreach m1 ≠ m2, m1 ¼ 1;…;K, and m2 ¼ 1;…;K. Global minimum ξ� ¼ M þ L − 1. Population: For ℓ ¼ 1;…;S computePð0Þℓ ¼ f�w1

ℓ ;…; �wKℓ g, where ð�wi

ℓÞk ¼ Bernoullið0:5Þ.Iteration While, n < N and ξ� > 0 For ℓ ¼ 1;…;S compute GðnÞ

ℓ ¼ XPðnÞℓ , GðnÞ

ℓ ¼ GðnÞℓ B, EðnÞ

ℓ ¼ ðD − GðnÞℓ Þ∘ðD − GðnÞ

ℓ Þ ðeðnÞℓ Þj ¼ minmðEðnÞℓ Þjm,

ðξðnÞÞℓ ¼ eðnÞℓ0 , ðLðnÞÞℓj ¼ argminmðEðnÞℓ Þjm. End For ℓ� ¼ argminℓðξðnÞÞℓ. If ðξðnÞÞℓ� < ξ�, then : ξ� ¼ ðξðnÞÞℓ�, f�w�1;…; �w�KgT ¼ PðnÞ

ℓ� ,ðB�Þkj ¼ ðBÞðnÞkðLÞℓ�j . End If fPðnþ1Þ

1 ;…;Pðnþ1ÞS g ¼ MuðCrðSlðfPðnÞ

1 ;…;PðnÞS g; ξðnÞÞÞÞ. End While

Output Optimal aperture codes �w�1;…; �w�K . Optimal linear coefficient B�.Note j ¼ 1;…; M þ L − 1 and k ¼ 1;…; L.

Fig. 7. (Color online) Slice of the data cube fs is modulated by the vector r and then reorganized into the matrix X. The output at the detector iscalculated as gi ¼ X�wi.


5. SIMULATIONS AND RESULTSA. Code Aperture OptimizationTo test the methodology developed in Section 4, the algorithmin Table 1 is used to find the optimal aperture codes for sev-eral desired spectral band sets λ, for increasing number ofmeasurement shots. The experiments use a test data cubeFs with L ¼ 24 spectral bands in 504nm–757nm and 256 ×256 spatial dimensions. A set of 24 prime numbers sk is gen-erated such that Eq. (25) is satisfied. The other parameters ofthe algorithm in Table 1 are: population size S ¼ 100, mutationpercentage pm ¼ 0:01, and maximum number of iterationsN ¼ 5000. In the experiment, the following desired spectralbands λ are used in the optimization

λ1 ¼ ½100010100000000000110000� λ2 ¼ ½100010001000100010001000�λ3 ¼ ½101000101000101000101000� λ4 ¼ ½110000001100000011000000�: ð34Þ

From numerous simulations, it can be observed that the algo-rithm converges for K < L when the vector λ is circularly per-iodic. When λ is aperiodic, the algorithm converges for K ¼ L.Figure 8 shows the performance evolution ξ� as a function ofthe iterations when the vectors λ2 and λ4 are used as input ofthe GA. Figure 9 illustrates the optimal aperture codes �w�i

when the vectors λ1, λ2, λ3, and λ4 are used as input to theoptimization algorithm. Additionally, Fig. 10 shows an 128 ×128 matrix whose elements are constructed by replicates ofthe optimal code apertures and coded by a random compo-nent. Each row of these matrices is constructed by selectingrandomly an optical code �wi, such that the entire matrix con-tains information of the all optimal codes. In addition to theoptimal codes, the optimization provides the optimal coeffi-cient matrix B� used to construct the elements of the optimalvirtual measurement ϱ�. Figure 11 illustrates the optimal ma-trix B� when λ4 is used; due to space limitations only a portionof this matrix is shown. Note that the patterns in Fig. 11 ex-hibit periodicity, except in the beginning and end, which ex-hibit edge structures that are different from that of the mainregion.

B. Filter Bank RepresentationThe filter bank spectral model aims at reconstructing the en-tire data cube from the V compressive measurements whereeachmeasurement has information only from one of V disjointspectral subsets with the constraint of using only K ≤ L shots.Each subset can be reconstructed separately and the resultscan be merged. The disjoint spectral subsets are specificallydesigned such that the shift invariance property of the codeapertures developed in previous sections is used.

More specifically, suppose that the L long vector λ has cir-cular period N , define U as the maximum number of conse-cutive one elements in a period of λ, and define V as V ¼ ⌈ N

U ⌉.Then the complete data cube can be reorganized in V com-pressive measurements with information only from the bandsgiven by fλ0; λ1;…; λV−1g, where the elements of each subsetλn is given by

ðλnÞk ¼ ðλÞðhk−nUiÞL k ¼ 1;…; L: ð35Þ

The filter bank representation is attained as follows: (a) gen-erate V vectors fλ0;…; λV−1g, where each vector λi is a shiftedversion of λ0 ¼ λ; (b) verify that λ0∨λ1∨…∨λV−1 ¼ uL, where∨ is the boolean sum; (c) the spectral components given bythe different λi can be reconstructed separately and the re-sults can be merged with the other reconstructions. Since½�w�1;…; �w�K � are the optimal codes and B� is the optimalcoefficient matrix to recover λ, then each combinationf½�w�1;…; �w�K �T ;Bi�g can be seen as a compressive filter bankto recover λi, where Bi� is a reordered version of the matrixB�. A representation of this technique was illustrated in Fig. 2.Define the “modp” λ pattern which elements are given by

ðλÞk ¼ modðk; pÞ; k ¼ 1;…; L; ð36Þ

Fig. 8. Performance evolution ξ� as a function of the iterations whenthe vectors (a) λ2 and (b) λ4 in Eq. (36) are used as input of the GA.


where mod is the modulo operation, p is the circular period ofλ, and L is the total number of bands. Here, it is assumed thatthe ratio L

p is an integer. These “modp” patterns can be used todecompose the data cube in the subsets fλ0; λ1;…; λp−1g,where λ0 is equal to λ in Eq. (36) and the others λi are givenby Eq. (35) with U ¼ 1. Notice that the λi subsets satisfy theabove (b) condition necessary to apply the filter bank decom-position. Decomposing the data cube in the “modp” subsetsrequires p shots. The respective optimal code apertures to se-lect the modp patterns is referred to as “modp filter bank codeapertures.” For example, referring to Eq. (34), λ2 is a mod4pattern and the codes of Fig. 9(b) are a mod4 filter bank codeapertures.

C. Reconstruction AlgorithmA number of CS reconstruction algorithm are available in theliterature [18–21]. The gradient projection for sparse recon-struction algorithm (GPSR) [19] was used to reconstructthe spectral data cubes in our simulations as it exhibits fastercomputational speed. This algorithm reconstructs the datacube f by solving f ¼ Ψfargminθ0∥ϱ� −HΨθ0∥22 þ τ∥θ0∥1g,where ϱ� are the optimized measurements. The operation that

involves the sparse matrix H are implemented through the in-dexes of nonzero elements of the matrix H instead of using itdirectly. The base representation Ψ is the Kronecker productof three basis Ψ ¼ Ψ1 ⊗ Ψ2 ⊗ Ψ3, where the combinationΨ1 ⊗ Ψ2 is the 2D-Wavelet Symmlet 8 basis andΨ3 is the co-sine basis [7]. A data cube of satellite images is used, it has 24spectral channels and 256 × 256 pixels of spatial resolution.Reconstruction was realized using a desktop with 3:06GHzIntel Core 2Duo processor and 4 GByte, 1067MHz DDR3RAM. The simulation tool was Matlab 2007a version 7.4.0for MacOS version 10.6.4. The simulations were realizedusing double numerical precision. The maximum number ofiterations in the GPSR algorithm is 700 and the stoppingcriterion as recommend in [19] is the objective function.Figures 12(b)–12(d) depict the reconstruction results forthe first spectral band of the data cube applying the techniqueof compressive filter bank. The mod2, mod12, and mod24 filterbank code apertures are used in the decomposition. Clearlythe SNR increases with the number of shots used.

A comparison between the compressive filter bank techni-que and a random multishot approach was realized. The ran-dom multishot technique consists of using random codeaperture patterns to realize the compressive measurements[12,13]. In that technique, no optimization of the code aper-tures was carried out. Figures 13(b)–13(d) show the recon-struction of the same first band of the data cube using therandom codes for K equal to 2, 12, and 24 shots. Figure 14

Fig. 11. Part of the matrix B� is shown. The lth column of B� repre-sents the optimal coefficient to construct the measurement elementðϱ�Þl. The vector λ4 is used as input of the optimization algorithm.

Fig. 10. 128 × 128 realization of the optimal code apertures for thevectors (a) λ1, (b) λ2, (c) λ3, (d) λ4 given in Eq. (34).

Fig. 9. Optimal code apertures �w�i for the vectors (a) λ1, (b) λ2, (c) λ3, and (d) λ4 indicated in Eq. (34).


illustrates the general results in terms of SNR for the multishotand filter bank approaches as a function of the number ofshots K . Notice that, in the multishot case, the complete spec-tral data cube is reconstructed at once. In the filter bank ap-proach, on the other hand, the complete data cube is dividedin K disjoints subsets where the nth subset λn is given by

ðλnÞj ¼�1 for j¼n; Kþn;2Kþn;…;

�LK −1

�Kþn

0 otherwise:; ð37Þ

where n ¼ 0;…; K − 1. Each desired subset is reconstructedindependently from others. The reconstructed subset bandsare then merged and the total SNR is calculated. The results

show that, when the number of shots increases, the filter bankapproach is superior in quality of reconstruction. Figure 15shows the respective reconstruction time for the same codeapertures used in Fig. 14. For the reconstruction techniquerandom multishot, the reconstruction time decreases with in-creasing number of shots because when more shots are usedthe system of equations are less undetermined and the objec-tive function decreases more quickly through the iterations. Inthe filter bank case, the reconstruction times are shown usingboth a single processor and a processor for the reconstructionof each subset of bands. The approach using several proces-sors is coined “parallel filter bank.” The parallel filter bankapproach provides up to two orders of magnitude faster re-construction than the random multishot approach.

In the second experiment, we observe the spectral selectiv-ity of the optimized code apertures. In this case the recon-struction of the 1st and 18th bands are of interest. Thevector λ4 in Eq. (34) is used as input of the optimization

Fig. 15. Reconstruction times for the full data cube as a function ofthe number of shots using (a) random code apertures (random multi-shot), (b) modp filter bank optimized code apertures (filter bank), and(c) modp filter bank code apertures using a processor for each subsetof bands (filter bank parallel).

Fig. 13. Reconstruction of the first spectral band of the 24 spectralband data cube. (a) Original; (b) random code aperture, 4 shots;(c) random code aperture, 12 shots; and (d) random code aperture,24 shots.

Fig. 12. Reconstruction of the first spectral band of the 24 spectralband data cube. (a) Original; reconstruction using (b) mod4 filter bankcode apertures, 4 shots; (c) mod12 filter bank code apertures, 12 shots;(d) mod24 filter bank code apertures, 24 shots.

Fig. 14. Mean PSNR for the reconstructed data cube as a functionof the number of shots. The techniques of random multishot andcompressive modp filter bank are shown.


algorithm. In this experiment, the results for K equal to 4, 8,12, and 16 shots are shown inFigs. 16(c)–16(f) and17(a)–17(d),respectively. Note that Fig. 8(b) shows a minimum numberof K ¼ 8 shots for convergence of the optimizationalgorithm. However, Figs. 16(c) and 16(d) show that it is

possible to reconstruct the desired spectral bands withfewer number of shots.

6. CONCLUSIONSWe have developed the underlying theory and optimizationframework for a new CAASI that adds the spectral selectivityproperty to CASSI. The mathematical model of CAASI ex-presses the output signal as the product of a reordered versionof the source input and the code aperture. Using the newCAASI representation, an optimization algorithm is developedthat finds the optimal code apertures to be used in a multishotsystem such that spectrally selective compressive measure-ments are attained. The optimization aims at minimizing theL0 norm between a data cube and a desired cube containingonly the spectral bands of interest. A GA solves the L0 opti-mization problem and finds the minimum number of shotsfor convergence. The optimization algorithm converges toK < L shots when the subset of desired bands exhibits somecircular periodicity, which is originated because any assump-tion is made over the random component of the code aperture.The designed code apertures exhibit a circular shift invarianceproperty that permits to divide the reconstruction of the com-plete data cube in V ≤ K less complex reconstruction pro-blems. This property of the optimal codes can be seen as acompressive filter bank decomposition. The filter bank ap-proach reconstructs either the complete data cube, or a selec-tive subset of bands, with higher peak signal to noise ratio(PSNR) and takes up to two orders of magnitude less proces-sing time than the traditional multishot approach with randomcode apertures.

APPENDIX A: MATRIX Γi

Using the specific structure of D, each element in Eq. (15) canbe written as ðgiqÞj ¼

PMLl¼1ðDÞj;lðWiÞl;lðρÞl: This equation can

be expressed as

ðgiqÞj ¼

8>><>>:

Pjk¼1ðWiÞmj

k;mjkðρÞmj

kfor j¼ 1;…L− 1P

Lk¼1ðWiÞml

k;mjkðρÞmj

kfor j¼ L;…MP

Lk¼j−Mþ1ðWiÞmj

k;mjkðρÞmj

kfor j¼M þ 1;…M þL− 1

;

ðA1Þ

where mjk indexes the kth nonzero component in the jth row

of D. Note that Eq. (A1) can be written more succinctly as

ðgiqÞj ¼XLk¼1

ðWiÞmjk;m

jkðρÞmj

k; ðA2Þ

where we have pointed the remaining mjk indexes in Eq. (A1)

to those rows ofD to have zero elements, such that they do notaffect the sum in Eq. (A2). Define Γi as

ðΓiÞj;k ¼ ðWiÞmjk;m

jkðρÞmj

k

k ¼ 1;…; L; and j ¼ 1;…; M þ L − 1;ðA3Þ

where Γi contains all the information required to calculate giq.Note that most of the elements of the ðM þ L − 1Þ × L matrixΓi are nonzero, in contrast to the ðM þ L − 1Þ × NLÞ matrix Dwhose elements are mostly zero. Using Eq. (A3), Eq. (A2) re-duces to ðgiqÞj ¼

PLk¼1ðΓiÞj;k. Figure 4 illustrates the sheared

Fig. 17. Reconstruction of the 1st and 18th spectral bands indicatedin Fig. 16(a) and 16(b). Reconstruction of the respective band usingthe vector λ4 in Eq. (36) for (c), (d) 12 shots; (e), (f) 16 shots.

Fig. 16. Reconstruction of the 1st and 18th spectral band of the 24band data cube. (a) Original 1st band (b) original 18th band; recon-struction of the respective band using the vector λ4 in Eq. (34).(c), (d) 4 shots; (e), (f) 8 shots.


structure of the elements of the matrix Γi, which are summedalong the axial direction to obtain the elements ðgiqÞj in thedetector.

APPENDIX B: RANGE OF THE bij ELEMENTSUsing d ¼ XduL in Eq. (29), Eq. (30) can be rewritten as

ej ¼ xdj uL −XKi¼1

bijxTj �wi; j ¼ 1;…; M þ L − 1; ðB1Þ

where xdj is the jth row of Xd. Observe that ej ¼ 0 in Eq. (B1)when x0j uL ¼ P

Ki¼1 bijxj �w

i or equivalently

XLk¼1

ðxdj Þk ¼XKi¼1

bijXLk¼1

ðxjÞkð�wiÞk: ðB2Þ

Using the fact that xdj contains only the desired componentpresent in xj , Eq. (B2) can be written as

Xk∈λ

ðxjÞk ¼XKi¼1

bijXLk¼1

ðxjÞkð�wiÞk; ðB3Þ

where λ is the set of desired bands. From Eq. (26), the ele-ments of xj are prime numbers arranged in a specific order.Denote s0k as the set of prime numbers contained in xj , thenEq. (B3) can be written as

Xk∈λ

s0k ¼XKi¼1

bijXLk¼1

s0kð�wiÞk: ðB4Þ

To illustrate the importance of the prime numbers, supposefirst that no linear combination of the measurements is used.In this case bij ¼ 1 for i ¼ j and 0 otherwise, then Eq. (B4) canbe written as

Xk∈λ

s0k ¼XLk¼1

s0kð�wiÞk: ðB5Þ

Observe in Eq. (B5) that the left term is a sum of prime num-bers. Observe also that these prime numbers satisfy Eq. (25),hence there is not a linear combination of different primenumbers on the right hand side of Eq. (B5) that satisfies it.To solve Eq. (B5), it is necessary that the binary ð�wiÞk coeffi-cients eliminate the excess prime numbers on the right handside of Eq. (B5), which is the desired functionality of theð�wiÞk terms.

Now suppose that bij ∈ R in Eq. (B4). In this case, it is notmandatory that there are the sameprime numbers in both sidesof Eq. (B4) to satisfy the equation. In this case the prime num-ber representation and the ð�wiÞk coefficients lose their func-tionality. To illustrate this situation, suppose that in Eq. (B4)s0k ¼ f7; 9; 11; 13g, λ ¼ f1; 0; 1; 0g, �w1 ¼ ½ 0 1 0 0 �, �w2 ¼½ 1 0 0 0 �, andK ¼ 2, then replacing these values results in

7þ 11 ¼ b1jð9Þ þ b2jð11Þ; ðB6Þ

which can be solved by b1j ¼ 2, b2j ¼ 0. Observe that differentprime numbers appear in both sides of Eq. (B6). The ð�wiÞk coef-

ficients, in this case, lose their function of eliminating theexcess prime numbers on the right hand side of Eq. (B4). Tocorrect this situation, the range of the bij elements is restrictedto f−1; 0; 1g. This constraint guarantees the correct functional-ity of prime number representation and the filter functionalityof the ð�wiÞk coefficients in Eq. (30).

APPENDIX C: GENETIC ALGORITHMAPPROACHThe genetic algorithm (GA) approach has as inputs the subsetof desired spectral bands λ, the number of shots K , the primenumbers sk for k ¼ 1;…; L, and the matrix B ∈ f−1; 0; 1gK×K,where K ¼ 3K . The mth column of B matrix bm contains thecoefficients of a possible linear combination of the measure-ments. Observe that the columns of B satisfy bm1 ≠ bm2 foreach m1 ≠ m2. Other parameters of the GA are the numberof candidates of initial population, the mutation constant,and the maximum number of iterations N . The GA takesan initial population of solution candidates Pð0Þ

ℓ ¼f�w1

ℓ ;…; �wKℓ gT for ℓ ¼ 1;…;S, where S is the number of ele-

ments of the population. The elements of the initial populationare realizations of i.i.d. Bernoulli variables, it is ð�wi

ℓÞk ¼Bernoullið0:5Þ for k ¼ 1;…; L. The variable ℓ indexes the indi-vidual of the population and the variable n the number of theiteration. The matrixGn

ℓ contains the CAASI output for the ℓthindividual and the matrix Gn

ℓ consists of all of the possible lin-ear combination of Gn

ℓ rows. The point wise square of the dif-ference between the desired outputs D and the matrix Gn

ℓ isthe matrix En

ℓ , which contains the errors for each linear com-bination. The minimum value of each En

ℓ row is saved in thevector enℓ and its respective index in LðnÞ. The GA calculatesthe performance ðξðnÞÞℓ ¼ eℓ0 for each candidate usingEq. (31). The candidates thus are selected (Sl) through a fit-ness ðξðnÞÞℓ based process, then the crossover (Cr) and muta-tion (Mu) operations are realized over the solutions [22]. Ineach iteration, the index ℓ� and the performance ðξðnÞÞℓ� ofthe best candidate are calculated and compared with the per-formance of the global best solution ξ�. If ðξðnÞÞℓ� < ξ�, thenthe new best solution is updated with f�w�1;…; �w�KgT ¼ PðnÞ

ℓ�along with the elements of B�, which are given by ðB�Þkj ¼ðBÞðnÞkðLÞℓ�j for k ¼ 1;…; L and j ¼ 1;…; M þ L − 1. The algo-rithm finishes when the number maximum of iterations Nis reached or the global minimum performance ξ� is equalto 0. The outputs of the GA are the optimal aperture codes�w�1;…; �w�K and the matrix B�. A resume of the GA ispresented in Table 1.

APPENDIX D: CIRCULAR SHIFTINVARIANCE OPTIMIZED CODEAPERATUREThe circular shift invariance of the optimized code aperturesfor a desired set of bands λ originates in the structure of thedesired signal d used in the code aperture optimization. Sup-pose that there are two sets of desired bands named λ and λ0.Furthermore, the vector λ0 is a mth circular shift version λ,which elements are related by ðλ0Þk ¼ ðλÞðhk−miÞL k ¼ 1;…; L.Each subset of bands λ and λ0 and the sets of prime numbers


sk and s0k can be used to construct the desired signals d and d0,respectively. Suppose that the optimal code apertures �w�i tospectrally select the λ subset are known. From Eq. (B5) theseoptimal codes satisfy

Pk∈λsk ¼

PLk¼1 skð�w�iÞk. The unknown

optimal codes �wi to spectrally select the λ0 subset must satisfy

Xk∈λ0

s0k ¼XLk¼1

s0kð�wiÞk: ðD1Þ

Notice that it is possible to define the s0k prime numbers as

s0k ¼ sðhk−miÞL; k ¼ 1;…; L: ðD2Þ

Given that both right sides of Eqs. (D1) and (D2) are equal andgiven that sk and s0k are related by Eq. (D1), then the optimalaperture codes �wi for the subset λ0 can be calculated bycircularly shifting the known �w�i codes. More specificallythe elements of the optimal codes to spectrally select the λ0

subset are given by ð�wiÞk ¼ ð�w�iÞðhk−miÞL for k ¼ 1;…; L:

ACKNOWLEDGMENTSThis work was supported by the U.S. Office of Naval Research(ONR) under contract NOOO14-1O-C-0 199. The authorsthank Dr. Michael Duncan and Dr. Keith Krapels for severalsuggestions.

REFERENCES1. A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady,

“Spectral image estimation for coded aperture snapshot spectralimagers,” Proc. SPIE 7076, 707602 (2008).

2. A. A. Wagadarikar, R. John, R. Willett, and D. Brady, “Single dis-perser design for coded aperture snapshot spectral imaging,”Appl. Opt. 47, B44–B51 (2008).

3. A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady, “Videorate spectral imaging using a coded aperture snapshot spectralimager,” Opt. Express 17, 6368–6388 (2009).

4. E. Candès, J. Romberg, and T. Tao, “Robust uncertainty princi-ples: Exact signal reconstruction from highly incompletefrequency information,” IEEE Trans. Inf. Theory 52, 489–509(2006).

5. D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory52, 1289–1306 (2006).

6. J. L. Paredes, G. Arce, and Z. Wang, “Ultra-wideband com-pressed sensing: channel estimation,” IEEE J. Sel. Top. SignalProcess. 1, 383–395 (2007).

7. M. F. Duarte and R. G. Baraniuk, “Kronecker product matricesfor compressive sensing,” in Proceedings of the IEEE Interna-

tional Conference on Acoustics, Speech and Signal Processing

(ICASSP) 2010 (IEEE, 2010), pp. 3650–3653.8. J. L. Paredes, G. R. Arce, and L. E. Russo, “Multichannel image

compression by bijection mappings onto zero-trees,” IEEETrans. Image Process. 11, 223–233 (2002).

9. Y. Wu, C. Chen, Z. Wang, P. Ye, G. Arce, D. Prather, and G.Schneider, “Fabrication and characterization of a compressive-sampling multispectral imaging system,” Opt. Eng. 48,123201 (2009).

10. D. Kittle, K. Choi, A. A. Wagadarikar, and D. J. Brady, “Multi-frame image estimation for coded aperture snapshot spectralimagers,” Appl. Opt. 49, 6824–6833 (2010).

11. I. Alabboud, G. Muyo, A. Gorman, D. Mordant, A. McNaught, C.Petres, Y. R. Petillot, and A. R. Harvey, “New spectral imagingtechniques for blood oximetry in the retina,” Proc. SPIE 6631,1–10 (2007).

12. D. Kittle, “Compressive spectral imaging,” Master’s thesis(Duke University, 2010).

13. Y. Wu, I. O. Mirza, G. R. Arce, and D. W. Prather, “Developmentof a digital-micromirror-device-based multishot snapshot spec-tral imaging system,” Opt. Lett. 36, 2692–2694 (2011).

14. P. Ye, H. Arguello, and G. Arce, “Spectral aperture codedesign for multi-shot compressive spectral imaging,” in Digital

Holography and Three-Dimensional Imaging, OSA TechnicalDigest (CD) (Optical Society of America, 2010), paper DWA6.

15. H. Arguello and G. R. Arce, “Code aperture design for compres-sive spectral imaging,” in Proceedings of the 18th European

Signal Processing Conference (EUSIPCO 2010) [EuropeanAssociation for Signal Processung (EURASIP), 2010], pp.1434–1438.

16. J. Romberg, “Compressive sensing by random convolution,”SIAM J. Imaging Sci, 1098–1128 (2009).

17. D. J. Brady, Optical Imaging and Spectroscopy (Wiley, 2009).18. S. J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky,

“An interior-point method for large-scale l1-regularizedleast squares,” IEEE J. Sel. Top. Signal Process., 1 606–617(2007).

19. M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradientprojection for sparse reconstruction: Application to compressedsensing and other inverse problems,” IEEE J. Sel. Top. SignalProcess. 1, 586–597 (2007).

20. J. L. Paredes and G. R. Arce, “Compressive sensing signalreconstruction by weighted median regression estimates,” IEEETrans. Signal Process. 59, 2585–2601 (2011).

21. Z. Wang and G. Arce, “Variable density compressed imagesampling,” IEEE Trans. Image Process. 19, 264–270 (2010).

22. D. E. Goldberg, Genetic Algorithms in Search, Optimization

and Machine Learning (Kluwer, 1989).23. S. Boxwell, S. G. Fox, and J. F. Román, “Design and optimization

of optical components using genetic algorithms,” Opt. Eng.43 (2004).

24. T. Shirakawa, K. L. Ishikawa, S. Suzuki, Y. Yamada, andH. Takahashi, “Design of binary diffractive microlenses withsubwavelength structures using the genetic algorithm,” Opt.Express 18, 8383–8391 (2010).


code aperture optimization for spectrally agile compressive imagingarce/files/research/... ·...

Documents