multispectral imaging in medicine

7/31/2019 Multispectral Imaging in Medicine

1/11

Multispectral Imaging in Biology

and Medicine: Slices of LifeRichard M. Levenson* and James R. Mansfield

CRI, Inc., Woburn, Massachusetts

Received 15 December 2005; Accepted 11 May 2006

Multispectral imaging (MSI) is currently in a period oftransition from its role as an exotic technique to its beingoffered in one form or another by all the major micros-copy manufacturers. This is because it provides solutionsto some of the major challenges in fluorescence-basedimaging, namely ameliorating the consequences of the

presence of autofluorescence and the need to easilyaccommodate relatively high levels of signal multiplexing.MSI, which spectrally characterizes and computationallyeliminates autofluorescence, enhances the signal-to-back-ground dramatically, revealing otherwise obscured tar-gets. While this article concentrates on examples derivedfrom liquid-crystal tunable filter-based technology, theintent is to showcase the advantages of multispectral ima-ging in general. Some technologies used to generate mul-tispectral images are compatible with only particular opti-cal configurations, such as point-scanning laser confocalmicroscopy. Band-sequential approaches, such as thoseafforded by liquid-crystal tunable filters (LCTFs), can beconveniently coupled with a variety of imaging modal-

ities, which, in addition to fluorescence microscopy,include brightfield (nonfluorescent) microscopy as wellas small-animal, noninvasive in-vivo imaging. Brightfieldmicroscopy is the chosen format for histopathology,

which relies on immunohistochemistry to provide mole-cularly resolved clinical information. However, in contrast

to fluorescent labels, multiple chromogens, if they spa-tially overlap, are much harder to separate and quantitate,unless MSI approaches are used. In-vivo imaging is a rap-idly growing field with applications in basic biology, drugdiscovery, and clinical medicine. The sensitivity of fluo-rescence-based in-vivo imaging, as with fluorescence mi-croscopy, can be limited by the presence of significantautofluorescence, a limitation which can be overcomethrough the utilization of MSI. q 2006 International Societyfor Analytical Cytology

Key terms: in-vivo imaging; fluorescence; autofluores-cence; immunohistochemistry; multiplexing

Multispectral imaging is the acquisition of spectrallyresolved information at each pixel of an imaged scene.Many different technologies can be employed to generatesuch information, ranging from multi-position filter-wheels,gratings and prisms, laser-scanning single point spectro-graphs, electronically adjustable tunable filters, Fourier-transform imaging spectrometry, and computed tomo-graphic imaging spectroscopy (reviewed in Ref. (1)). Thisreport will highlight the use of liquid crystal tunable filter-(LCTF-) based multispectral imaging approaches, along

with application-specific analysis tools, for a variety of ima-

ging tasks, but should also be read as a presentation of theadvantages of MSI in general.

FLUORESCENCE MICROSCOPY

Fluorescence imaging of paraffin-embedded, formalin-fixed tissues is often confounded by interfering autofluor-escence, which can reduce the ability to detect the fluoro-phore(s) of interest. Unfortunately, what is worse is thattissue autofluorescence can easily be mistaken for a signalof interest, leading to erroneous results. Tissue autofluor-

escence is present to some degree at all excitation wave-lengths, although it is strongest when UV- or blue-excita-tion ranges are employed. Formalin-fixation greatly en-hances autofluorescence, and components such as collagen,red blood cells, and neuronal constituents can be particu-larly bright (25).

The ability to accurately quantitate the fluorescenceemission of a labeled target requires that the signal beingmeasured comes only from the fluorophore of interest,and not from a mixture of autofluorescence and fluoro-phore. Numerous efforts have been made to chemically

eliminate tissue autofluorescence (4,6,7), none of whichhave been entirely successful, sometimes because they failto completely eliminate the effects of autofluorescence

*Correspondence to: Richard M. Levenson.E-mail: [email protected]

The authors of this report are both employees of CRI, Inc.Grant sponsor: NIH; Grant numbers: 1 RO1 CA108468 and 1 R44

CA88684.

Published online in Wiley InterScience (www.interscience.wiley.com).DOI: 10.1002/cyto.a.20319

q 2006 International Society for Analytical Cytology Cytometry Part A 69A:748758 (2006)


2/11

and sometimes because they create more variability in thesample than they remove.

Using multispectral imaging to separate the contribu-tions of the various fluorophores in a sample into theirown images, or channels, is an effective method of im-proving both contrast (or the signal-to-noise ratio) and the

quantitative accuracy of the measurement. Although MSImethodologies are, by their nature, applicable to all fluor-ophores, the combination of MSI methods with quantumdot-based labels and the like may be particularly fruitful(810). Quantum dots, when properly prepared and deri-

vatized, are relatively immune to the effects of photo-bleaching. Moreover, they have comparatively narrowemission bandwidths, established by the size of the quan-tum dots cores, and can have large Stokes shifts. Thismakes the multiplexing of fluorophores much simpler,since a single excitation wavelength can be used to exciteall the species present simultaneously.

IN-VIVO IMAGING

Over the past decades, imaging has become a criticalcomponent of medicine with striking advances in MR- andCT-based imaging methods. For the most part, however,these imaging modalities reveal anatomical rather thanmolecular features, and while this has proved useful,many of the molecular and cellular changes that occur atthe onset of a disease are not detectable with purely ana-tomical imaging. Even when disease states become evi-dent in anatomical imaging methods, the information canbe hard to interpret since it is not directly related to mo-lecular entities, such as proteins or expressed genes; in-

vivo molecular imaging is a recent development that aimsto provide such information. While not usually covered incytometry journals, the topic of in-vivo imaging is relevant

to this discussion of spectral methodology, not onlybecause the same complement of imaging hardware andsoftware tools can be applied, but also because in-vivoimaging techniques must often be validated with micros-copy-based methods, which can themselves take advan-tage of spectral information.

For a variety of reasons (including cost and regulatoryobstacles), it is likely that in the near future much of mo-lecular imaging will remain restricted to use in whole-ani-mal basic research and the preclinical phases of the drugdevelopment cycle, where such approaches have alreadyproved to be of tremendous utility. The ability to performrelevant, minimally or noninvasive imaging helps reducecosts and enables longitudinal studies of multiple pro-

cesses and parameters in individual animals. PET and MRI,and more recently, optical imaging have been adapted tofacilitate these studies through the use of specialized con-trast agents and imaging probes that either identify theexistence of certain genetically modified cells or tissues,or that produce signals roughly proportional to regionalmolecular abundance or the rate of specific events in mo-lecular pathways (1118). In human subjects, optical ima-ging is likely to remain restricted to relatively superficialtargets because of the absorbing and scattering propertiesof tissue in the visible and near-infrared (NIR). However,

these issues are less significant in preclinical (typicallymouse) animal models: their smaller sizes allow the detec-tion of sufficient photon flux at the surface of the animalto allow detection of signals with acceptable signal-to-noise ratios.

Optical molecular imaging systems and methodologies

have been developed that use both bioluminescent (19)and fluorescent (20) signals. Bioluminescent systems typi-cally use luciferase genes coupled with luciferin substratesas reporters. The major attraction of this approach is thatalthough absolute light levels generated by the targetsmay be low, photons are generated generally only whereluciferase is present, leading to low background signals. Incontrast, fluorescence-based imaging requires an externallight source to stimulate the emission of light from theprobe, and may be accompanied by bright backgroundsignals arising from the animals intrinsic autofluores-cence. However, fluorescence is a more flexible technol-ogy, since it permits the use of a far wider range of probes,labeling methods and targets, and can be used with labels

that emit in the NIR, the spectral sweet spot for deep tis-sue in-vivo imaging. The number of photons emitted isorders of magnitude greater than with bioluminescence;the presence of autofluorescence is what generally limitsachievable target-to-background ratios (21).

MSI fluorescence-based methodologies can be used toimage practically any of the fluorophores used in biomedi-cal research, with best results typically achieved when theemission wavelengths of the dye are between 500 and950 nm (22). These fluorophores can be xenografted intoeither a tumor implanted in an animal or can be generatedby transgenic animals; they can be covalently bound toantibodies, peptides or other agents that bind to targets;or they can simply be fluorescent compounds that are

introduced into an animal. An example of antibody-tar-geted spectral imaging and analysis can be found in thereport by Gao et al., examining the distribution of quan-tum-dot-labeled antitumor antibodies in mice (23).

BRIGHTFIELD MICROSCOPY

While fluorescence microscopy has long been themethod of choice for true molecular imaging, it is notfavored in the clinical realm populated by pathologistsand their colleagues, who typically prefer brightfield tech-niques employing immunohistochemistry (IHC) to deter-mine the distribution and abundance of specific molecularfeatures in cells and tissues. Typically, a single target mole-cule is detected using antibodies linked to chromogenic

read-out systems, and the tissue is counterstained with ageneral stain such as hematoxylin or other contrastingagent to provide anatomic context. Detection and quanti-tation of single signals has been possible for a number of

years using simple color cameras and appropriate separa-tion and analysis software (for example, see Refs. (24,25)).Preparation of double- and even triple-stained IHC sam-ples are now feasible for routine use thanks to the adventof automatic staining machines and appropriate labelingreagents, but to date these are only employed if the targetsthat are stained do not spatially overlap (co-localize), since

749MULTIPURPOSE MULTISPECTRAL MULTIPLEXING

Cytometry Part A DOI 10.1002/cyto.a


3/11

neither our eyes, nor conventional color cameras, are ca-pable of easily resolving mixtures of chromogens. Theconventional approach to obtaining information on anumber of molecular targets from a single sample is to cutserial sections and stain each one with a different anti-body. While this is straightforward in principle, there are

drawbacks. The microscopist needs to look at multiplesections (some of which may no longer contain the tissueof interest) and correlate distribution of the antigens ei-ther mentally or with some potentially complicated ima-ging scheme. Of more importance, perhaps, is the factthat information on multiple markers is not available on acell-by-cell basis, but only on populations of cells, sinceindividual cells rarely span more than one section. Thus,correlative studies that look for co-occurrence of molecu-lar phenotypes within cells (the true functional units) arenot possible without multiplexing. As will be shown,spectral imaging is capable of allowing single cell multi-plexed imaging using spatially overlapping chromogens inbrightfield.

MATERIALS AND METHODSSpectral Imaging Hardware

All the examples shown here were acquired using CRis(Woburn, MA, 01801) commercially available LCTF-basedimaging systems. The core technology has been describedelsewhere (2628) and compared in some detail to otherspectral imaging techniques (1). Some of the features ofLCTFs that make them suitable choices for many situationsare as follows. They are band-sequential filters that areeasily coupled to focal plane array detectors employingCMOS, standard CCD, or EMCCD technologies, amongothers. By sequentially tuning the filter and exposing the

sensor, complete images are acquired at each wavelength,band by band. Unlike some other techniques, for example,those that use prisms or gratings to disperse and collectall wavelengths simultaneously, this design allows the userto vary the exposure time as a function of wavelength,thus optimizing signal-to-noise in situations where sensi-tivity (emitted photons convolved by imaging receivercharacteristics) varies over the spectral range. Moreover,the wavelengths acquired can be arbitrarily spacedthrough the spectral range of interest, allowing the user tomaximize signal-to-noise by acquiring only the most in-formative bands (29). Other advantages include the ab-sence of moving parts, excellent optical properties yield-ing near-diffraction-limited images, spectral stability to

fractions of a nanometer, and high reliability.As always, there are also some disadvantages related to

band-sequential approaches in general and/or LCTFs inparticular. A band-sequential approach implies that thecomplete image stack (or cube) is built up over time;thus, if significant sample- or camera-movement occursduring the acquisition, or if high temporal resolution isneeded to capture certain events, like calcium signalingtransients, simultaneous multiband acquisition strategiescould be more appropriate. Also, photobleaching duringacquisition can be a concern, since some wavelengths will

be acquired after others, and therefore will be subject toillumination longer before being detected. However, aslong as some signal is still detectable, unmixing can beaccomplished, and the relative intensity losses can beaccounted for, if absolute quantitation (something of a chi-mera) is desired. Overall light throughput can be a con-

cern: LCTFs use polarization in their spectral selectionprocess, and transmission efficiencies are typically in the30%-range, in comparison to traditional interference filtersthat can transmit 90% of incoming light. However, themetric of success in imaging is usually not total photonscaptured, but achievable signal-to-noise (or signal-to-back-ground), and except where high speed is required, thebenefit of spectral information gained generally outweighsthe impact of lower transmission efficiencies. For exam-ple, compare a grayscale to a color image: as much as 2/3or so of the available light is lost in a color sensor due tothe presence of redgreenblue filter masks; however, theuseful information content of a color image, captured withfewer photons, can be vastly more than that of a mono-

chrome image of the same scene.

Image Acquisition

The Nuance and Maestro multispectral imaging systemsused in this work both incorporate a LCTF opticallycoupled to a 1.3 megapixel CCD camera (Sony ICX285CCD chip). The images in this manuscript were acquiredusing the full CCD frame at either 1 3 1 binning (1,360 31,024 pixels) or 23 2 binning (6803 512 pixels).

1. Fluorescence microscopy: In general, a NuanceTM

spectral imaging system (CRi) is mounted onto a conven-tional fluorescence microscope equipped with a filter

cube comprised of a standard excitation filter and dichroicmirror and a long-pass emission filter, 615 or so images

were taken at fixed exposure times (typically around100 ms per image) every 10 nm throughout the spectralrange. The resulting datasets are saved as a series of 12-bitmonochrome TIFFs. The fluorescence data in this article

were acquired using a Zeiss Axioplan microscope, a 203objective and a DAPI filter cube equipped with a 480-nmlong-pass emission filter.

2. Brightfield microscopy: Before acquiring a spectraldataset in brightfield, an autoexposure routine was per-formed while imaging a blank area of a slide to determinethe exposure time necessary to nearly fill (to about 90%)the CCD wells at each wavelength. A white cube, or ref-

erence cube, was then acquired, followed by spectral ima-ging of the sample, with both cubes being acquired usingthe same exposure times. Finally, the spectral data wassimultaneously flat-fielded and converted from transmis-sion to optical density units by taking the negative log ofthe ratio of the sample divided by the white cube usinga Beers Law conversion (30).

3. In-vivo imaging: CRis MaestroTM instrument wasused to acquire the data in a manner analogous to thatdescribed for spectral fluorescent microscopy except thatno dichroic mirror was included in the light path and the

750 LEVENSON AND MANSFIELD



4/11

excitation light was brought to the sample using the Mae-stro fiber-optic illumination system. Exposure times of200500 ms per spectral band are typical. The spectralranges used are indicated in the figures.

Image Analysis

RGB (redgreenblue) color images were synthesizedfrom the spectral cube by mapping the spectral data intothose color channels. Either true-color (in which spectralregions are mapped faithfully into their correspondingRGB channels) or false-color displays can be generated;the latter are useful when signals in the near-infrared

(NIRby definition mostly invisible to human vision) areacquired. All the images identified as RGB images in thisreport are derived from the spectral datasets and not fromconventional color sensors. Spectral library development,including automated tools to identify spectral features,spectral unmixing, and composite image creation wereperformed as described elsewhere (22,31). Images wereunmixed with no further manipulation except being scaledfor display.

Unique aspects of the methods used here include purespectral computation and automated spectral feature detec-

tion. Compute Pure Spectrum (CPS) is used to extractthe authentic spectrum of a fluorophore if it is contami-nated with autofluorescence in the sample (the typicalcase). In essence, it subtracts the autofluorescence signalfrom the mixed signal to yield the pure label spectrum.

Automated spectral feature extraction (real componentanalysis or RCA) is described in detail in a recent reference(22). Finally, as utilized in the example shown in Figure 3, itis possible to subtract a spectral background from an entiredatacube with a specified intensity, when the data is in fluo-rescence or absorbance (but not transmittance) format. Forexample, this can be used to remove excess hematoxylinstain from cytoplasm but leave behind the denser signals inthe nucleitypically with little effect on the relative inten-sities of other specific labels.

RESULTSFluorescence Microscopy

Two examples are presented for fluorescence micros-copy. The first (Fig. 1) demonstrates the ability of spectralimaging to separate a dim fluorescein signal from muchbrighter and spectrally similar tissue autofluorescence in aformalin-fixed, paraffin-embedded prostate specimen. Theantibody targeted the basal cells present in normal pros-

FIG. 1. Fluorescein-labeled basal cells in formalin-fixed prostate tissue. Panel A shows an RGB representation of the dataset, with the fluorescein signalbarely visible due to abundant autofluorescence. The spectral graphs (B) indicate the spectra of the autofluorescence (AF) signal (in pink); the mixed spec-trum of fluorescein plus AF (cyan) and the computed fluorescein spectrum (green). Unmixing using the AF and computed fluorescein spectra, images weregenerated to reveal the specific staining of the basal cells (C) and a composite image compr ised of specific staining plus AF (in pink) is shown in D.




5/11

tate glands. The RGB image in panel A shows the appear-ance of the sample as one would see it through the eye-pieces of the microscope or as color camera would cap-ture the image. The spectral graphs (B) indicate the spec-tra of the autofluorescence (AF) signal (in pink); thespectrum of fluorescein plus AF (cyan), and the computedfluorescein spectrum (green). Note that the fluoresceinspectrum completely overlaps with the AF spectrum;employing a simple narrow-band emission filter system toimage this sample, as is often recommended for samples

with a high degree of autofluorescence, would not elimi-nate the contrast problem caused by the AF. Using the AFand computed fluorescein spectrum as inputs to the

unmixing step, images were generated to reveal the speci-fic staining of the basal cells (C), and a composite imagecomprised of specific staining plus AF (in pink) is shownin D. Software tools are provided that allow for individualplanes of the composite image to be turned on or off, andbe adjusted in terms of brightness and contrast. Here, fordisplay clarity, the AF signal was dimmed relative to thefluorescein signal (compare A to D). However, these dis-play adjustments do not affect the unmixed data them-selves, which can be used to generate quantitative mea-surements.

There are several regions in the RGB image where theAF signal is considerably brighter than the fluorescein sig-nal. MSI can enhance the detectability of weak fluoro-phores by isolating their contribution, even when theirspectral signatures overlap completely with that of AF. Inother words, before unmixing, the specific signal mayconstitute a small increment over a bright base; after un-mixing, the signal, even if it is small, is contrasted againsta near-zero background.

Figure 2 illustrates the application of spectral unmixingto another challenging specimen, in this case, brain (cere-bellar region) labeled with two quantum-dot-coupled anti-bodies directed against glial fibrillary acid protein (GFAP,

605-nm QDot, yellow) and neurofilamin (NF, 655-nmQDot, red; data courtesy Ventana Medical Systems,Tucson, AZ). Nuclei were stained with DAPI. While DAPInormally generates a blue signal, because this dataset wasacquired in the spectral range, 530680 nm, only its greenemission tail is detected here. In addition to these speci-fic signals, a ubiquitous greenyellow AF was also detectedand removed from the final image to improve contrast.

Panel A shows an RGB image of the fluorescence of thesample. The yellow and red striations of GFAP and neurofi-lamin (NF) protein, respectively, are visible, although not

FIG. 2. Formalin-fixed cerebellum labeled with two quantum dots. Panel A shows an RGB image of the sample with glial fibrillary acidic protein (GFAP)immunolabeled with a 605-nm Qdot, and neurofilamin (NF) immunolabeled with a 655-nm Qdot. Nuclei were labeled with DAPI. In addition to the specificlabels, tissue autofluorescence is present. The insert shows the spectra that were derived from this datacube (see Materials and Methods) and used tounmix into the four component images (B through D), whose border colors correspond to the colored spectral graphs and to the pseudocolors used toform the composite image, F. Thus, B identifies the nuclear signal, C, the 605-nm GFAP signals, D, the 655-nm NF signals, and E, tissue autofluorescence.Because it is unmixed in the black channel, it is invisible in the panel F, which accounts for the greater clarity in panel F vs. the original (panel A). Samplecourtesy of Ventana Medical Systems.




6/11

clearly, against a ubiquitous greenish autofluorescencebackground. Specific nuclear autofluorescent signals arealso present and can be seen most prominently at theright of the image. The insert shows the spectra that werederived from this datacube using the automated spectralspecies detection tool, RCA (see Materials and Methods),and used to unmix into the four component images (Bthrough D), whose border colors correspond to thecolored spectral graphs and to the pseudocolors used to

form the composite image, F. Thus, B identifies the speci-fic nuclear autofluorescence signal, C, the 605-nm GFAPsignals, D, the 655-nm NF signals, and E, generic autofluor-escence. Because it is unmixed in the black channel, itis invisible in the panel F, which accounts for the greaterclarity in panel F vs. the original (panel A).

Brightfield Immunohistochemistry

Brightfield imaging of IHC-labeled specimens with bothspatially nonoverlapped and overlapped signals is illu-

strated in Figures 3 and 4. Figure 3 illustrates the practicaladvantage of being able to detect and separate commonlyused brown (DAB, or 3,30-diaminobenzidine) and red (FastRed) chromogens, which may be difficult to unravel

visually in densely packed, complex scenes, particularlywhen there is co-localization of the chromogens. In Fig-ure 3, the sample is thymus tissue stained to displayCD41 and CD81 cells. In a mature thymus these shouldbe distinct cell populationsthus no signal-overlap is

expected. As can be seen in the RGB image, an overly darkhematoxylin counterstain is also present. Panel B illustrateshow some of the hematoxylin can be digitally removed bysubtracting an amount sufficient to essentially isolate thesignal to the cell nuclei (see Materials and Methods). PanelC shows the spectra for hematoxylin, DAB, and Fast Redused to unmix the subtracted data into the individual com-ponents shown in panels D, E, and F. Panel G is a compos-ite image of the unmixed components, a detail of which isshown in H. Finally, panel I illustrates how the hematoxy-lin channel can be turned off, to display only the signals

FIG. 3. Chromogenic double-labeling for CD41 and CD81 cells in thymus. CD41 were stained with 3,30-diaminobenzidine (DAB, brown) and CD81cells were stained with Fast Red, and counterstained with hematoxylin. RGB image is shown in panel A. Subtracting excess hematoxylin signal digitally isillustrated in Panel B (see Materials and Methods). Spectra (panel C) for hematoxylin, DAB, and Fast Red were used to unmix the subtracted data into theindividual components (panels D, E, and F). Panel Gis a composite image of the unmixed components, a detail of which is shown in H. Finally, panel I illus-trates how the hematoxylin channel can be turned off, to display only the signals from the two IHC chromogens. Sample courtesy Dr. Chris van der Loos.




7/11

from the two IHC chromogens. As can be seen, the CD41and CD81 cells are separate populations, since only puregreen and red signals are present.

Figure 4 illustrates a breast cancer specimen with threechromogens overlapping in the cell nuclear compartment.The hematoxylin counterstain is present in all nuclei,

while antibody labeling of estrogen receptors (ER, in

brown) and progesterone receptors (PR, in red) is presentis partially overlapping subsets of tumor cells. Panel Ashows the original specimen, and the spectra of the threecolor signals used for unmixing into the separate channelsis shown in B. The hematoxylin, DAB, and Fast Red signals

were unmixed without evident crosstalk. Panel C showsthe hematoxylin signal which can be used to identify thelocation of all nuclei. Panels D, E, and F show binarymasks indicating the location of the PR, ER, and co-loca-lized signals respectively. The masks are created by settingthresholds on the individual unmixed chromogen images

and then overlaying these on the original RGB image fordisplay.

An alternative way of displaying the data is shown inpanels G, H, and I, which show in detail a region from theupper center of the whole image. The data are inverted sothat the signals look like and behave like fluorescence sig-nals (thus, green and red combine to form yellow). The

PR and ER signals have been colored red and green asbefore. The combined image (panel I) therefore showsany double-labeled nuclei as yellow. Further quantitativeimage analysis techniques can be used to generate dataindicating more detailed information about the presenceand degree of co-expression on a cell-by-cell basis.

In-Vivo Imaging

The benefits of spectral imaging are especially markedfor the detection of faint fluorophore signals commingled

FIG. 4. Chromogenic double-labeled estrogen receptor (ER) and progesterone receptor (PR) in breast cancer. ER is labeled with DAB and PR is labeledwith Fast Red in the presence of a hematoxylin nuclear counter stain. Panel A shows the original specimen and the spectra of the three color signals usedfor unmixing into the separate channels is shown in B. Panel C shows the hematoxylin signal which can be used to identify the location of all nuclei. PanelsD, E, and F show binary masks indicating the location of the PR, ER, and co-localized signals, respectively. An alternative display method is illustrated inpanels G, H, and I, which show in detail a region from the upper center of the whole image. The data are inverted so that the green and red signals combineto form yellow where they overlap. Sample courtesy of Chris Kerfoot, Mosaic Laboratories.




8/11

with spectrally overlapping autofluorescence. An exampleis shown in Figure 5, which presents a pair of nude micethat have been injected subcutaneously with three fluoro-phores (FITC, TRITC, and Cy3.5) and that also exhibit skinautofluorescence and food autofluorescence. The chal-lenge here is both to detect and separate these overlap-ping fluorophores from each other, which, except forFITC, are barely visible in the RGB image (panel A). Panel

B shows the AF spectrum in pink, and the purified spec-tra of the FITC (green), TRITC (blue), Cy3.5 (red), andfood (yellow) generated using the compute-pure-spectra(CPS) tool as described in Materials and Methods. Thesespectra are consistent with published spectra. Panels Cthrough G show the unmixed components: FITC (panelC); TRITC (panel D); and Cy3.5 (panel E); food (panel F);and skin autofluorescence (panel G). All panels show sig-nals which are well isolated from the others. Panel H is acomposite image of the unmixed components, presentedin their respective pseudocolors.

For variety, in Figure 6, we show a different problematicautofluorescence source, namely pericardiac fat in a pigheart imaged ex vivo. The experiment consisted of inject-ing multiple fluorescently labeled beads into a cardiac ar-tery in an anesthetized pig to investigate the influence ofparticle size on perfusion patterns. The labels consisted ofthree sets of fluorescently tagged latex beads ranging insize from 0.1 to 20 lm in diameter (only two of which can

be seen with the excitation settings used in this imagingexample). After injection, the heart was removed for spec-tral imaging. The complete results will be reported else-

where; here we demonstrate the separation of a bright sig-nal from fat from spectrally similar green and orange latexbeads, as well as from the dim autofluorescence of generaltissue autofluorescence.

An RGB image of the heart is shown in Panel A. Thelarge greenorange region is the area of distribution of theinjected fluorescent beads. The bright green object to thetop right is pericardiac fat, and the rest of the heart is visi-

FIG. 5. Nude mice with two different species of autofluorescence and three subcutaneous fluorophore signals. The mice were injected subcutaneouslywith FITC, TRITC, and Cy3.5. The RGB image is shown in panel A. Panel B shows the AF spectrum (pink), and purified spectra of the other componentswith the AF contr ibutions computationally removed (see Materi als and Methods, FITC, green; TRITC, blue; Cy3.5, red; food, yellow). Panels C through Gshow the unmixed components with essentially no cross-talk despite spectral overlap. FITC, Panel C; TRITC, Panel D; Cy3.5, Panel E; Food, Panel F; andskin autofluorescence, Panel G. Panel H is a composite image of the unmixed components displayed in their respective pseudocolors.




9/11

ble due to general tissue autofluorescence. Panel B indi-cates the spectra corresponding to these features: general

AF, pink; bead 1, green; bead 2, red; fat AF, yellow. Notehow all these species overlap spectrally. Panels C, D, E, andF are the spectrally unmixed images corresponding to bead1, bead 2, fat AF, and general AF, respectively. Little cross-talk is present. Ideally, the general AF should have been uni-

formly grey even where the beads were located. However,it is likely that the dense bead presence may have blockedthe excitation light reaching the tissue and/or the emittedlight returning from it. Finally, a composite image (minusthe general AF signal for clarity) is shown in panel G.

To achieve these results, it proved necessary to measurethe spectra of the beads (plus heart AF) in situ and then tocompute the pure spectra of the individual fluorescentbeads as described earlier. Using the spectra of the beadsmeasured in suspension (rather than in the heart) did notprovide optimal unmixing results, even though the mea-

sured and computed spectra were fairly similar (data notshown). This reinforces the advisability of using spectra asmeasured during imaging particular samples, rather thanrelying on previously determined spectra collected underdifferent conditions of excitation, absorbance, and scatter-ing. The source of the autofluorescence signal arising inthe fat has not been determinedliterature is scarce on

the subject. It is possible that fat-soluble dietary compo-nents with appropriate fluorescence properties have accu-mulated in fatty tissues in this pig. For example, maize hasbeen shown to express several fluorescent species emit-ting in the green and yellow range (32), consistent withthe fat-signals seen here.

DISCUSSION

There are some specific aspects of MSI in fluorescencemode that merit further discussion. In this mode, spectralunmixing benefits measurements of label fluorescence in

FIG. 6. Excised porcine heart injected in vivo with fluorescent perfusion markers. Green and red beads with different dimensions and thus different tissuedistribution patterns were injected in-vivo into the anterior circumflex artery of a pig. After sacrifice, the heart was spectrally imaged. An RGB image of theheart is shown (Panel A). The large yellowgreen region is the area of bead-distribu tion. The bright green object to the top right represent emissions frompericardiac fat, and the rest of the heart is visible due to general tissue autofluorescence. Panel B indicates the spectra corresponding to these features: gen-eral AF, pink; bead 1, green; bead 2, red; fat-AF, yellow. Panels C, D, E, and F are the spectrally unmixed images corresponding to bead 1, bead 2, fat-AF, andgeneral AF, respectively. Little crosstalk is present. A composite image (minus the general AF signal for clarity) is shown in panel G. Sample courtesy Dr.Roger Hajjar, Massachusetts General Hospital.




10/11

two ways. First, there is the advantage of improved quanti-tative accuracy by means of the appropriate partitioningof the optical signal into its various sources (target(s) andautofluorescence(s)). The presence of noise in the mea-surements will affect the overall accuracy of spectralunmixing by adding uncertainty to the mathematical ma-

trix-inversion procedure (33). However, exposures aretaken at multiple wavelengths, thereby effectively increas-ing the number of photons detected, and reducing theeffective shot-noise. The degree to which this is a benefitis related to how distinct (orthogonal) the spectral vectorsare that reflect the various fluorescent sources in asample.

With respect to choice of imaging sensor, it is importantto realize that when either autofluorescence or thedesired signals are relatively bright, one may be free to usemid-range ($5,000$15,000) rather than high-end sensors,since for shot-noise-limited situations, in which the chiefnoise-source scales with the brightness of the capturedlight, unmixing will improve signal-to-noise much more

than will moving to a low-noise camera. In low-light cases,the approach typically used for increasing signal-to-noiseis to employ more sensitive, highly cooled, low-read-noise(and therefore expensive) CCD cameras. However, forcontrast-limited situations as opposed to read-noise lim-ited situations, increased sensor sensitivity merely resultsin capturing autofluorescence more rapidly. Camera noisetypical of mid-range scientific cameras in such cases is usu-ally immaterial since shot-noise will be much larger thancamera noise for all reasonable exposure times (i.e., thoseshort enough to make thermal noise insignificant) (34).

CONCLUSION

We have shown that a spectral imaging approach based

on band-sequential acquisition is suitable for many differ-ent imaging modalities, including microscopy in bothbrightfield and fluorescence, and whole-animal in-vivoimaging. In brightfield, at least three spatially and spec-trally overlapping chromogens can be quantitatively re-solved, thereby enabling multiparameter molecular ima-ging with labeling techniques generally favored by pathol-ogists and others who, for various practical and/orhistorical reasons, are not inclined to use fluorescence-based imaging approaches. In fluorescence microscopy,

we show the removal of significant autofluorescence pres-ent in typical clinical tissue specimens as well as demon-strate the utility of MSI with multiplexed quantum-dotlabeled specimens. In addition to resolving multiple spe-

cies, autofluorescence can be either suppressed or ex-ploited as an endogenous signal. Finally, fluorescence-ima-ging of small animals or excised organs benefits from mul-tispectral data acquisition and analysis.

In all cases, the technology used for these exampleswas straightforward and robust. The microscope systemscan easily be moved from microscope to microscope, andthe current software for both the microscope and in-vivoimaging systems is very similar, thus flattening the learn-ing curve for users who may want to take advantage ofboth imaging platforms.

ACKNOWLEDGMENTS

The authors acknowledge their research partners andcustomers who have generously allowed them to sharetheir images, including Ventana Medical Systems (for Fig.2); Dr. Chris van der Loos, Academical Medical Center,

Amsterdam; (for Fig. 3); Chris Kerfoot, Mosaic Laborato-

ries (for Fig. 4); Umar Mahmood and Jenny Tam, Massa-chusetts General Hospital (for Fig. 5); and Roger Hajjar,Massachusetts General Hospital (for Fig. 6). They alsothank the editor and the anonymous reviewers for theircareful reading and insightful comments.

LITERATURE CITED1. Bearman G, Levenson R. Biological Imaging Spectroscopy. In: Vo-Dinh

T, editor. Biomedical Photonics Handbook. Boca Raton: CRC Press;2003. p 8_18_26.

2. Bottiroli G, Croce AC, Locatelli D, Marchesini R, Pignoli E, Tomatis S,Cuzzoni C, Di Palma S, Dalfante M, Spinelli P. Natural fluorescence ofnormal and neoplastic human colon: A comprehensive ex vivostudy. Lasers Surg Med 1995;16:4860.

3. Zonios GI, Cothren RM, Arendt JT, Wu J, Vandam J, Crawford JM,Manoharan R, Feld MS. Morphological model of human colon tissuefluorescence. IEEE Trans Biomed Eng 1996;43:113122.

4. Baschong W, Suetterlin R, Laeng RH. Control of autofluorescence of archi-val formaldehyde-fixed, paraffin-embedded tissue in confocal laser scan-ning microscopy (CLSM). J Histochem Cytochem 2001;49:15651572.

5. Lowry A, Wilcox D, Masson EA, Williams PE. Immunohistochemicalmethods for semiquantitative analysis of collagen content in humanperipheral nerve. J Anat 1997;191(Part 3):367374.

6. Clancy B, Cauller LJ. Reduction of background autofluorescence inbrain sections following immersion in sodium borohydride. J Neu-rosci Methods 1998;83:97102.

7. Neumann M, Gabel D. Simple method for reduction of autofluores-cence in fluorescence microscopy. J Histochem Cytochem 2002;50:437439.

8. Ho YP, Kung MC, Yang S, Wang TH. Multiplexed hybridization detec-tion with multicolor colocalization of quantum dot nanoprobes. NanoLett 2005;5:16931697.

9. Gao X, Nie S. Quantum dot-encoded beads. Methods Mol Biol 2005;303:6171.

10. Voura EB, Jaiswal JK, Mattoussi H, Simon SM. Tracking metastatic tu-mor cell extravasation with quantum dot nanocrystals and fluores-

cence emission-scanning microscopy. Nat Med 2004;10:993998.11. Weissleder R, Mahmood U. Molecular imaging. Radiology 2001;219:

316333.12. Ntziachristos V, Ripoll J, Wang LV, Weissleder R. Looking and listening

to light: The evolution of whole-body photonic imaging. Nat Biotech-nol 2005;23:313320.

13. Hoffman R. Green fluorescent protein imaging of tumour growth,metastasis, and angiogenesis in mouse models. Lancet Oncol 2002;3:546556.

14. Contag CH, Bachmann MH. Advances in in vivo bioluminescence ima-ging of gene expression. Annu Rev Biomed Eng 2002;4:235260.

15. Bornhop DJ, Contag CH, Licha K, Murphy CJ. Advance in contrastagents, reporters, and detection. J Biomed Opt 2001;6:106110.

16. Weissleder R, Tung CH, Mahmood U, Bogdanov A Jr. In vivo imagingof tumors with protease-activated near-infrared fluorescent probes.Nat Biotechnol 1999;17:375378.

17. Herschman HR. Molecular imaging: Looking at problems, seeing solu-tions. Science 2003;302:605608.

18. Montet X, Ntziachristos V, Grimm J, Weissleder R. Tomographic fluo-

rescence mapping of tumor targets. Cancer Res 2005;65:63306336.19. Rice BW, Cable MD, Nelson MB. In vivo imaging of light-emitting

probes. J Biomed Opt 2001;6:432440.20. Graves EE, Ripoll J, Weissleder R, Ntziachristos V. A submillimeter re-

solution fluorescence molecular imaging system for small animal ima-ging. Med Phys 2003;30:901911.

21. Troy T, Jekic-McMullen D, Sambucetti L, Rice B. Quantitative compari-son of the sensitivity of detection of fluorescent and bioluminescentreporters in animal models. Mol Imaging 2004;3:923.

22. Mansfield JR, Gossage KW, Hoyt CC, Levenson RM. Autofluorescenceremoval, multiplexing, and automated analysis methods for in-vivo flu-orescence imaging. J Biomed Opt 2005;10:41207.

23. Gao X, Cui Y, Levenson RM, Chung LW, Nie S. In vivo cancer targetingand imaging with semiconductor quantum dots. Nat Biotechnol2004;22:969976.




11/11

24. Ruifrok AC. Quantification of immunohistochemical staining by colortranslation and automated thresholding. Anal Quant Cytol Histol1997;19:107113.

25. Ruifrok AC, Johnston DA. Quantification of histochemical stainingby color deconvolution. Anal Quant Cytol Histol 2001;23:291299.

26. Hoyt C. Liquid crystal tunable filters clear the way for imaging multip-robe fluorescence. Biophotonics Int 1996;4951.

27. Miller PJ, Hoyt CC. Multispectral imaging with a liquid crystal tunablefilter. Proc SPIE-Int Soc Opt Eng 1995;2345:354365.

28. Gat N. Imaging spectroscopy using tunable filters: A review. ProcSPIE-Int Soc Opt Eng 2000;4056:5064.

29. Miller PJ, Harvey AR. Signal to noise analysis of various imaging sys-tems. Proc SPIE-Int Soc Opt Eng 2001;4259:1621.

30. Mansfield JR, Sowa MG, Scarth GB, Mantsch HH. The use of fuzzy C-means clustering in the analysis of spectroscopic imaging data. AnalChem 1997;69:33703374.

31. Farkas DL, Du C, Fisher GW, Lau C, Niu W, Wachman ES, LevensonRM. Non-invasive image acquisition and advanced processing in opti-cal bioimaging. Comput Med Imaging Graph 1998;22:89102.

32. Lin Y, Irani NG, Grotewold E. Sub-cellular trafficking of phytochem-icals explored using auto-fluorescent compounds in maize cells. BMCPlant Biol 2003;3:10.

33. Zimmermann T, Rietdorf J, Pepperkok R. Spectral imaging and itsapplications in live cell microscopy. FEBS Lett 2003;546:8792.

34. Mansfield JR, Hoyt CC, Miller PJ, Levenson RM. Distinguishedphotons: Increased contrast with multispectral in vivo fluorescenceimaging. Biotechniques 2005;39(Suppl.):S25S29.



multispectral imaging in medicine

Documents