hyperspectral image projector - itea
TRANSCRIPT
1Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Hyperspectral Image Projector
Joseph P. Rice
Optical Technology DivisionNational Institute of Standards and Technology
Gaithersburg, MD 20899 USA
Collaborators: L. Brandon Shaw, Rafael Gattass, Jas Sanghera, NRLJorge Neira, Steve Brown, David Allen, Allan Smith, NIST
Michael Kehoe, Casey Dodge, Casey Smith, Rand Swanson, Resonon Inc.
This work is funded by the U.S. Department of Defense Test Resource Management Center (TRMC) Test and Evaluation / Science and Technology (T&E/S&T) Program through MIPR8DDAT3A622,
Frank Carlen, Multi-Spectral Test Focus Area Executing Agent
2Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Problem Statement
Test and Evaluation (T&E) NeedMultispectral and Hyperspectral sensor T&E requires scene projectors that can:
Present synchronized, co-registered, spectrally & spatially realistic scenes – in the solar-reflected, visible-near-infrared (VNIR) and short-wave-infrared (SWIR)– in the thermally-emitted, mid-wave-infrared (MWIR) spectral range
Provide high spatial and spectral resolution and large scene areaOperate at real-time speedPresent high-temperature scene contentIncorporate high brightness sources to examine stray light entry & effects
Science and Technology (S&T) ChallengesProject dynamic, 2D spatial images, with pixel-by-pixel programmable spectral contentDevelop a high output, broadband MWIR sourceControl, measure, and correct stray light in the projector and in the reference instruments used to characterize the projected scenes
TRL Start/Stop: 3/6
End State is an integrated and calibrated prototype 2D VNIR/SWIR/MWIR Hyperspectral Image Projector (HIP)
3Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Target Pixel
TrackDirection
Spatial Stray Light
Spectral Stray Light
Spectral and SpatialStray Light
Sensor Focal Plane Array
Sensor Optics
FOV
The HIP Probes Spectral and Spatial Stray Light Paths of the Sensor Under Test
Consider the stray light contributions to thenadir pixel in, for example, the typical pushbroomsensor as depicted here:
•In the real world, most pixels, like the red pixel, provide stray light to the sensor
•This stray light could masquerade as a targetof interest, causing false positives
•Or it could clutter the scene, causing missedidentifications.
•The HIP properly probes such stray light paths, since it includes such pixels in its test
4Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Broadband Fiber Light SourceProvides high brightness in a small spot for high spectral resolutionFused-silica fiber supercontinuum source for VNIR/SWIRChalcogenide fiber supercontinuum source for MWIR
Spectral EngineFiber source light is spectrally dispersed across a digital micromirror device (DMD)Image reflected from on-pixels of DMD is spatially integrated and determines spectrum
Spatial EngineUses another DMD to project spectra into Unit Under Test (UUT) per 2D spatial abundance images, through a collimator, in sync with spectral engine
Reference InstrumentWell-calibrated imaging spectrometer to measure the spectrum of each spatial pixel at the output, relative to NIST reference standards, to provide truth data
HIP Background
Spectral Engine(Spectra Generator)
Reference InstrumentBroadband Fiber Light Source
Spatial Engine(Image Projector) Unit Under Test
Computer
VNIR/SWIR: 450-2500 nm, MWIR: 3000 – 5000 nm
5Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Principle of the Spectral Engine
Dispersing element Spatial Light Modulator(SLM) Recombine the Light
SLM Light Guide
Available High-Resolution Spatial Light Modulators:• Digital micromirror devices (DMD)• Liquid crystal on silicon (LCOS) arrays
6Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Digital Micromirror Device (DMD)
An array of MEMS micromirror elementsThe HIP uses these in the Spectral Engine
and in the Spatial EngineDeveloped by Texas Instruments (TI)
1024 x 768 elements, +/- 12 degree tilt angle Aluminum mirrors13.68 micron pitch< 24 microseconds mechanical switching time
For the prototype HIP DMDs, TI D3000 XGA DMDs: 11 kHz binary frame rate is achievedThe standard glass windows have been replaced by windows optimized for each spectral rangeBrassboard control algorithms use LabVIEW top-level software with a USB interface to a standard PCFor the prototype, we are exploring real-time drive electronics options for these DMDs
MEMS = Micro-Electro-Mechanical System
Mirror -12 degreesMirror +12 degrees
7Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
How the DMD is used in a spectrally tunable source: Monochromator Mode
Near IRUV Visible
1024 mirrors
768
mi r
ror s
Wavelength
Inte
nsity
mirror array
Mirrors in "On"Position
8Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Near IRUV Visible
1024 mirrors
768
mir
r ors
Wavelength
Inte
nsity
mirror array
How the DMD is used to create an arbitrarily programmable spectrum
9Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
VNIR/SWIR Supercontinuum Fiber Source
Utilizes non-linear effects in a photonic crystal optical fiber to greatly broaden the spectrum of a 1064 nm pump laser
Broadband light is generated in a single-mode (<5 μm core diameter) photonic crystal (holey) optical fiber
No etendue issues as with lamps or blackbodiesIdeally suited for coupling to a spectral engine
High power and high spectral resolution:
3 mW/nm spectral power density from 450 nm to 1700 nm
10Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
HIP Prototype Concept
NIST ReferenceInstruments
Supercontinuum Fiber Laser
Light SourceVNIR/SWIR
Spatial EngineVNIR/SWIR Sensor
Under Test
Computer
VNIR Spectral Engine
SWIR SpectralEngine
MWIR Spectral Engine
MWIR Spatial Engine
MWIR Sensor Under Test
450-2500 nm
3000 – 5000 nm
Chalcogenide Supercontinuum
Light Source
Acronyms: VNIR = Visible Near Infrared; SWIR = Short Wave Infrared; MWIR = Mid Wave Infrared
11Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
HIP Prototype Specifications
Parameter Target Specification
Spectral Range450 - 2500 nm (VNIR-SWIR)3000 – 5000 nm (MWIR)
Spectral Resolution 5 nm VNIR8 nm SWIR8 nm MWIR
VNIR/SWIR/MWIR Sync. Accuracy 1 microsecond
Spatial Format 1024 H 768 V
Projected FOV 7.9 H 5.9 V
Spatial Resolution 0.135 mrad
Average Spectral Radiance VNIR/SWIR 1000 W/m2srμmMWIR: 750 W/m2srμm
Bit Depth/ Frame Rate 12 bits/250 Hz
Contrast Ratio VNIR/SWIR: 1000:1 MWIR: 250:1
Wavelength Accuracy 2 nm
Radiance Accuracy 2%
12Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Design Concept for VNIR/SWIR Spectral Engine
SupercontinuumFiber Light
Source
DMD2
Recombining System
Dispersing Prisms
DMD1
VNIR SWIR
Input
Output toSpatial Engine
SWIR
VNIR
• Splits input into VNIR and SWIR• One DMD (1024x768) for each range• Optical modeling in Zemax and FRED
– Prisms selected– Design meets requirements
Spectral rangeTransmittanceSpectral resolution
• Mechanical modeling in Solid Works
VNIR/SWIR spectral engine fits in 19” rack-mountable chassis
Source Spectrum
13Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
HIP VNIR-SWIR Spectral Engine
Inside view from back:Outside view from front:
Supercontinuum Light Source
VNIRSpectralEngine
SWIRSpectralEngine
14Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
HIP VNIR-SWIR Spectral Engine
Supercontinuum Light Source
VNIRSpectralEngine
SWIRSpectralEngine
Source Collimator
ElectronicsSection
Output
}}
View from back:
DichroicBeamsplitter
DichroicBeamcombiner
15Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
VNIR-SWIR Spatial Engine Optical Design
Sensor
16Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
VNIR-SWIR Spatial Engine Mechanical Design
DMD
DMD Driver Electronics PC Boards Cooling Fan
ReplaceableCollimator
Purged, SealedOptics SectionBeam Pipe
For Direct CouplingFor Spectral Engine
ElectricalConnectionsTo SpectralEngine
17Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
VNIR-SWIR HIP System
Brassboard
VNIR Spectral Engine
SWIR Spectral Engine
Spatial Engine
Collimator For Projectionto UUT
Super-continuumSource
Power Supply
18Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
VNIR-SWIR HIP
Liquid Light Guide:•Couples HIP Spectral Engineto HIP Spatial Engine
HIPSpectral Engine
(rear view)
HIPSpatial Engine
TranslationStageBrassboard
19Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
VNIR Spectral Resolution Test Data
Meets the < 5 nm spec
Spectral Resolution of Test Instrumentation is 0.2 nm
20Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
SWIR Spectral Resolution Test Data
Meets the < 8 nm spec
Spectral Resolution of Test Instrumentation is 0.5 nm
21Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Spectral Matching Performance
• Image on the Spectral Engine DMD that gives the spectrum above
• Measured and target spectra with supercontinuum fiber light source with the breadboard visible-band spectral engine
• NIST-developed software has been used to automatically calibrate this and match to target spectrum
Wavelength
Inte
nsity
22Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
The HIP Can Match Typical Reflected-Solar Radiance Spectra
• HIP has enough light to simulate a bright sunny day outside• Left panel shows the maximum HIP spectral radiance (all DMD micromirrors on)• Left panel shows the Top-of-Atmosphere (TOA) Earth reflected radiance spectrum• Right panel is an expanded scale: Red shows HIP output matched to some typical Earth-reflected spectra in the Visible-Near IR (VNIR)
23Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Hyperspectral Imager Test at the HIP
Used a VNIR pushbroom Hyperspectral Imager (HSI) from collaborators at University of ColoradoInput data was a real scene collected by HSI Projected by the HIP and measured by the HSI.HSI scanned HIP to simulate ground track motion
HIP Projected, HSI Measured:
HSI
With Greg Kopp , Paul Smith, and Joey Espejo of the University of Colorado
24Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
MWIR Spectral Engine Design
25Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
MWIR Spatial Engine Design
26Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Monochromatic HIP Projection Algorithm
BSQ Cube1 1024x768xM
Load M ImagesOnto Spatial Engine
DMD
Project into Sensor
Issues:•Not light efficient to use monochromatic bands: low brightness•Need to cycle through all M monochromatic bands for each cube: slow
Maximum cube rate is f/M, where f is single image frame rate.
M is typically 100 to 1000
M X
BSQ (Band Sequential): A stack of 2D images, each at one of M different monochromatic wavelengths
27Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Spectral Engine DMD in Monochromator Mode
Near IRUV Visible
1024 mirrors
768
mi r
ror s
Wavelength
Inte
nsity
mirror array
Mirrors in "On"Position
28Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Near IRUV Visible
1024 mirrors
768
mir
r ors
Wavelength
Inte
nsity
mirror array
Spectral Engine DMD in Broadband Mode
29Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
HIP Image Cube Projection Concept
Spatial Engine• Projects images with
component spectra into sensor
• Reference instrument characterizes output
ReferenceInstrument
Sensor Under Test
Supercontinuum Light Source
Spectral Engine• Uses light from supercontinuum source• Produces programmable spectra that
match component spectra• Directs these to Spatial Engine
0.0
0.2
0.4
0.6
0.8
1.0
400 450 500 550 600 650 700 750
Pow
er
Wavelength (nm)
Example Spectrum
Input HyperspectralImage Cube
ComponentSpectra
Principle-components-type algorithm reduces image cube to Component Spectraand Abundance Images
AbundanceImages
30Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
InputImage Cube
Example: AVIRIS Image Cubeof San Diego Naval Air Station
Given an image cube, ENVI/SMACC (COTS software) is used to find the Component Spectra and their Abundances
ES-3
ES-4
ES-2 ES-5
ES-6
Component Abundances
ES-1
ES-1
Component Spectra
ES-2
ES-3
ES-5ES-4
ES-6
Compressive Projection
Spectrum of each pixel is a linear combination of these component “eigenspectra”.
Then we need only project N = 6 broadband spectra instead of M = 30+ monochromatic spectra.
RGB Image:
31Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Prototype HIP Projection Algorithm
BSQ Cube1 1024x768xM
SMAAC to1024x768xN
BSQ (Band Sequential): A stack of 2D images, each at one of M different monochromatic wavelengthsSMAAC: An algorithm to reduce image cube to N component spectra and corresponding spatial images
Project into Sensor
Match N SpectraDisplay N Spectral
Images ontoSpectral Engine DMD
Display N SpatialImages onto
Spatial Engine DMD
N X
•M is typically between 100 and 1000; N is typically < 10Maximum cube rate is f/N, where f is single image frame rate.
•Spectral image display in sync with spatial image display•All spectral and spatial images displayed during sensor exposure time
N X
Faster and brighter than monochromatic projection
32Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Demonstrated CompatibilityBetween MSP2 and HIP
•MSP2 is real-time scene generator software developed by Signature Research•Interface exercise between Signature Research (MSP2) and NIST (HIP)
VNIR Image cube rendered from MSP2 as 1024 x 768 x 150 bandsSuccessfully read into ENVI (RGB false-color image below added in ENVI)Pre-processed for HIP projection using SMAAC
ENVI: ENvironment for Visualization of Imagery (COTS software for analyzing image cubes)
MSP2 data cube from Denise Talcott and Marshall Weathersby of Signature Research, Inc.
33Hyperspectral Image Projector (HIP) 20July2011 ITEAOptical Technology Division
Summary
The Hyperspectral Image Projector (HIP) is being developed by NIST and collaborators to address scene presentation issues for realistic T&E of multispectral and hyperspectral imaging sensorsProjects a 2-D image into the sensor whereby the spectrum of each pixel can be independently controlled, to simulate realityConcept has been demonstrated in the VNIR-SWIRPrototype operating in VNIR/SWIR/MWIR currently under developmentPlanning to demonstrate HIP prototype by end of FY2012To be delivered to U.S. Army Redstone Test Center
This work is funded by the U.S. Department of Defense Test Resource Management Center (TRMC) Test and Evaluation / Science and Technology (T&E/S&T) Program through MIPR8DDAT3A622,
Frank Carlen, Multi-Spectral Test Focus Area Executing Agent