lanl hyperspectral image processing goals collect analyze react on deployed platform barriers : data...
TRANSCRIPT
LANL Hyperspectral Image LANL Hyperspectral Image ProcessingProcessing
Begin periodic
evaluation of clutter
Subtract mean
spectral vector
NxJxL ops
1 row of i-grams is complete
N pixels x 2LInterferogram accumulator
N x (2L)2 buffer w/ rolling pointer
Readouts: N spatial pixels
by 2L i-gram samples
Alarms
N FFTs
N x 2L ln(2L) ops
Divide sum
by NxJ
L opsIncrement
mean vector sums
N x L ops
Fast on-board processing for broad area search
Factor covariance
matrix
Order L3 opsForm
covariance matrices
NxJxL2 ops
Average: L2 ops
Buffer spectra
Evaluate matched filter(s)
NxJxLxM ops
NxJxL
Buffer full
Every J timesteps
M(2L2+L) ops
Form matched filter(s)
M signatures
Threshold exceeded
MxNxJ ops
Record detection
L channels; J rows in buffer; M spectral signatures; N pixels in swath
Every timestep: Every J steps:
Goals•Collect•Analyze•Reacton Deployed Platform
Barriers:•data intensive •compute intensive
HIRIS Example:Sensor Data at 67MB/sec
Problems with conventional solution:It takes 3.75 minutes to process 3 sec. of data on a high end multiprocessor server after the data has been returned to the ground.There is a critical need for real-time feedback at the sensor.
HIRIS data processing chain
Solution•Reconfigurable Computers
Parallel OperationsHigh Memory Bandwidth
•Algorithms tailored for hardware
HIRIS - Hyperspectral InfraRed Imaging System
HIRIS is an airborne technology demonstration of an imaging Fourier transform spectrometer.
Goal: Broad Area Search with Passive Hyperspectral ImagerFly a Michelson interferometer with an add-on imaging system on an airplane to detect chemical plumes at 20km.
•Imaging Fourier transform spectrometry from 8 to 17 microns.•Need pointing and tracking system to stare for several seconds with very high
accuracy and low jitter.•Need extremely low NESR to detect and distinguish effluent gases in LWIR
Status:•Numerous HIRIS flight campaigns with two instruments on two airplanes.•Successful detection of gaseous chemical plumes by spatial/spectral analysis.•Active program to refine analysis and exploitation algorithms and test them on
both flight and modeled data.•LANL modeling and exploitation has produced an end-to-end model, analysis
of HIRIS data, and a number of new algorithms.
Data Source:Fourier Transform Imaging Spectrometer
Instrument Data Characteristics:
• Spatial Resolution: 128x128 pixels• 14 Bits of data per pixel• Mirror Positions: 4000+• Number of mirror positions not
necessarily a power of 2• Mirror Scan Time over all locations:
3 seconds• Each spatial image is collected before
the mirror is moved again.• Image Data cube of contains an
Interferogram of theoptical spectrum at each
spatial pixel.• The FOV of the instrument may move
slightly between image collections. Such translations need to be removed, preferably at the sub-pixel level
Raw Data Rate: 67 Megasamples/second
Difficulties with hyperspectral image analysis
Complexity of spectra - spectral clutter:•Atmosphere: To model all features at high resolution is a huge job.•Background surfaces: Variety of spectral emissivities.•Gases: Molecular band structures are function of temperature.
Mitigation: Construct a matched filter for each gas incl. Water vapor.
Complexity of background - spatial clutter•Material variation, both under plume and on plume•Variations of water vapor over short distances
Mitigation: Preclassify scene, perform spectral analysis by class Fit plume shape iteratively to physical dispersion model.
Spectrum of 1Pixel
Background
Categorize hyperspectral image analysis
1. Precise Chemical Analysis: (2 alternative approaches)1. Statistical
Principal Component Analysis followed by Adaptive Matched FilterPast and current main focus of LANL effortProcessing chain validated and being used
2. ExplicitClassify and find spectral end-members followed by Spectral FittingCurrent LANL research effortClassify with K-Means Alg. End Member with Pixel Purity Index Alg and Nfindr Alg.
2. Detection1. Real-time analysis with no human analyst input
Based on further Hardware Acceleration of Statistical Approach but reduced precision because:• Front End data reduction• Less precise background compensation since no analyst
Future LANL research effort
Desired: All of the following must be performed in less than 3 seconds:• Step 1: Register each image slice with the next• Step 2: Perform a FT on each pixel’s Interferogram
• -> 1634 4096 pt FTs• Step 3: Combine and average symmetric portions of transforms
• Actual number of valid wavenumber bins is 251• Step 4: Apply wavelength and pixel dependent gains and offsets
• Calibration table generated from subset of data and instrument parameters• Step 5: Sum the datacube in the wavenumber dimension and produce a
broadband image• Step 6: Compute the * Covariance Matrix of the image background• Step 7: Use the covariance matrix on 32 library spectral templates to produce 32
matched filters.• Step 8: Produce 32 images by performing a * Spectral Dot Product of the matched
filters on the datacube.
Statistical Analysis Overview (Example)
* Focus of ACS acceleration to date
(a) Original Image
(b) Image plus 5% plume of SO2
(c) Using SO2 spectrum to look for the plume
(d) Using orthogonalized SO2 spectrum to look for the plume
Example of Statistical Analysis with Simulated SO2 plume
L=630
N=128
J=128
DP1 DP2 DPM
L(n
)
...
Covariance
MACS = NJL(L/2) = 3.25e9
RCC Computation Time = NJL(L/2)/Mfclk = 1.3s fclk = 50MHz, M=50
(M is number of DPs in FPGA)
Benchmark = 71.5s (733MHz PC)
Acceleration 55X
…L2,L1
Local mem
Work in Progress – Acceleration of Statistical Analysis
L=630
N=128
J=128
DP1 DP2 DPM
F1
co
effs
...
...
Matched Filter
Total MACS = NJLM = 516e6
TCOMP = NJL/fclk = 0.2s fclk = 50MHz
Benchmark = 20.9s (733MHz PC)
Acceleration 105X
…p2,p1
MACs
M Matched filters
F2
co
effs
FM
co
effs
Local mem
Work in Progress – Acceleration of Statistical Analysis
class 0
class 1
class 2
class 3
class 4
Completed Work – Acceleration of Explicit Analysis
K-MEANS: Unsupervised image classification: ‘cluster’ the pixels into a few number of classes such that pixels belonging to the same class have similar spectral properties
Initialization: Randomly assign pixels to clusters.
Step 1: Find new cluster centers using current cluster assignments.
Use the mean of pixels assigned to that cluster.
Step 2: Assign pixels to clusters.
Use minimum distance from pixel to cluster center.
Iterate Steps 1 and 2
Completed Work – Acceleration of Explicit Analysis
Pixel Purity Index (PPI): Identify “pure” pixels (or end-members) from multi-spectral images
•Set N random D-vectors: skewers
•for each skewer:
–compute a dot-product with all the pixels
–the two pixels having produced the highest and the lowest dot-product are potential pure pixels
•Select the “pure” pixels
–i.e. the pixels which have been selected several times as potential pure pixels
Future Work – Acceleration of Detection Algorithm
Goal: Chemical Detection at the frame rate of the sensor
Approach: Evaluate ways to accelerate statistical processing chain if we are willing to give up a little in sensitivity.
Plan:
Work with HIRIS experts to identify end to end processing steps
Modify processing by
a) performing data reduction at front end
b) Matching processing chain to FPGA strengths
c) More automated background compensation
Allow some reduction in sensitivity because
a) we remove the human analyst from the process
b) front end data reduction
Develop software model of hardware processing chain
Take advantage of BYU JHDL Module Generators
Target Hardware: New USC/ISI Virtex2 based RCC “Osiris”
Future Work – Acceleration of Detection Algorithm
Tasks• Generate Detailed Task List
• Explore Architectural Trades of a Heterogeneous System
• Develop Software Model of Detection Algorithm
• Code an Implementation on “Osiris”
• Caveat: Acceleration will depend on a FFT. While the FFT is feasible, development of a high performance FFT is beyond the scope of this effort. An early task is to address this issue.
Level of Effort• LANL - 1 FTE: Herb Fry, Kevin McCabe, Tony Nelson
• ISI - 1 FTE: Ron Riley, Matt French
• BYU – 1 student?
Future Work – Acceleration of Detection Algorithm
Schedule• June : Detailed Task List
• June – August : Develop initial software model
• June – August : Identify Modules
• July : Address high performance FFT issue (what size FFTs are available and how fast are they?)
• End of July : Checkpoint and Demo definition
• August – November : Refine software model and begin detailed design
• November : Workshop
• Spring ’02 : JHDL Module and VHDL coding for “Osiris”
• August ’02 : Demo: probably a detection “movie”
Future Work – Acceleration of Detection Algorithm
Dependencies
• FFT Solution
• New ISI “Osiris” RCC
• BYU resources