AISR Briefing // Michael Turmon, JPL 1
Statistical Object Identification, Tracking, and Analysis
Michael Turmon
Jet Propulsion Laboratory/Caltech
AISR Program Meeting
NASA Ames Conference Center
4 April 2005
AISR Briefing // Michael Turmon, JPL 2
Object Tracking and Analysis: Overview
Identification: Find the objects in multispectral science images Tracking: Link identified objects in series of images Trajectory Analysis: Model and classify object tracks
Identification
Allow scientists in domains like solar physics and atmosphere & ocean circulation to understand great volumes of temporal data in directly informative terms
Sunspot and Facula Regions in a Solar Quadrant15 November 1998 and the next five days; using MDI imagery
Tracking Trajectory analysis
Move scientists beyond looking at pixels to understanding phenomena
AISR Briefing // Michael Turmon, JPL 3
Project Activities
Scope: Demonstrate object analysis technology in three application areas: solar physics, geophysics (GPS), oceans/atmospheres
Highlights (June 2001 – June 2004)– Sunspots tracked over seven years’ images
- 100 GB of imagery distilled into 2500 object histories, < 1GB
- Related high-cadence track dataset also analyzed
– Integration of object tracks with DS9 browser
– Fast Kalman models developed and tested for GPS time series
– New hidden Markov classification of seismograph time series - Developed and used new constrained optimization methods
Future activities– Perform analysis of high-cadence solar data
– Tune segmentation and object tracking models for HMI
AISR Briefing // Michael Turmon, JPL 4
Technology Overview
Identification– Per-class mixture models drive Markov random field segmentation
- Trained using combination of expert-provided labels and unclassified pixels
– Aggregate connected components into objects Tracking
– Compute index of object overlap (past -> future)– Associate current objects to past objects to optimize total overlap
Object analysis– Continuous and discrete modeling of object path and characteristics– Basic subroutines are Kalman smoother and forward-backward recursion
AISR Briefing // Michael Turmon, JPL 5
Photogram
Magnetogram SNQ
Key:S(pot)F(acula)Q(uiet sun)
Flexible, general methods using statisticalmodels to identify objects in images S
F
Q
1: Experts identify classesin sample images
2: Learned model performsclassification automatically
MagneticField
Lig
htIn
tens
ity
Labeling
Labeling by inferredstatistical model
Q
SF
Identification: Integrating Multimode Imagery
Can not distinguish classes from just one observable– Move beyond ad hoc threshold rules to allow arbitrary class separators
Select model by using sample images labeled by scientists
AISR Briefing // Michael Turmon, JPL 6
Unlabeled Data
Labeled Data
sunspot
quiet
facula
faculaquiet
sunspot
Previous Feature->Class Map New Feature->Class Map
Identification: Partly-Labeled Data
Hand Labeling: time-consuming, expensive, asks much of scientists. Data from some feature classes (e.g., background) is easy to identify;
small amounts of labeled data can be obtained with care.- E.g., scatter plot at left: 15K quiet examples + 607 sunspot + 340 facula
- Technical challenge: ensure atypical distribution of labeled data does not affect learned class proportions.
– Developed methods using partly-classified data to bootstrap vast amounts of unlabeled data, seamlessly in same clustering algorithm.
- Selected 100K examples from 10B total, 30K labeled — mostly quiet background.
Yields >20% improvement in sunspot classification accuracy, and >25% improvement in facula classification accuracy.
AISR Briefing // Michael Turmon, JPL 7
Identification: ResultsTurmon et al., “Statistical Pattern Recognition for Labeling Solar Active Regions:
Application to SoHO/MDI Imagery,” Astrophysical Journal, March 20 2002, 396-407.
AISR Briefing // Michael Turmon, JPL 8
Feature Identification: Publications
The mixture modeling work appeared in:- Mixtures-2001, “Recent Developments in Mixture Modelling,” Hamburg
- Compstat-2004, Prague, as “Symmetric Normal Mixtures”
Work comparing our MDI labelings to other observatories:– Harry Jones (Kitt Peak Nat’l Solar Obs.) & Steve Walton (San Fernando Obs.)
- J. Pap, H. Jones, M. Turmon & L. Floyd, “Study of the SOHO/VIRGO Irradiance Variations using MDI and Kitt Peak images,” Proc. SOHO-11 Workshop, Davos, 2002.
- H.P. Jones, M. Turmon, et al. “A comparison of feature classification methods for modeling solar irradiance variation,” 34th COSPAR Scientific Assembly, 2002.
– Laslo Gyorfi at Debrecen Observatory, Hungary- L. Gyori, T. Baranyi, M. Turmon & J.M. Pap, "Comparison of image-processing
methods to extract sunspots,” Proc. SOHO-11 Workshop, Davos, 2002.
- L. Gyori, T. Baranyi, M. Turmon & J.M. Pap, “Study of differences between sunspot area data determined from ground-based and space-borne observations,” Adv. Space Res., April 2004.
Connections with irradiance- J. M. Pap, M. Turmon et al., “Magnetic Field & Long-Term Solar Irradiance
Variations Over Solar Cycles 21 to 23,” AGU, 2003, San Francisco (poster).
AISR Briefing // Michael Turmon, JPL 9
Feature Identification: Infusion
This software will be used in the HMI data pipeline at Stanford– HMI imager will fly on board SDO, the first LWS mission
- http://sdo.gsfc.nasa.gov and http://hmi.stanford.edu
– HMI’s data volume is unprecedented in Solar Physics- 4096x4096 pixel images every 90 seconds
- These data volumes make it more important to focus attention
– HMI/SDO is the successor to MDI/SoHO, on which these results are based
The software has also been baselined by CNES Picard– Picard has been given the go-ahead by CNES for 2007 launch
- http://smsc.cnes.fr/PICARD/
– Picard will measure tiny variations in solar diameter and shape- Active region recognition and rejection is important to its delicate results
– Software must clear ITAR and licensing hurdles
AISR Briefing // Michael Turmon, JPL 10
Object Tracking Methods
Associate objects in beforeand after images
Correlation-based tracker– Motion model: deterministic drift plus stochastic uncertainty
– For sunspots or cyclones, have motion and correlation on the sphere
– Correlation measure between a in A and b in B is D(a,b)
Solve assignment problem to match A up to A’:
with P a permutation matrix
Solution by linear programming
For our applications, key is to get deterministic drift correct
A B
Before After
AISR Briefing // Michael Turmon, JPL 12
Object Tracking: Sunspots over seven years
Coordinates, size, intensity of 2500 sunspots from 1996-2003– Over 100-fold reduction in data volume
June through July 1999 shown above– Ordered by central meridian passage (CMP) time
– Successive sightings overplotted; extent indicated by bounding box
Enable quantitative studies of spot taxonomy
La
titu
de
Time (CMP) ––>Zoom view
AISR Briefing // Michael Turmon, JPL 13
Object Tracking: High-rate data
We also have tracks from four, three-month periods of high-cadence data from SOHO/MDI
– Continuous telemetry gives full-disk images every minute
– Unprecedented temporal resolution: 1500 images/day or 3GB/day
– Small features we identify and track are tracers for motion of plasma in photosphere
High-cadence data give more samples for each region of interest
AISR Briefing // Michael Turmon, JPL 14
Object Analysis Methods: General
Premise: Learn from objects by modeling their evolution as noisy differential equations of several related types
– Hidden Markov models: Finite-state machine controls time series- Divide behaviors into classes according to hidden discrete state u(t)
– Kalman filters: Continuous-state machine controls time series- Explain or model behavior by hidden state vector u(t)
– Track clustering: Discrete variable C selects object type- Extends clustering (the most useful
baseline discovery algorithm) to thetemporal domain
All methods in this family generalizethe basic HMM/Kalman model
– Crucially: subroutine re-use
– Kalman smoother and forward-backward