value of information 1 st year review. ucla 2012 kickoff aro muri on value-centered information...
TRANSCRIPT
Valueof
Information
1st year review. UCLA 2012
Kickoff
ARO MURI on Value-centered Information Theory for Adaptive Learning, Inference, Tracking, and Exploitation
Valueof
Information
1st year review. UCLA 2012
Numerical computation of Non-Comm. VoI Metrics & Spectra of
Random Graphs
Co-PI Raj Rao NadakuditiUniversity of Michigan
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
Mission Informationand
Objectives
Non-commutativeInfo Theory
Info-geometric learning
Consensuslearning
Info theoreticsurrogates
Information-driven Learning. Jordan (Lead); Ertin, Fisher,
Hero, Nadakuditi
Bounds, models
and
learning algorith
ms
Scalable, Actionable
VoI measures
Research programInfo-driven learning
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
• Principal component analysis • Direction-finding (e.g. sniper localization) • Pre-processing/Denoising to SVM-based classification (e.g. pattern, gait & face recognition) • Regression, Matched subspace detectors• Community/Anomaly detection in networks/graphs
• Canonical Correlation Analysis• PCA-extension for fusing multiple correlated sources
• LDA, MDS, LSI, Kernel(.) ++, MissingData(.)++
• Eigen-analysis Spectral Dim. Red. Subspace methods• Technical challenge:
• Quantify eigen-VoI (Thrust 1) and Exploit quantified uncertainty (Thrust 2) for eigen-analysis based sensor fusion and learning
Eigen-analysis methods & apps.
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationRole of Non-Comm. Info theory
• For noisy, estimated subspaces, quantify:• Fundamental limits and phase transitions• Estimates of accuracy possibly, data-driven• Rates of convergence, learning rates • P-values • Impact of adversarial noise models
• “Classical” info. measures in low-dim.-large sample regime• e.g. f-divergence, Shannon mutual info., Sanov’s thm.
vs.• Non-comm. info. measures in high-dim.-relatively-small-
sample regime• Non-commutative analogs of above
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationAnalytical signal-plus-noise model
• Low dimensional (= k) latent signal model
• Xn is n x m noise-only Gaussian matrix
• c = n/m = # Sensors / # Samples
• Theta ~ SNR
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationEmpirical subspaces are unequal
• c = n/m = # Sensors / # Samples
• Theta ~ SNR, X is Gaussian
• Insight: Subspace estimates are biased!• “Large-n-large-m” versus “Small-n-large-m”
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
A non-commutative VoI metric (beyond Gaussians)
• Xn is n x m unitarily-invariant noise-only random matrix• Theorem [N. and Benaych-Georges, 2011]:
• μ = Spectral measure of noise singular values• D = D-transform of μ “log-Fourier” transform in NCI
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationNumerically computing D-transform
• Desired:• Allow continuous and discrete valued inputs• O(n log n) where n is number of singular values• Numerically stable
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationEmpirical VoI quantification
• Based on an eigen-gap based segment, compute non-comm VoI subspaces
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationAccomplishment - I
• Uk are Chebyshev polynomials• Series coefficients computed via DCT in O(n log n)• Closed-form G transform (and hence D transform)
series expansion!• “Numerical computation of convolutions in free probability theory” (with Sheehan
Olver)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
Information
• For noisy, estimated subspaces, quantify:• Fundamental limits and phase transitions• Estimates of accuracy possibly, data-driven• Rates of convergence, learning rates • P-values • Impact of adversarial noise models• Impact of finite training data
• Facilitate fast, accurate performance prediction for eigen-methods!
• Transition: MATLAB toolbox
Broader Impact
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationSpectra of Networks
• Role of spectra of social and related networks:• Community structure discovery• Dynamics • Stability
Open problem: Predict graph spectra given degree sequence
Broader Impact: ARL CTA & ITA, ARO MURI
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationNon. Comm. Prob. for Network Science
• Role of spectra of social and related networks:• Community structure discovery• Dynamics • Stability
Open Solved problem: Predict spectra of a graph given expected degree sequence
Answer: Free multiplicative convolution of degree sequence with semi-circle
“Spectra of graphs with expected degree sequence” (with Mark Newman)
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationAccomplishment - II
• Predicting spectra (numerical free convolution – Accomplishment I)
• “When is a hub not a hub (spectrally)?” • New phenomena, new VoI analytics
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationPhase transition in comm. detection
• Unidentifable: If cin – cout < 2 • cin = Avg. degree “within”; cout = Avg. degree “without”
Valueof
Information
1st year review. UCLA 2012
Kickoff
1st year review. UCLA 2012
Valueof
InformationYear 2 plans
• Accomplishments– Numerical computation of Non-Comm convolutions– Predicting spectra of complicated networks
• Impact– Information fusion
• Numerical computation of Non-Comm. Metrics• Performance prediction• New VoI analytics for networks• Predicting graph spectra from degree sequence
– Information exploitation• Selective fusion of subspace information