1: brigham young university, electrical and computer ...bjeffs/publications/jeffs_ska_05.pdf ·...
TRANSCRIPT
Analysis of the Peeling Algorithm
Brian D. Jeffs1 and Sebastiaan van der Tol2
1: Brigham Young University, Electrical and Computer [email protected]
2: TU Delft, Circuits and Systems, [email protected]
The Bad News:
LOFAR is uncalibratable ...
The Good News:
with conventional algorithms.
Next Generation WidefieldInstrument Calibration Challenges
• Larger apertures.• Many more array elements.• Wider range of frequencies.
• Ionospheric interaction. • Calibration may be source
direction dependent.• Calibrated UV data may not
be possible.
Each station antenna sees the entire sky.
7200 dual-pol antennas.
Multiple simultaneous beams are formed in different directions.
~6˚ beam mainlobe
© ASTRON
LOFAR is a Widefield Instrument
LOFAR Geometry
• 72 stations.
• 100 km aperture.
• Significant ionospheric variation across the array complicates calibration.
• Nonisoplanatic iono-sphere across calibration sources and stations.
• Very low frequencies: 30 - 240 MHz.
© ASTRON
The LOFAR Calibration Problem
• At low frequencies the ionosphere perturbs phase and gain.
• Calibration terms must be estimated for each bright source & station.
• Calibration for other objects is interpolated.
• Physical constraints must be applied.
Full array aperture
Station beamfield of view
Ionosphere
LOFAR stationLOFAR station
Objects in field of viewsee different ionosphericphase and gain
Calibration is Direction Dependent
• Each station sees a differentdirection-dependent blur.
• Calibration on several brightpoint sources in the field of view is required.
x
G
K
m=1 m=M
Matrix Form Data Model
V: visibility matrix, computed over a series of time-frequency intervals. Observed.
G: calibration complex gain matrix. One column per calibrator source. Unknown.
K: Fourier kernel, geometric array response. sq is source direction vector. rm is station location. Known.
B: Calibrator source intensity. Known.
D: Noise covariance. Unknown.
=
=
⋅=
=
=
+⋅⋅=
=
MQ
mqcf
qm
QMM
Q
QMM
Q
d
d
b
b
ikkk
kk
gg
gg
nnE
OO
L
MM
L
L
MM
L
oo
11
2,
,1,
,11,1
,1,
,11,1
H
H
,
}exp{,
)()(]}[][{
DB
rsK
G
DKGBKGxxV
π
The Single Snapshot Calibration Ambiguity
• For conventional arrays without direction dependent ionospheric phase perturbation calibration is possible with a single Vk,n observation.
• Not so for LOFAR, there is an essential ambiguity.
• Calibration is Impossible with a single visibility snapshot!
1
H
HH
))(()(~
matrixunitary a ,
)()(
21
21
21
21
−•− ⋅⋅=
=
+⋅⋅=
KBUBKGUG
IUU
DKGBUUBKGV
oo
oo
H
Rotation by U is invisible in V
Each U produces a different calibration
Solutions to Calibration Ambiguity
• Time-frequency diversity• Fringe rotation over time and frequency changes
visibilities while calibration gains are nearly constant.• Low order polynomial fitting over time-frequency.• Peeling.
• Single snapshot calibration• Compact core.• Deterministic Frequency dependence.• Known gain magnitudes, |G|.
The Direct Least Squares Solution
• Problems:Direct optimization is computationally intractable.Too many parameters.Requires good initialization. Where do you get it?Does not exploit known smoothness structure over k,n.Due to ambiguity, solution is not unique if θk,n has same degrees of freedom as Gk,n.
[ ]2
,,,,,,,
,1,1
)}{()}{(minargˆ
,,
,1,1
∑∑ ⋅⋅−=Θ
=Θ
k n
Hnknknknknk
TTNK
T
NK
KGBKGV ooL
L
θθ
θθ
θθ
The Peeling Approach
• Current proposed LOFAR calibration method.• Replace joint estimation of G for Q sources with a series
of single source calibration problems.• Exploit relative fringe rotation rates among calibrator
sources.• Assume calibration gains are constant over a t-f cell. • Computationally efficient.• References:
1. J.E. Noordam, “LOFAR calibration challenges,” Proceedings of the SPIE, vol. 5489, Oct. 2004.2. J.E. Noordam, “Peeling the Visibility Onion, the optimum way of self-calibration,” ASTRON tech. report MEM-078 June 2003.
Basic Peeling Algorithm Steps
• Over a time-frequency cell of nearly constant gains, rotate all visibilities to phase center the brightest remaining source.
}{diag}{diag~,,,,,, nkqnknkqnk kVkV =
“Image plane” equivalent
Basic Peeling Algorithm Steps
• Over a time-frequency cell of nearly constant gains, rotate all visibilities to phase center the brightest remaining source.
• Average centered visibilities to suppress non-centered sources.
∑∑=k n
nkq KN ,~1ˆ VV
“Image plane” equivalent
Basic Peeling Algorithm Steps
• Over a time-frequency cell of nearly constant gains, rotate all visibilities to phase center the brightest remaining source.
• Average centered visibilities to suppress non-centered sources.
• Solve as a conventional single source calibration.
Hqqq b ggVg
g−= ˆminˆ
“Image plane” equivalent
Basic Peeling Algorithm Steps
• Over a time-frequency cell of nearly constant gains, rotate all visibilities to phase center the brightest remaining source.
• Average centered visibilities to suppress non-centered sources.
• Solve as a conventional single source calibration.
• Subtract the calibrated source from visibilities.
}diag{ˆˆ}diag{ ,,,,,, nkqHqqnkqqnknk b kggkVV −=
“Image plane” equivalent
Basic Peeling Algorithm Steps
• Over a time-frequency cell of nearly constant gains, rotate all visibilities to phase center the brightest remaining source.
• Average centered visibilities to suppress non-centered sources.
• Solve as a conventional single source calibration.
• Subtract the calibrated source from visibilities.
• Repeat for next brightest source.
“Image plane” equivalent
2-D Polynomial Model over time-frequency for Ionospheric Variation
• Variations in G are smooth over time and frequency
• Estimating the smaller parameter set, p, improves performance.
• Over a large window p does not depend on k,n.
{ })(exp
)(}{
112
02012
201000
112
02012
201000,
nknnkk
nknnkknk
tfttffi
tfttff
ΦΦΦΦΦΦ
ΓΓΓΓΓΓ
+++++
+++++=
⋅oθG
[ ]TT11
T00
T11
T00 }diag{,}vec{,,}vec{,}vec{,,}vec{ Dp ΦΦΓΓ LL=
Challenges
• Phase centered averaging does not completely remove non-centered sources.
• “Contamination” from non-centered sources biases estimate of .
• Solution: Multiple passes of peeling.
• Computational burden eliminates some candidate approaches.
• Many local minima in polynomial optimization makes a good initial solution critical.
qg
Why Study the Cramer-Rao Bound?
• Array calibration is a statistical parameter estimation problem.
• The CRB reveals the theoretical limit on estimation error variance.
• Absolute frame of reference: no algorithm can beat the CRB.
• Answers the questions:
• Is the existing algorithm adequate?
• Is there hope for finding a better solution, “How close are we?”
• Permits trading off performance and computational burden.
• Can be computed even if no algorithm exists yet.
• The BIG question: Can LOFAR be reliably calibrated?
CRB Definition
• Notationx: a vector of random samples with joint probability
density p(x |θ ).any unbiased estimator for θ. covariance matrix for .
M: Fisher information matrix.
• The Cramer-Rao theorem:
• Error variance is lower bounded by !
:θ:θC
1
T
21
ˆ)|(ln
−
−
∂∂∂
−=≥θθ
θxMC pEθ
}diag{θC
θ
A Simple Example
• Estimate a constant in additive white Gaussian noise:
+w[n] x[n]
θ
0 10 20 30 40 50 60 70 80 90 1000
1
2
3
4
5
6
7
8
9
10
n
x[n]
θ
.)ˆvar(2
Nσθ ≥CRB result:
T]]1[,],0[[ −= Nxx Lx
This is the well known variance for the sample mean!
A Second Simple Example
• Line fitting in additive white Gaussian noise:
][][ 21 nwnnx ++= θθ
+w[n] x[n]
θ1+θ2n
0 10 20 30 40 50 60 70 80 90 1003
4
5
6
7
8
9
10
11
n
x[n]
θ1
θ2 We now have no intuition on estimation error for θ1 and θ2!
• Variance on constant term θ1 is now higher. → estimating more parameters increases error.
• Variance of slope term, θ2, drops more rapidly with N.→ θ2 is easier to estimate. → x[n] is more sensitive to θ2 due to multiplication by n.
A Second Simple Example: Insights
[ ] [ ])1(
12)var(,)1(
)12(2)var(
,
6)12)(1(
2)1(
2)1(
1
2
2
2,21
2
2
1,11
1
2
−==
+−
==
−−−
−
=
−−
NNNNN
NNNNN
NNN
σθσθ
σ
MM
M
3
2
2
2
112)var(lim4)var(lim
NN NN
σθσθ ≥≥∞→∞→
Now it Gets a Little Messy
O
Constraint Jacobian for packed central core
Block Fisher information
Blockclosedforms
Now it Gets a Little Messy
The important points:• Closed form CRB expressions have been derived for
most important LOFAR calibration models.• Though expressions are complex, computer codes
have been developed to evaluate them.• These solutions exist now and could be made available
for astronomers to predict calibration performance for a given observations.
Peeling Simulation Results
• Polynomial fit over a 10 s by 100 kHz “snippet”window.
• Two sources.• Plot is for zero order
coefficient, 16th station, 1st
source.• 30 station array with 100
km aperture.• Performance is typical of
all other parameters.• Three peeling passes. -40 -35 -30 -25 -20 -15 -10 -5 0
-60
-50
-40
-30
-20
-10
0
10
20
SNR in dB for brightest calibrator
Par
amet
er e
rror v
aria
nce
in d
B
Sample error and CRB for parameter no. 76
sample err varcheat err varCRB
CRB Analysis for a realistic Scenario
• Point LOFAR beam in arbitrary direction.• Model station beam pattern and noise level accurately.• Calibrate on 5 brightest sources in beam mainlobe and
5 brightest in sidelobes.• Use 2-D 1st order polynomial fitting over a “snippet” of
10 seconds and 500 kHz.• Calculate CRB for polynomial coefficients.• Project coefficient CRB to corresponding complex gain
error variance.
EWI Circuits and Systems
Full Sky Map
3C461
3C144
3C405
3C147
3C274
Full Sky Map
EWI Circuits and Systems
Field of View
6667686970717254
54.5
55
55.5
56
56.5
57
4C+55.08
4C+54.06
4C+56.09
4C+56.10
4C+55.09
Field of View
Right Ascension (deg)
Dec
linat
ion
(deg
)
EWI Circuits and Systems
Station Beam Pattern
Station Beam Pattern
−20
−18
−16
−14
−12
−10
−8
−6
−4
−2
0
Multiple Source CRB for 2-D Polynomial Coefficients
Time (s)
Fre
quen
cy (
kHz)
Relative gain error − Antenna m=2 / Source q=3 (4C+55.08)
1 2 3 4 5 6 7 8 9 10 11
50
100
150
200
250
300
350
400
450
500
10
20
30
40
50
60
70
80
90
100
• Coefficient error variance relative to conventional single source calibration.
• Note that calibra-tion fails without at least two seconds and 150 kHz of diversity.
Multiple Source Complex Gain Error CRB
Relative gain error − Antenna m=2 / Source q=3 (4C+55.08)
1
2
3
4
5
6
7
8
9
10
• Use estimated polynomial coefficients to calculate complex gains.
• Calculate error variance CRB over full time frequency range.
• Largest error is near domain edges.
Conclusions
• The BIG answer: Yes, LOFAR can be calibrated.
• Given a range of time-frequency observations and compact core geometry: there are no theoretical roadblocks to achieving useful calibration estimates.
• Does peeling work for direction dependent calibration?
• Yes, so far so good.
• Ongoing progress in reducing cross-source contamination: multiple pass peeling & demixing.
• Low SNR performance improvements being studied.
Optional Slides
• Use these only if (by some miracle) there is extra time.
LOFAR Calibration with Compact Central Core
• The full array can be calibrated given a compact central core.• The wisdom of the LOFAR design is confirmed by CRB analysis
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
x 105
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1x 105
-3000 -2000 -1000 0 1000 2000 3000-3000
-2000
-1000
0
1000
2000
3000
Full array Compact core
LOFAR Calibration with Compact Central Core
• Calibration succeeds for Q ≤ Mc+1. Q calibrator sources and an Mc element core.
0 10 20 30 40 50 60 70 80 900
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5x 10
12 CRB for 10 element array,4 element subarray and 6 sources
Mea
n S
quar
ed E
rror
Parameter Index
CRBstart of phasesstart of noise powers
0 5 10 15 20 25 30 35 40 45 500
0.5
1
1.5x 10
−3 MSE for 10 element array,4 element subarray and 3 sources
Mea
n S
quar
ed E
rror
Parameter Index
ML Monte CarloCRBstart of phasesstart of noise powers
×1012×10-3