e-vlbi: a brief overview alan r. whitney mit haystack observatory
DESCRIPTION
VLBI astronomy example ASTRONOMY Highest resolution technique available to astronomers – tens of microarcseconds Allows detailed studies of the most distant objects GEODESY Highest precision (few mm) technique available for global tectonic measurements Highest spatial and time resolution of Earth’s motion in space for the study of Earth’s interior Earth-rotation measurements important for military/civilian navigation Fundamental calibration for GPS constellation within Celestial Ref Frame VLBI Science Plate-tectonic motions from VLBI measurementsTRANSCRIPT
e-VLBI: A Brief Overview
Alan R. WhitneyMIT Haystack Observatory
Traditional VLBIThe Very-Long Baseline Interferometry (VLBI) Technique(with traditional data recording)
The Global VLBI Array(up to ~20 stations can be used simultaneously)
Chautauqua 2001
Quasars, hotspots, polarization
VLBI astronomy example
ASTRONOMY• Highest resolution technique available to
astronomers – tens of microarcseconds• Allows detailed studies of the most distant
objects
GEODESY• Highest precision (few mm) technique
available for global tectonic measurements• Highest spatial and time resolution of
Earth’s motion in space for the study of Earth’s interior
•Earth-rotation measurements important for military/civilian navigation•Fundamental calibration for GPS constellation within Celestial Ref Frame
VLBI Science
Plate-tectonic motions from VLBI measurements
Scientific Advantages of e-VLBI
• Bandwidth growth potential for higher sensitivity– VLBI sensitivity (SNR) proportional to square root of Bandwidth
resulting in a large increase in number of observable objects(only alternative is bigger antennas – hugely expensive)
– e-VLBI bandwidth potential growth far exceeds recording capability(practical recordable data rate limited to ~1 Gbps)
• Rapid processing turnaround– Astronomy
• Ability to study transient phenomena with feedback to steer observations– Geodesy
• Higher-precision measurements for geophysical investigations• Better Earth-orientation predictions, particularly UT1, important for
military and civilian navigation
Practical Advantages of ‘e-VLBI’
• Increased Reliability– remove recording equipment out of field– remote performance monitor & control capability in near real-time
• Lower Cost– Automated Operation Possible
• eliminates manual handling and shipping of storage media– Near Real-time Processing
• forestalls growth of storage-capacity requirements with bandwidth growth – Elimination of recording-media pool (millions of $’s!)
• Avoid unexpected media-shipping interruptions and losses
Elements of e-VLBI Development
• Phase 1: Develop eVLBI-compatible data system– Mark 5 system development at MIT Haystack Observatory being supported by
NRAO, NASA, USNO plus four international partners– Prototypes now deployed in U.S. and Europe
• Phase 2: Demonstrate 1 Gbps e-VLBI using Bossnet(w/ DARPA and NASA support)
– ~700km link between Haystack Observatory and NASA/GSFC– First e-VLBI experiment achieved ~788Mbps transfer rate
• Phase 3: Establish adaptive network protocol(newly awarded NSF grant to Haystack Observatory; collaboration with MIT Lab for Computer Science and MIT Lincoln Laboratory);
– New IP-based protocol tailored to operate in shared-network ‘background’ to efficiently using available bandwidth
– Connect with telescopes worldwide (U.S., Europe, Japan)
Mark 5 VLBI Disk-Based Data System (Phase 1)
• 1 Gbps continuous recording/playback to/from set of 8 inexpensive (ATA) disks• Developed at MIT Haystack Observatory with multi-institutional support• Mostly COTS components• Two removable ‘8-pack’ disk modules in single 5U chassis• With currently available 200GB disks – capacity of single ‘8-pack’ 1.6TB; expected to increase to 2.5TB by early 2003 at cost of ~$1/GB• GigE connection for real-time and quasi-real-time e-VLBI operations• Inexpensive: <$20K• ~20 Mark 5 systems now installed at stations and correlators
Haystack
Westford
NASA/GSFCUSNO
(correlator)
(correlator)
Bossnet 1 Gbps e-VLBI demonstration experiment(Phase 2)
Initial experiment
Future
WS/Mark 5
Summit1i
Summit5i
Westford Antenna Millstone RPE Bldg (Lorraine)
Haystack Correlator
GigEGigE
GigE GigESummit
5i
GigE
Congress StBoston
GlowNet
Lincoln Lab (Lorraine/Steve)
A1
A2
B5
B1
D2
Cisco1500
A4
C3B2
B4
C1
B3
D3
Cisco1500
Summit5i
Summit5i
Amp
E1
E3
Bossnet(Steve/Jim) ISI-E
E2 E6
A3
Figure 1: e-VLBI Path - Haystack to ISI-E
C2
GigEB6
WDM
WDM
GigE GigE
F2Amp Amp
GlowNetD1
E8
F1
GigE
WDM
evlbi011.drw20 Dec 01
WS/Mark 5
WS/Mark 5
1 8
14
13
16
15
28-1A
28-1B
LL
(Shared with LL onscheduled basis)
To be upgraded from2 to 3
To be replaced byJuniper M20 Jan 02 with OC-48 to Bossnet
1310
TestWS
To be providedby Haystack
MITCampus
1510
Plan install by 1 Jan 02
E4
E7
E5
GigE
Other users(no control)
GigE
To be updatedto OC48 Jan02
Scheduling onBossnet calendar
Schedule to becoordinated with LL
Schedule to becoordinated with LL
Dedicated -no scheduling
Dedicated -no scheduling
Details of path from Haystack to ISI-E – work in progress!
Trans-ponder
ISI-E (Tom)
G1
Figure 2: e-VLBI Path - ISI-E to GSFC/GGAO
JuniperM40
JuniperM160
JuniperM160
Cisco3508
Switch
Summit5i
WS/Mark 5
UMCP (Jerry or Dan)
GSFC/Bldg 28 (Pat) GGAO/Bldg 202 (Pat) GGAO Antenna Trailer (Bill)
WDM
WDM
WDM GigEGigEGigEGigE
OC-48Bossnet
G3G5
G10 H3
H7
J3
J5
K1
K2
K3
K4
K7 L1
L2
L3
L4
evlbi011.drw20 Dec 01
G2G4 G8
H8
H9
J4
H7
OC-48
GigEGigE
GigEGigE
Max @ ISI-E (Jerry)
Borrowed OC48interface - may notbe able to keep
Switch
GigEGigEBackup plan
Preferred e-VLBI path,but may not be in place.Pat working with Jerry and Dan.
Other users ( no control)
No jumbo frames
Max @ ISI-E (Jerry)
LuxN
Summit5i
TunerGigE GigE
OC-48
Plan to update toOC-48 ~Jan 02
Summit5i
On order MAC on orderdue Jan-Feb 02
Dell Linux currently at Haystack willalso be available here
1600 ft of fiberto be installedJan 02
Other users(no control)
1310
G9
G11
G12 H1TestWS?
G6
G7
TestWSMac with
Yellowdog Linux;need details
K5
K6
J1
GigE
Plan mid Jan 02
J2
Need details
Eventually replacedwith Mark 5 unit
Need details
H6
TestWS?
H5 Need details
TestWS?
H4
Scheduling onBossnet calendar
No scheduling;sufficient capacityfor e-VLBI
Scheduling necessaryif this path used
Will try to scheduleeither K2 or K3exclusively fore-VLBI experiment
Dedicated - noscheduling necessary
H2
J6
Details of path from ISI-E to GSFC Antenna – work in progress
Performance test results – Haystack/GGAO
Average sustained rate >900 Mbps over 10 hours
Westford-GGAO e-VLBI results
• First near-real-time e-VLBI experiment conducted on 6 Oct 02– Recorded data at 1152 Mbps on Westford-GGAO baseline– GGAO disk-to-disk transfer at average 788 Mbps transfer rate– Immediate correlation on Haystack Mark 4 correlator– Nominal fringes observed
• Direct data transfer experiment conducted on 24 Oct 02– Direct transfer of GGAO data to disk at Haystack at 256 Mbps– Immediate correlation with Westford data– Nominal fringes observed
• Next step – full real-time e-vlbi– Mark 5 system is capable of transmitting in real-time– But, still need additional work on correlator software to synchronize
correlator operation to real-time– Hope to conduct first experiment in early 2003
• Conclusion– e-VLBI at near Gbps speeds over ordinary shared networks is possible
but still difficult
Westford-to-Kashima e-VLBI experiment
• Westford/Kashima experiment conducted on 15 Oct 02– Data recorded on K5 at Kashima and Mark 5 at Westford at 256 Mbps– Files exchanged over Abilene/GEMnet networks
• Nominal speed expected to be ~20 Mbps, but achieved <2 Mbps for unknown reasons - investigating
– File formats software translated– Correlation on Mark 4 correlator at Haystack and PC Software correlator
at Kashima– Nominal fringes obtained– Further experiments are anticipated
Plans for UT1 Intensive e-VLBI• Daily ~1 hour VLBI sessions between Kokee Park, Hawaii and
Wettzell, Germany are used for UT1 measurements• Data are time sensitive since they are used for predicting UT1• Currently requires ~4 day turnaround shipping media• These measurements are an ideal candidate for routine e-VLBI
– Short daily session collect <100 GB of data– Even 100 Mbps will allow transfer in a few hours
• Work now in progress to make necessary connections– Network being organized from Kokee Park to USNO;
connection speed OC-3– Data from Mark 5 system in Wettzell will be carried to Univ. of
Regensberg, about 1 hour drive; connection speed OC-3– Negotiations ongoing for extension of MAX network to USNO with GigE
connection
New IP Protocols for e-VLBI (Phase 3)
• Based on observed usage statistics of networks such as Abilene, it is clear there is much unused capacity
• New protocols are being developed to utilize networks in ‘background’ mode for applications such as e-VLBI– Take advantage of special characteristics of e-VLBI data– Will ‘scavenge’ and use ‘secondary’ bandwidth– Will give priority to ‘normal’ users– Requires a new ‘end-point adaptive strategy’
• Work being carried out by MIT Haystack Observatory in collaboration with MIT Laboratory for Computer Science and MIT Lincoln Laboratory– 3-year program; will demonstrate e-VLBI connections both nationally and
internationally
Typical bit-rate statistics on Abilene network
Conclusion: Average network usage is only a few % of capacity
Usage >20Mbps less than 1% of the time
100 500 Mbps
0.1
1.0
0.01
0.001
Typical distribution of heavy traffic on Abilene
Conclusion: Heavy usage of network tends to occur in bursts of <2 minutes
secs200 400 1000
1.0
0.9
0.8
0.7
<10% of ‘bulk’ transfers exceed ~100 secs
Impact of e-VLBI Program
• Opens new doors for astronomical and geophysical research.
• Represents an excellent match between modern Information Technology and a real science need.
• Motivates the development of a new shared-network protocol that will benefit other similar applications.
• Drives an innovative IT research application and fosters a strong international science collaboration.