e-VLBI Development at Haystack Observatory

Alan WhitneyChet RuszczykKevin DudevoirJason SooHoo

MIT Haystack Observatory

24 March 2006EVN TOG meeting

JIVE

e-VLBI Developments at Haystacks• August 2004:

– Haystack link upgraded to 2.5 Gbps– Real-time fringes at 128 Mbps, Westford and GGAO antennas, Haystack

Correlator• February 2005

– Real-time fringes Westford-Onsala at 256 Mbps– Used optically-switched light paths over part of route

• Starting April 2005– Start routine e-VLBI transfers from Tsukuba and Kashima (~240Mbps)

• Starting ~June 2005– Semi-automated regular e-VLBI UT1 Intensive data transfers from Wettzell to

ISI-E (disks hand-carried to USNO for correlation)– A few start-up problems, but now operating fairly smoothly

• September 2005– All CONT05 data from Tsukuba transferred to Haystack via e-VLBI

(many TB!)• Fall 2005

– Major effort in progress to connect NyAlesund to Haystack through NASA/GSFC at up to 100Mbps for e-VLBI

• November 2005– Global real-time e-VLBI demos at 512 Mbps at iGRID/SC2005 conferences;

used optically-switched dedicated light paths

Challenges

• Network bottlenecks well below advertised rates• Performance of transport protocols

– untuned TCP stacks, fundamental limits of regular TCP• Throughput limitations of COTS hardware

– Disk-I/O - Network• Complexity of e-VLBI experiments

– e-VLBI experiments currently require significant network expertise to conduct• Time-varying nature of network• Standard formats for transfer of data and control information between

different VLBI systems• ‘Last-mile’ connectivity to telescopes

– Most telescopes are deliberately placed in remote areas– Extensive initiatives in Europe and Japan to connect;

U.S. is lagging

Current Projects at Haystack Observatory

• Network interfacing equipment for e-VLBI– Mark 5 VLBI data system

• Standardization (VSI-E)

• Intelligent Applications– Automation of e-VLBI transfers an ongoing process

– Development of optimization-based algorithms for intelligent applications ongoing (EGAE)

– Intelligent optically-switched networks (DRAGON)

• e-VLBI test experiments

• Production e-VLBI– Put e-VLBI into routine use – progressing well in limited venues

VSI-E

• Goals:– Efficient transport mechanism– Standard protocols– Internet-friendly transport– Scalable Implementation– Ability to transport individual data-channel streams as individual packet

streams– Ability to make use of multicasting to transport data and/or control information

in an efficient manner• could be used in the future for support of distributed correlation

VSI Model

RTP Architecture

VSI-E Status

• Beta version of VSI-E is ready for evaluation for transfers from Kashima to Haystack (has been delayed by network problems);expect result soon

• Plan to submit VSI-E protocol for approval by Internet Engineering Task Force (IETF) as international standard after agreement within e-VLBI community and successful demonstrations

New Application-Layer Protocols for e-VLBI

• Based on observed usage statistics of networks such as Abilene, it is clear there is much unused capacity

• New protocols are being developed which are tailored to e-VLBI characteristics; for example:– Can tolerate some loss of data (perhaps 1% or so) in many cases

– Can tolerate delay in transmission of data in many cases

• EGAE allows user to specify a profile of experiment requirements, which are then translated into network requirements and strategies

• ‘Experiment-Guided Adaptive Endpoint’ (EGAE) strategy developed at Haystack Observatory under NSF grant– Now used in some routine e-VLBI transfers

EGAE Progress

• ‘Production’ e-VLBI facility has been established at Haystack to support routine e-VLBI transfers

• EGAE is now supporting routine non-real-time e-VLBI data transfers from Tsukuba

• EGAE will soon be used for routine e-VLBI transfers from Wettzell and NyAlesund

Real-time e-VLBI SC05 DemoNov 2005

Real-time transmission and processing of data from antennas in Westford, MA, Greenbelt, MD, and Onsala, Sweden at 512 Mbps/antenna

All except Kashima equipped with Mark 5 data systems; Kashima uses Japanese K5, included via VSI-E

Correlation results displayed inreal-time at SC05 meeting

Antenna/Correlator Connectivity• JIVE Correlator (6 x 1 Gbps)• Haystack (2.5 Gbps)• Westford, MA (10 Gbps to Haystack; 1 Gbps to outside world)• Kashima, Japan (1 Gbps)• Usuda, Japan (1 Gbps?)• Nobeyama, Japan (2.5 Gbps?)• Koganei, Japan (2.5 Gbps?)• Tsukuba, Japan (1 Gbps)• GGAO, MD (1 Gbps)• Onsala, Sweden (1 Gbps)• Torun, Poland (1 Gbps)• Westerbork, The Netherlands (1 Gbps)• Medicina, Italy (1 Gbps)• Jodrell Bank (1 Gbps)• Arecibo, PR (155 Mbps)• Wettzell, Germany (~30 Mbps)• Kokee Park, HA (nominally ~30 Mbps, but currently disconnected)• TIGO (~2 Mbps)

In progress• Hobart – agreement reached to install high-speed fiber• NyAlesund – work in progress to provide 100Mbps link for e-VLBI data

to Haystack Observatory• Forteleza – funds secured for fiber connection at 2.5Gbps; early 2006• Metsahovi – 1Gbps in 2006

International e-VLBI workshops

• Held every year since 2002, rotating between U.S., Europe, Japan and Australia

• Successful e-VLBI workshop held in Sydney, Australia in July 2005

– Hosted by CSIRO/ATNF

– 60 attendees from 9 countries

– Many up-to-date developments in e-VLBI and network research

– e-VLBI Technical Working Group re-formed

– Many thanks to all for a fine workshop

• Next e-VLBI workshop will be held at Haystack, 17-20 Sept 2006

– YOU ARE INVITED!