connect. communicate. collaborate support by gÉant2 and nrens for evlbi in europe john chevers,...

Connect. Communicate. Collaborate

Support by GÉANT2 and NRENs for eVLBI in Europe

John Chevers, DANTEIn collaboration with:

Toby Rodwell (DANTE)

Steve Parsley, Arpad Szomoru (Joint Institute for VLBI in Europe)

Paul Burgess, Simon Casey, Richard Hughes-Jones, Adam Mathews, Stephen O’Toole, Ralph Spencer (University of Manchester)

TNC2005 Poznan, Poland 8th June 2005

Connect. Communicate. CollaborateUser Support in GÉANT2

• GÉANT2 has 3 million+ users in 34 countries in Europe and beyond• How to support them?• Pan-European projects with ‘demanding’ requirements

– International user groups– High bandwidth requirement?– Specialised service requirement?

• Aim to introduce new levels of accountability, transparency and service• Custom solutions for the particular user group• To manage the lifecycle of the networking aspects of each user project• Customer focused


User Support Mechanism for GÉANT2

• Initial Contact– Establish eligibility of user application– Inform all NRENs involved– Assign point of contact

• User Questionnaire– Requirement gathering

• Evaluation– Consultation between DANTE and NRENs– Define stakeholders


User Support Mechanism for GÉANT2 (II)

• Planning and Implementation– Action plan– Design of OPN– Risk Register

• Continued Support– A service charter for the user– Change management

QUALITY OF CUSTOMER SERVICE IS VITAL!

Connect. Communicate. CollaborateNetworking for eVLBI in Europe

• European eVLBI Network: collaboration of radio observatories in Europe, Asia and South Africa

• Presently 5 telescopes connected: Torun, Onsala, Cambridge, Westerbork, Jodrell Bank

• Many more to be connected soon• Large data flows, monitoring required• A demanding user!


The future of the distant past:using networks for VLBI

• Connecting observatories to create a telescope as big as Europe!

• Continental-scale baselines produce the most detailed images ever of the most distant objects!

• Improving the resolution of observations to see further back in time

• Eliminating out-moded tape technology and speeding up results

• Harnessing the power of many radio telescopes in real time –only possible with GÉANT2

• Allowing new ‘Window of Opportunity’ experiments: dynamic scheduling

• Distributing results to users across the world

Connect. Communicate. CollaborateGÉANT2/NREN eVLBI Network Connect. Communicate. Collaborate

se1 - Juniper

uk1 - Juniper

nl1 - Juniper

cz1 - Juniper

pl1 - Juniper

de1 - Juniper

it1 - Juniper

de2 -Juniper

STM-64 (10Gbps)

ws4.it Dell

geant-gw3lond-scr3

man'-bar

Cisco Cisco

warr-scr

Cisco

read-scr3

Milan GSR

BolognaGSR

Bologna

GARR bwctl Mk5 MB

STM-16 (2.5Gbps)

Gigabit Ethernet

ManCompCen

Transmode

3ComCR1

AR5

JIVE, NL

JODRELL BANK,

UK

Mk5s

Mk5s

BOLOGNA, IT

ws4.ukDell

Mk5 Torun

oso-gw

chalmers2

goteborg2

boras2jonkoping1stockholm1stockholm2

se-kth

Mk5

Cisco

ONSALA, SE

TORUN, PL

Unknown

Netherlight

Connect. Communicate. CollaborateGÉANT2 User Support for eVLBI

• The EVN-NREN forum: the communities meet face-to-face!– Planning– Problem solving– Discussion of results– Increasing understanding

• GÉANT Weathermap and Taxametro tools• Counters on GÉANT links• Diagnostic tools (BWCTL) in GÉANT and NREN network nodes• PERT account created –letting SMEs loose on the problem!• Also use of specialised tools eg UDPmon

Connect. Communicate. CollaborateProgress in eVLBI 2004-2005

• 01/04 First disc-buffered eVLBI image • 04/04 Three-telescope eVLBI session• 06/04 eVLBI network stress tests (iPerf)• 09/04 4 telescope eVLBI• 09/04 First real-science eVLBI • 12/04 Fringes at 256 Mbps JB-Tr• 02/05 Network data transfer tests (BWCTL) between 6 eVLBI sites• 03/05 eVLBI science session involving 6 telescopes: Wb, Ar, JB, Cm, On, Tr• 03/05 EXPReS Proposal:

– ‘Production’ eVLBI– 16 telescopes – High Data Rates– Global eVLBI

• 06/05 Connection of Medicina observatory: brings total to 6 telescopes in 5 countries across Europe


4th Year MPhys ProjectAdam Mathews Steve O’Toole

Throughput vs packet spacing Manchester: 2.0 GHz Xeon Dwingeloo: 1.2 GHz PIII Near wire rate, 950 Mbps Packet loss CPU Kernel Load Receiver

UDP Throughput Nov 2003Manchester-SuperJANET4-GEANT-Surfnet-Dwingeloo

Gnt5-DwMk5 11Nov03-1472 bytes

0

2

4

6

8

10

12

0 5 10 15 20 25 30 35 40Spacing between frames us

% P

acket

loss

Gnt5-DwMk5

DwMk5-Gnt5

Gnt5-DwMk5 11Nov03/DwMk5-Gnt5 13Nov03-1472bytes

0

200

400

600

800

1000

1200


Recv W

ire r

ate

Mbits/s

Gnt5-DwMk5

DwMk5-Gnt5

Gnt5-DwMk5 11Nov03 1472 bytes

020406080

100


% K

erne

l R

ecei

ver

Effect on local traffic

Connect. Communicate. Collaborate 1472 byte Packets man -> JIVE Relative 1-way delay

Packet Loss & 1-way Delay

1472 bytes w12 Gnt5-DwMk5 21Oct03

0

2000

4000

6000

8000

10000

12000

2000 2100 2200 2300 2400 2500 2600 2700 2800 2900 3000Packet No.

1-w

ay d

elay

us

1472 bytes w12 Gnt5-DwMk5 21Oct03

0

2000

4000

6000

8000

10000

12000

0 1000 2000 3000 4000 5000Packet No.

1-w

ay d

elay

us


Measure the time between lost packets in the time series of packets sent.

Lost 1410 in 0.6s Is it a Poisson process? Assume Poisson is stationary

λ(t) = λ Use Prob. Density Function:

P(t) = λ e-λt

Mean λ = 2360 / s[426 µs]

Plot log: slope -0.0028expect -0.0024

Could be additional process involved

Packet Loss DistributionIs it Poisson?

packet loss distribution 12b bin=12us

0

10

20

30

40

50

60

70

80

12 72 132

192

252

312

372

432

492

552

612

672

732

792

852

912

972

Time between lost frames (us)

Num

ber

in B

in

Measured

Poisson

packet loss distribution 12b

y = 41.832e-0.0028x

y = 39.762e-0.0024x

1

10

100

0 500 1000 1500 2000

Time between frames (us)

Num

ber

in B

in


eVLBI Data Transfers over GÉANT:Long term tests of Bandwidth

Jodrell-UKERNA-GÉANT-SURFnet-JIVE

1472 byte UDP packets, Dual Xeon (Manchester) to Mark5A (JIVE)

550

600

650

700

750

800

850

900

950

01/11/2004 12:00 09/11/2004 12:00Time of day

Wir

e r

ate

(M

b/s

)

12us

18us

20us


eVLBI Data Transfers over GÉANT: Long term tests of Packet Loss

Jodrell-UKERNA-GÉANT-SURFnet-JIVE

• 6 runs out of 747 showed any loss• 1097 packets lost out of 81 million sent – 1.3x10-3%

1472 byte UDP packets, Dual Xeon (Manchester) to Mark5A (JIVE)

0

0.2

0.4

0.6

0.8

1

1.2

01/11/2004 12:00 09/11/2004 12:00

Time of day

% L

os

t 12us18us20us


UDP Tests Jan 2005 Simon Casey (PhD project)

Between JBO and JIVE in Dwingeloo, using production networkPeriod of high packet loss (3%):


GEANT – UKLight Comparison May 2005 Simon Casey (PhD project)

Between JBO and JIVE in DwingelooUsing production network Using UKLight network

gig6-jivegig1_int_1May05

050000

100000150000200000250000300000350000

0 10 20 30 40Spacing between frames us

Pkt

s re

-ord

ered

50 bytes

100 bytes 200 bytes

400 bytes 600 bytes

800 bytes 1000 bytes


1472 bytes


0

20

40

60

80

100


% P

acke

t lo

ss

50 bytes

100 bytes 200 bytes

400 bytes 600 bytes



1472 bytes


0100200300400500600700800900

1000


Recv W

ire r

ate

Mbit/s

50 bytes

100 bytes

200 bytes

400 bytes

600 bytes

800 bytes

1000 bytes

1200 bytes

1400 bytes

1472 bytes

gig7-jiveg1_UKL_31May05

0

0.2

0.4

0.6

0.8

1


Pac

ket

re-o

rder

ed

50 bytes

100 bytes 200 bytes

400 bytes 600 bytes



1472 bytes


0100200300400500600700800900

1000


Rec

v W

ire r

ate

Mbi

t/s

50 bytes

100 bytes

200 bytes

400 bytes

600 bytes

800 bytes

1000 bytes

1200 bytes

1400 bytes

1472 bytes


0

20

40

60

80

100


% P

acke

t lo

ss

50 bytes

100 bytes 200 bytes

400 bytes 600 bytes



1472 bytes

Connect. Communicate. CollaborateResults of Jodrell-JIVE eVLBI Transfer Tests

• Overall link quality Manchester/Jodrell to JIVE very good – near line rate possible

• Occasional negligible losses, very infrequent noticeable losses

• Packet re-ordering observed• Monitoring equipment will help to discover point in the link

where losses occur• These tests had no ‘exclusive’ access to machines –

possible for other users to have an effect• Real eVLBI sessions should be able to control access of

machines


Conclusions and the FUTURE

• Data rates of 256Mbps feasible on real eVLBI runs• Data rates of 900Mbps possible with UDP protocol• The EXPRes EU eVLBI proposal: 16 telescopes in Europe,

Asia, Australia, Latin America and North America JBO-Onsala test link at 4 Gbps, submitted March 19th 2005

• eVLBI tests in EVN at ~ 1 per month• Network tests at up to 950 Mb/s – statistics of packet loss• Investigate alternatives to TCP, UDP – tsunami, VSI-E etc.• ESLEA tests on MB-NG and UKLight• eVLBI will be routine in 2006!!


Backup Slides


•Packet loss distribution:

( )t

p d

Cumulative distribution

Cumulative distribution of packet loss, each bin is 12 sec wide

Long range effects inthe data?

Poisson


Divide time series of packets into 1000 slices of 50 packets

Total lost packets 1410 Average number / slice =

1.4

Calc Poisson Probability P(n, µ) = µ n e -µ

n!

Curves close but not exact Could be more than 1

process

VLBI Project: Packet Loss – Is it Poisson?

0

50

100

150

200

250

300

350

400

0 5 10 15n num lost in sub-sample

N(n

)

run12b

1

1.3

1.4

1.8


Aggregated Variance Method Divide time series length N into

blocks of size m Calc mean of each section

Xm(k) k= 1 … N/m Calc variance VXm of these

Xm(k) Vary m size of the blocks

Plot on log-log & fit slope β Hurst parameter H

β = 2H -2 Measure:

β = -0.355 H 0.822 H =1 no long range dependence

VLBI Project: Packet Loss – Long Range Effects?

y = -0.355x + 2.8826

0

0.5

1

1.5

2

2.5

3

3.5

4

0 0.5 1 1.5 2 2.5 3sub-sample size Log10( m )

Agg

rigat

e-va

rianc

e L

og10

( X(m

) )

connect. communicate. collaborate support by gÉant2 and nrens for evlbi in europe john chevers,...

Documents