performance measurement and monitoring internet2 fall member meeting philadelphia, pa eric l. boyd...

Performance Measurement and Monitoring

Internet2 Fall Member MeetingPhiladelphia, PA

Eric L. Boyd – Internet2

Nicolas Simar – Dante

Agenda

• Internet2/G2 JRA1/ESnet Interaction Update

• Internet2 Performance Activities

• GÉANT2-JRA1 Performance Activities

• Next Steps / Open Issues

perfSONAR: Overview

• Joint effort of ESnet, GÉANT2 JRA1 and Internet2 PAT

• Webservices network performance framework• Network measurement tools• Network measurement archives• Distributed scheduling/authorization• Multi-domain policy

perfSONAR: Accomplishments

• Architecture document (Fall ’04)• Framework Design document (Spring ’05)• Prototype (Summer – Fall ’05)• Workshops in Brussels (09/04), Zurich (04/05),

Ann Arbor (05/05) and Poznan (08/05)• Development Environment (05/05)• Communications:

– Paper submissions (e2emon, TNC05 and ICSOC05)– TNC05 presentation

perfSONAR: Current Work

• Current work is focused on developing a prototype that will allow interface utilization data to be shared.

• Relatively simple use case, but demonstrates the feasibility of sharing data across multiple administrative domains.

perfSONAR: Services (1)• Measurement Point Service (MP)

• Measurement Archive Service (MA)

• Look-up Service (LS)

• Authentication Service (AS)

• Transformation Service (TS)

• Topology Service (ToS)

• Resource Protector Service (RP)

perfSONAR : Services (2)• Measurement Point Service

– Publishes measurement data to one or several consumer.

– Capability of starting a new tests.– Initiates active measurements, queries passive

measurement devices or packet capture .– Integrated within the tools or wrapper around legacy

tools.• Measurement Archive Service

– Storage of measurement data (over long period of time).

– A MP or a TS publish data to the MA.

perfSONAR: Services (3)• Lookup Service

– Allows the client to discover the existing services and other LS services.

– Dynamic: Services registration themselves to the LS and mention their capabilities, they can also leave or be removed if a service gets down.

• Authentication Service– Internet2 MAT, GN2-JRA5– Authentication functionality for the framework– Users can have several role, the authorisation is done

based on the user role.– Trust relationship between networks.

perfSONAR Services (3)• Transformation Service

– Transform the data (aggregation, concatenation, correlation, translation, etc).

• Topology Service– Make the network topology information available to

the framework.– Find the closest MP, provide topology information for

visualisation tools

• Resource protector– Arbitrate the consumption of limited resources.

Here is who I am, I’d like to access MA AHere is who I am, I’d like to access MA B

Where Link utilisation along - Path a,b,c,d,e,f?a,b,c: Network A – LS A, c,d,e,f : Network B, MA B, AS B

Client

Network A Network B

LS A LS BMA A MA B

AS A AS B

a b

c de f

Where Link utilisation along - Path a,b,c?a,b,c : Network A, MA A, AS A Token MB

Get link utilisation c,d,e,fHere you go

Token MA

Get Link utilisation a,b,cHere you go

Useful graph

Get link utilisation a, b, cHere you go.

Useful graph

Agenda

• Internet2/G2 JRA1/ESnet Interaction Update

• Internet2 Performance Activities

• GÉANT2-JRA1 Performance Activities

• Next Steps / Open Issues

Current Projects

• Performance Tools– BWCTL– NDT– OWAMP– Thrulay

• Performance Measurement Framework– piPEs -> perfSONAR– GGF NMWG

• Member Outreach– Network Performance

Measurement Workshops

– Performance Tool Cookbooks

• Bulk Transport– Design Space– Prototype

OWAMP (1-way ping)

• LOTS of new deployments (Network Performance Workshop Attendees)

• New “developers” release to support latest version (14) of owdp spec• TTL (hop count)• Early terminated sessions handled more gracefully• Sender will skip sending “late” records and shares that information with

receiver

BWCTL (Bandwidth Test Controller)

• LOTS of new deployments (Network Performance Workshop Attendees)

• New alpha release• Bug fixes• Ported to Solaris, OS X• NTP dependency eliminated

NDT and Thrulay

• NDT– Duplex Mismatch Detection– Grant from NIH / NLM

• Thrulay– UDP Mode– Google Summer of Code enhancements

GGF NMWG

• Version 1 of the schema “all but done”– Employed by piPEs, Advisor, AMP,

MonALISA, and SLAC

• Version 2 of the schema continues to evolve– perfSONAR work benefits from and informs

this project

Network Performance Measurement Workshops (1)

• Goals– Installed base of BWCTL/Iperf, OWAMP, and

NDT at GigaPoP and regional campuses.– Begin integration into IT support processes.– Create and installed base for perfSONAR

deployment.– Give each participant tool-specific cookbooks.

Network Performance Measurement Workshops (2)

• Completed– SOX / GaTech (03/05)– CENIC / UCLA (06/05)– JT – Vancouver (07/05)– MAGPI / FMM (09/05)

• Planned– OARNet / OSU (09/05)– APAN (01/06)

• Under Consideration– MERIT, MAX, Winter Joint Techs, …

Other Outreach

• Next Generation Measurement Panel/BoF (07/05)

• Bridging the Gap Workshop (08/05)

• Application-Community-Specific Network Performance Workshops (mini-version of the workshop, focusing on use of tools, not installation) being developed

Bulk Transport Tool (1)

• Killer App for High Performance Networks (i.e. why else do we need fat pipes)

• Remedies for TCP’s maladies• Tuning: buffers, window scaling, timestamps,

SACK• Use multiple streams• Something Else• Replace the kernel and use different congestion

control• Replace all the routers and kernels

Bulk Transport Tool (2)

• Many alternative TCP/IP congestion control algorithms

• Modified kernels are incompatible with regular kernel security patches

• Get the benefits of kernel-level modifications to TCP/IP congestion control algorithms in a user level tool, avoiding security issue with alternate kernels

Bulk Transport Tool (3)

• Design Space Document

• Early Stage Prototype

• Google Summer of Code efforts

Agenda

• Internet2/G2 JRA1/ESnet Interaction Update

• Internet2 Performance Activities

• GÉANT2-JRA1 Performance Activities

• Next Steps / Open Issues

GÉANT2 JRA1

• 15 European NRENs putting resources together to work on multi-domain monitoring and on performance measurements.– Arnes (SI), Belnet (BE), Carnet (HR), Cesnet (CZ),

DFN (DE), FCCN (PT), GARR (IT), GRnet (GR), ISTF (BG), Uninett (NO), PSNC (PL), redIRIS (ES), Renater (FR), Surfnet (NL), SWITCH (CH), DANTE.

• Works with – The Performance Enhancement Response Team.– The multi-domain L2 circuits activity to provide them multi-domain measurement data.

Requirements• Questionnaires sent to NRENs, Projects and

End-users (system administrators).– 45 answers received (15 each).

• Requirements – Strong interests to access monitoring information form

multiple networks. – Readiness to open access to measurement data

varies greatly.– On-demand capability (to and from other domains).– Be able to monitor the services deployed.

• IP QoS, IPv4/v6, multicast

– Great variety of tools deployed.

Requirements [2]• Requirements

– Monitoring Information:• RTT and OWD.• Bandwidth Utilisation and achievable TCP

throughput.• RTT and OWD packet loss.• Delay variation.• Interfaces error and drops.• Routing/path information.

Tools• Work on the integration of few selected

measurement tools into the framework. – Evaluate the different tools before selecting one.– DFN IPPM (OWD), BWCTL/iperf, RRD –

December 05.– Enhancement of those measurement tools.– Investigate netflow and packet capture uses.

• Make use of existing visualisation tools – NEMO (Uninett), CNM (DFN).

• Will build (new) visualisation tools based on users feedback.

Other tasks

• Work on data aggregation and concatenation.

• Deployment of a test infrastructure.

• Multi-domain L2 circuit monitoring (SDH and Ethernet based circuits).

Agenda

• Internet2/G2 JRA1/ESnet Interaction Update

• Internet2 Performance Activities

• GÉANT2-JRA1 Performance Activities

• Next Steps / Open Issues

perfSONAR: Prototype• Phase 0

– Simplistic client which requests data to a MA (RRD filesystem) using web-services (Done)

• Phase 1– Include simplistic LS web-services (on-going) – Visualisation – map based (on-going)

• Phase 2– Request additional data (OWD, packet drops)– Dynamic registration to LS– Ping MP

• Phase 3 (Dec 05)– Handel attributes for other services– Distributed LS data across several domains– MP get’s on-demand capability (BWCTL)– Make use of the attributes to offer different functionalities to the

users

What’s Next?

• Current Status:– Regular discussions– Development is underway– About to use perfSONAR to provide data to an international

project (link capacity and link utilisation).

• We are at a key moment of the collaboration:– Distributed development process emerging– Making compromises between the vision and the technology– Where should we cut corners on the prototype?– Does the prototype form the basis of the deployed system?

Questions?

• Eric Boyd, Internet2 (eboyd@internet2.edu)

• Nicolas Simar, DANTE / GÉANT2 – JRA1 (Nicolas.Simar@dante.org.uk)