13-Oct-2003

Internet2 End-to-End Performance Initiative: piPEs

Eric Boyd, eboyd@internet2.edu

Matt Zekauskas, matt@internet2.edu

Internet2 InternationalTask Force Meeting

13-Oct-2003 2Internet2 Measurement Architecture for End to End Performance

Performance Measurement and Monitoring Architectures

What are we (Internet2) planning?

piPEs (performance monitoring framework)How can we interoperate?

Common input/output schemas/interfaces

(GGF NMWG)

Common tool development

(Open source; under discussion)How are we collaborating internationally?

(DANTE/GEANT, UCL, HENP)

13-Oct-2003 3Internet2 Measurement Architecture for End to End Performance

What is “piPES”?

End-to-End Performance Initiative Performance Environment System (E2E piPES)

Goal: To allow end-users and network operators to determine performance capabilities, locate problems, and contact the right person to get a problem resolved.

Approach: Collaborative project combining the best work of many organizations, including DANTE/GEANT, NLANR/DAST, SLAC, UCL.

13-Oct-2003 4Internet2 Measurement Architecture for End to End Performance

Sample Use Vision

User has a perceived problem (or wants to pre-test an end-to-end path)

Bring up Web page on end point

Input kind of application and other endpoint

Ask (local) piPES testing and analysis engine, “Will it work?”[Domain Interface]

13-Oct-2003 5Internet2 Measurement Architecture for End to End Performance

What’s in the network?Using Internet2 as an example

Within Abilene, deploy test points at every router node

• They run a full mesh of periodic tests (latency, traceroute, throughput)

• They have access to local utilization data and sampled flow data

Similar points in gigaPoPs and then campuses

• They run periodic tests among each other (depending on local preferences)

• They run occasional tests into backbone nodes

13-Oct-2003 6Internet2 Measurement Architecture for End to End Performance

Sample Use Vision, continued

Testing & analysis engine discovers performance measurement points along the path, determines a set of test results required

It uses the results of periodic testingIt schedules a local test to nearest test point, and any other tests where the data is stale

The results are used to “divide and conquer”, pointing to a suspect network segment and a contact point

The contact is given the results to further investigate the problem

13-Oct-2003 7Internet2 Measurement Architecture for End to End Performance

Where are we now?(What is Internet2 implementing?)

Backbone router measurement points

Recurring testing written into database;IPv6 and IPv4

On-demand testing

Web and “web services” output

Enough for knowledgeable human, or consumption by other projects (including those aimed at user interface)

13-Oct-2003 8Internet2 Measurement Architecture for End to End Performance

Role of Internet2

Membership contributing; work on holes

Provide central support where it makes sense, for example

• Backbone viewpoints, architecture• Central knowledge repository

Help with path problems, connection with University/gigaPoP networking personnel

13-Oct-2003 9Internet2 Measurement Architecture for End to End Performance

E2Epi Work

piPEs architecture (collaborative)One-way measurement tools (e.g. OWAMP)

For intermediate servers• Scheduling• Database• Authentication

Export data via web serviceSpecific reference servers or beacons

13-Oct-2003 10Internet2 Measurement Architecture for End to End Performance

Aside: Other E2Epi Work

Understand applications and their performance requirements

• Technical Advisory Group

Provide best practices/experience for network operators

• Collecting Performance Stories• Campus Network Infrastructure Guide• Pointers to relevant projects

13-Oct-2003 11Internet2 Measurement Architecture for End to End Performance

Aside:Reference Servers / Beacons

(TCP) Performance debugging• Conjecture: 80% of problems related to

–Host tuning (mostly buffers)–Duplex mismatch [path]–Other physical connection problem [path]–NDT: http://miranda.ctd.anl.gov:7123/

H.323 conferencing• Goal: portable machines that tell you if system likely to work (and if not, why?)

–http://e2epi.internet2.edu/Beacon/BeaconOverview.html–ViDeNet Scout, http://scout.video.unc.edu/

13-Oct-2003 12Internet2 Measurement Architecture for End to End Performance

E2E piPEs Architecture

End UserInterface

Network AdministratorInterface

CulpritDB

“TurnYourself

In” CulpritInterface

ID Self

AAA

QueryCulprit

IDCulprit

Specify end points &application family

Request TestResultTesting

EngineAnalysisEngine

Specify Test &End Nodes

SendConclusion

InitiatorAdministrative

DomainInterface

Reply Ready,Rejected, Patience

Virtual PerformanceMeasurement DB (local to

Domain)

AcceptorAdministrative

DomainInterface

AAA and ToolAgreementRequest Test

Result

InitiatorPMC

AcceptorPMC

AAA & ScheduleAgreement

TargetPMP

SourcePMP

AAA & Test

DatabaseGatekeeper

Web-based

DisplayEngine

HumanAnalysisEngine

13-Oct-2003 13Internet2 Measurement Architecture for End to End Performance

E2E piPEs Architecture v1.0

End UserInterface

Network AdministratorInterface

CulpritDB

“TurnYourself

In” CulpritInterface

ID Self

AAA

QueryCulprit

IDCulprit

Specify end points &application family

Request TestResultTesting

EngineAnalysisEngine

Specify Test &End Nodes

SendConclusion

InitiatorAdministrative

DomainInterface

Reply Ready,Rejected, Patience

Virtual PerformanceMeasurement DB (local to

Domain)

AcceptorAdministrative

DomainInterface

AAA and ToolAgreementRequest Test

Result

InitiatorPMC

AcceptorPMC

AAA & ScheduleAgreement

TargetPMP

SourcePMP

AAA & Test

DatabaseGatekeeper

Web-based

DisplayEngine

HumanAnalysisEngine

13-Oct-2003 14Internet2 Measurement Architecture for End to End Performance

Intra-‘PMP’-Module Protocol

DomainInterface

RequestInterface

ResponseInterface

E

C

D

B

Source PMCSink PMC

A

F

Source PMP

G

Sink PMP

J

K

I

HM

N

L

O P

DBGatekeeper

UCL,SLAC,Dante,

CLRC-DL,UKERNA

Internet2

VirtualDB

13-Oct-2003 15Internet2 Measurement Architecture for End to End Performance

piPEs + Abilene Measurement Rollout (ongoing)

AMIPMP

AMIPMP

AMIPMP

OWAMPPERF

DB

Visualization Package

OWAMPIPERF

Traceroute

OWAMPIPERF

Traceroute

OWAMP, IPERF, Traceroute

TraceroutePERF

DB

IPERFPERF

DB

SNMPPERF

DB

Flow, SNMP

RouterInterface

RouterInterface

Flow, SNMP

RouterInterface

Flow, SNMP

FLOWDATA

DB

13-Oct-2003 16Internet2 Measurement Architecture for End to End Performance

piPEs / AMI Rollout (ongoing)

AMIPMP

AMIPMP

AMIPMP

OWAMPPERF

DB

Visualization Package

OWAMPIPERF

Traceroute

OWAMPIPERF

Traceroute

OWAMP, IPERF, Traceroute

Web Service

TraceroutePERF

DB

IPERFPERF

DB

SNMPPERF

DB

Flow, SNMP

RouterInterface

RouterInterface

Flow, SNMP

RouterInterface

Flow, SNMP

FLOWDATA

DB

13-Oct-2003 17Internet2 Measurement Architecture for End to End Performance

piPEs / AMI Rollout (near future)

AMIPMP

AMIPMP

AMIPMP

OWAMPPERF

DB

Visualization Package

OWAMPIPERF

Traceroute

OWAMPIPERF

Traceroute

OWAMP, IPERF, Traceroute

Web Service

TraceroutePERF

DB

IPERFPERF

DB

SNMPPERF

DB

Flow, SNMP

RouterInterface

RouterInterface

Flow, SNMP

RouterInterface

Flow, SNMP

FLOWDATA

DB

CampusPMP

CampusPMP

13-Oct-2003 18Internet2 Measurement Architecture for End to End Performance

E2E piPEs Architecture (Grid)

InitiatorAdministrative

DomainInterface

Virtual PerformanceMeasurement DB (local to

Domain)

AcceptorAdministrative

DomainInterface

AAA and ToolAgreement

InitiatorPMC

AcceptorPMC

AAA & ScheduleAgreement

TargetPMP

SourcePMP

AAA & Test

DatabaseGatekeeper

NLANR/DAST Advisor (Testing/Analysis Engine)

Abilene Observatory (Web-basedDisplay Engine)

SLAC (High Energy Physics Routing)

UCL (GRID / Globus Interface)

Applications (Application ProblemDetection)

AMP DataRIPE Data

Surveyor Data

13-Oct-2003 19Internet2 Measurement Architecture for End to End Performance

Coordinating Interoperably

A few levels• Awareness of other projects• Cooperate on architecture development• Ensure important interfaces are shared/publisheduse our data!

• Share code• Install each others nodes

Please comment on architecture

TF-NGN focus on inter-domain issues

13-Oct-2003 20Internet2 Measurement Architecture for End to End Performance

Coordinating

Measurement Schema(lots of work in GGF)

Authentication and Authorization• Roles for access (End user, test buddy, NOC)• For us, try Shibboleth for implementation

How discover PMP/domain interface

13-Oct-2003 21Internet2 Measurement Architecture for End to End Performance

Coordinating (help)

Debugging algorithms using data

Designing system to scale

Balance centralization and distributed database requirements

13-Oct-2003 22Internet2 Measurement Architecture for End to End Performance

References

http://e2epi.internet2.edu

http://abilene.internet2.edu/observatory/

13-Oct-2003 23Internet2 Measurement Architecture for End to End Performance

13-Oct-2003 24Internet2 Measurement Architecture for End to End Performance

Intra-‘PMP’ Module Components

Domain Interface: Web Service Interface to Request Performance Data

Performance Measurement Controller (PMC): Schedules Tests

Performance Measurement Point (PMP): Performs Tests, Stores Results in Database

Source: Initiates Test Request

Target: Accepts Test Request & Starts Test

13-Oct-2003 25Internet2 Measurement Architecture for End to End Performance

Domain Interface

Request Interface:Accepts External Result RequestsCompares Requestor Role to PolicyRejects Request or Queries Response Interface

Response Interface:Accepts Result/Tool RequestsCompares Requester Identity, Source Role to PolicyDecides if Tool is AvailableRejects Request or Supplies ‘Capability’

13-Oct-2003 26Internet2 Measurement Architecture for End to End Performance

Initiator / Acceptor Performance Measurement Controller

Initiator PMC:Supplies Capability, Identity, ToolAcceptor PMC:Accepts/Rejects/Delays Request Based on PolicyContacts Target PMP to Initiate TestAccepts/Rejects Request Based on PMP Response

13-Oct-2003 27Internet2 Measurement Architecture for End to End Performance

Target / Source Performance Measurement Point

Source PMP:Accepts/Rejects Requests to Start Test based on IdentityStarts TestsTarget PMP:Accepts Test from Source PMPStores Results LocallySends Data to DB Gatekeeper

13-Oct-2003 28Internet2 Measurement Architecture for End to End Performance

Database Gatekeper

Accepts/Rejects Requests to Store Data based on Identity

Accepts/Rejects Requests to Release Data based on Role, Identity

Supplies Performance Data

13-Oct-2003 29Internet2 Measurement Architecture for End to End Performance

piPEs Tool Deployment on Abilene

All tests on IPv4 and IPv6

OWAMP: Deployed on 10/11 nodes (nms4)

IPERF – UDP: In deployment beta on 2 nodes (nms1)

IPERF – TCP: In deployment beta on 2 nodes (nms1)

Traceroute: In deployment beta on 2 nodes (nms4)

Router Data: Deployed on all 11 nodes (router interface)

Flow Data: Deployed on 10/11 nodes (nms3)

13-Oct-2003 30Internet2 Measurement Architecture for End to End Performance

Goal of this talk

Performance Measurement and Monitoring Architectures:

What are networks planning and how can we coordinate interoperability?

[DELETE THIS SLIDE WHEN DONE]