e2epi pipes update eric l. boyd. 2 decomposing the monolithic measurement architecture
TRANSCRIPT
E2Epi piPEs UpdateE2Epi piPEs Update
Eric L. BoydEric L. Boyd
2
Decomposing the Monolithic Measurement ArchitectureDecomposing the Monolithic Measurement Architecture
Data Analysis
Data Publication
Data Gathering Data
Gathering
Data
Public
ation
Data
Analysis
Monolothic Measurement
Archicture Measurement Framework
Measurement Tools
Analysis Tools
API
API
3
BWCTL (Jeff Boote)http://e2epi.internet2.edu/bwctlBWCTL (Jeff Boote)http://e2epi.internet2.edu/bwctl
bwctlclient
Verify Time/Return Results
TestStream
bwctldResource
Broker
bwctldResource
Broker
bwctld
bwctld
iperf
bwctld
bwctld
iperf
4
OWAMP (Jeff Boote)http://e2epi.internet2.edu/owampOWAMP (Jeff Boote)http://e2epi.internet2.edu/owamp
Server
owpingclient
[control]
owampd[Resource Broker]
Initialconnection
owampd[control]
OWD testendpoint
OWD testendpoint
Client
5
NDT (Rich Carlson)NDT (Rich Carlson)
• Network Diagnostic Tester• Developed at Argonne National Lab• Ongoing integration into piPEs framework
• Redirects from well-known host to “nearest” measurement node
• Detects common performance problems in the “first mile” (edge to campus DMZ)
• In deployment on Abilene:• http://ndt-seattle.abilene.ucaid.edu:7123
6
E2Epi Network Performance WorkshopE2Epi Network Performance Workshop
• Announcement: Internet2 has begun a new series of workshops
• Focus: Performance Tools• BWCTL, OWAMP, NDT• Installation• Use• Integration into IT support process
• Approach• Hands on• Interactive
7
Workshop DeliverablesWorkshop Deliverables
• Working installation of BWCTL, NDT, and OWAMP at GigaPoP
• Working installation of BWCTL, NDT, and OWAMP at each campus
• Exchange of “keys” among campuses and GigaPoP
• Community of Collaborative approach to performance diagnosis
8
Workshop Host / AudienceWorkshop Host / Audience
• Each workshop has a host, expected to be a campus / GigaPoP
• Audience is network engineers from a GigaPoP and associated campuses
• Host must identify audience, make invitations, provide networking
9
Workshop HistoryWorkshop History
• Developed/taught by Internet2 PAT Team (Spring ’05)
• SOX, hosted by GaTech (March ’05)• http://e2epi.internet2.edu/network-perf-wk/
gatech/agenda.html
• CENIC, hosted by UCLA (June ’05)• Joint Techs – Vancouver (July ’05)• Orthogonal to focus on GigaPoPs• Good opportunity to “kick the tires”
• After that, TBD …
10
Hosting a Network Performance WorkshopHosting a Network Performance Workshop
• Contact:• Eric Boyd ([email protected])
• Further Details:• http://e2epi.internet2.edu/network-perf-
wk/workshops-list.html
11
Performance Measurement: Project PhasesPerformance Measurement: Project Phases
• Phase 1: Tool Beacons (Stable)• BWCTL (Complete), http://e2epi.internet2.edu/bwctl• OWAMP (Complete), http://e2epi.internet2.edu/owamp• NDT (Complete), http://e2epi.internet2.edu/ndt• Deployment Workshops (In Progress),
http://e2epi.internet2.edu/network-perf-wk/workshops-list.html
• Phase 2: Measurement Domain Support (Working Prototype)• piPEs Alpha (Prototype), http://e2epi.internet2.edu/pipes/ • Abilene Measurement Infrastructure Deployment (Complete),
http://abilene.internet2.edu/observatory
• Phase 3: Federation Support (Current Focus)• AA (Prototype – optional AES key, policy file, limits file)• Discovery (Measurement Nodes, Databases) (Prototype – nearest NDT
server, web page)• Test Request/Response Schema Support (Prototype – GGF NMWG
Schema)
12
Metcalf’s LawMetcalf’s Law
• Robert Metcalf's law states that the "value" or "power" of a network increases in proportion to the square of the number of nodes on the network.
• Our version: The value of a performance measurement framework scales with the square of the deployment footprint
13
Vision StatementVision Statement
• Goal #1: A Federation of Measurement Frameworks• Approach: Develop a common language for
measurement frameworks• Metcalf’s Law: Successful new measurement
frameworks will be designed to speak the language• Goal #2: A Measurement Framework
Federation• Approach: Develop a decentralized, locally
administered, participatory framework• Metcalf’s Law: Autonomous network operators gain
more value by joining than “going alone”• These goals are not mutually exclusive …
14
Federation of Measurement FrameworksFederation of Measurement Frameworks
• Working within the GGF NMWG
• Version 1 of the schema is all but “done”
• Version 2 of the schema is under development
• Efforts such as piPEs, AMP, MonALISA, and NLANR DAST’s Advisor are interoperable
15
Measurement Framework Federation (1)Measurement Framework Federation (1)
• Tool beacons for on-demand and regularly scheduled tests are released:• BWCTL (http://e2epi.internet2.edu/bwctl)• Resource Allocation Broker for NLANR DAST’s Iperf (
http://dast.nlanr.net/Projects/Iperf/)
• NDT (http://e2epi.internet2.edu/ndt)• OWAMP (http://e2epi.internet2.edu/owamp)
• And deployed:• http://e2epi.internet2.edu/pipes/pmp/pmp-dir.ht
ml• 17 organizations listed as of today
16
Measurement Framework Federation (2)Measurement Framework Federation (2)
• piPEs Measurement Framework alpha is released:• http://e2epi.internet2.edu/pipes/
• And deployed on Abilene Observatory:• Shows results of 11 x 11 mesh of Iperf and
OWAMP tests among Abilene nodes• http://abilene.internet2.edu/observatory/
• Anyone can test to Abilene from their own BWCTL (need a key) or OWAMP node• http://e2epi.internet2.edu/pipes/ami/pmp-info.html
• Federation of NDT servers deployed on Abilene, with redirect to nearest server
17
Internet2 / GÉANT2 JRA1:Exchange of IdeasInternet2 / GÉANT2 JRA1:Exchange of Ideas
• UCL E2E Monitoring Workshop 2003• http://people.internet2.edu/~eboyd/ucl_workshop.h
tml• Internet2, DANTE, CANARIE biannual
meetings (12/03, 07/04) and ESnet (01/05)• Transatlantic Performance Monitoring
Workshop 2004 (03/04)• http://people.internet2.edu/~eboyd/
transatlantic_workshop.html• Brussels Workshop – Architecture (09/04)• Zurich Workshop – Prototype (04/05)• Ann Arbor Workshop – Prototype (05/05)
18
Internet2 / GÉANT2 JRA1:Working TogetherInternet2 / GÉANT2 JRA1:Working Together
• Contribution to the GGF NM-WG• Both Internet2 and Dante provide a significant
contribution to the effort (four to five people contributing regularly, 2 from Europe, 2-3 from US)
• Contribute to design, early adoption and prototyping, feedback
• Installation/evaluation of tools • Internet2 OWAMP, Internet2 BWCTL, Internet2
piPEs framework• DFN IPPM
• Caltech <-> CERN Demo• March ’04 and November, December ‘04
• Haystack, USA <-> Onsala, Sweden
19
Internet2 / GÉANT2 JRA1:Joint Development (1)Internet2 / GÉANT2 JRA1:Joint Development (1)
• Rather than build two separate interoperable measurement frameworks, why not jointly develop a single measurement framework?
• Steps:• Agree to joint open source development √• General Framework Design √• Prototype (Summer ’05)• Detailed Design• Implementation
• Seek participation from NRENs & campuses, particularly Internet2 & ESnet members
• Twice weekly conference calls• Very active mailing list• 2-3 face-to-face meetings per year
20
So what are we building?So what are we building?
• You could call it piPEs v2.0 or GFD or “SONAR”• Working Name: “Services Oriented
Network ARchitecture”• Plus sonar is used to “ping” :-D
• A Services-Based Measurement Framework for Building Dynamic, Self-Organizing Performance Communities
21
Design GoalsDesign Goals
• Services-oriented measurement framework• Dynamic, self-organizing characteristics identical to
that of the network as a whole• Recognize and facilitate the ability of independent
network entities to set policies and limits on the use of measurement resources locally
• Encourage and facilitate the use of measurement resources by users interested in network paths that traverse remote administrative domains
• Facilitate the widespread adoption of new performance tools in a broad, E2E framework
• Allow framework to evolve over time
22
Services Oriented ArchitectureServices Oriented Architecture
• In a simple scenario, each domain consists of a set of services. All services are well defined and independent
• Services within a domain represent the domain with the help of Authentication and Authorization – they respond to requests only if the Authentication service of the domain has authenticated the user and the policy of the given service authorizes it.
23
Basic ServicesBasic Services
• Lookup
• Authentication
• Measurement Point
• Measurement Archive
• Resource Protector (Authorization)
• Transformation• Topology
24
Lookup ServiceLookup Service
• Initial discovery• Multicast / Anycast• Well known servers• Required servers (by administrative configuration)• Previously detected servers (organized in a P2P
network – lookup services find out about other lookup services…
• Lookup is not simply by name• Type of service, Community, Network path,
Organization, Type of authentication required, etc.
• Response contains• Contact information, Available services, Authentication
required, etc.
25
Authentication ServiceAuthentication Service
• Registers with lookup• Client requests “kind” of authentication token
based on lookup results• Authentication grants time-limited token used to
request service• Attribute service created to protect privacy and
support role-based authorization• Allow new measurement points to be created as
easily as possible• Allow new data consumers access as easily as
possible
26
Measurement Point ServiceMeasurement Point Service
• Service to wrap measurement tools• Interacts with resource protectors to
protect shared resources• Registers with lookup service and
specifies the authentication credentials required to interact
• Registers with lookup service to indicate types of tests it can perform
• Accepts requests for tests
27
Measurement Archive ServiceMeasurement Archive Service
• Subscribes to some set of data – either from a measurement point or from an aggregation service
• May publish the derived data sets
28
Resource Protector ServiceResource Protector Service
• Enables centralizing of resource allocation (not globally - this is within spheres of administrative control)
• Multiple measurement points interact with a given resource protector to limit the shared resources
• Resource protectors can be chained to control aggregations of shared resources across larger frameworks.
29
Transformation ServiceTransformation Service
• Pipelines data between other components in the framework
• Subscribes and Publishes data• Provides:• Aggregation• Correlation• Caching• Duplication• Filtering• Translation
• Event generation• Data analysis
30
Topology ServiceTopology Service
• Special case of a Transformation Service• Network topology information is necessary for
measurement system optimization• Creates overviews/”maps” to illustrate network• Layered approach (domain level through to
wavelengths and physical level)• Collects raw data from measurement points and
pushes topology information into the measurement archive (allows topologically based queries by lookup service)
31
SummarySummary
• Open Source Shared Development• Sourceforge-based Sub-Projects• Modified Berkeley Licensing (?)
• Common Service-based Architecture • Architecture spans superset of deployment
use cases• ~Quarterly face-to-face meetings• ~Twice-Weekly phone conferences• Split development according to interest,
resources
32