ARCHSTONEARCHSTONE
Advanced Resource Advanced Resource Computation for Hybrid Service Computation for Hybrid Service
and TOpology NEtworksand TOpology NEtworks
Project Review MeetingProject Review MeetingLBNL, Berkeley, CALBNL, Berkeley, CA
October 5October 5thth, 2010, 2010
Personnel
• USC/ISI
• Tom Lehman
• Xi Yang
• ESnet
• Chin Guok
• Eric Pouyoul
• Inder Monga
• UNM
• Nasir Ghani
• Feng Gu
Current Network Provisioning
• Current Dynamic Network capabilities and deployments allow us to provision point-to-point "Ethernet" circuits or Layer 3 MPLS tunnels
lower layer is statically
provisioned
Current Network Provisioning
• The single layer provisioning model can be extended to Multi-Domain via the InterDomain Controller Protocol (IDCP)
• www.controlplane.net
• Using systems like OSCARS, DRAGON
• While the technology of the networks may differ, there is a single "service":
• Ethernet Framed Point-to-Point Circuit
lower layer is statically
provisioned
Desired - Vertical Intra-Domain
• Provision services at lower layer to create a topology element (link between routers) at the higher layer
• Subsequently provision any remaining bandwidth at the higher level
• may provision a 10Gbps LSC link in response to a request for 5Gbps immediate need
• remaining 5Gbps available for subsequent service requests
Desired - Multi-Service/Layer/Domain Provisioning
• Provisioning at multiple layers within a domain
• A variety of services (layer 3, layer2, layer 1, different technologies)
• Multiple peering at different technology layers between domain
ARCHSTONE Objectives
• ARCHSTONE objective is to enable this "single- layer point-to-point circuit provisioning" to evolve to "multi-layer topology provisioning"• The topology component is a very important paradigm shift
• Currently a provisioning event is single point-to-point circuit
• ARCHSTONE objective is to view service requests as topologies which may be a single point-to-point link, or a multi-node/link construct which requires multiple coordinated provisioning events to instantiate
• In addition, we want to be able to compute and instantiate topologies which allow for flexible and extensible "constraint" space or "multiple dimension computation"
• These dimensions include time domain, AAA, technology layers, performance, protection, restoration, measurement, and monitoring.
ARCHSTONE Focus and Approach
• Advanced Network Service Interface
• "Request Topology" and "Service Topology" concepts
• Common Network Resource Description schema
• Multi-Dimensional Topology Computation Element (MX-TCE)
• High Performance computation with flexible application of constraints
• Use OSCARSv6 as base infrastructure and development environment
Network Service
Requester
Network Provider
Agent
Network Resource
Description
Network Network Service Service InterfaceInterface
request
replyMX-TCE
OSCARSv6
MX-TCE
•Time Domain•AAA•Management (SNMP) Data•Administrator Requirements
Virtual Organization Specific View
Friday, 3amMaintenance View
IP View
SDN View
What paths are possible between A and B, between time i and ii, with specific policy.A B
User Specified Multiple ViewsESnet
IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
OSPF-TE
Topology
OSPF-TE
Topology
OSPF-TE
Topology
OSPF-TE
TopologyTopology
OSPF-TE
MX-TCE
•Time Domain•AAA•Management (SNMP) Data•Administrator Requirements
•Time Domain•AAA•Management (SNMP) Data•Administrator Requirements
Virtual Organization Specific View
Friday, 3amMaintenance View
IP View
SDN View
What paths are possible between A and B, between time i and ii, with specific policy.A BA B
User Specified Multiple ViewsESnet
IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
OSPF-TE
Topology
OSPF-TE
Topology
OSPF-TE
Topology
OSPF-TE
TopologyTopology
OSPF-TE
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
ESnet IP
ESnet IP
ESnet SDN
ESnet SDN
WDM Layer
WDM Layer
OSPF-TE
Topology
OSPF-TE
Topology
OSPF-TE
Topology
OSPF-TE
TopologyTopology
OSPF-TE
Service-Oriented MX-TCEService-Oriented MX-TCE
The core capability of the project is Multi-X Topology Computation Element (MX-TCE)
• Handling multi-dimensional information and constraints that are typically not considered in current network PCE engines
• Computing both paths and topologies for multi-layer networks
• Designed to integrate into OSCARSv6 and provide basic PCE services
• Also designed to be used as advanced Topology Computation engine for other applications and purposes
Network Service RequesterNetwork Service Requester
Network Provider AgentNetwork Provider Agent
Physical TopologyPhysical Topology
Abstracted TopologyAbstracted Topology
Service Topology (in green)Service Topology (in green)
Service TopologyService TopologyRequested TopologyRequested Topology
Abstraction Process
Path Computation
Network Services Interface
ARCHSTONE New Features/Objectives
• advanced Network Service Interface• request- and service- topology based
• calculate whole topologies (not just circuits)• service and user specific topology views• detailed explicit route objects containing all required information to
initiate and complete a multi-layer, multi-technology provisioning event
• pseudo-provisioning operations where the impacts of various potential provisioning scenarios can be evaluated.
• accept complex questions and return complex answers regarding service and topology provisioning
• operate in multi-layer, multi-technology, multi-vendor environments• maintains topology on all Layers (Layers 0-3)
ARCHSTONE New Features/Objectives
• integrate multi-dimensional information from the control, management, security, business, and administrative planes into the single computational space• utilize advanced algorithms within a complex computational space
• designed with large scalability and speed in mind
• abstracted data representations with vendor specific interface modules
• provide "services" for other “value-added” systems in science and network application spaces
Multi-Layer Networking – ARCHSTONE Role
• "Multi-Layer Networking, An Architecture Framework"• http://hybrid.east.isi.edu/twiki/pub/HybridMLN/Pubs/multi-layer-
architecture-v9.0.pdf• Capability Planes
• The CapabilityPlanes covered by this work include the following:• Control Plane requirements, design, and implementation• Service Plane Interface• Control Plane processing of information from the Service Plane, AA Plane,
Management Plane, Service Plane, and Data Plane.
DataPlane
ServicePlane
Management Plane
ControlPlane
AAPlane
ApplicationPlane
Generic DataPlane Layer
ControlPlane
AAPlane
ManagementPlane
ServicePlane
ApplicationPlane
Network Service Interface (NSI) Requirements
OGF Standards Status and ARCHSTONE Contributions
OSCARS version 6 Overview
Chin Guok (ESnet)
Requirements for Flexible Network Services
Network architectures and services - Increasing Complexity• Growing need to consider many more dimensions (or constraints) for
control and provision of network resources
• User services becoming more complex and varied - virtual organization specific resource control, on-demand creation of network topologies across multiple layers, extended user-to-network "conversations"
Essentials for multi-layer network control• Next-generation networks tend to be architected as a heterogeneous
“multi-layer, multi-technology” construct
• Multiple types of services simultaneously over common infrastructures
• Need to hide network details from users, i.e., “services virtualized”
Where are we now?• Initial renditions of user requested service provision paradigms have
been realized in DOE ESnet SDN, Internet2 DCN, and others.
• Challenging issues unresolved: rich service interface definition, dynamic topology computation, multi-layer control etc.
Atomic and Composite Network Services Architecture
Atomic Service (AS1)
Atomic Service (AS2)
Atomic Service (AS3)
Atomic Service (AS4)
Composite Service (S2 = AS1 + AS2)
Composite Service (S3 = AS3 + AS4)
Composite Service (S1 = S2 + S3)
Ser
vice
Abs
trac
tion
Incr
eas
es
Ser
vice
Abs
trac
tion
Incr
eas
es
Ser
vice
Usa
ge
Sim
plif
ies
Network Service Plane
Service templates Service templates pre-composed for pre-composed for
specific applications specific applications or customized by or customized by advanced usersadvanced users
Atomic services used Atomic services used as building blocks for as building blocks for composite servicescomposite services
Network Services Interface
Multi-Layer Network Data Plane
Examples of Atomic Services
Security Service (e.g. encryption) to ensure data integrity
Measurement Service to enable collection of usage data and performance stats
Monitoring Service to ensure proper support using SOPs for production service
Store and Forward Service to enable caching capability in the network
1+11+1
Topology Service to determine resources and orientation
Resource Computation Service* to determine possible resources based on multi-dimensional constraints
Connection Service to specify data plane connectivity
Protection Service to enable resiliency through redundancy
Restoration Service to facilitate recovery
(*MX-TCE)
Example Workflow Using Multiple Atomic ServicesNetwork Service Network Service
RequesterRequesterNetwork Service Network Service
ProviderProvider
ComputationComputation
Is there a 1GigE connection between A and Z for 2 hours within the next 24 hours?
ConnectionConnection
ConnectionConnection
MeasurementMeasurement
ComputationComputation
ProtectionProtection
Yes. That is available in the following time-windows: [1, 5], [8, 11], [19,24]. NS-ID = 1000
(SID=AZ001) Reserve 1GigE connection between A and Z at [1,3].
Reserved: GRI = es.net-A-Z-123, Service Topolgy = {A..Z}NS-ID = 1001
1+1+11
(SID=AZ001) Instantiate the Service Topology for GRI = es.net-A-Z-123.
Service Topology instantiated
(SID=AZ001) Add measurement points to Service Topology for GRI = es.net-A-Z-123.
Measurement points added to Service Topology at {A..X..Y..Z}
SID = AZ001SID = AZ001
(SID=AZ001) Are there resources for 1:1 path-protection for GRI = es.net-A-Z-123.
Yes!
Confirmed.
(SID=AZ001) Apply 1:1 protection to Service Topology GRI = es.net-A-Z-123.
End of SessionEnd of SessionSID=AZ001SID=AZ001
Standards Contribution
• Primary effort focused on Open Grid Forum (OGF)• OGF Network Services Interface (NSI) WG*
- Helped define service plane as an abstracted representation of transport plane resources (i.e. service topology)
- Convinced WG to include service types within the NSI protocol to cater for composable services model
- Develop the idea of a Service Termination Point (STP) which hides the complexity of multi-layer transport termination from the user
- Co-authors of the OGF NSI Architecture Framework document (recently submitted for general comments)
- Connection Service and NSI Protocol definition is in progress with good participation
• OGF InfraStructure On-Demand (ISOD) RG*- Explore the complex workflows in integrated scheduling of network, compute,
and storage resources
* Both WG and RG are co-chaired by people involved in the ARCHSTONE project
ESnet OSCARS
• On-demand• Private• Bandwidth guarantees• Advance reservations• ESnet SDN control
plane
OSCAR 0.6 Architecture
Notification Broker• Manage Subscriptions• Forward Notifications
AuthN• Authentication
Resource Manager• Manage Reservations
• Auditing
Coordinator• Workflow Coordinator
PCE*• Constrained Path
Computations(*MX-TCE)
Topology Bridge• Topology Information
Management
IDC API• Manages External WS
Communications
Path Setup• Network Element
Interface
Lookup• Lookup service
AuthZ*• Authorization
• Costing
*Distinct Data and Control Plane Functions
Web Browser User Interface
21
Example Graph of PCE Modules And Aggregation
Aggregate Tags 3,4
Aggregate Tags 3,4
Aggregate Tags 1,2
Aggregate Tags 1,2
PCERuntime
PCE 1PCE 1
Tag 1Tag 1
PCE 3PCE 3
Tag 1Tag 1
PCE 2PCE 2
Tag 1Tag 1
PCE 4PCE 4
Tag 2Tag 2
PCE 5PCE 5
Tag 3Tag 3
PCE 6PCE 6
Tag 4Tag 4
PCE 7PCE 7
Tag 4Tag 4
User + PCE1 + PCE2 + PCE3
Constrains (Tag=1)
User + PCE1 + PCE2 + PCE3
Constrains (Tag=1)
User + PCE1 + PCE2 Constrains
(Tag=1)
User + PCE1 + PCE2 Constrains
(Tag=1)
User + PCE1 Constrains (Tag=1)
User + PCE1 Constrains (Tag=1)
User ConstrainsUser Constrains
User + PCE4 Constrains (Tag=2)
User + PCE4 Constrains (Tag=2)
User + PCE4 Constrains (Tag=2) )
User + PCE4 Constrains (Tag=2) )
User + PCE4 + PCE6 Constrains
(Tag=4)
User + PCE4 + PCE6 Constrains
(Tag=4)
User + PCE4 + PCE6 + PCE7
Constrains (Tag=4)
User + PCE4 + PCE6 + PCE7
Constrains (Tag=4)
User + PCE4 + PCE5 Constrains
(Tag=3)
User + PCE4 + PCE5 Constrains
(Tag=3)
User ConstrainsUser Constrains
NB: Constraints = Network Element Topology Data NB: Constraints = Network Element Topology Data
Intersection of [Constrains (Tag=3)] and [Constraints
(Tag=4)] returned as Constraints (Tag =2)
Intersection of [Constrains (Tag=3)] and [Constraints
(Tag=4)] returned as Constraints (Tag =2)
ARCHSTONEMX-TCE
MX-TCE – Component Architecture
MX-TCE
PolicyManager
ResvManager
TEDBManager
TEWGTEWG
ComputationThread
TCE Client
Request Reply
ComplexPolicyEvaluator
ResourceDeltaModifier
PolicyPruningModifier
push
pull
ExternalPolicyServer
ExternalScheduler
push
pull
TEDBExternal Storage
WS, API, OSPF, SQL, XML(pull or push)
ReservationMaker
Focus is on speed and flexible incorporation of constraints into the computation process
MX-TCEMulti-Dimensional Topology Computation Engine
• TEDB Manager - manages the Traffic Engineering Database (TEDB)
• Policy Manager - manages policy data that are locally configured with the MX-TCE as well as from the rules imported from external policy server(s)
• Reservation Manager - manages reservation data that come from both the local reservations made with the MX-TCE and the reservations made with external scheduler(s)
• Computation Thread - implements path and topology computation algorithms that process the working graph and associated constraints and provide answers to questions asked by the TCE clients
For Details: Multi-X Topology Computation Element (MX-TCE) Architecture and Design
MX-TCE – Path and Topology Computation Work Flow
For Details: Multi-X Topology Computation Element (MX-TCE) Architecture and Design
• Path Computation• Constrained Search• Graph Transformation• Heuristic Search
• Topology Computation• Bridged star/tree• Meshed Multi-Connection
• Constraints• Prunable Constraints• Additive Constraints• Non-additive Constraints• Adaptation Constraints
MX-TCE – Flexible Use Cases within OSCARSv6 and by other Applications
All in One Computation
Intra-Domain Focused Computation
Topology Data Focused Computation Feeds Results to an Aggregate PCE
MX-TCEAll-in-one Constrained Path
Computation
CoordinatorWorkflow Coordinator
Topology BridgeTopology Information
Management
AuthZAuthorization Policy
Management
Resource ManagerManage Reservations and
Scheduling
Aggregate PCEAggregating Constrained Path
Computation Results
CoordinatorWorkflow Coordinator
Topology BridgeTopology Information
Management
AuthZAuthorization Policy
Management
Resource ManagerManage Reservations and
Scheduling
MX-TCETopology TE Constrained
Path Computation
PCE-1Policy Constrained
Computation
PCE-2Scheduling Constrained
Computation
PCE-NOther Constraints
MX-TCEIntra-domain Constrained Path
Computation
CoordinatorWorkflow Coordinator
Topology BridgeTopology Information
Management
AuthZAuthorization Policy
Management
Resource ManagerManage Reservations and
Scheduling
PCEInter-domain Constrained
Path Compuation
MX-TCE can also be used by other Applications
Standalone MX-TCE
NS
I A
PI
NS
I A
PI
NS
I A
PI
NS
I A
PI
Client Application
What topologies are available next week for 3 hours?
Here's Two
ARCHSTONEProject Documents
• ARCHSTONE AAA Information Model
• ARCHSTONE Multi-Dimension TEDB Adaptations Requirements
• ARCHSTONE Multi-Layer/Multi-Technology Provisioning Requirements
• ARCHSTONE Network Service Interface Requirements
• ARCHSTONE Standards Contributions
• ARCHSTONE Multi-Layer Networking Standards and Topology Definition Study
• ARCHSTONE Multi-Layer Networking Path and Topology Computation Study
• ARCHSTONE Multi-X Topology Computation Element (MX-TCE) Architecture and Design
• ARCHSTONE Multi-X Traffic Engineering Database (MX-TEDB) Architecture and Design
• ARCHSTONE Network Service Interface (NSI) Architecture and Design
http://archstone.east.isi.edu
ARCHSTONEProject Status and Schedule
• Completed first year activities
• Design and Architecture of Network Service Interface (NSI) and Multi-Dimensional Topology Computation Element (MX-TCE)
- Message formats and schema
- Topology Schemas (request topologies, service topologies, multi-layer/point/constraint)
- Software module architectures and designs
• Deployed software development and test environment on the ESnet Testbed
- OSCARS version 6 based
• Began additional detailed design and implementation
• Plans for Year 2
• Complete initial implementations of NSI and MX-TCE
• Deploy and test in Testbed
• Deploy and test on actual networks in a test and demonstration mode
Multi-Layer Network Topology Standards and Protocol Review
• Multi-Layer Network Topology Standards Study Objectives• Review state of standards bodies and industry groups in the area
of Multi-Layer Network topology descriptions and associated protocols
• The purpose of this activity was to review and understand these industry standards, as it applies to the ARCHSTONE advanced services: • and then be able to leverage this work to the maximum extent
possible
Multi-Layer Standards Overview
Internet Engineering Task Force Optical Internetworking Forum International Telecom Union
Open Grid Forum DICE (Dante-Internet2-CANARIE-Esnet)
Multi-Layer Network Topology Standards Review
• Internet Engineering Task Force (IETF) Standard• Open Shortest-Path First-Traffic Engineering (OSPF-TE)• Border Gateway Protocol (BGP)• Resource Reservation Protocol-Traffic Engineering (RSVP-TE)• Constraint-Based Routing Label Distribution Protocol (CR-LDP)• Path Computation Element (PCE)• Psuedowire Working Group (PW3E)
• Optical Internetworking Forum (OIF) Standards• User Network Interface (UNI) 1.0• User Network Interface (UNI) 2.0• External Network-Network Interface (E-NNI) 1.0 Signaling• External Network-Network Interface (E-NNI) 2.0 Signaling• External Network-Network Interface (E-NNI) OSPF-Based Routing 1.0
• International Telecommunications Union (ITU-T) Standards
• Optical Grid Forum (OGF) - Network Description Language (NDL)
Standard Bodies InteractionsStandard Bodies Interactions
OIF Networking WG’sOIF Networking WG’sUNI, NNI specificationsUNI, NNI specificationsOIF Networking WG’sOIF Networking WG’s
UNI, NNI specificationsUNI, NNI specificationsITU-T SG-15, SG-13 WGArchitectures, L1 VPN
ITU-T SG-15, SG-13 WGArchitectures, L1 VPN
IETF WG’sArchitectures, protocols,
L1 VPN
IETF WG’sArchitectures, protocols,
L1 VPN
Multi-Layer / Multi-Domain Activities
Liason ActivitiesLiason Activities
Multi-Layer Standards and Technology ReviewSummary
• ARCHSTONE work will leverage the industry standards• particularly in the area of the information model for topology descriptions• will also leverage the built in vendor use of standards based protocols (or
vendor proprietary versions)
• However, the R&E community needs to go further (and faster) then what is occurring in the standards bodies and associated vendor implementations. In particular, the standards bodies: • are not working on multi-layer, multi-constraint path computation details• have completed very little work on application of an Authentication,
Authorization, Accounting (AAA) model to the control plane• have completed very little work on scheduling of provisioned services• have not converged on Inter-AS interdomain E-NNI routing or signaling
protocols• are not addressing scalability and security to the degree required for the
R&E community
Grid-Networking & E-Science Community
• Open Grid Forum (OGF)• Network markup language (NML) WG to define hybrid layer networks
• Network descriptor language (NDL) for multi-layer network definition:- Use RDF web-interchange standard, define an ontology- 5 schemas: Topology, layer, capability, domain, physical
• Network Measurement Working Group (NMWG) Format- XML Based, utilized by PerfSONAR and OSCARS currently
• Dante-Internet2-CANARIE–ESnet (DICE) • Inter-domain controller protocol (IDCP) implementation:
- Domain-level controller entities defined (akin to PCE)- Signaling reservations, resource discovery, topology exchange, etc
• Based on NMWG for topology descriptions
• ARCHSTONE will utilize NMWG based framework with extensions as required for the advanced services
ARCHSTONE - Use of ESnet Testbed
• Infinera has multiple capabilities that ARCHSTONE can leverage to enable advanced service provisioning:
• Protection/Restoration: Includes Dual-TAM Digital SNC Protection (D-SNCP), Single TAM D-SNCP, Unprotected Restorable
• Circuit Monitoring: Capabilities to monitor health of circuit via TL1, possibly other methods as well
• Unidirectional Digital Multicast: Point-to-Multipoint service allows for native broadcast services
• Topology Retrieval: Complete real-time topology can be retrieved with single TL1 command. Will also explore possible extensions to enable peering with routing protocols
• Signaling: GMPLS UNI which may support limited services (probably no explicit route specification), Network Management System API, and TL1
• For each of these we will explore how to integrate them into advanced ARCHSTONE services
ARCHSTONE - Use of ESnet Testbed
• ARCHSTONE is interested in the multi-layer provisioning opportunities on the testbed
• Routers (Junipers) over WDM (Infinera)
• Main focus has been Infinera Equipment familiarization/evaluation and set up of software development/experiment/test environment
• ARCHSTONE is designed to integrate vendor capabilities into the service model
• ARCHSTONE NSI Protection Atomic Service may be realized via one of the native Infinera protection mechanisms
• An ARCHSTONE NSI "Service Topology" may utilize the native Infinera point-to-multipoint broadcast capability in response to a "Request Topology" with this capability requested
• The objective here is not to be dependent on a specific vendor feature set, but to show how unique vendor capabilities can be integrated into the ARCHSTONE service model
• OSCARSv6 has been deployed on the testbed as a development environment