TeraPathsTeraPaths: establishing end-to-end QoS paths - the user perspective

Presented byPresented by

Dimitrios Katramatos, BNLDimitrios Katramatos, BNL

2

Outline

What is TeraPaths?What is TeraPaths? Motivation Concept and implementation View of the world

How can it be used?How can it be used? Admins:

Installation and configuration Security model

Users: Web interface API/CLI Integration

Status/futureStatus/future

3

Motivation

The problem: The problem: support efficient/reliable/predictable peta-scale support efficient/reliable/predictable peta-scale

data movement in modern high-speed networksdata movement in modern high-speed networks Capacity is not limitless

Multiple data flows with varying priority

Default “best effort” network behavior can cause performance and

service disruption problems

Solution:Solution: enhance network functionality with QoS features to enhance network functionality with QoS features to

allow prioritization and protection of data flowsallow prioritization and protection of data flows Classify traffic

Schedule network usage

4

TheTeraPathsTeraPaths Service: Reserve End-to-End Paths with Guaranteed Bandwidth

WAN

WAN web services

TeraPaths

TeraPaths

5

Prioritized vs. Best Effort Traffic

Network QoS with Three Classes: Best Effort, Class 4, and EF

0

200

400

600

800

1000

1200

0 100 200 300 400 500 600 700 800 900 1000

Time (Seconds)

Utilized Bandwidth (Mbit/second)

Best Effort Class 4 Expedite Forward TOTAL Wire Speed

6

How?

Make arrangements at end (TeraPaths-controlled) sitesMake arrangements at end (TeraPaths-controlled) sites Check local site availability and reserve temporarily Check remote site availability and reserve temporarily Pick alternative time slot if requested slot taken and iterate

Make arrangements with WANMake arrangements with WAN Request MPLS tunnel or dynamic circuit with matching bandwidth and

duration

Confirm and commitConfirm and commit …if all parties agree… …otherwise fail

Timer activated tasks configure network devicesTimer activated tasks configure network devices Verify status of all reservations and run configuration tasks (WAN services

do their part of the setup) Fail otherwise

7

Conceptual View of the Network

TeraPaths

TeraPaths

TeraPaths

TeraPaths

Site A

Site B

Site C

Site D

WAN 1

WAN 2

WAN 3

service invocation

data flow

peering

WAN chain

8

TeraPaths Testbed ( )

current

9

Administration

Software installation Software installation Web service modules on SJSAS

JVM, MySQL

Software configuration Software configuration Server security

Module locations

Database

Site router configurationSite router configuration Classes of service, policing, PBR

DSCP trust

VLANs (for L2 support – dynamic WAN circuits)

10

TeraPathsTeraPaths Web Services Architecture

Internal Services

Public Services

Web Interface

Admin Module

NDCNDCNDC • • •

Database

protected network

API

remote

local

WAN Services

• • •

WAN Services

• •

•

proxy

proxy

• •

•

CLI s/w client

11

Security Model

Current: “circle of trust”Current: “circle of trust” Client-side and server-side authentication

Routers get configured through ssh from specific controlling hosts (preferably physically connected)

Servers “talk” only to known servers (certificates) Hosts only “talk” to known hosts (firewalls) Servers only accept users with known certificates (DOE grid)

Local user login User needs at least one local account to enter

In the works: grid-style In the works: grid-style GUMS/VOMS

12

Site Setup

13

Necessary Info

User data for AAAUser data for AAA

Data flow IDData flow ID Source IP and port Destination IP and port IPs and ports can be ranges (multiple flows) Direction (unidirectional/bidirectional) Protocol

Bandwidth (class of service)Bandwidth (class of service) Multiple flows will share (best effort within the class)

Start time and durationStart time and duration Minute resolution

14

Web Interface (i)

15

Web Interface (ii)

16

Web Interface (iii)

17

Web Interface (iv)

18

Web Interface (v)

19

Web Interface (iii-b)

20

Web Interface (vi)

21

API/CLI and Integration

Web Interface uses APIWeb Interface uses API

CLI uses APICLI uses API

Users can invoke CLI/API from their scripts/applicationsUsers can invoke CLI/API from their scripts/applications

Popular data transfer software plug-ins (e.g. dCache)Popular data transfer software plug-ins (e.g. dCache) Life is easier because transfer tool knows flow id data

API will be extended for use by other network path-building API will be extended for use by other network path-building

and scheduling services (e.g., services being developed in and scheduling services (e.g., services being developed in

US LHCNet, UltraLight/PLaNetS)US LHCNet, UltraLight/PLaNetS)

22

Status and Future

Currently: basic software readyCurrently: basic software ready API and web interface, simple negotiation Statically allocated bandwidth classes L3 paths (MPLS tunnels) through ESnet Elementary AAA BNL UMich

In the works, futureIn the works, future CLI, extended API, configurable negotiation Dynamic bandwidth allocation L2 paths (dynamic circuits) through ESnet and Internet2 Grid-style AAA Admin module to facilitate site setup Expansion to T2 sites

http://www.racf.bnl.gov/terapathshttp://www.racf.bnl.gov/terapaths