Sponsored by the National Science Foundation 1April 8, 2014,

Testbeds as a Service: GENI

Heidi Picher DempseyInternet2 Annual Meeting

April 8, 2014 www.geni.net

Sponsored by the National Science Foundation 2April 8, 2014,

GENI provides a virtual lab for networking and distributed systems research and education

• GENI started with exploratory, rapid prototyping 5 years ago• GENI design assumes federation of autonomously owned and operated systems• Yearly prototyping cycle for an idea: develop, integrate and operate• Experimenters use the testbed while we are building it out• Even prototypes have “activist” users, and must evolve to satisfy those users or fade

away. Two of five original design frameworks predominate now.• “Horizontal” dataplane slicing as a service (or sometimes just engineered)• “Vertical” control plane APIs to negotiate and allocate resources

Fundsin handNeedsfunding

Sponsored by the National Science Foundation 3April 8, 2014,

GENI: Infrastructure for Experimentation

GENI provides compute, network, and wireless resources that can be connected in experimenter-specified Layer 2 topologies.

Sponsored by the National Science Foundation 4April 8, 2014,

GENI: Infrastructure for Experimentation

GENI provides compute resources that can be connected in experimenter specified Layer 2 topologies.

Sponsored by the National Science Foundation 5April 8, 2014,

Multiple GENI Experiments run Concurrently

Resources can be shared

between slices

Experiments live in isolated “slices”

Sponsored by the National Science Foundation 6April 8, 2014,

GENI is “Deeply Programmable”

I install software I want throughout my network slice (into routers, switches, …) or control

switches using OpenFlow

Experimenters can set up custom topologies, protocols and switching of flows

Sponsored by the National Science Foundation 7April 8, 2014,

United States GENI Resources

• Flexible network / cloud research infrastructure

• Over 1800 users (March, 2014)• 52 racks installed/in progress• 10 WiMAX (50 LTE planned)

• VMs, bare metal nodes, SDN switches (OpenFlow 1.0), software routing or OVS OpenFlow, WiMAX/LTE base stations and clients all available to experimenters.

MetroResearch

Backbones

InternetISPU N I V E R S I T YU N I V E R S I T Y

U N I V E R S I T YU N I V E R S I T Y

Regional Networks Campus

g

g

gLegend

GENI-enabled hardware

Layer 3Control Plane

Layer 2Data Plane

Sponsored by the National Science Foundation 8April 8, 2014,

International Federation Activities

• International Federation API for clearinghouse functions.

• Supported for multiple clearinghouses.

• Joint GENI/FIRE (US/EU) capability demonstrated this year (not operating yet)

• GENI dataplane slices extend to other research networks by special arrangement now (Japan, Korea, Australia)

• Shared monitoring prototype planned for 2014

• Investigating/prototyping standards for experimenter-driven dataplane resource negotiation and provisioning

• NSI, OSCARS• GENI Stitching• SDN Exchanges

Sponsored by the National Science Foundation 9April 8, 2014,

Campus Requirements for Shared Services

• Sites provide space, power, security (as with other campus IT resources)• Provide at least 1Gbps OpenFlow/SDN path from rack to campus boundary *• Provide connection from rack to on-campus resources (varies by campus, usually SDN)*• Operate with up-to-date GENI-specified software (e.g. AM API, OpenStack) • Provide no-cost access to rack resources for GENI authorized users at other campuses• Provide points of contact for GENI response team (see

http://groups.geni.net/geni/attachment/wiki/ComprehensiveSecurityPgm/Aggregate Provider Agreement v3.pdf ) * * No STP or MAC learning

Sponsored by the National Science Foundation 10April 8, 2014,

Process Requirements for Shared Services

• Standard installation processeshttp://groups.geni.net/geni/wiki/GENIRacksHome/RacksChecklistStatus

• System Acceptance Testing• Production: InstaGENI, ExoGENI • Provisional: Dell (OpenGENI), Cisco

• Shared site resource and access detailshttp://groups.geni.net/geni/wiki/GeniAggregate

Sponsored by the National Science Foundation 11April 8, 2014,

Process Requirements (cont.)

• Site confirmation tests with logs and RSPECshttp://groups.geni.net/geni/wiki/GENIRacksHome/InstageniRacks/ConfirmationTestStatus

http://groups.geni.net/geni/wiki/GENIRacksHome/ExogeniRacks/ConfirmationTestStatus

• Emergency Stop and Legal, Law Enforcement and Regulatory Event Coordination (GMOC at Indiana University)

• Shared monitoring infrastructure and shared operations (6 major ops groups)

Sponsored by the National Science Foundation 12April 8, 2014,

University of Kentucky Ops Monitoring

Dashboard of Collector

Sponsored by the National Science Foundation 13April 8, 2014,

Network Engineering Requirements for Shared Services

• L2 dataplane engineering– campuses, regional, core and

international networks– many vendors and technologies – 1-100GBE interfaces (GENI shares

with other R&E projects)– Shared or exclusive experimenter

VLANs on interfaces depending on experiment (mostly exclusive)

• SDN (OpenFlow 1.0) switches with experimenter’s and sometimes R&E network’s controllers (many vendors, varying implementation of standards)

• Standard Internet control plane • Internet2 AL2S cross-connects and ION

http://groups.geni.net/geni/wiki/GENIOESSTopologiesPerformance - IONtoAL2SPerformance

Sponsored by the National Science Foundation 14April 8, 2014,

Network Engineering (continued)

• GENI dynamic stitching available at 12 racks via Internet2 IONhttp://groups.geni.net/geni/wiki/GeniNetworkStitchingSites

• Over 16,000 stitched slivers since November, 2013

• OESS GENI Stitching Aggregate coming soon

• Stitching operations monitoring prototyping with MAX, Internet2

Sponsored by the National Science Foundation 15April 8, 2014,

Slice credentials

Software: Clearinghouse and Aggregates

• Clearinghouse: manages users, projects and slices– Standard credentials shared via custom API or new Common CH API– GENI supported accounts: GENI Portal/CH, PlanetLab CH, ProtoGENI CH

• Aggregate: provides resources to GENI experimenters– Typically owned and managed by an organization– Speaks the GENI Aggregate Manager API (AM API)– http://groups.geni.net/geni/wiki/GAPI_AM_API_V3 most recent version– http://trac.gpolab.bbn.com/gcf download reference implementation (gcf), OMNI command line client– Examples: PlanetLab, Emulab, GENI racks on various campuses

Create & Register Slice

Researcher

Aggregate Manager API - listResources - createSliver … Aggregate

ManagerAggregate Resources

users

slices

clearinghouse

projects

Tool

Sponsored by the National Science Foundation 16April 8, 2014,

GENI Slivers

• Sliver: One or more resources provided by an aggregate– e.g. Bare machines, virtual machines, VLANs

Backbone #1

Backbone #2

Campus#3

Campus#2

Access#1

CommercialClouds

CorporateGENI suites

Other-NationProjects

ResearchTestbed

Campus My GENI Slice

My slice contains slivers from many aggregates.

Sponsored by the National Science Foundation 17April 8, 2014,

Software: RSpecs

• RSpecs: Lingua franca for describing and requesting resources– “Machine language” for negotiating resources between experiment

and aggregate– Experimenter tools eliminate the need for most experimenters to

write or read Rspec

<?xml version="1.0" encoding="UTF-8"?><rspec xmlns="http://www.protogeni.net/resources/rspec/2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.protogeni.net/resources/rspec/2 http://www.protogeni.net/resources/rspec/2/request.xsd" type="request" > <node client_id="my-node" exclusive="true"> <sliver_type name="raw-pc" /> </node></rspec> RSpec for requesting a single node

Sponsored by the National Science Foundation 18April 8, 2014,

Reserving Resources using RSpecs and the AM API

• Experimenter tools and aggregates talk to each other using resource specifications (RSpecs) and the GENI Aggregate Manager API (GENI AM API)

• Advertisement RSpec: What does an aggregate have?• Request RSpec: What does the experimenter want?• Manifest RSpec: What does the experimenter have?

AggregateManager

Exp

erim

ente

rTo

ol

ListResources(…)

Advertisement RSpec

CreateSliver(Request RSpec, …)

Manifest RSpec

ListResources(SliceName, …)

Manifest RSpec

What do you have?

I have …

I would like …

You have …

What do I have?

You have …

Sponsored by the National Science Foundation 19April 8, 2014,

GENI Design Activities

• Open to all http://groups.geni.net/geni/wiki/GeniDesign

• Mailing lists http://lists.geni.net/mailman/listinfo• geni-annouce@geni.net most general• Regular calls and design reviews announced

through interest group mailing lists• IRC/chat (mostly operations)http

://groups.geni.net/geni/wiki/HowTo/ConnectToGENIChatRoom

• GENI Engineering Conferences (3 per year)• June 21-24, 2014 University of California, Davis

(Travel grants available)