sponsored by the national science foundation geni and cloud computing niky riga geni project office...

Sponsored by the National Science Foundation

GENI and Cloud Computing

Niky RIga

GENI Project Office

[email protected]

Sponsored by the National Science Foundation 2IC2E‘14 – 10 March 2014 www.geni.net

Cloud, HPC, and Distributed Computing

• Why Cloud / Distributed Computing– Performance– Locality– Dynamic resource allocation– Reliability

Slide by Paul Ruth

Image by: http://www.iec.ch/etech/2012/etech_0512/pic_tech/tech-2_network_lrg.jpg


GENI Ricks and SDN

SDN and Clouds

GENI and other Cloud Providers

International Collaborations

GENI and Cloud Research



• Combine multiple resources– Compute, Network, Storage

• Need to distribute tasks– PBS/Torque, MPI, HTCondor, Hadoop, Slirm – Tightly or loosely coupled

• Cloud (GENI)– Allocate compute, network, and storage– GENI can allocate WAN resources

Slide by Paul Ruth


Cloud Providers

Observatory

Wind tunnel

Workflow

Slide by Paul Ruth



GENI Racks as Clouds

• A GENI Rack– Raw PCs– 100s of VMs– Separates control and data

planes– Flexibility on host OSes– Hosts isolated experiments

Racks can act as cloud providers, or to deploy clouds


GENI Racks

InstaGENI ExoGENI *

Nodes per rack 5 10workers + 1head

Cores per rack 60 120

Network interfaces 4x 1Gbit 2x 10Gbit with SR-IOV

Storage 1 TB local 150GB+500 GB local + 6 TB SAN

Switches HP ProCurve 5406 (Vlan-based OpenFlow)

IBM G8264R (Port-based OpenFlow)

* Listed are the specs for ExoGENI IBM-based racks.

InstaGENI: Less powerful more placesExoGENI: More powerful fewer placesEach rack has two raw PCs


High Throughput Computational Genomics

MotifNetwork– Jeffery L. Tilson (RENCI)

– Identification of functional

domains and the identification of

conserved functional

relationships across large

numbers of genomes.

– Workflow ensemble

(Pegasus/HTCondor)

– Task data sets on the order of

50-100 GB.

– Implicit iteration

– Scales to 1000s of tasksSlide by Paul Ruth


Urgent Computing

ADCIRC– Brian Blanton (RENCI)

– Storm surge and tide

model

– Finite element model

– MPI tightly coupled

– Approved by FEMA for

computing storm surge

flood hazard simulations

– Used for Digital Flood

Insurance Rate Maps

(DFIRMs)

– Scales to 10000+ MPI

processesSlide by Paul Ruth



SDN and Clouds



SDN and Clouds


Cloud Research and Networking

Cloud deployments rise new network challenges– Within one Datacenter– In distributed Datacenters

Google’s SDN WAN

SDN enables easy, customizable network innovation


GENI OpenFlow Deployment

OpenFlow-enabled hardware switch at:– Each GENI Rack– Backbone and regional networks (between racks)


GENI Experiment: Virtual Desktop Clouds


Prasad Calyam,U. of Missouri

Program realtime load-balancing functionality

deep into the network to improve QoE

GENI Experiment: Virtual Desktop Clouds



SDN and Clouds

GENI and other Testbeds




Interconnecting

grid

internet


Interoperate

internet

e.g. LabWiki

GENI AM API

GENI Tools


NowCast System

Slide by Mike Zink, UMass Amherst

Short-term weather prediction (1-15 mins)

Forecasts as we know them:• Data from many

sensors: Radar, satellite, balloons,

• Usually for large • regions• Takes super

computers to calculate


Multi-radar NetCDF Data

Nowcast Processing

1. Spin up system in Amazon commercial EC2 and S3 services on demand

“raw” live data

Generate “raw” live dataViSE/CASA radar nodes

http://stb.ece.uprm.edu/current.jsp

ViSE views steerable radars as shared, virtualized resourceshttp://geni.cs.umass.edu/vise

Nowcast images for display

Weather NowCastingUniversity of Massachusetts

David Irwin et al

Create and run realtime “weather service on demand”as storms turn life-threatening



SDN and Clouds


International Collaborations International Collaborations


International Collaboration

Modified slide from: http://groups.geni.net/geni/attachment/wiki/GEC18Agenda/MonPlenary/

GEC18_brecht_vermeulen_International_Federation.pdf

Efforts to provide L2, highspeed, SDN network between International testbeds


GENI Experiments: TransGEO

The Intercloud will be about the seamless movement of computation

Design and development of protocols, security procedures, architectures,

economic models that permit computation to move to data, rather than the other way around


GENI Experiments: TransGEO

Compute how “Green” a City using satellite images

Transcontinental Federation of Cloud Systems with Private 10Gb/s transcontinental network linking sites

Perform distributed query on TransCloud


QUESTIONS?

sponsored by the national science foundation geni and cloud computing niky riga geni project office...

Documents

geni racks

clouds geni

cloud researchsponsored

paul ruthsponsored

cloud research sdn

sdn sdn

paul ruthcloud

mpi processes slide