1 testbeds breakout tom anderson jeff chase doug comer brett fleisch frans kaashoek jay lepreau hank...

27
1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica Joe Touch Amin Vahdat

Upload: randell-jacobs

Post on 04-Jan-2016

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

1

Testbeds Breakout

• Tom Anderson• Jeff Chase• Doug Comer• Brett Fleisch• Frans Kaashoek• Jay Lepreau• Hank Levy

• Larry Peterson• Mothy Roscoe• Mehul Shah• Ion Stoica• Joe Touch• Amin Vahdat

Page 2: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

2

GENI Requirements

• Virtualizable – So users can share infrastructure

• Programmable – So users can provide arbitrary functionality

• Supports painless user opt-in and opt-out – So we can get real workload

• Federation – So new devices, clusters, edge networks can be plugged

in

• Software development support– So we can make our stuff real and available to each other– So we can build on each other's work; this includes

(especially) management software

Page 3: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

3

Questions

• What do we need?

• How can we contribute to substrate?

• What basic services can we provide?

Page 4: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

4

What do we need? (1)

• Significant storage and computation infrastructure make it possible to deploy Google and Yahoo like services 20-30 clusters > 256 node per cluster > 256 TB per site

• Many smaller clusters with heterogeneous connectivity make it possible to deploy Akamai like services

• Others: 1000s of hosts, sensor nodes, mobile devices, embedded devices

Page 5: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

5

What do we need? (2)

• Allow users to easily opt-in and opt-out with their resources to/from the testbed

• Enable testbed to organically grow to include– Wireless networks– Sensor networks – Community Networks– …

Page 6: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

6

How can we contribute to substrate? (1)

• Provide a “virtual network system” abstraction:– Virtualize all resources: CPU. Memory,

storage, network– Virtualization within constraints (e.g., 20 ms

delay, 2 Mbps links)

• Challenge: Map virtual system networks onto physical resources while meeting time and resource constraints

Page 7: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

7

How can we contribute to substrate? (2)

• Resource management & allocation – How to allocate resources (virtual network

systems) when testbed is oversubscribed?

• Challenge: Develop flexible policies and mechanisms– E.g., reservation in both time and space,

market-based allocation, …

Page 8: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

8

How can we contribute to substrate? (3)

• Support for auditing, debugging– How to discover users with malicious intend,

misconfigurations, bugs?

• Challenges:– Efficient and scalable infrastructure that at

limit would allow all nodes to log all messages, virtual machine checkpoints, etc

– Extensible monitoring infrastructure; provide hooks for users to add their own monitoring or logging code

Page 9: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

9

What services can we provide? (1)

• PKI infrastructure• Certification authority• Auditing services• Name server (DNS++)• Resource location and discovery

Page 10: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

10

What service can we provide? (2)

• Citeseer• Source forge• Usenet news• arXiv.org• Conference submission • Fastlane• Data distribution service• Spam filters• Distributed firewalls• Open search engine (Open Google?)

Page 11: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

11

Page 12: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

12

Goals

• Flexibility/Control

• Isolation

• Realism

• Fairness

• Security

• Support for tracing, replaying

Page 13: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

13

What should a Testbed Include?

• PlanetLab++– Large number of node (1000s), heterogeneous

connectivity

• Optical networks

• Sensor nodes

• Mobile hosts (PDAs, Phones, etc)

• Data centers (Google, Yahoo, part of the Internet fabric)

Page 14: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

14

• Soft-radios• Four classes of wireless• All things for all people is difficult• Configurable testbeds• Heterogeneous separate testbed• What’s it at this site?

– Storage to do management

• Contribute with software, maintain and support

• Operational and manage this

Page 15: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

15

What we need?

• Sensornodes

• Open environment– Organically evolve testbeds

• Distribution, heterogeneity, scale

Page 16: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

16

What else we need (Software)?

• Databases

Page 17: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

17

How can we contribute?

• Management?

Page 18: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

18

Flexibility

• Need to be have complete control on infrastructure node– Run various OSes– Port numbers– Real-time – Root privileges

Page 19: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

19

Isolation

• One user shouldn’t be able to interfere with the experiments of other users

• At multiple levels– CPU– Memory – Disk– Bandwidth (both outgoing and ingoing)

Page 20: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

20

Realism

• Real users, real applications

• Negotiate with ISPs to send traffic across testbed– How to guarantee that ISPs traffic won’t be

screwed

• Recreate catastrophic failures, attacks

Page 21: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

21

Security

• Prevent using testbed to initiate attacks– Malicious users– Misconfigurations

• Challenge: minimal impact on flexibility, performance

Page 22: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

22

Management

• How to allocate resources to users in a fair and easy to understand (predictable?) way

• Flexible polices and mechanisms– Reservation in both time and space– Biding, trading resources – Economic-based allocation

Page 23: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

23

Support for tracing, replaying

• Ideally, log everything:– Traffic– Virtual machine checkpoints

• Enable replaying, forensic

• Hard

Page 24: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

24

• Virtualized testbeds– Network and edge devices network– Virtual machine and virtual network– Virtualization within constraints (20ms)

• Abstract away heterogeneous software

• Specify requirements map on real resources

Page 25: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

25

• Auditing/logging

• Flexible monitoring

• Secure hooks for monitoring

Page 26: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

26

Extensible testbeds

• Flexible routing infrastructure

• Integrate everything

• Community networks

• Useful control system

Page 27: 1 Testbeds Breakout Tom Anderson Jeff Chase Doug Comer Brett Fleisch Frans Kaashoek Jay Lepreau Hank Levy Larry Peterson Mothy Roscoe Mehul Shah Ion Stoica

27

• Resource allocation• Model for incentives• Incentives to X add resources

• PKI infrastructure• Certified authority• Auditing services• Name servers• Resource location and discovery