bonfire: features, sites and tools
DESCRIPTION
Slides for a general webinar about BonFIRE, the features offered, the sites making up this multi-site testbed and the tools available for experimenters using the facility. A video with audio is available on YouTube: http://youtu.be/0ulgvs32wvITRANSCRIPT
Building service testbeds on FIRE
BonFIRE Webinar
29th May 2013
Delivered by Kostas Kavoussanakis, EPCC
BonFIRE2 2
Overview
• Basic experiment lifecycle
• BonFIRE key functionalities
• Testbed sites
• Client tools
• Experiment support mechanism
Building service testbeds on FIRE
BonFIRE Experiment Lifecycle
BonFIRE4 4
Prerequisites
• You must have an account in BonFIRE.– If you don’t, please go to:
• http://www.bonfire-project.eu/involved• https://portal.bonfire-project.eu/register/
• Your public key must be uploaded.– If not, you can find more information here:
• http://doc.bonfire-project.eu/R3.1/getting-started/upload-ssh-key.html
• You must have access to a SSH environment able to connect through one the BonFIRE SSH gateways– If not, you can find more information here:
• http://doc.bonfire-project.eu/R3.1/getting-started/ssh-gateway-config.html
BonFIRE5 5
1. Create the experiment
You can choose between different tools
Portal
Command Line
Experiment Manager
Restfully
CURL with OCCI
• Don’t forget to set walltime properly. After it expires, the experiment (and its resources) will be SHUTDOWN and DELETED.
BonFIRE6 6
Experiments
ResourceManager
Experiment: 356Walltime: 3600
Experiment with Walltime 3600
BonFIRE7 7
2. Set up the experiment
• Create networks, VMs and Storages
• If you want monitoring, don’t forget to add an Aggregator.– http://doc.bonfire-project.eu/R3.1/monitoring/howto.html
BonFIRE8 8
Experiments
ResourceManager
Experiment 356
Experiment 357
MyDataBlock
DataBlock
SAVE
BonFIRE9 9
Managed Experiments for initial deployment
ResourceManager
Experiment 356
Experiment name: myExperiment
compute: Name: compute1 Location: uk-eppc Disk: OS Network: BonFIRE WAN
Compute: Name: compute2 Location: fr-inria Disk: OS Network: BonFIRE WAN
ExperimentManager
ManagedExperiment 104
BonFIRE10 10
3. Configure the computes
• Log in to the computes and configure the,
• Bear in mind that you can SAVE an image and it can be reused as many times as you want in the same location.
• Images can’t be moved between locations, think about scripts for repeating the same commands.
BonFIRE11 11
Creating and configuring VMs
OS
OSMySQL
OS
OS
> > apt_get mysql
OSMySQL
OSMySQL
>
OS
> apt_get mysql
OSMySQL
OSMySQL
SAVE
SAVE
BonFIRE12 12
Contextualisation, elasticity and connectivity
<context> <role>server</role> …</context>
<context> <role>client</role> <serverIP> 192.168.2.2 </serverIP> …</context>
<context> <role>client</role> <serverIP> 192.168.2.2 </serverIP> …</context>
<context> <role>client</role> <serverIP> 192.168.2.2 </serverIP> …</context>
BonFIREWAN
BonFIREWAN
BonFIRE13 13
Data Blocks
DataBlock
OSMySQL
OS
OS
OS OSMySQL
SAVE
MyDataBlock
BonFIRE14 14
SSH Gateways and VPN
BonFIRE WAN
JohnSmith/*****
LDAP ServerLDAP ServerVPN serverVPN server
SSH GatewaySSH Gateway
SSH GatewaySSH Gateway
SSH GatewaySSH Gateway
JohnSmith/*****
JohnSmith/*****
> ssh 192.168.4.2
BonFIRE15 15
4. Perform your experiment
• Here you decide what to do!
• If the experiment contains Virtual Wall (iMinds) resources, don’t forget to put in RUNNING state the experiment.
BonFIRE16 16
5. Monitoring
• Access the Zabbix GUI, tunneled through the Portal.
• Monitor Zabbix parameters, or configure your own.– BonFIRE offers application, VM and infrastructure monitoring from
the same API
• Use the monitoring API to make elasticity and application decisions on the fly
BonFIRE17 17
6. Wrap up your experiment
• The experiment can finish in two different ways:– Expiry time (Walltime) ends.– You have finished before walltime arrives, and you
decide to stop or delete the experiment.
• Before this time arrives, save the images and datablocks that you want to use again. Then, shutdown the VM where the image/datablock is being used.– http://doc.bonfire-project.eu/R3.1/reference/experiment-lifecycle.ht
ml
BonFIRE18 18
7. After the experiment
• Access the monitoring data that you have saved– http://doc.bonfire-project.eu/R3.1/monitoring/getting-data/export-csv
.html
• Reuse the images that you have configured and saved
Building service testbeds on FIRE
BonFIRE Key Functionalities
BonFIRE20 20
Four pillars of BonFIRE
BonFIRE21 21
BonFIRE22 22
Three layers of monitoring
Physical Machine
VirtualMachine
VirtualMachine
VirtualMachine
BonFIRE23 23
Agents and Aggregators
VM Host 1
VM Host 2
VM Host 2
VM Host 1
VMVM
VMVM
ExperimentAggregatorExperimentAggregator
Data
VM data
Application Data
Intrastructure data
VM Host 1
Agent
Agent
Site Aggregator
Site Aggregator
Site Aggregator
Site AggregatorAgent
Agent Agent
Agent
VM Host 2
BonFIRE24 24
Zabbix GUI
BonFIRE25 25
More Observability
BonFIRE26 26
BonFIRE27 27
Emulated networks using the Virtual Wall
Client 1Client 1
Client 2Client 2
Client 3Client 3
Client 4Client 4
10Mbps10Mbps 1000Mbps1000MbpsServerServer1000Mbps1000Mbps
100Mbps50ms latency3% loss rate
100Mbps50ms latency3% loss rate
100Mbps100Mbps
100Mbps50ms latency5% loss rate
100Mbps50ms latency5% loss rate
100Mbps50ms latency10% loss rate
100Mbps50ms latency10% loss rate
100Mbps50ms latency15% loss rate
100Mbps50ms latency15% loss rate
100Mbps50ms latency20% loss rate
100Mbps50ms latency20% loss rate
100MbpsTraffic: TCP
Packet size: 200 bytesThroughput: 20 packet/s
100MbpsTraffic: TCP
Packet size: 200 bytesThroughput: 20 packet/s
100MbpsTraffic: TCP
Packet size: 200 bytesThroughput: 30 packet/s
100MbpsTraffic: TCP
Packet size: 200 bytesThroughput: 30 packet/s
100MbpsTraffic: TCP
Packet size: 200 bytesThroughput: 40 packet/s
100MbpsTraffic: TCP
Packet size: 200 bytesThroughput: 40 packet/s
BonFIRE28 28
epcc
vmhost0
vmhost1
ResourceManager
Controlled placement
Location: epccInstance type: lite
OpenNebula
Location: epccInstance type: lite
Host: vmhost1Location: epcc
Instance type: large
Location: epccInstance type: large
Host: vmhost1
BonFIRE29 29
Exclusive physical machines
INRIA
node1 node2 node3
node7node6
node8 node9
OpenNebula
ResourceManager
ReservationSystem
Give me3 physical machines
from 26/4/12 10:00 to
28/4/12 20:00
Cluster: 34563P23P40p92
Location: inriaCluster: 34563
p23 p40
p92
Cluster: 34563
Location: inriaCluster: 34563
Host: p40
node5
BonFIRE30 30
Controlled Contention
VM Host 1 VM Host 2
ExclusivePhysicalMachine
VM under testVM under test
VM under testVM under test VM under testVM under test
CoCoMaCoCoMa
Memory Use
IO Use
BonFIRE31 31
BonFIRE32 32
Custom instance types
BonFIRE33 33
epcc
vmhost0
vmhost1
Custom instance type
ResourceManager
OpenNebulaLocation: epcc
Instance type: customCPU = 16
VPCU = 16Memory = 16GBHost = vmhost1
BonFIRE34 34
Elasticity as a Service (EaaS)
ElasticityManagerElasticityManager
AggregatorAggregator
Aggregator IP: 192.168.4.233Upscale: {system.cpu.usage.last(0)}>70Min: 1Max: 5Type: liteDisk: disk_nameLocation: uk-epcc Load Balancer: HAProxy, port 80, uk-epcc
LoadBalancer
LoadBalancer
VMVM
VMVM
VMVM
BonFIRE35 35
BonFIRE36 36
Ease of Use
• If it’s not easy, it won’t be used
• But what is easy?
• BonFIRE interpretations:
BonFIRE37 37
BonFIRE Portal
BonFIRE38 38
Experiment Descriptor{ "name": "myExperiment", "description": “Controlled Network", "duration": 60, "resources": [ { "network": { "name": "myNetwork", "locations": ["be-ibbt"], "address": "192.168.0.0", "size": "C", "lossrate": 0, "bandwidth": 700, "latency": 0 } }, { "compute": { "name": "server", "locations": ["be-ibbt"], "instanceType": "Large-EN", "min": 1, "resources": [ {"storage": "@iperf-demo2"}, {"network": "@BonFIRE WAN"}, {"network": "myNetwork"} ], "contexts": [] } }, { "compute": { "name": "client", "locations": ["be-ibbt"], "instanceType": "Large-EN", "min": 1, "resources": [ {"storage": "@iperf-demo2"}, {"network": "@BonFIRE WAN"}, {"network": "myNetwork"} ], "contexts": [ {"IPERF_SERVER": ["server","myNetwork"]} ] } } ]}
BonFIRE39 39
300 pages of User Doc
Building service testbeds on FIRE
BonFIRE Testbed Sites
BonFIRE41 41
Infrastructure in general
• Open Access Sites– Virtual Wall: iMinds (IBBT)– OpenNebula: EPCC / INRIA / USTUTT– HP Cells: HPLabs
• Side wide interconnects– Private: BonFIRE WAN (VPN network)– Public: GEANT
• Special BonFIRE flavours– Heterogeneous resources– Permanent and on-request infrastructure– Virtual Wall: controlled networks (emulated)– HP Cells: securely isolated cells contain storages, computes, …
BonFIRE42 42
Site Capacities
Permanent Cores Memory Storage Nodes
HPLabs 128 136 GB 5 TB 32
iMinds 400 400 GB 32 TB 100
HLRS 154 452 GB 12 TB 17
EPCC 96 256 GB 3.6 TB 2
INRIA 48 256 GB 2.4 TB 4
On-Request Cores Memory Storage Nodes
HPLabs 384 144 GB 32 TB 96
iMinds 64 64 GB 32 TB 16
HLRS
EPCC
INRIA 1672 2.7 TB 42 TB 160
BonFIRE43 43
Infrastructure Map
BonFIRE44 44
• Sites have different computing architectures
• Different varieties of VMs are described through instance types
• Sites support different instance types
Name VCPU cores Memory Features
Lite 0.5 256MB CPU may be < 1
Small 1 1GB
Medium 2 2GB
Large 2 4GB
Large+ 2 4GB Higher CPU clock speed (over 3GHz)
Large-EN 4 4GB Emulated network
Xlarge 4 8GB
XLarge+ 4 8GB Higher CPU clock speed (over 3GHz)
Custom Free Free Integer values
Which site to use?
Site Lite Small Medium Large Large+ Large-EN XLarge Xlarge+ Custom
HP + + + + +
iMinds ++
HLRS + + + + ++ + ++ +
EPCC + + + ++ ++ +
INRIA + + + +
BonFIRE45 45
Which site to use?
• Depends on your requirements
• Special characteristics– Controlled networks only available at iMinds– Public IPs are not available at iMinds and HLRS– Reservation of nodes only available at INRIA
• Reservation at Inria Exclusive access to node• https://api.bonfire-project.eu/locations/fr-inria/reservations
• Access methods– SSH & VPN– Each gateway serves for every BonFIRE WAN IP address
BonFIRE46 46
Information about site status
• BonFIRE health map indicates general availability– Nagios system tests– http://nebulosus.rus.uni-stuttgart.de/nagvis/frontend/nagvis-js/index.
php?mod=Map&act=view&show=bonfire-full
• Log files– OpenNebula status– OpenNebula Virtual machines– Experiment Manager log
• Mailing list– [email protected]
Building service testbeds on FIRE
BonFIRE Client Tools
BonFIRE48 48
Interacting with BonFIRE
Resource Manager
Site Z
Site Y
Site X
OC
CI
EnactorcURL
Experimenter
Create compute
XML
All communication via an open RESTful interface
Don’t need to write XML and use OCCI directly - we have several client tools available!
BonFIRE49 49
Client tools overview
BonFIRE Portal• Web interface to the BonFIRE API• Very simple to use and get started in BonFIRE
Experiment descriptors• Write the initial resource deployment in JSON or OVF• Create descriptor step-by-step on Portal and edit later
Restfully• General-purpose RESTful client• Ruby scripting or Ruby shell interactions with the BonFIRE API
Command line tools• Interactive, manual or scripted interaction with the BonFIRE API
BonFIRE50 50
BonFIRE Portal
Step-by-step, wizard-like creation of resources
BonFIRE51 51
BonFIRE Portal
BonFIRE52 52
BonFIRE Portal Monitoring
BonFIRE53 53
BonFIRE Portal pros and cons
Pros• Very easy to use• Great starting point for
understanding how things work in BonFIRE
• Gives you the overview of your experiments and resources
Cons• Slow to set up large-scale
experiments• Manual creation of each
resource• No scripted events
possible for interacting with services
BonFIRE54 54
Experiment descriptors
Experiment descriptors can specify the initial deployment of resources.
Additional features include:• VM contextualisation• IP dependency resolution• Configuration of monitoring metrics
Two representation languages available• JSON• OVF
BonFIRE55 55
Experiment descriptors
BonFIRE56 56
JSON experiment description snippet
{ "name": "An experiment", "description": "An experiment description ", "duration": 60, "resources": [ { "compute": { "name": "VM", "locations": [ "fr-inria" ], "instanceType": "small", "min": 1, ...
20,
BonFIRE57 57
Experiment descriptor pros and cons
Pros• Good for large-scale
deployment of resources• Easy to create via the
Portal and edit offline• Intuitive and purposefully
created for BonFIRE• Submit via Portal or
command line
Cons• Cannot script other
elements of experiment set-ups like interacting with experiment services
• Cannot get connection to resources when created to perform actions
BonFIRE58 58
Restfully
• A general-purpose client tool for RESTful APIs• Abstracts the details of exchanging HTTP requests• Discovers resources at run-time• Can be used in a Ruby shell to interactively query the
BonFIRE API• Can be used in Ruby scripts to automate deployment of
resources and interactions with the resources
Written in Ruby
BonFIRE59 59
Ruby shell example
ruby-1.8.7-p249 > pp root.locations#<Collection:0x813dd534 uri=https://api.bonfire-project.eu/locations RELATIONSHIPS parent, self ITEMS (0..5)/5 #<Resource:0x813c6b04 uri=https://api.bonfire-project.eu/locations/be-ibbt> #<Resource:0x813b5868 uri=https://api.bonfire-project.eu/locations/de-hlrs> #<Resource:0x813a45cc uri=https://api.bonfire-project.eu/locations/fr-inria> #<Resource:0x81393330 uri=https://api.bonfire-project.eu/locations/uk-epcc> #<Resource:0x81382094 uri=https://api.bonfire-project.eu/locations/uk-hplabs>> => nil
BonFIRE60 60
Ruby script example
session = Restfully::Session.new(
:configuration_file => "~/.restfully/api.bonfire-project.eu.yml",
:gateway => "ssh.bonfire.grid5000.fr",
:keys => ["~/.ssh/id_rsa"])
begin
experiment = session.root.experiments.find{|e|
e['name'] == "Demo SSH" && e['status'] == "running"
} || session.root.experiments.submit(
:name => "Demo SSH",
:description => "SSH demo using Restfully - #{Time.now.to_s}",
:walltime => 8*3600 # 8 hours
)
BonFIRE61 61
Restfully pros and cons
Pros• Good for large-scale
experiments• One script for deployment
of resources and gives you connection to resources to interact with, e.g., to install software, start services, etc.
• Interactive scripting possible (user input on command line)
Cons• You have to create the
scripts from scratch• Learning curve for Ruby
can be steep, but we provide examples online
• Manually need to resolve deployment constraints
BonFIRE62 62
Command line tools
• Purposefully built for BonFIRE• Written in Python
– Bindings are easy to get by reading our online documentation
• Support for BonFIRE experiment lifecycle:– bfexperiment– bfcompute– bfstorage– …
• Abbreviations possible, instead of long URIs– /locations/uk-epcc/networks/42 42
BonFIRE63 63
Command line tools examples
Create experiment• bfexperiment create <name> [-D <description>] [-W <walltime>]
[-G <group> ...]• bfexperiment create my-experiment -D description -W 60 -G
bonfire
Create storage resource• bfstorage create <name> <location> [<experiment>] [-D
<description>] [-S <size>] [-T {datablock|shared}] [-F <fstype>] [-G <group> ...] [-P] [-R]
• bfstorage create my-storage fr-inria 42 -D storage-desc -S 1024 -T datablock -F ext3 -G bonfire
BonFIRE64 64
Command line tools pros and cons
Pros• Intuitive to use• Scripting possible, e.g.,
Bash on Linux or Batch (MSDOS) on Windows
Cons• Scripting is more limited
than Restfully– Connecting to interact with
resources possible but not as easy as with Ruby
• Less declarative than experiment descriptor– Manually need to resolve
deployment constraints
BonFIRE65 65
Resources
• Client tools: http://doc.bonfire-project.eu/integration/client-tools/overview.html
Building service testbeds on FIRE
Experiment Support
BonFIRE67 67
Support Mechanism
Documentation• General Information:
http://bonfire-project.eu/ • User Documentation:
http://www.bonfire-project.eu/docs
User Forumhttps://forum.bonfire-project.eu/
Support Ticketing [email protected]
Building service testbeds on FIRE
Thank you for your attention
BonFIRE69 69
Licence and Acknowledgements
Copyright © 2013, EPCC, The University of Edinburgh, on behalf of the BonFIRE Consortium.
Licensed under Creative Commons “Attribution-NoDerivs”.
BonFIRE is funded by the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement numbers 257386 and 287938.
www.bonfire-project.eu