Jolyon WhiteGEC9 2nd November 2010

A Tutorial Introduction to OML

Introduction, Aims

• What is OML?– The Orbit Measurement Library– Most current version: OML v2.4.0

• (but v2.5.0 is due out in a few days)– A client library (liboml2) for instrumenting your

applications; plus– A measurement server for collecting and storing

measurements, remotely.

By the end of this tutorial you should be able to:• Understand the OML system architecture• Understand how to run OML applications

– How to configure your app’s measurements– How to interpret the stored results

• How to use OML to instrument your experiment applications

2

OML – the Orbit Measurement Library

• Open Source, under active development– Started at WINLAB, work continuing at NICTA (Sydney)– *NIX target (currently Linux, Mac OS X, on i386, amd64,

and ARM)• Network research generates lots of data• Need a way to get data to a central location for

storage & analysis– Need a better option than local files + scp

• Main design aim: Hit the “power vs. simplicity” sweet spot:

3

Simple

Limited

Labour-intensive, error-prone

OMLSimple + Powerful

By Design

(Too) Complex

Expensive OAM

Robust

Lots of features

OML Deployments

4

Rutgers University, New Jersey

NICTA, Sydney

Deutsche Telekom Labs@ TU Berlin

BOWL Testbed

National Broadband Network100Mbs FTTH

VoD Trial

IREELNetwork EducationTeaching Platform

Rail Bridge Monitoring Sensors

NSW Road Traffic Authority

Parking DiscoveryRutgers

Marco Gruteser

OML System Architecture

5

Separation of Concerns

• Instrumentation– Adding measurement points to an application

• Collection– Running an experiment, collecting measurements

• OML makes a clean distinction between these two activities

• Application writer and application user might be different people

• OML supports users to do both activities effectively

• An application’s measurement collection is configurable at run-time– By the experimenter (application user)

6

OML Architecture – Types

• All measured values in OML are typed– True for the whole measurement pipeline

• Supported types:

• OML_STRING_VALUE is for short strings (<255 bytes)

• OML_BLOB_VALUE is for long blobs (max ~ 232 bytes)

• … more types planned (e.g. IP addresses)

7

• Numeric:– OML_INT32_VALUE– OML_UINT32_VALUE– OML_INT64_VALUE– OML_UINT64_VALUE– OML_DOUBLE_VALUE

• String/arbitrary data:– OML_STRING_VALUE– OML_BLOB_VALUE

OML Architecture – Measurement Points

• Data enters the OML measurement system via a Measurement Point (MP)– Group related measurements

• Each MP has a name to identify it• Every time the application wants to record a

measurement, it injects a value into the MP• “value” == typed tuple with named fields• E.g.

8

MP udp_in = ( ts : OML_DOUBLE_VALUE,flow_id : OML_INT32_VALUE,seq_no : OML_UINT32_VALUE,pkt_length : OML_UINT32_VALUE,src_host : OML_STRING_VALUE,src_port : OML_STRING_VALUE )

OML Architecture – Client + Server

Application or

Service

Measurement points Filters Measurement streams

OML Server

Database

(SQL)

Database tables

File

OML client library9

OML Architecture – Measurement Collection

OML Server10

Application

Experiment Node

OML Server

Application

Experiment Node

Application

Experiment Node

Application

Measurement

s

Destination for each stream configured at run time(XML config file)

So far, so good

• OML’s client/server architecture is simple

but

• Most complicated part of OML comes from– Filtering– Measurement streams– Schemas

11

OML Architecture – Filters

• Filters are for:– Selection– Transformation

• Filters are executed by liboml2, i.e. within the same process as the application

• Take input from an MP– Can SELECT one field or multiple fields

• Compute a new value based on the input– TRANSFORM to a new value

• Input can be multiple fields from one MP• Output can have multiple fields – tuple

12

OML Architecture – Filters

• OML has numerous standard filters built-in• Example: averaging filter (avg)

• Each filter has an output schema• Types in the output schema can be either:

– A specific type (e.g. OML_UINT32_VALUE); or– “whatever type you gave me as input”

• Some filters are picky (e.g. avg only accepts numeric types) 13

avgavg : DOUBLEmax : DOUBLEmin : DOUBLE

ts : DOUBLEflow_id : INT32seq_no : UINT32pkt_length : UINT32src_host : STRINGsrc_port : STRING

MP udp_in:

OML Architecture – Measurement Streams

• Each filter takes input from one MP• Filters are grouped based on destination (more

later)• A Measurement Stream (MS) groups all the filter

outputs from one MP to one destination• Each MS has a schema

– Combination of schema of filter outputs

14

MP (A, B, C)A

B

C

(S, T)

(U, V, W)

(X, Y)

(S, T, U, V, W, X, Y)

MS Schema

OML Architecture – Schemas and the DB

• An OML app declares schemas for each MS to the remote server– Handled automatically by liboml2

• Each application has a name• Each MS schema has a name• Each schema field has a name and a type• Names are derived from:

– App name– MP name– MP field name– Filter output field name

• One MS One database table

15

OML Architecture – Schemas and the DB

• Example: app name is “otr2”

• Schema:

• SQL issued to the database:

16

avgavg : DOUBLEmax : DOUBLEmin : DOUBLE

ts : DOUBLEflow_id : INT32seq_no : UINT32pkt_length : UINT32src_host : STRINGsrc_port : STRING

MP udp_in:

otr2_udp_in : pkt_length_avg:double pkt_length_max:double pkt_length_min:double

CREATE TABLE otr2_udp_in ([other stuff], pkt_length_avg REAL, pkt_length_max REAL, pkt_length_min REAL);

OML Architecture – Filters (again)

• Filters operate in either count- or interval-sampling mode

• Filter can accumulate state over the sampling period

• Filter generates an output at end of sampling period

• E.g. 1) every 10 samples• E.g. 2) every 3.5 seconds

• For instance, if count=10, avg filter outputs the average of the last 10 samples, then resets its internal state.

• See ‘—oml-samples’ and ‘—oml-interval’ command line options

17

Collection:Using and Configuring OML

Applications

18

Configuring Client Applications

• Two options– Command line– XML config file

• Mandatory configuration items:– Node ID (--oml-id) – identify source of a measurement– Experiment ID (--oml-exp-id) – group related

measurements in one database– Destination address (local file name or remote host:port)

• --oml-file <file_name>• --oml-server <hostname>:<port>

• Experiment ID == Database Name

19

Node A’

Node B

Node A

Configuring Client Applications

20

App1, ID B, E1

App2, ID B, E1

App1, ID A, E2

App3, ID A, E2

App1, ID A, E1

App2, ID A, E1E1

E2

OML Server

Command Line Configuration

$ nmetrics_oml2 --oml-id node1 \ --oml-exp-id monitor \ --oml-file cpu.txt \ --cpu --ram

21

OMLoptions

Nmetricsoptions

$ nmetrics_oml2 --oml-id node1 \ --oml-exp-id monitor \ --oml-server

10.0.0.200:3003 --cpu --ram

Sampling policy

• Count-based sampling: --oml-samples <n>• Interval-based: --oml-interval <seconds>

22

$ nmetrics_oml2 --oml-id node1 \ --oml-exp-id monitor \ --oml-server 10.0.0.200:3003

\ --oml-interval 2.5

--cpu --ram

Sampling policy and filter configuration

• Command line config establishes default filters• One filter for each field of each MP• Default filter type depends on field type:

– Numeric MP field Averaging filter– Other types (e.g. string) ‘First’ filter

• The ‘first’ filter outputs the first value in the sampling period

• Throws away the rest

• BUT: for ‘--oml-samples 1’, numeric fields get a first filter instead

23

Custom configuration – config file

• XML• First: establish destinations – <collect …/>• Second: select MP – <mp … />• Third: create filters for each MP – <f … />• Example: one destination, one MP, one filter

24

<omlc exp_id=‘monitor’ id=‘node1’> <collect url=“tcp://10.0.0.200:3004”> <mp name=“memory” interval=“2”> <f pname=“rx_packets” fname=“avg”/> </mp> </collect></omlc>

Config file – longer example

25

<omlc exp_id=‘monitor’ id=‘node1’> <collect url=“tcp://10.0.0.200:3004”> <mp name=“memory” interval=“2”> <f pname=“free” fname=“avg”/> </mp> </collect> <collect url=“tcp://localhost:3003”> <mp name=“network” interval=“0.5”> <f pname=“name” /> <f pname=“rx_packets” fname=“avg”/> <f pname=“rx_bytes” fname=“avg”/> <f pname=“rx_dropped” fname=“avg”/> </mp> </collect> <collect url=“file:cpu.txt”> <mp name=“memory” samples=“1” /> </collect></omlc>

Packaged applications

• oml2-nmetrics – libsigar wrapper (node monitoring)

• oml2-trace – libtrace wrapper (including radiotap)• oml2-wlanconfig – wrapper around wlanconfig(1)• oml2-gps – interface to gpsd(1) for GPS location

data• oml2-iperf – instrumented version of iperf

– Currently iperf version 1.7– Version 2.0 (and maybe 3.0) under development

• otg2 / otr2 – Orbit traffic generator & receiver– Background traffic generator

• Open to contributions! 26

OMF Integration

• OMF provides support for OML applications• Launching on remote nodes• Automatically set node ID• Automatically set experiment ID• Configure measurement collection from OMF

experiment script• Access and visualize results

– Through OMF Aggregate Manager

27

Visualization

28

Instrumentation:Writing/Modifying Applications to

Use OML

29

Writing OML Applications

• OML applications link against liboml2• Liboml2 provides API for:

– Defining Measurement Points;– Injecting measurement samples into MP’s

• Liboml2 also executes filters• API consists of 5 main functions:

– omlc_init() – initialize library– omlc_add_mp() – define measurement points– omlc_start() – start measurement sampling + filtering

system– omlc_inject() – inject a sample into a measurement point– omlc_close() – shut down the OML client library

30

OML Application Phases

31

Initialize

Establish MP’s

Main application

loopRecord

measurements

omlc_add_mp()

omlc_init()

omlc_inject()

End application

omlc_close()

OML Initialization

• omlc_init() processes command line vector– Parses and removes all ‘—oml-’ options– Sets up internal library configuration

• Must call omlc_init() before other OML functions• Example:

32

#include <oml2/omlc.h>

int main (int argc, const char **argv){ int result = omlc_init(“myapp”, &argc, argv, NULL);

/* . . . */

/* Process application’s own options */

/* Do the application */

return 0;}

Establishing Measurement Points

• After initialization, call omlc_add_mp() to create MP’s

• MP is defined as a C array:

• Final element is a sentinel to terminate the array (important!)

• OmlMPDef array is an input to omlc_add_mp()33

OmlMPDef mpdef [] = { { “label”, OML_STRING_VALUE }, { “pkt_count”, OML_UINT32_VALUE }, { “throughput”, OML_DOUBLE_VALUE }, { NULL, (OmlValueT)0 } /* Terminator */ }

Establishing Measurement Points

• Example call omlc_add_mp():

34

#include <oml2/omlc.h>

int main (int argc, const char **argv){ int result = omlc_init(“myapp”, &argc, argv, NULL);

OmlMPDef mpdef [] = { { “label”, OML_STRING_VALUE }, { “pkt_count”, OML_UINT32_VALUE }, { “throughput”, OML_DOUBLE_VALUE }, { NULL, (OmlValueT)0 } /* Terminator */ };

OmlMP *mp = omlc_add_mp(“packets”, mpdef);

/* Define more MP’s (no limit on calls to omlc_add_mp() */

return 0;}

Starting measurement and the main loop

• After all MP’s are initialized, call omlc_start() to kick off measurement sampling

• Can’t call omlc_add_mp() again after calling omlc_start()

• Can’t call omlc_inject() until after calling omlc_start()

35

Application main loop

36

int main (int argc, const char **argv){ int result = omlc_init(“myapp”, &argc, argv, NULL); OmlMPDef mpdef [] = { ... }; OmlMP *mp = omlc_add_mp(“packets”, mpdef);

omlc_start() /* Enable measurement system */

while (1) { char *label; uint32_t pkt_count; double throughput; OmlValueU v[3]; // same size as MP

/* do some application logic; compute values for the 3 variables above */

omlc_set_string(v[0], label); omlc_set_uint32(v[1], pkt_count); omlc_set_double(v[2], throughput); omlc_inject (mp, v); } return 0;}

Rules on naming

• Application name, MP names, and MP field names must be valid C identifiers

• i.e. start with an underscore or letter, followed by alpha-numeric + underscore characters

• No spaces allowed• Reason 1: spaces in names make schemas harder

to parse • Reason 2: these names appear in database table

+ column names• Reason 3: we do code generation

37

Easier app definition with oml2_scaffold

• oml2_scaffold(1)– Generate C-code for MP definitions from declarative spec– Can also declare command line options for your app

38

defApplication('app:myapp', ’myapp') do |a| a.version(1, 0, 0) a.shortDescription = 'Application to count packets' a.description = %{This application counts packets and measures throughput}

a.defProperty('address', 'address to bind to', ?a, :type => :string) a.defProperty('port', 'port to bind to', ?p, :type => :int, :default => 2947)

# Define one MP a.defMeasurement(”packets") do |m| m.defMetric('label', 'string', 'Packet label') m.defMetric('packets', 'uint32', 'Number of packets received') m.defMetric('throughput', 'double', 'Packet throughput') endend

oml2_scaffold(1)

• oml2_scaffold automatically generates:– OmlMPDef arrays– A global struct of OmlMP pointers, g_oml_mps– A function to register all MP’s, oml_register_mps()

• oml2_scaffold can also generate:– A libpopt compatible command line arguments

specification– A skeleton main.c– A Makefile

• The skeleton can actually be built and run using the Makefile

• Application description can be used by OMF

39

oml2_scaffold(1)

• More information:– man oml2_scaffold – Unix man page– http://omf.mytestbed.net/doc/oml/html/oml2_scaffold.htm

l• Tutorial

– http://omf.mytestbed.net/projects/oml/wiki/OML2_scaffold_Tutorial

40

Instrumentation – General Strategies

• Write from scratch– Easy: build application around oml2_scaffold description

• Existing application – with source code– Moderate: Analyze code:– Find where to initialize OML – before app processes its

command line– Find out what you want to measure

• Create MP’s• Insert omlc_inject() statements where needed

– E.g. iperf, see tutorial:• http://omf.mytestbed.net/projects/oml/wiki/Quick_Start_Tutor

ial

• Existing application – no source code– Use fork(2) & pipe(2), then parse the app’s stdout– Same, but use oml4r.rb – Ruby implementation of text

protocol41

The OML Proxy Server:Handling disconnection

42

Measurement with Mobile Nodes

• Sometimes only one wireless interface• No dedicated control network

– Measurement traffic affects experiment outcome• Sporadic connectivity

– What do we do with measurement traffic when disconnected?

• Sometimes fixed network experiments suffer similar problems– E.g. if measurement traffic > measurement network BW– Fixed nodes with only one interface

43

Measurement with Mobile Nodes

Fixed Testbed

Experiment network(s)

Control/measurement network

Measurement server

Proxy Server

• Buffer measurements on command– Don’t transmit to remote server

• Same protocol as oml2-server– Transparent to client applications

Proxy server OML ServerApplication

PAUSERESUME

45

Future directions

• Refactor server into a library (in progress)– Clients can also be endpoints– Servers can also be clients– Hierarchy of measurement generators & collectors

• Streaming queries• Alternative transports: IPFIX main priority• Alternative database backends: PostgreSQL in

v2.6.0• Make oml2_scaffold more betterer

– Generate injection function for each MP no need for OmlValueU array

– Generate OMF application descriptions– Make coffee

• Bindings to other languages: Ruby, Python, Java (Android!)

46

Get it now! – Links

• Project pages– OML: http://omf.mytestbed.net/projects/oml– OML Applications: http://omf.mytestbed.net/projects/omlapp

• Debian/Ubuntu packages– http://omf.mytestbed.net/projects/oml/wiki/Installing_OML_pa

ckages– http://omf.mytestbed.net/projects/omlapp/wiki/Installing_pac

kages• Packages for Fedora Core 8 known to exist

(PlanetLab)• Source tarballs

– http://omf.mytestbed.net/projects/oml/files– http://omf.mytestbed.net/projects/omlapp/files

• From the source repo:– git clone git://mytestbed.net/oml.git– git clone git://mytestbed.net/oml-apps.git

47