openstack congress and datalog (english)

of 25 /25
© 2015 VMware Inc. All rights reserved. OpenStack Congress & Datalog 2 nd Tokyo OpenStack Meetup at Vmware K.K. Motonori Shindo (@motonori_shindo) CTO Ambassador / Technical Leader VMware

Upload: motonori-shindo

Post on 14-Aug-2015

398 views

Category:

Technology


2 download

TRANSCRIPT

Page 1: OpenStack Congress and Datalog (English)

© 2015 VMware Inc. All rights reserved.

OpenStack Congress & Datalog 2nd Tokyo OpenStack Meetup at Vmware K.K.

Motonori Shindo (@motonori_shindo) CTO Ambassador / Technical Leader VMware

Page 2: OpenStack Congress and Datalog (English)

Self Introduction •  Motonori Shindo

•  Bio –  Tokyo Electric Power Co (TEPCO), School of Computer Science

at Carnegie Mellon University, Ascend Communications, CoSine Communications, Proxim, Fivefront, Nicira, VMware

2

Page 3: OpenStack Congress and Datalog (English)

What is OpenStack Congress ? •  One of the projects in OpenStack to provide “Policy as a Service”.

•  Why called “Congress” ? –  Because that’s where policy is defined J

3

Page 4: OpenStack Congress and Datalog (English)

Why does Congress live in OpenStack? •  Congress is a generic policy engine so it works as standalone (i.e. without OpenStack)

•  That said, in order to define a meaningful / useful policy, some sort of information (“data source”) upon which policy can be defined is needed.

•  OpenStack has a rich set of data sources that can be consumed by Congress, so it is a great place for Congress to live!

4

Page 5: OpenStack Congress and Datalog (English)

What is “Policy” •  No single answer but let’s think of it as something that dictates how the system should behave

in order to conform to:

–  Law / Regulations

–  Business rule

–  Application requirement

–  Geographical constraint

–  Security requirement

–  …

5

A generic language that can dictates these policies is needed!

Page 6: OpenStack Congress and Datalog (English)

Datalog •  Declarative Language based on First Order Logic

–  Often used as a query language

•  Syntactically it is similar to Prolog but it has different semantics : –  No Function Symbols –  Guarantee to terminate –  Order of rule definition is irrelevant –  No “List” construct –  No Cut (!) and fail operators

6

Page 7: OpenStack Congress and Datalog (English)

Datalog Syntax

7

<atom> :- <literal 1>, <literal 2>, <literal 3> … , <literal N>.

Head Body

Page 8: OpenStack Congress and Datalog (English)

Safety Properties of Datalog •  All variables that appear in the head must also appear in the body in the rule as non-arithmetic

positive literal.

•  All variables that appear in the body as negative literal must also appear in other positive literals.

•  Example of non-Safety rules –  q(X, Y, Z) :- r1(X,Y), X < Z. –  q(X, Y, Z) :- r1(X,Y), not r2(X, Y, Z).

•  Example of Safety rules –  q(X, Y, Z) :- r1(X, Y), r2(Y, Z), X < Z. –  q(X, Y, Z) :- r1(X,Y), not r2(X, Y, Z), r3(Y, Z).

8

Page 9: OpenStack Congress and Datalog (English)

Datalog (Prolog) Example 1

9

parent(motonori, manzo). parent(motonori, keiko). male(manzo). male(motonori). female(keiko). father(X, Y) :- parent(X,Y), male(Y). mother(X, Y) :- parent(X,Y), female(Y). ?- father(motonori, X). father(motonori, manzo).

Page 10: OpenStack Congress and Datalog (English)

Datalog (Prolog) Example 2

10

adjacent(a, b). adjacent(b, c). adjacent(c, d). adjacent(a, d). adjacent(e, f). reachable(X, Y) :- adjacent(X, Y). reachable(X, Y) :- adjacent(X, Z), reachable(Z, Y). ?- reachable(b, d). reachable(b, d). ?- reachable(a, f).

a b

d

f

c

e

Page 11: OpenStack Congress and Datalog (English)

What Congress can do today (and in the future) •  Monitoring

–  Check the current status of Cloud against policy and report error if there’s a mismatch

•  Enforcement –  Take an action in order to avoid policy violation –  Proactively / Reactively / Interactively

•  Auditing –  History management of policy and policy violation

11

Page 12: OpenStack Congress and Datalog (English)

Datalog in Congress

•  Syntax

•  Restrictions –  Recursion is not supported (at least for the time being)

12

<policy> ::= <rule>* <rule> ::= <head> COLONMINUS <literal> (COMMA <literal>)* <head> ::= <atom> <head> ::= EXECUTE[<atom>] <literal> ::= <atom> <literal> ::= NOT <atom> <atom> ::= TABLENAME LPAREN <arg> (COMMA <arg>)* RPAREN <arg> ::= <term> <arg> ::= COLUMNNAME=<term> <term> ::= INTEGER | FLOAT | STRING | VARIABLE

Page 13: OpenStack Congress and Datalog (English)

Extension in Congress •  Tables in certain data source may have many number of columns. When writing policy using

such a table it is cumbersome to write all those columns explicitly.

•  Full form:

•  Simplified form:

13

port(id) :- neutron:ports(id, tenant_id, name, network_id, mac_address, admin_state_up,        status, device_owner, fixed_ips, security_groups).

port(id) :- neutron:ports(id=id).

Page 14: OpenStack Congress and Datalog (English)

Drivers that are currently supported for Congress •  OpenStack Ceilometer •  OpenStack Cinder

•  OpenStack Glance (v2) •  OpenStack Ironic

•  OpenStack Keystone

•  OpenStack Murano

•  OpenStack Neutron (v2) •  OpenStack Nova

•  OpenStack Switft •  Cloud Foundry

•  Plexxi

•  vCenter

14

Page 15: OpenStack Congress and Datalog (English)

Example 1: Congress Policy (for monitoring)

15

error(vm, network) :- nova:virtual_machine(vm), nova:network(vm, network), nova:owner(vm, vm_owner), neutron:owner(network, network_owner), not neutron:public_network(network), not same_group(vm_owner, network_owner) same_group(user1, user2) :- ad:group(user1, group), ad:group(user2, group)

Page 16: OpenStack Congress and Datalog (English)

Example 2: Congress Policy (for enforcement)

16

Execute[neutron:disconnectNetwork(vm, network)] :- error(vm, network)

Execute[nova:pause(x)] :- nova:servers(id=x, status=“ACTIVE”)

Page 17: OpenStack Congress and Datalog (English)

Congress -- Policies

17

Page 18: OpenStack Congress and Datalog (English)

Congress – Data Sources

18

Page 19: OpenStack Congress and Datalog (English)

Congress – Data Sources

19

Page 20: OpenStack Congress and Datalog (English)

Congress – Data Sources

20

Page 21: OpenStack Congress and Datalog (English)

Live Demo

21

Goal : Detect a policy violation when a VM is spun up with a flavor lager than or equal to 4GB of memory

Page 22: OpenStack Congress and Datalog (English)

STEP 1: •  Create the following two rules under “classification” policy by CLI:

22

% openstack congress policy rule create classification 'large_flavor(id) :- nova:flavors(id, name, vcpus, ram, disk, ephemeral, rxtx_factor), gteq(ram, 4096)' % openstack congress policy rule create classification 'error(id, name) :- nova:servers(id, name, host_id, status, tenant_id, user_id, image_id, flavor_id), large_flavor(flavor_id)'

Page 23: OpenStack Congress and Datalog (English)

STEP 2: •  Launch a VM with a flavor “m1.nano” and confirm that there’s no policy violation detected by

Congress.

23

Page 24: OpenStack Congress and Datalog (English)

STEP 3:

24

•  Launch another VM with a flavor “m1.large” and confirm Congress detected a policy violation with VM ID and its name.

Page 25: OpenStack Congress and Datalog (English)

Questions