how to build a neutron plugin (stadium edition)

19
How to write a Neutron Plugin (if you really need to) Salvatore Orlando Mitaka Design Summit Tokyo, Japan NEUTRON STADIUM EDITION

Upload: salvatore-orlando

Post on 11-Apr-2017

464 views

Category:

Software


2 download

TRANSCRIPT

Page 1: How to build a Neutron Plugin (stadium edition)

How  to  write  a  Neutron  Plugin (if  you  really  need  to)

Salvatore  Orlando  Mitaka  Design  Summit  -­‐  Tokyo,  Japan

NEUTRON

STADIUM

EDITION

Page 2: How to build a Neutron Plugin (stadium edition)

Summary

• Confusing  you  with  plugin,  service  plugins  and  drivers  

• More  confusion:  the  Neutron  stadium  

• Thriving  in  the  Neutron  stadium

Page 3: How to build a Neutron Plugin (stadium edition)

A  Neutron  plugin  in  a  Nutshell

• Implements  one  or  more  “plugin  interfaces”  • Handles  requests  from  the  API  layer

API

Plugin

Agents,   physical/virtual  

appliances,  controllers,  etc.

AuthNAuthZ

ValidationDispatch

API  request

Page 4: How to build a Neutron Plugin (stadium edition)

Core  and  service  plugins– Core:  Implements  the  “core”  Neutron  API  – L2  networking  +  IPAM  – Service:  provides  additional  network  services  – Eg.:  L3,  load  balancing,  firewall,  VPN  – services  can  also  be  provided  by  core  plugin  by  implementing  the  relevant  extensions

API

Plugins

Core L3 FW

Core  Plugin

Core L3 FW

Core  Plugin

Core L3 FW

Core  plugin

L3  plugin

FW  plugin

FW  plugin

Page 5: How to build a Neutron Plugin (stadium edition)

Plugins  with  drivers

• Can  execute  a  given  request  on  different  backend  • backend  interaction  is  entirely  delegated  to  the  driver  – ML2  • Openvswitch,  linuxbridge,  hyper-­‐V,  tail-­‐F  NCS,  Arista,  …  – Load  Balancing  reference  plugin  – Firewall  reference  plugin  – VPN  reference  plugin

Page 6: How to build a Neutron Plugin (stadium edition)

Plugin  or  ML2  driver

Page 7: How to build a Neutron Plugin (stadium edition)

The  Neutron  Stadium

Page 8: How to build a Neutron Plugin (stadium edition)

Plugins  in  the  Neutron  stadium• Out  of  tree  • each  plugin  (or  driver)  lives  in  its  own  code  repository  

• Self  managed  • Each  plugin  (or  driver)  has  a  distinctdevelopment  and  core  team  

• Limited  oversight  from  Neutron  core  team  

• (Optional)  coordinated  release

Page 9: How to build a Neutron Plugin (stadium edition)

Introducing  the  HDN  plugin

• HDN:  Human  Defined  Networking  – https://github.com/salv-­‐orlando/hdn    

• Rediscover  the  human  face  of  IT  – REST  API  requests  are  converted  into  emails  sent  to  the  networking  guy(s)  in  your  IT  department  

– Asynchronous  and  eventually  consistent  request  processing  – Karma-­‐based  request  prioritisation:  the  nicer  you  are  to  the  IT  guy,  the  sooner  your  requests  will  be  processed

Page 10: How to build a Neutron Plugin (stadium edition)

The  HDN  architecture

API  request

Neutron  REST  Interface

Message  bus  (email)  #TODO:  Phone,  Fax

Human-­‐powered  networking  engine

Page 11: How to build a Neutron Plugin (stadium edition)

HDN  Plugins

• Core  +  Service  plugins  • 1:1  mapping  between  services  and  plugins  

• Plugins  interact  via  callback  registry  

• Support  L3  &  Security  group  community  extensions  

• Provides  a  plugin-­‐specific  extension  for  task  management

Core L3 Sec  Groups

Core  plugin

L3  plugin

Security  Group    plugin

Admin  tasks

HDN  tasks  plugin

Page 12: How to build a Neutron Plugin (stadium edition)

Devstack  Integration

• Implement  a  devstack  plugin  • Override  default  values  • Execute  shell  scripts  at  given  stages  of  devstack  execution  • Specify  core  and  service  plugin,  set  values  for  plugin  configuration  • Run  subproject  database  migrations  

• Supply  plugin-­‐specific  entry  points  

• References:  • https://github.com/salv-­‐orlando/hdn/tree/master/devstack    • http://docs.openstack.org/developer/devstack/plugins.html    • http://docs.openstack.org/developer/devstack/plugin-­‐registry.html

Page 13: How to build a Neutron Plugin (stadium edition)

Database  schema• Plugins  and  drivers  can  extend  neutron’s  DB  schema  • But  not  alter  neutron’s  schema  

• Schema  migrations  managed  via  alembic  • Each  sub-­‐project  (like  HDN),  get  its  own  alembic  environment  • Each  sub-­‐project  can  have  expand/contract  branches  just  like  neutron  • Dependencies  among  migrations  in  distinct  project  cannot  (yet)  be  specified  

• References:  • https://github.com/salv-­‐orlando/hdn/tree/master/hdn/db/migration  • http://docs.openstack.org/developer/neutron/devref/alembic_migrations.html  • http://docs.openstack.org/developer/neutron/devref/contribute.html#db-­‐model-­‐migration-­‐testing  

Page 14: How to build a Neutron Plugin (stadium edition)

Leveraging  Callbacks• Simplify  interaction  among  plugins  • Available  both  for  local  and  remote  calls  (e.g.:  plugin/agent  interaction)  • plugins  do  not  need  to  be  aware  of  each  other  • plugins  subscribe  for  events  by  registering  callbacks  with  the  registry  • registry.subscribe(do_something, resources.PORT, events.BEFORE_DELETE)

• plugins  generate  events  by  notifying  the  registry  • registry.notify(resources.PORT, events.BEFORE_DELETE, notify_func, **kwargs)

• References:    • http://docs.openstack.org/developer/neutron/devref/callbacks.html    • http://docs.openstack.org/developer/neutron/devref/rpc_callbacks.html

Page 15: How to build a Neutron Plugin (stadium edition)

Callbacks  in  HDN

• Whenever  a  resource  is  created,  trigger  an  AFTER_CREATE_EVENT  • The  ‘hdn-­‐tasks’  extension  registers  callbacks  to  create  tasks  when  such  events  occur  

• Similar  behaviour  for  update  and  delete  events  • Whenever  a  task  is  completed  by  a  HDN  operator,  a  TASK_COMPLETE  event  is  triggered  

• Core  and  service  plugins  register  callbacks  to  update  the  status  of  the  affected  resource  (or  delete  it)  according  to  the  outcome  of  the  task

Page 16: How to build a Neutron Plugin (stadium edition)

How  to  deal  with  breakages

• Fact:  changes  in  neutron  will  break  your  plugin  quite  often  • Be  prepared  for  failures  • Constantly  test  your  plugin.  Breakages  are  often  detected  simply  by  unit  tests.  

• Running  tests  on  neutron  patches  on  under  review  can  provide  early  warning  to  plugin  maintainers  • it  might  be  sufficient  just  running  unit  tests,  as  most  breakages  are  indeed  very  trivial  -­‐  for  instance  a  method  signature  change,  or  a  change  in  return  values

Page 17: How to build a Neutron Plugin (stadium edition)

Testing

• Rule  of  thumb:  if  Neutron  tests  it,  your  subproject  should  too  • Unit  &  functional  tests:  provide  as  much  coverage  as  possible  • In-­‐tree  API  job  • Add  API  tests  for  every  extensions  the  subproject  introduces  

• Scenario  tests  • Run  any  neutron  tempest  scenario  that  covers  your  subproject  • Define  new  scenarios  for  every  use  case  your  subproject  supports  

• DB  schema  validation  • Tests  for  verifying  the  DB  models  are  in  sync  with  the  migrations

Page 18: How to build a Neutron Plugin (stadium edition)

Other  stadium  challenges

• Release  management  • http://docs.openstack.org/developer/neutron/devref/sub_project_guidelines.html#sub-­‐project-­‐release-­‐process    

• Requirements  • The  “requirement  contract” https://github.com/openstack/requirements/blob/master/README.rst  

• Oslo  incubator  • Do  NOT  import  code  from  neutron.openstack.common  

• CI  • http://docs.openstack.org/developer/neutron/policies/thirdparty-­‐ci.html  

• Documentation

Page 19: How to build a Neutron Plugin (stadium edition)

Thanks for your time!

Questions,  comments,  complaints,  rants  and  rotten  tomatoes  welcome!