open stack nova reverse engineer

OpenStack Nova ‘Reverse Engineer’

2016 May 07Hieu LE Vietnam OpenStack Community OrganizerFujitsu Vietnam - PODC

Copyright 2016 Fujitsu Vietnam Limited

Starting Point (1)

1 Copyright 2016 Fujitsu Vietnam Limited

Video Tutorial: how to setup remote debugging environment for OpenStack Nova,

request from some community members.

Ref: https://youtu.be/Z0PmKyeZjjA

https://youtu.be/Z0PmKyeZjjA

Starting Point (2)

@NamNH (Fujitsu) contribute to documentation: “Debugging OpenStack Neutron”

Ref: https://vietstack.wordpress.com/2016/04/12/how-to-remote-debugging-openstack-neutron/

Hope that he will present at next

technical meetup about Neutron ;)


https://vietstack.wordpress.com/2016/04/12/how-to-remote-debugging-openstack-neutron/

Agenda


1. Architecture overview

2. Data Flows

3. Nova Code Structure Components

4. Conclusion

3

“to provide massively scalable, on demand,

self service access to compute resources”

OpenStack Nova mission

http://docs.openstack.org/developer/nova/project_scope.html



“to provide massively scalable, on demand,

self service access to compute resources”

OpenStack Nova mission



Virtual Servers Containers Bare Metal Servers Driver Parity


Agenda



2. Data Flows


4. Conclusion

6

Architecture overview

Model:

Nova Network

Neutron

Cells (v1, 2)

Core components

DB: sql database for data storage.

API: receive HTTP requests, converts commands and communicates with other components via the oslo.messaging queue or HTTP

Scheduler: decides which host gets each instance

Compute: manages communication with hypervisor and virtual machines.

Conductor: handles requests that need coordination(build/resize), acts as a database proxy, or handles object conversions.

Cells: used by some large deployments to partition compute nodes into smaller groups, coupled with a database and queue. DEFAULT FROM MITAKA WITH CELLS V2


Architecture overview

Other nova components

Network: manages ip forwarding, bridges, and vlans

Metadata: serves metadata for instances

Cert: serves X509 certificates, using for euca-bundle-image and only needed for EC2 API

Console auth and proxies: noVNC, SPICE, XVP, RDP.. for remote console purpose toward instances

EC2: *DEPRECATED AND MOVE TO NEW EC2-API PROJECT FROM MITAKA* using EC2 API with nova services.


Architecture overview (1)


1

2 3 45

7

6

Ref:

http://docs.openstack.org/developer/nova/architecture.html




1

2 3 45

7

6

Ref:




Normal Workflow


MQ and DB is under heavy-load.

Cells

There are Austin summit video about Nova scheduler performance.

https://www.openstack.org/videos/video/dive-into-nova-scheduler-performance-where-is-the-bottleneck

https://docs.google.com/presentation/d/1UG1HkEWyxPVMXseLwJ44ZDm-

ek_MPc4M65H8EiwZnWs/edit?ts=571fcdd5#slide=id.g12d2cf15cd_2_84

https://www.openstack.org/videos/video/dive-into-nova-scheduler-performance-where-is-the-bottleneck

https://docs.google.com/presentation/d/1UG1HkEWyxPVMXseLwJ44ZDm-ek_MPc4M65H8EiwZnWs/edit?ts=571fcdd5#slide=id.g12d2cf15cd_2_84



Ref: OpenStack Japan User Group



API DB

Compute DB


Cells: aim for easily scaling and providing fault tolerance by partition compute nodes into smaller groups with separate database and queue, instead of legacy deployment model with single logical database and message queue.

Cells v1:

Optional

Not all features supported

Sync instance between DBs

Cells v2:

Default deployment model is one v2 cell

New API database

Tools to migrate from cells v1 **NOT IMPLEMENTED IN MITAKA, PLAN IS IN NEWTON**


Agenda



2. Data Flows

3. Nova code structure

4. Conclusion

15

Façade


WSGIService

oslo_service lib

Paste filters

Paste/Paste.Deploy lib via api-paste.ini

WebOb lib for WSGI req/res wrapper

1. RESTful req 2. WSGI Req

APIRouter

RoutesMiddleware

3. Filtered req

Routes lib

Extension

Controller:index/show/delete/

create..

Extension


create..

Extension


create..

4. Req dispatching

stevedore lib for managing ext, ctrl

Expose via pbr entrypoint

oslo_conf

Façade/Decorator pattern

oslo_message

RPC

Façade design pattern

Nova-compute API (nova/nova/compute/api.py): API, HostAPI, InstanceActionAPI, KeypairAPI…


Ref: Wikipedia/façade_pattern

OpenStack Oslo common lib

oslo.cache: a library for caching based on dogpile

oslo.concurrency: a project with helpers managing external processes and task synchronization. E.g: check host resources.

oslo.config: an extensive library for parsing configuration files and command line arguments.

oslo.log: a logging configuration library.

oslo.service: helper library that provides functionality for running OpenStack services. nova-api service

oslo.messaging: a library that provides a messaging API which supports RPC and notifications over a number of different messaging transports.

oslo.serialization

Ref: https://wiki.openstack.org/wiki/Oslo


https://pypi.python.org/pypi/dogpile.cache

https://wiki.openstack.org/wiki/Oslo

pbr

Python Build Reasonableness: using setuptools to deploy your code and expose via entry points.

setup.cfg

policy.json: will be checked each time an API is called

Can be modified for reasonable purpose


WSGI – PEP 0333

nova/wsgi.py

WSGI Application

Python callable passed two arg (WebOb)

• Env

• Start_response function callback function

WSGI Server

Call the application

WSGI Middleware (Paste)

Both a server and application

Use to wrap request

Paste Deploy will automatically find and config WSGIapplications and servers


Ref: WSGI – Aseries of Tubes by Ian Bickings (The

Open Planning Project)

Paste

http://pythonpaste.org/

Check api-paste.ini file below and see DEMO


Middleware filter Apps

Filter flow

http://pythonpaste.org/

Stevedore (1)

Extension manager library

https://github.com/openstack/stevedore

http://docs.openstack.org/developer/stevedore/


https://github.com/openstack/stevedore

http://docs.openstack.org/developer/stevedore/

Stevedore (2)

Check nova/api/openstack/compute/disk_config.py

Easier to write new API

Easier to write new plugin


Stevedore (3)

Where is extension manager ?

nova/api/openstack/__init__.py – ApiRouterv21 class

nova/compute/monitors/__init.py__

and more …


Routes

Mapping URLs to controllers+actions. URLs are mapped to ‘action’ methods on ‘controller’ classes in nova/api/openstack/__init__.py, class ApiRouter._init_ and ApiRouterv21.__init__


Stevedore ext mgr

Map URL from alias

/v2.1/{tenant_id}/os-aggregates

Ref: api-ref site OpenStack

Agenda



2. Data Flows


4. Conclusion

27

Nova code structure


documentation

For packaging, deploying and testing (using tox)

Source Codes

For setup with devstack

Configuration files

XenServer plugin for using with nova

Nova code structure

Configuration files


Rootwrap.d conf: allow a service-specific

unprivileged user to run a number of actions as

the root user, in the safest manner possible.

https://wiki.openstack.org/wiki/Rootwrap

Nova API config (paste + policy)

Nova Cells config samplehttp://docs.openstack.org/mitaka/config-reference/compute/cells.html

Docs

Logging sample config file

Nova config generator file. Using oslo-config-generator

tool or tox –egenconfig for a sample of nova.conf file.

Nova.conf sample file will list all nova config options

and default value of each option.


http://docs.openstack.org/mitaka/config-reference/compute/cells.html

Nova code structure

Configuration files


Rootwrap.d conf: allow a service-specific

unprivileged user to run a number of actions as

the root user, in the safest manner possible.


Nova API config (paste + policy)

Nova Cells config samplehttp://docs.openstack.org/mitaka/config-reference/compute/cells.html

Docs

Logging sample config file

Nova config generator file. Using oslo-config-generator

tool or tox –egenconfig for a sample of nova.conf file.

Nova.conf sample file will list all nova config options

and default value of each option. (~810 option)


http://docs.openstack.org/mitaka/config-reference/compute/cells.html

Nova source code


Executable system script, bundle and expose via pbr.

e.g.: nova-api, nova-compute..

Centralize configuration options for all nova subcomponents

(WIP Newton version)

Nova compute virt (virtualization) driver driver parity approach

Nova source code


Executable system script, bundle and expose via pbr.

e.g.: nova-api, nova-compute..

Centralize configuration options for all nova subcomponents

(WIP Newton version)

Nova compute virt (virtualization) driver driver parity approach

HANDS-ON DEMO

Agenda




3. Data Flows

4. Conclusion

33

Conclusion


WSGIService

oslo_service lib

Paste filters

Paste/Paste.Deploy lib via api-paste.ini

WebOb lib for WSGI req/res wrapper

1. RESTful req 2. WSGI Req

APIRouter

RoutesMiddleware

3. Filtered req

Routes lib

Extension


create..

Extension


create..

Extension


create..

4. Req dispatching

stevedore lib for managing ext, ctrl

Expose via pbr entrypoint

oslo_conf

Façade/Decorator pattern

oslo_message

RPC

open stack nova reverse engineer

Presentations & Public Speaking