Introduction to Tempest

OpenStack Users January 2013 Meetup – India

Presenters: Nithya, Sarad & Suresh

Agenda

• Introduction To Tempest

• Explore Tempest

• Getting Started with Tempest in Openstack

• Role of Tempest in Openstack CI

• Pro’s & Con’s

• Q & A

What is tempest?

• Tempest – It’s an Openstack Integration test suite

• Based on unittest2 framework & currently uses Nosetest runner

• Used by community as gating on commits to trunk

• In a nutshell, all it does is run tests against OpenStack service endpoints by exercising API calls & validate the response

Tempest explored

Tempest

Nova

Glance

Swift

keystone

Quantum

Cinder

• Test Types – Smoke Tests – Positive Tests – Negative Tests – Stress Tests – White box Tests

• Tempest base – httplib2 – Rest based framework – Nose test runner

(testr)

Tempest Directory Structure • Common - Rest client & ssh client

• Services - services modules (nova, identity, image and network)

• Nova implements the OS Nova API

• Identity implements the OS Keystone API

• Image implements the OS Glance API

• Network implements the OS Quantum API

• Tests - Contains actual tests.

– Rely on the above services (via the Manager) to connect to the system.

– Test classes and methods have also access to the tempest configuration.

– Test classes must subclass unittest, and they may use nose and unittest specific decorators

• etc

– tempest.conf

Sample Test workflow

Test case call base class setup

Create server call

Servers_client (json/xml)

REST request

Rest_client

Config.py Openstack_manager.py

Supported Clients

•REST – Httplib •BOTO – ec2 •Novaclient •glance client •swift client •keystone client •SSH – Paramiko •Cloudfile •Quantum client

How to Contribute

• Prerequisites – Signup CLA – Launchpad account

• Upload your SSH keys to Launchpad • Gerrit imports your SSH keys from Launchpad • Subscribe to main openstack mailing list

– devstack installed locally • Follow the steps given in http://devstack.org/guides/single-

machine.html

• Code submission – Clone tempest code – Create a new tempest branch for the bug id – Make required changes – Commit the changes – Submit for review – For resubmitting the changes use commit amend

Continuous Integration Monitor version control system for changes. Whenever a change is

detected, automatically compile and test the application. Notify

developers when things go wrong to get a fix immediately.

Monitor version control system for changes. Whenever a change is

detected, automatically compile and test the application. Notify

developers when things go wrong to get a fix immediately.

Watch code

Build Product

Run Tests

Publish results

• CI Tool - Jenkins

• Openstack CI - https://jenkins.openstack.org/

• Gate tests – Tempest

Openstack CI & Tempest

Report success / failure at review.openstack.org

Run tempest tests

Checks out latest code and setup devstack

Jenkins prepares a Virtual Machine

Code approved by core reviewers

Change proposed to OS component

• https://review.openstack.org/#/c/17770/

• Tempest gated with – Gate-tempest-merge

– Gate-tempest-pep8

– Gate-tempest-devstack-vm

• https://jenkins.openstack.org/view/Gate/job/gate-tempest-devstack-vm/

Tempest Pros and Cons

Pro’s

• Modular

• One unified suite to test all openstack components.

• Easily Maintainable

• Less complex to create tests

• Supports most of the client interfaces

Con’s

• Currently tests are not running in parallel

• Only smoke tests are being run as part of gating process.

• Stress tests development at nascent state

• Tests for service workflows at the minimal

Q&A

References

• Tempest Presentation by Jay Pipes - https://docs.google.com/presentation/d/1M3XhAco_0u7NZQn3Gz53z9VOHHrkQBzEs5gt43ZvhOc/edit?pli=1#slide=id.p

• Openstack CI – http://ci.openstack.org/

• Devstack-gate - https://github.com/openstack-infra/devstack-gate/blob/master/README.md

• Tempest source - https://github.com/openstack/tempest