Giving your website a command line interface

Michael Hudson-Doylemichael.hudson@linaro.org

Linaro aims to make Linux work better on ARM processors

Linaro and its mission

The ProblemThe ARM ecosystem is very fragmented, and the kernel has a lot of copy and paste code

"Gaah. Guys, this whole ARM thing is a f*cking pain in the ass."— Linus Torvalds, 17 Mar 2011https://lwn.net/Articles/437170/

Enter Linaro!"Linaro is a not-for-profit software

engineering company investing in core Linux software and tools for ARM

SoCs."

Also about educating the members in how to do open source development...

LAVA - Linaro Automated Validation

A bit part of Linaro is about automated validation:

•Find regressions earlier•Also benchmark toolchain improvements•Maybe even power management changes too...

LAVAWe have a bunch of hardware

LAVASome scripts and tricks that can boot a board with a new kernel and run some tests.

Quick Demo(ever the optimist)

LAVAAnd a website that lets you see whats going on

The Problem (finally!)We want to do things like trigger test runs when a kernel build finishes.

This basically means some kind of Remote Procedure Call (RPC).

ParanoiaFor a bunch of reasons, we need some kind of security in our system:

•The boards in our lab are a limited resource•Some risk of mischief•Eventually may have test results from unreleased hardware or benchmarks with licenses that forbid publication of results

Protocol Choices•We use XML-RPC•We didn't think about this very hard but it is well supported in most languages •Will probably add JSON-RPC support at some point for easier browser access

First idea: OAuthAn open protocol to allow secure API authorization in a simple and standard

method from desktop and web applications.

– http://oauth.net/

The great thing about standards...

<bob2> kennethreitz: oauth is a font of villany and dispair -- #python, Jun 09 11:55:08

Also doesn't solve our problem

OAuth specifies that various aspects of the request are signed, but not, crucially for us, the body of the request – an important detail, because in XML-RPC the body of the request is where all the important stuff is.

Transport Layer Security, here we come

If you're going as far as to cryptographically sign something, it's not much further to go to actually just encrypt it!

And what does everyone know about encryption?

Don't implement it yourself

(i.e. use HTTPS)

Back to BasicAnd if you're operating over HTTPS, you might as well just just good old RFC 2617 Basic Authentication...

... but with tokens rather than passwords

Tokens > PasswordsBecause we expect the RPC to be invoked from build systems and so on, there is a moderate chance of the token being leaked – so it should not let you take over the owning user's account.

In the future, a token might only let you access some APIs.

Also, we use SSO...In addition we use Launchpad's SSO service for authentication, so most users don't have a LAVA password!

Show me the code!On the server side, we've built a library that lets you add a authenticating XML-RPC to a Django project:

https://launchpad.net/linaro-django-xmlrpc

It includes views and models (and very very simple templates) for creating and managing tokens.

Server side codeexample/api.py:from linaro_django_xmlrpc.models import ExposedAPIfrom linaro_django_xmlrpc.globals import mapper

class ExampleAPI(ExposedAPI): def whoami(self): if self.user: return self.user.username else: return None

mapper.register(ExampleAPI)

in your urlconf: url(r'', include('linaro_django_xmlrpc.urls')),

Client side libraryThis isn't properly factored yet really (it's it all mashed up with our toolkit for doing command line tools), but the code is in "lava-tool":

https://launchpad.net/lava-tool

It uses python-keyring for token management.

Client-side codefrom lava_tool.authtoken import \ AuthenticatingServerProxy, KeyringAuthBackend

auth_backend = KeyringAuthBackend()auth_backend.add_token( "user", "http://server/RPC2/", token)

sp = AuthenticatingServerProxy( "http://user@server/RPC2/", auth_backend=auth_backend)print server.whoami()

Demo

(assuming the first one wasn't a disaster)

ConclusionThe lesson:

Don't try to be clever – just use HTTPS and Basic auth.

The code:

lp:linaro-django-xmlrpclp:lava-tool

Thanks for listening!

Any Questions?