gamers do rest - djangocon 2014
DESCRIPTION
I gave a more comprehensive version of this talk at EuroPython 2014: http://www.slideshare.net/limpangel/gamers-do-rest-europython ------------------------------------------------------------------------------- The presentation is meant to give an overview (sprinkled with implementation details) of how we use Django to build RESTful APIs in Demonware and how we manage to reliably serve millions of gamers all over the world that play Activision-Blizzard’s successful franchises Call of Duty and Skylanders. Topics the presentation will touch (some only briefly because of time constraints): - tech stack overview - API design - configuration handling - middleware usage for logging, metrics and error handlingTRANSCRIPT
Gamers do RESTnot
by Angel Ramboi
This is one of our recent graphs of online users.As you can see their number never even comes close to zero.
And these guys get really excited during launch time.
About Demonware
Dublin
Shanghai
Vancouver
What do we do?
We enable gamers to find one another and
shoot each other in the face
What do we do?leaderboardsmatchmaking
anticheataccounts management
and more … 70+ services
We hire superheroes
http://www.demonware.net/jobs
Why REST?
interoperability
scalability
Tech stack overview
Django 1.6Python 2.7
MySQL 5.6 (sharded)CentOS
Apache+mod_wsgi
API design● We tend to follow the REST principles outlined in Roy
Fielding's thesis● GET, POST, PUT, DELETE verbs for API CRUD● HTTP for the communication protocol● JSON for representation● Pragmatic approach
○ “good enough” > perfect
API design (example)GET /v1.0/users/1/ HTTP/1.1
Accept: application/json
{
"userName": "cmac1",
"email": "[email protected]",
"firstName" : "Connor",
"lastName" : "MacLeod",
"dateOfBirth": "1518-03-18",
"immortalityAttained": "1536-11-05",
"country": "GB",
"gender": "male",
"link": { "href": "/v1.0/users/1/only-one/" }
}
Process and tools
Code and deployments
YAML 1.2
---
django:
DEBUG: False
ALLOWED_HOSTS: ["*"]
TIME_ZONE: UTC
LANGUAGE_CODE: en-us
USE_I18N: True
SECRET_KEY: “It’s a secret!!!”
TEMPLATE_LOADERS:
- django.template.loaders.filesystem.Loader
INSTALLED_APPS:
- django.contrib.contenttypes
[...]
Cross project&
Validation
App configuration
App configurationValidation example
minimum_age = Option(
type={
'type': 'integer',
'valid': [['>=', 0]]
},
default=13,
description='The minimum age of a user this client can create.')
{
"title": "Example Schema",
"type": "object",
"properties": {
"username": {
"type": "string",
"pattern": "^[a-z0-9_-]{3,15}$"
},
"age": {
"description": "Age in years",
"type": "integer",
"minimum": 0
}
}
}
JSON validation
http://json-schema.org/
{
"title": "Example Schema",
"type": "object",
"properties": {
"firstName": {
"type": "string",
"maxLength": 100
},
"lastName": {
"type": "string",
"maxLength": 100
}
},
"required": ["firstName", "lastName"]
}
JSON validation
http://json-schema.org/
{ "title": "Example Schema",
"type": "object",
"properties": {
"name": { "type": "string" },
"gender": {
"type": "string",
"enum": ["male", "female", "other"],
"exceptions": {
"required": errors.GenderMissingError,
"type": errors.InvalidGenderError,
"enum": errors.InvalidGenderError
}
}
},
"required": ["gender"]
}
JSON validation
http://json-schema.org/
class ErrorHandlingMiddleware(object):
def process_exception(self, request, exception):
return format_and_render_error(
request,
exception
)
Error handling
{ "error": {
"msg": "Request data validation failed, see context for more details.",
"code": 227000,
"name": "Error:ClientError:InvalidRequest:DataInvalid",
"context": [
{
"msg": "Email [email protected] already exists",
"code": 288000,
"name": "Error:ClientError:Conflict:EmailExists"
},
{
"msg": "Username cmac1 already exists",
"code": 289000,
"name": "Error:ClientError:Conflict:UsernameExists"
}
]}}
Error handling
Logging
Logging
Logging// Bad message - not suitable/useful for production.logger.debug(“Variable x={}”.format(var))
// Good message - suitable for production.logger.error(
“Request {request} failed unexpectedly for reason {reason} resulting in client error {error}”
.format({
“request”: req,
“reason”: expl,
“error”: client_error_code})
)
Logging
2014-05-10T22:58:56.394565+00:00 level=error project=highlander app=users view=get_UsersView client=127.0.0.1 method=GET path=/v1.0/users/2 msg=Error:NotFound(No user with user_id 2 could be found. There can be only one!)
Metrics
Metricsclass MetricsMiddleware(object):
def process_request(self, request):
request.metrics_start_time = time.time()
def process_response(self, request, response):
if hasattr(request, 'metrics_start_time'):
time_in_request = (time.time() - request.metrics_start_time) * 1000
metrics.write(
name='request_time',
value=time_in_request
)
return response
Auth● We use JSON Web Tokens● JOSE is a framework intended to provide a method to securely transfer
claims:○ https://github.com/Demonware/jose○ https://pypi.python.org/pypi/jose/
or just:% pip install jose
Summary
Rest is awesomeBe Pragmatic
Monitor EverythingWe are hiring!!!
Questions?