gamers do rest - djangocon 2014

Download Gamers do REST - DjangoCon 2014

Post on 11-Jun-2015




1 download

Embed Size (px)


I gave a more comprehensive version of this talk at EuroPython 2014: ------------------------------------------------------------------------------- The presentation is meant to give an overview (sprinkled with implementation details) of how we use Django to build RESTful APIs in Demonware and how we manage to reliably serve millions of gamers all over the world that play Activision-Blizzard’s successful franchises Call of Duty and Skylanders. Topics the presentation will touch (some only briefly because of time constraints): - tech stack overview - API design - configuration handling - middleware usage for logging, metrics and error handling


  • 1. Gamers do REST not by Angel Ramboi

2. This is one of our recent graphs of online users. As you can see their number never even comes close to zero. 3. And these guys get really excited during launch time. 4. About Demonware Dublin Shanghai Vancouver 5. What do we do? We enable gamers to find one another and shoot each other in the face 6. What do we do? leaderboards matchmaking anticheat accounts management and more 70+ services 7. We hire superheroes 8. Why REST? interoperability scalability 9. Tech stack overview Django 1.6 Python 2.7 MySQL 5.6 (sharded) CentOS Apache+mod_wsgi 10. API design We tend to follow the REST principles outlined in Roy Fielding's thesis GET, POST, PUT, DELETE verbs for API CRUD HTTP for the communication protocol JSON for representation Pragmatic approach good enough > perfect 11. API design (example) GET /v1.0/users/1/ HTTP/1.1 Accept: application/json { "userName": "cmac1", "email": "", "firstName" : "Connor", "lastName" : "MacLeod", "dateOfBirth": "1518-03-18", "immortalityAttained": "1536-11-05", "country": "GB", "gender": "male", "link": { "href": "/v1.0/users/1/only-one/" } } 12. Process and tools 13. Code and deployments 14. YAML 1.2 --- django: DEBUG: False ALLOWED_HOSTS: ["*"] TIME_ZONE: UTC LANGUAGE_CODE: en-us USE_I18N: True SECRET_KEY: Its a secret!!! TEMPLATE_LOADERS: - django.template.loaders.filesystem.Loader INSTALLED_APPS: - django.contrib.contenttypes [...] Cross project & Validation App configuration 15. App configuration Validation example minimum_age = Option( type={ 'type': 'integer', 'valid': [['>=', 0]] }, default=13, description='The minimum age of a user this client can create.') 16. { "title": "Example Schema", "type": "object", "properties": { "username": { "type": "string", "pattern": "^[a-z0-9_-]{3,15}$" }, "age": { "description": "Age in years", "type": "integer", "minimum": 0 } } } JSON validation 17. { "title": "Example Schema", "type": "object", "properties": { "firstName": { "type": "string", "maxLength": 100 }, "lastName": { "type": "string", "maxLength": 100 } }, "required": ["firstName", "lastName"] } JSON validation 18. { "title": "Example Schema", "type": "object", "properties": { "name": { "type": "string" }, "gender": { "type": "string", "enum": ["male", "female", "other"], "exceptions": { "required": errors.GenderMissingError, "type": errors.InvalidGenderError, "enum": errors.InvalidGenderError } } }, "required": ["gender"] } JSON validation 19. class ErrorHandlingMiddleware(object): def process_exception(self, request, exception): return format_and_render_error( request, exception ) Error handling 20. { "error": { "msg": "Request data validation failed, see context for more details.", "code": 227000, "name": "Error:ClientError:InvalidRequest:DataInvalid", "context": [ { "msg": "Email already exists", "code": 288000, "name": "Error:ClientError:Conflict:EmailExists" }, { "msg": "Username cmac1 already exists", "code": 289000, "name": "Error:ClientError:Conflict:UsernameExists" } ]}} Error handling 21. Logging 22. Logging 23. Logging // Bad message - not suitable/useful for production. logger.debug(Variable x={}.format(var)) // Good message - suitable for production. logger.error( Request {request} failed unexpectedly for reason {reason} resulting in client error {error} .format({ request: req, reason: expl, error: client_error_code}) ) 24. Logging 2014-05-10T22:58:56.394565+00:00 level=error project=highlander app=users view=get_UsersView client= method=GET path=/v1.0/users/2 msg=Error: NotFound(No user with user_id 2 could be found. There can be only one!) 25. Metrics 26. Metrics class MetricsMiddleware(object): def process_request(self, request): request.metrics_start_time = time.time() def process_response(self, request, response): if hasattr(request, 'metrics_start_time'): time_in_request = (time.time() - request.metrics_start_time) * 1000 metrics.write( name='request_time', value=time_in_request ) return response 27. Auth We use JSON Web Tokens JOSE is a framework intended to provide a method to securely transfer claims: or just: % pip install jose 28. Summary Rest is awesome Be Pragmatic Monitor Everything We are hiring!!! 29. Questions?