swampdragon presentation: the copenhagen django meetup group
TRANSCRIPT
What is real time?
• Computing that delivers processed data within a required deadline.
• Hard deadline
• Firm deadline
• Soft deadline
3
The web an real time challenges
• HTTP is stateless (by design)
• Request/Response (we had a good run until next time)
• Session/Cookies just identify who you are so the response is use specific.
5
How have we progressed?
• Static pages.
• Dynamic content (storing in databases and serving content with asp, asp, php, ruby python)
• AJAX
6
Short polling
function poll(){ $.post(“some_end_point”, function(data) { console.log(data); // do something with your data setTimeout(poll,5000); /* after receiving you data wait 5 seconds and make another request*/ }); }
7
Long polling
• A variation of short polling
• Response process (your view) holds the connection open for a longer period.
• Responds when it has new data or closes connection and waits for a new request.
8
Web Sockets
• Connection stays open between emitters and receivers on the browser and the server.
• Emitters send messages when an event occurs.
• Can broadcast messages.
9
The problem with Django
• Django is synchronous/blocking
• DB queries contribute to the blocking nature.
10
Solutions
• Paint stuff green.
• Nginx
• Greenlets
• Add asynchronous functionality through third party apps (Tornado)
11
Nodejs/ExpressjsWebsocket with socket.io
NoSQL with MongoDB
var app = require('express')(); var server = require('http').Server(app); var io = require('socket.io')(server); !server.listen(80); !app.get('/', function (req, res) { res.sendfile(__dirname + '/index.html'); }); !io.on('connection', function (socket) { socket.emit('news', { hello: 'world' }); socket.on('my other event', function (data) { console.log(data); }); });
What is SwapDragon• Near real-time functionality for Django via web socket
support.
• Tornado
• SockJS-tornado
• SockJS
• tornado-redis
• redis
SockJS
• HTML5 Websocket
• Client and sever socket implementation
• SockJS-tornado (Python implementation)
What makes it awesome?
• You can add it to an existing Django project
• Simple routers
• Self publish in models
• Allows you to work with Django sessions
What makes it questionable? (just my opinion…man)
• It has it’s own serialisers.
• Documentation is a little sketchy
• Your assumptions can block something (more my own fault than anything else)
Getting startedpip install swampdragon
dragon-admin startproject <myproject>
INSTALLED_APPS = ( ... 'swampdragon', ) !!# SwampDragon settings SWAMP_DRAGON_CONNECTION = ('swampdragon.connections.sockjs_connection.DjangoSubscriberConnection', '/data')
#!/usr/bin/env python import os import sys !!from swampdragon.swampdragon_server import run_server !os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings") !host_port = sys.argv[1] if len(sys.argv) > 1 else None !run_server(host_port=host_port)
server.py
Adding SD to your modelsfrom django.db import models !# Publishes a new object as soon as it is created from swampdragon.models import SelfPublishModel !from .serializers import QuestionSerializer, ChoiceSerializer !class Question(SelfPublishModel, models.Model): # Serializes the model serializer_class = QuestionSerializer question_text = models.CharField(max_length=200) pub_date = models.DateTimeField('date published') !!class Choice(SelfPublishModel, models.Model): serializer_class = ChoiceSerializer question = models.ForeignKey(Question) choice_text = models.CharField(max_length=200) votes = models.IntegerField(default=0) !
Your Serializersfrom swampdragon.serializers.model_serializer import ModelSerializer #from .models import Question, Choice !class QuestionSerializer(ModelSerializer): choice_set = 'ChoiceSerializer' class Meta: model = 'polls.Question' # Fields you want to publish via the router publish_fields = ('question_text', ) # Any fields you have added that you would ike to update via the router update_fields = ('bar', ) !!class ChoiceSerializer(ModelSerializer): class Meta: model = 'polls.Choice' # Fields you want to publish via the router publish_fields = ('question_text', ) # Any fields you have added that you would ike to update via the router update_fields = ('bar', )
Your Routersfrom swampdragon import route_handler from swampdragon.route_handler import ModelRouter from .serializers import QuestionSerializer, ChoiceSerializer from .models import Question, Choice !class QuestionRouter(ModelRouter): route_name = 'questions-list' serializer_class = QuestionSerializer model = Question ! # return a single question by 'pk' def get_object(self, **kwargs): return self.model.objects.get(pk=kwargs['pk']) ! # return all your questions def get_query_set(self, **kwargs): return self.model.objects.all() !!!route_handler.register(QuestionRouter) !
... !class ChoiceRouter(ModelRouter): serializer_class = ChoiceSerializer route_name = 'choice-route' ! # get a single choice by 'pk' def get_object(self, **kwargs): return self.model.objects.get(pk=kwargs['pk']) ! # return all your choices for a question def get_query_set(self, **kwargs): return self.model.objects.filter(question__id=kwargs['q_id']) !route_handler.register(ChoiceRouter)
In the browser<!-- Swamp dragon --> <script type="text/javascript" src="http://localhost:9999/settings.js"></script> <script type="text/javascript" src="{{ STATIC_URL }}swampdragon/js/vendor/sockjs-0.3.4.min.js"></script> <script type="text/javascript" src="{{ STATIC_URL }}swampdragon/js/swampdragon.js"></script> <script type="text/javascript" src="{{ STATIC_URL }}swampdragon/js/datamapper.js"></script> <script type="text/javascript" src="{{ STATIC_URL }}swampdragon/js/swampdragon-vanilla.js"></script>
index.html
var dragon = new VanillaDragon({onopen: onOpen, onchannelmessage: onChannelMessage}); !!function onChannelMessage(channels, message) { $('#polls').append('<a href="'+message.data.id+'" class="list-group-item">'+ message.data.question_text + '</a>'); } !!function onOpen(){ dragon.subscribe('questions-list', 'question', null, function(response) { //console.log(response); }, function(response) { console.log("Failed to subscribe"); }); ! dragon.getList('questions-list', {}, function(context, data){ console.log(data) for (i in data){ $('#polls').append('<a href="/polls/'+data[i].id+'" class="list-group-item">'+ data[i].question_text + '</a>'); } }, function(respone){ console.log("Could not get"); }); }
Access usersSWAMP_DRAGON_CONNECTION = ('swampdragon_auth.socketconnection.HttpDataConnection', '/data') !!# Access your users self.connection.get_user() !self.connection.user
pip install swampdragon-auth
https://github.com/jonashagstedt/swampdragon-auth
Deployingupstream swampdragon { server 127.0.0.1:9000; server 127.0.0.1:9001;
}
server { listen 80; server_name sd.myhost.tld; ! # WebSocket. location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Scheme $scheme; proxy_set_header Host $http_host; proxy_pass http://swampdragon; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } }
[program:mysite_swampdragon] command=/path/to/virtualenv/bin/python /path/to/project/manage.py socketserver 127.0.0.1:900%(process_num)01d user=myuser autostart=true autorestart=true stderr_logfile=/path/to/logfiles/sd.log stdout_logfile=/path/to/logfiles/sd.log stopsignal=INT process_name=%(program_name)s_%(process_num)01d numprocs=2
http://swampdragon.net/blog/deploying-swampdragon/
Linkshttp://swampdragon.net/
https://github.com/tornadoweb/tornado
https://github.com/sockjs/sockjs-client
http://redis.io/