Download - Django in the Real World
![Page 1: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/1.jpg)
django
Django in the Real World
James Bennett ● Jacob Kaplan-Mosshttp://b-list.org/ http://jacobian.org/
PyCon 2009http://jacobian.org/speaking/2009/real-world-django/
1
![Page 2: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/2.jpg)
So you’ve written a web app...
2
![Page 3: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/3.jpg)
Now what?
3
![Page 4: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/4.jpg)
API MeteringBackups & SnapshotsCountersCloud/Cluster Management Tools
Instrumentation/Monitoring FailoverNode addition/removal and hashingAutoscaling for cloud resources
CSRF/XSS ProtectionData Retention/ArchivalDeployment Tools
Multiple Devs, Staging, ProdData model upgradesRolling deploymentsMultiple versions (selective beta)Bucket TestingRollbacksCDN Management
Distributed File Storage
Distributed Log storage, analysisGraphingHTTP CachingInput/Output FilteringMemory CachingNon-relational Key StoresRate LimitingRelational StorageQueuesRate LimitingReal-time messaging (XMPP)Search
RangingGeo
ShardingSmart Caching
Dirty-table management
http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
4
![Page 5: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/5.jpg)
What’s on the plateStructuring for deployment
Testing
Production environments
Deployment
The “rest” of the web stack
Monitoring
Performance & tuning
5
![Page 6: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/6.jpg)
Writing applications you can deploy, and
deploy, and deploy...
6
![Page 7: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/7.jpg)
The extended extended remix!
7
![Page 8: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/8.jpg)
The fivefold pathDo one thing, and do it well.
Don’t be afraid of multiple apps.
Write for flexibility.
Build to distribute.
Extend carefully.
8
![Page 9: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/9.jpg)
1
9
![Page 10: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/10.jpg)
Do one thing, and do it well.
10
![Page 11: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/11.jpg)
Application == encapsulation
11
![Page 12: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/12.jpg)
Keep a tight focusAsk yourself: “What does this application do?”
Answer should be one or two short sentences
12
![Page 13: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/13.jpg)
Good focus“Handle storage of users and authentication of their identities.”
“Allow content to be tagged, del.icio.us style, with querying by tags.”
“Handle entries in a weblog.”
13
![Page 14: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/14.jpg)
Bad focus“Handle entries in a weblog, and users who post them, and their authentication, and tagging and categorization, and some flat pages for static content, and...”
The coding equivalent of a run-on sentence
14
![Page 15: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/15.jpg)
Warning signsA lot of very good Django applications are very small: just a few files
If your app is getting big enough to need lots of things split up into lots of modules, it may be time to step back and re-evaluate
15
![Page 16: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/16.jpg)
Warning signsEven a lot of “simple” Django sites commonly have a dozen or more applications in INSTALLED_APPS
If you’ve got a complex/feature-packed site and a short application list, it may be time to think hard about how tightly-focused those apps are
16
![Page 17: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/17.jpg)
Approach features skeptically
17
![Page 18: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/18.jpg)
Should I add this feature?What does the application do?
Does this feature have anything to do with that?
No? Guess I shouldn’t add it, then.
18
![Page 19: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/19.jpg)
2
19
![Page 20: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/20.jpg)
Don’t be afraid of multiple apps
20
![Page 21: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/21.jpg)
The monolith mindsetThe “application” is the whole site
Re-use is often an afterthought
Tend to develop plugins that hook into the “main” application
Or make heavy use of middleware-like concepts
21
![Page 22: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/22.jpg)
The Django mindsetApplication == some bit of functionality
Site == several applications
Tend to spin off new applications liberally
22
![Page 23: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/23.jpg)
Django encourages thisInstead of one “application”, a list: INSTALLED_APPS
Applications live on the Python path, not inside any specific “apps” or “plugins” directory
Abstractions like the Site model make you think about this as you develop
23
![Page 24: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/24.jpg)
Should this be its own application?
Is it completely unrelated to the app’s focus?
Is it orthogonal to whatever else I’m doing?
Will I need similar functionality on other sites?
Yes? Then I should break it out into a separate application.
24
![Page 25: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/25.jpg)
Unrelated featuresFeature creep is tempting: “but wouldn’t it be cool if...”
But it’s the road to Hell
See also: Part 1 of this talk
25
![Page 26: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/26.jpg)
I’ve learned this the hard way
26
![Page 27: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/27.jpg)
djangosnippets.orgOne application
Includes bookmarking features
Includes tagging features
Includes rating features
27
![Page 28: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/28.jpg)
Should be about four applications
28
![Page 29: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/29.jpg)
OrthogonalityMeans you can change one thing without affecting others
Almost always indicates the need for a separate application
Example: changing user profile workflow doesn’t affect user signup workflow. Make them two different applications.
29
![Page 30: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/30.jpg)
ReuseLots of cool features actually aren’t specific to one site
See: bookmarking, tagging, rating...
Why bring all this crap about code snippets along just to get the extra stuff?
30
![Page 31: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/31.jpg)
AdvantagesDon’t keep rewriting features
Drop things into other sites easily
31
![Page 32: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/32.jpg)
Need a contact form?
32
![Page 33: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/33.jpg)
urlpatterns += (‘’, (r’^contact/’, include(‘contact_form.urls’)),)
33
![Page 34: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/34.jpg)
And you’re done
34
![Page 35: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/35.jpg)
But what about...
35
![Page 36: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/36.jpg)
Site-specific needsSite A wants a contact form that just collects a message.
Site B’s marketing department wants a bunch of info.
Site C wants to use Akismet to filter automated spam.
36
![Page 37: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/37.jpg)
3
37
![Page 38: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/38.jpg)
Write for flexibility
38
![Page 39: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/39.jpg)
Common senseSane defaults
Easy overrides
Don’t set anything in stone
39
![Page 40: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/40.jpg)
Form processingSupply a form class
But let people specify their own if they want
40
![Page 41: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/41.jpg)
Specify a default template
But let people specify their own if they want
Templates
41
![Page 42: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/42.jpg)
You want to redirect after successful submission
Supply a default URL
But let people specify their own if they want
Form processing
42
![Page 43: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/43.jpg)
Provide a URLConf in the application
Use named URL patterns
Use reverse lookups: reverse(), permalink, {% url %}
URL best practices
43
![Page 44: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/44.jpg)
Working with modelsWhenever possible, avoid hard-coding a model class
Use get_model() and take an app label/model name string instead
Don’t rely on objects; use the default manager
44
![Page 45: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/45.jpg)
Working with modelsDon’t hard-code fields or table names; introspect the model to get those
Accept lookup arguments you can pass straight through to the database API
45
![Page 46: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/46.jpg)
Learn to love managersManagers are easy to reuse.
Managers are easy to subclass and customize.
Managers let you encapsulate patterns of behavior behind a nice API.
46
![Page 47: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/47.jpg)
Advanced techniquesEncourage subclassing and use of subclasses
Provide a standard interface people can implement in place of your default implementation
Use a registry (like the admin)
47
![Page 48: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/48.jpg)
The API your application exposes is
just as important as the design of the sites
you’ll use it in.
48
![Page 49: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/49.jpg)
In fact, it’s more important.
49
![Page 50: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/50.jpg)
Good API design“Pass in a value for this argument to change the behavior”
“Change the value of this setting”
“Subclass this and override these methods to customize”
“Implement something with this interface, and register it with the handler”
50
![Page 51: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/51.jpg)
Bad API design“API? Let me see if we have one of those...” (AKA: “we don’t”)
“It’s open source; fork it to do what you want” (AKA: “we hate you”)
def application(environ, start_response) (AKA: “we have a web service”)
51
![Page 52: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/52.jpg)
4
52
![Page 53: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/53.jpg)
Build to distribute
53
![Page 54: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/54.jpg)
So you did the tutorialfrom mysite.polls.models import Poll
mysite.polls.views.vote
include(‘mysite.polls.urls’)
mysite.mysite.bork.bork.bork
54
![Page 55: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/55.jpg)
Project coupling kills re-use
55
![Page 56: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/56.jpg)
You have to replicate that directory structure every time you re-use
Or you have to do gymnastics with your Python path
And you get back into the monolithic mindset
Why (some) projects suck
56
![Page 57: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/57.jpg)
A good “project”A settings module
A root URLConf module
And that’s it.
57
![Page 58: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/58.jpg)
AdvantagesNo assumptions about where things live
No tricky bits
Reminds you that it’s just another Python module
58
![Page 59: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/59.jpg)
It doesn’t even have to be one module
59
![Page 60: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/60.jpg)
ljworld.comworldonline.settings.ljworld
worldonline.urls.ljworld
And a whole bunch of reused apps in sensible locations
60
![Page 61: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/61.jpg)
Configuration is contextual
61
![Page 62: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/62.jpg)
What reusable apps look like
Single module directly on Python path (registration, tagging, etc.)
Related modules under a package (ellington.events, ellington.podcasts, etc.)
No project cruft whatsoever
62
![Page 63: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/63.jpg)
And now it’s easyYou can build a package with distutils or setuptools
Put it on the Cheese Shop
People can download and install
63
![Page 64: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/64.jpg)
Make it “packageable” even if it’s only for your
use
64
![Page 65: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/65.jpg)
General best practicesBe up-front about dependencies
Write for Python 2.3 when possible
Pick a release or pick trunk, and document that
But if you pick trunk, update frequently
65
![Page 66: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/66.jpg)
I usually don’t do default templates
66
![Page 67: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/67.jpg)
Be obsessive about documentation
It’s Python: give stuff docstrings
If you do, Django will generate documentation for you
And users will love you forever
67
![Page 68: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/68.jpg)
5
68
![Page 69: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/69.jpg)
Embracing and extending
69
![Page 70: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/70.jpg)
Don’t touch!Good applications are extensible without hacking them up.
Take advantage of everything an application gives you.
You may end up doing something that deserves a new application anyway.
70
![Page 71: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/71.jpg)
But this application wasn’t meant to be
extended!
71
![Page 72: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/72.jpg)
Use the Python (and the Django)
72
![Page 73: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/73.jpg)
Want to extend a view?If possible, wrap the view with your own code.
Doing this repetitively? Just write a decorator.
73
![Page 74: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/74.jpg)
Want to extend a model?You can relate other models to it.
You can write subclasses of it.
You can create proxy subclasses (in Django 1.1)
74
![Page 75: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/75.jpg)
Model inheritance is powerful.
With great power comes great
responsibility.
75
![Page 76: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/76.jpg)
Proxy modelsNew in Django 1.1.
Lets you add methods, managers, etc. (you’re extending the Python side, not the DB side).
Keeps your extensions in your code.
Avoids many problems with normal inheritance.
76
![Page 77: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/77.jpg)
Extending a formJust subclass it.
No really, that’s all :)
77
![Page 78: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/78.jpg)
Other tricksUsing signals lets you fire off customized behavior when particular events happen.
Middleware offers full control over request/response handling.
Context processors can make additional information available if a view doesn’t.
78
![Page 79: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/79.jpg)
But if you must make changes to someone
else’s code...
79
![Page 80: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/80.jpg)
Keep changes to a minimum
If possible, instead of adding a feature, add extensibility.
Then keep as much changed code as you can out of the original app.
80
![Page 81: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/81.jpg)
Stay up-to-dateYou don’t want to get out of sync with the original version of the code.
You might miss bugfixes.
You might even miss the feature you needed.
81
![Page 82: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/82.jpg)
Make sure your VCS is up to the job of
merging from upstream
82
![Page 83: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/83.jpg)
Be a good citizenIf you change someone else’s code, let them know.
Maybe they’ll merge your changes in and you won’t have to fork anymore.
83
![Page 84: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/84.jpg)
What if it’s my own code?
84
![Page 85: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/85.jpg)
Same principles applyMaybe the original code wasn’t sufficient. Or maybe you just need a new application.
Be just as careful about making changes. If nothing else, this will highlight ways in which your code wasn’t extensible to begin with.
85
![Page 86: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/86.jpg)
Further reading
86
![Page 87: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/87.jpg)
Testing
87
![Page 88: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/88.jpg)
“”
Tests are the Programmer’s stone, transmuting fear into
boredom.
— Kent Beck
88
![Page 89: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/89.jpg)
Hardcore TDD
89
![Page 90: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/90.jpg)
“”
I don’t do test driven development. I do
stupidity driven testing… I wait until I do something
stupid, and then write tests to avoid doing it again.
— Titus Brown
90
![Page 91: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/91.jpg)
Whatever happens, don’t let your test suite break thinking, “I’ll go back and fix this later.”
91
![Page 92: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/92.jpg)
Unit testing
Functional/behaviortesting
Browser testing
unittest
doctest
django.test.Client, Twill
Windmill, Selenium
92
![Page 93: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/93.jpg)
You need them all.
93
![Page 94: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/94.jpg)
Unit tests“Whitebox” testing
Verify the small functional units of your app
Very fine-grained
Familier to most programmers (JUnit, NUnit, etc.)
Provided in Python by unittest
94
![Page 95: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/95.jpg)
from django.test import TestCasefrom django.http import HttpRequestfrom django.middleware.common import CommonMiddlewarefrom django.conf import settings
class CommonMiddlewareTest(TestCase): def setUp(self): self.slash = settings.APPEND_SLASH; self.www = settings.PREPEND_WWW
def tearDown(self): settings.APPEND_SLASH = self.slash; settings.PREPEND_WWW = self.www
def _get_request(self, path): request = HttpRequest() request.META = {'SERVER_NAME':'testserver', 'SERVER_PORT':80} request.path = request.path_info = "/middleware/%s" % path return request def test_append_slash_redirect(self): settings.APPEND_SLASH = True request = self._get_request('slash') r = CommonMiddleware().process_request(request) self.assertEquals(r.status_code, 301) self.assertEquals(r['Location'], 'http://testserver/middleware/slash/')
95
![Page 96: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/96.jpg)
django.test.TestCaseFixtures.
Test client.
Email capture.
Database management.
Slower than unittest.TestCase.
96
![Page 97: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/97.jpg)
Easy to write & read.
Produces self-documenting code.
Great for cases that only use assertEquals.
Somewhere between unit tests and functional tests.
Difficult to debug.
Don’t always provide useful test failures.
Doctests
97
![Page 98: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/98.jpg)
class Template(object): """ Deal with a URI template as a class:: >>> t = Template("http://example.com/{p}?{‐join|&|a,b,c}") >>> t.expand(p="foo", a="1") 'http://example.com/foo?a=1' >>> t.expand(p="bar", b="2", c="3") 'http://example.com/bar?c=3&b=2' """
def parse_expansion(expansion): """ Parse an expansion ‐‐ the part inside {curlybraces} ‐‐ into its component parts. Returns a tuple of (operator, argument, variabledict)::
>>> parse_expansion("‐join|&|a,b,c=1") ('join', '&', {'a': None, 'c': '1', 'b': None}) >>> parse_expansion("c=1") (None, None, {'c': '1'}) """
def percent_encode(values): """ Percent‐encode a dictionary of values, handling nested lists correctly:: >>> percent_encode({'company': 'AT&T'}) {'company': 'AT%26T'} >>> percent_encode({'companies': ['Yahoo!', 'AT&T']}) {'companies': ['Yahoo%21', 'AT%26T']} """
98
![Page 99: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/99.jpg)
****************************************************File "uri.py", line 150, in __main__.parse_expansionFailed example: parse_expansion("c=1")Expected: (None, None, {'c': '2'})Got: (None, None, {'c': '1'})****************************************************
99
![Page 100: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/100.jpg)
a.k.a “Behavior Driven Development.”
“Blackbox,” holistic testing.
All the hardcore TDD folks look down on functional tests.
But it keeps your boss happy.
Easy to find problems, harder to find the actual bug.
Functional tests
100
![Page 101: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/101.jpg)
Functional testing toolsdjango.test.Client
webunit
Twill
...
101
![Page 102: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/102.jpg)
django.test.ClientTest the whole request path without running a web server.
Responses provide extra information about templates and their contexts.
102
![Page 103: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/103.jpg)
def testBasicAddPost(self): """ A smoke test to ensure POST on add_view works. """ post_data = { "name": u"Another Section", # inline data "article_set‐TOTAL_FORMS": u"3", "article_set‐INITIAL_FORMS": u"0", } response = self.client.post('/admin/admin_views/section/add/', post_data) self.failUnlessEqual(response.status_code, 302)
def testCustomAdminSiteLoginTemplate(self): self.client.logout() request = self.client.get('/test_admin/admin2/') self.assertTemplateUsed(request, 'custom_admin/login.html') self.assertEquals(request.context['title'], 'Log in')
103
![Page 104: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/104.jpg)
The ultimate in functional testing for web applications.
Run test in a web browser.
Can verify JavaScript, AJAX; even design.
Test your site across supported browsers.
Web browser testing
104
![Page 105: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/105.jpg)
Browser testing toolsSelenium
Windmill
105
![Page 106: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/106.jpg)
Exotic testingStatic source analysis.
Smoke testing (crawlers and spiders).
Monkey testing.
Load testing.
...
106
![Page 107: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/107.jpg)
107
![Page 108: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/108.jpg)
Further resourcesTalks here at PyCon! http://bit.ly/pycon2009-testing
Don’t miss the testing tools panel (Sunday, 10:30am)
Django testing documentation http://bit.ly/django-testing
Python Testing Tools Taxonomy http://bit.ly/py-testing-tools
108
![Page 109: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/109.jpg)
Deployment
109
![Page 110: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/110.jpg)
Deployment should...Be automated.
Automatically manage dependencies.
Be isolated.
Be repeatable.
Be identical in staging and in production.
Work the same for everyone.
110
![Page 111: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/111.jpg)
apt/yum/... virtualenv Capistrano
easy_install zc.buildout Fabric
pip Puppet
zc.buildout
Dependency management Isolation Automation
111
![Page 112: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/112.jpg)
Let the live demo begin(gulp)
112
![Page 113: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/113.jpg)
Building your stack
113
![Page 114: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/114.jpg)
http://danga.com/words/
LiveJournal Backend: Today(Roughly.)
User DB Cluster 1
uc1a uc1b
User DB Cluster 2
uc2a uc2b
User DB Cluster 3
uc3a uc3b
User DB Cluster N
ucNa ucNb
Job Queues (xN)
jqNa jqNb
Memcached
mc4
mc3
mc2
mcN
...
mc1
mod_perl
web4
web3
web2
webN
...
web1
BIG-IP
bigip2
bigip1perlbal (httpd/proxy)
proxy4
proxy3
proxy2
proxy5
proxy1
Global Database
slave1
master_a master_b
slave2 ... slave5
MogileFS Database
mog_a mog_b
Mogile Trackers
tracker3tracker1
Mogile Storage Nodes
...
sto2
sto8
sto1
net.
djabberd
djabberd
djabberd
gearmand
gearmand1
gearmandN
“workers”
gearwrkN
theschwkN
slave1 slaveN
3
Brad Fitzpatrik, http://danga.com/words/2007_06_usenix/
114
![Page 115: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/115.jpg)
server
django
database
media
115
![Page 116: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/116.jpg)
Application serversApache + mod_python
Apache + mod_wsgi
Apache/lighttpd + FastCGI
SCGI, AJP, nginx/mod_wsgi, ...
116
![Page 117: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/117.jpg)
Use mod_wsgi
117
![Page 118: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/118.jpg)
WSGIScriptAlias / /home/mysite/mysite.wsgi
118
![Page 119: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/119.jpg)
import os, sys
# Add to PYTHONPATH whatever you needsys.path.append('/usr/local/django')
# Set DJANGO_SETTINGS_MODULEos.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'
# Create the application for mod_wsgiimport django.core.handlers.wsgiapplication = django.core.handlers.wsgi.WSGIHandler()
119
![Page 120: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/120.jpg)
A brief digression regarding the question
of scale
120
![Page 121: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/121.jpg)
Does this scale?
server
django
database
media
Maybe!121
![Page 122: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/122.jpg)
122
![Page 123: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/123.jpg)
Real-world example
Database A
175 req/s
Database B
75 req/s
123
![Page 124: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/124.jpg)
Real-world example
http://tweakers.net/reviews/657/6
124
![Page 125: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/125.jpg)
database server
web server
django
database
media
125
![Page 126: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/126.jpg)
Why separate hardware?Resource contention
Separate performance concerns
0 → 1 is much harder than 1 → N
126
![Page 127: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/127.jpg)
DATABASE_HOST = '10.0.0.100'
FAIL127
![Page 128: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/128.jpg)
Proxy between web and database layers
Most implement hot fallover and connection pooling
Some also provide replication, load balancing, parallel queries, connection limiting, &c
DATABASE_HOST = '127.0.0.1'
Connection middleware
128
![Page 129: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/129.jpg)
Connection middlewarePostgreSQL: pgpool
MySQL: MySQL Proxy
Database-agnostic: sqlrelay
Oracle: ?
129
![Page 130: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/130.jpg)
media server
database server
web server
django
database
media
130
![Page 131: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/131.jpg)
Media server traitsFast
Lightweight
Optimized for high concurrency
Low memory overhead
Good HTTP citizen
131
![Page 132: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/132.jpg)
Media serversApache?
lighttpd
nginx
132
![Page 133: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/133.jpg)
media server
database server
web server
django
database
media
The absolute minimum
133
![Page 134: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/134.jpg)
web server
django
database
media
The absolute minimum
134
![Page 135: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/135.jpg)
media server
web server cluster
load balancer
proxy
django
media
djangodjango
database server
database
135
![Page 136: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/136.jpg)
Why load balancers?
136
![Page 137: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/137.jpg)
Load balancer traitsLow memory overhead
High concurrency
Hot fallover
Other nifty features...
137
![Page 138: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/138.jpg)
Load balancersApache + mod_proxy
perlbal
nginx
138
![Page 139: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/139.jpg)
CREATE POOL mypool POOL mypool ADD 10.0.0.100 POOL mypool ADD 10.0.0.101
CREATE SERVICE mysite SET listen = my.public.ip SET role = reverse_proxy SET pool = mypool SET verify_backend = on SET buffer_size = 120kENABLE mysite
139
![Page 140: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/140.jpg)
you@yourserver:~$ telnet localhost 60000
pool mysite add 10.0.0.102OK
nodes 10.0.0.10110.0.0.101 lastresponse 123798744910.0.0.101 requests 9755456310.0.0.101 connects 12924243510.0.0.101 lastconnect 123798744910.0.0.101 attempts 12924474310.0.0.101 responsecodes 200 35810.0.0.101 responsecodes 302 1410.0.0.101 responsecodes 207 9910.0.0.101 responsecodes 301 1110.0.0.101 responsecodes 404 1810.0.0.101 lastattempt 1237987449
140
![Page 141: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/141.jpg)
media server cluster
web server cluster
load balancing cluster
proxy
django djangodjango
database server cluster
database
mediamediaproxyproxy
database database
cache cluster
cachecache
141
![Page 142: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/142.jpg)
“Shared nothing”
142
![Page 143: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/143.jpg)
BALANCE = None
def balance_sheet(request): global BALANCE if not BALANCE: bank = Bank.objects.get(...) BALANCE = bank.total_balance() ...
FAIL143
![Page 144: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/144.jpg)
Global variables are right out
144
![Page 145: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/145.jpg)
from django.cache import cache
def balance_sheet(request): balance = cache.get('bank_balance') if not balance: bank = Bank.objects.get(...) balance = bank.total_balance() cache.set('bank_balance', balance)
... WIN145
![Page 146: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/146.jpg)
def generate_report(request): report = get_the_report() open('/tmp/report.txt', 'w').write(report) return redirect(view_report)
def view_report(request): report = open('/tmp/report.txt').read() return HttpResponse(report)
FAIL146
![Page 147: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/147.jpg)
Filesystem? What filesystem?
147
![Page 148: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/148.jpg)
Further readingCal Henderson, Building Scalable Web Sites (O’Reilly, 2006)
John Allspaw, The Art of Capacity Planning (O’Reilly, 2008)
http://kitchensoap.com/
http://highscalability.com/
148
![Page 149: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/149.jpg)
Monitoring
149
![Page 150: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/150.jpg)
GoalsWhen the site goes down, know it immediately.
Automatically handle common sources of downtime.
Ideally, handle downtime before it even happens.
Monitor hardware usage to identify hotspots and plan for future growth.
Aid in postmortem analysis.
Generate pretty graphs.
150
![Page 151: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/151.jpg)
Availability monitoring principlesCheck services for availability.
More then just “ping yoursite.com.”
Have some understanding of dependancies (if the db is down, I don’t need to also hear that the web servers are down.)
Notify the “right” people using the “right” methods, and don’t stop until it’s fixed.
Minimize false positives.
Automatically take action against common sources of downtime.
151
![Page 152: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/152.jpg)
Availability monitoring tools
Internal tools
Nagios
Monit
Zenoss
...
External monitoring tools
152
![Page 153: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/153.jpg)
Usage monitoringKeep track of resource usage over time.
Spot and identify trends.
Aid in capacity planning and management.
Look good in reports to your boss.
153
![Page 154: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/154.jpg)
Usage monitoring toolsRRDTool
Munin
Cacti
Graphite
154
![Page 155: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/155.jpg)
155
![Page 156: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/156.jpg)
156
![Page 157: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/157.jpg)
Record information about what’s happening right now.
Analyze historical data for trends.
Provide postmortem information after failures.
Logging and log analysis
157
![Page 158: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/158.jpg)
Logging toolsprint
Python’s logging module
syslogd
158
![Page 159: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/159.jpg)
Log analysisgrep | sort | uniq ‐c | sort ‐rn
Load log data into relational databases, then slice & dice.
OLAP/OLTP engines.
Splunk.
Analog, AWStats, ...
Google Analytics, Mint, ...
159
![Page 160: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/160.jpg)
Performance (and when to care about it)
160
![Page 161: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/161.jpg)
Ignore performanceFirst, get the application written.
Then, make it work.
Then get it running on a server.
Then, maybe, think about performance.
161
![Page 162: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/162.jpg)
Code isn’t “fast” or “slow” until it’s been
written.
162
![Page 163: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/163.jpg)
Code isn’t “fast” or “slow” until it works.
163
![Page 164: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/164.jpg)
Code isn’t “fast” or “slow” until it’s actually
running on a server.
164
![Page 165: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/165.jpg)
Optimizing codeMost of the time, “bad” code is obvious as soon as you write it. So don’t write it.
165
![Page 166: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/166.jpg)
Low-hanging fruitLook for code doing lots of DB queries -- consider caching, or using select_related()
Look for complex DB queries, and see if they can be simplified.
166
![Page 167: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/167.jpg)
The DB is the bottleneckAnd if it’s not the DB, it’s I/O.
Everything else is typically negligible.
167
![Page 168: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/168.jpg)
Find out what “slow” means
Do testing in the browser.
Do testing with command-line tools like wget.
Compare the results, and you may be surprised.
168
![Page 169: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/169.jpg)
Sometimes, perceived “slowness” is actually
on the front end.
169
![Page 170: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/170.jpg)
Read Steve Souders’ book
170
![Page 171: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/171.jpg)
YSlowhttp://developer.yahoo.com/yslow/
171
![Page 172: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/172.jpg)
What to do on the server side
First, try caching.
Then try caching some more.
172
![Page 173: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/173.jpg)
The secret weaponCaching turns less hardware into more.
Caching puts off buying a new DB server.
173
![Page 174: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/174.jpg)
But caching is a trade-off
174
![Page 175: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/175.jpg)
Things to considerCache for everybody? Or only for people who aren’t logged in?
Cache everything? Or only a few complex views?
Use Django’s cache layer? Or an external caching system?
175
![Page 176: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/176.jpg)
Not all users are the sameMost visitors to most sites aren’t logged in.
CACHE_MIDDLEWARE_ANONYMOUS_ONLY
176
![Page 177: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/177.jpg)
Not all views are the sameYou probably already know where your nasty DB queries are.
cache_page on those particular views.
177
![Page 178: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/178.jpg)
Site-wide cachesYou can use Django’s cache middleware to do this...
Or you can use a proper caching proxy (e.g., Squid, Varnish).
178
![Page 179: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/179.jpg)
External cachesWork fine with Django, because Django just uses HTTP’s caching headers.
Take the entire load off Django -- requests never even hit the application.
179
![Page 180: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/180.jpg)
When caching doesn’t cut it
180
![Page 181: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/181.jpg)
Throw money at your DB first
181
![Page 182: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/182.jpg)
Web server improvements
Simple steps first: turn off Keep-Alive, etc.
Consider switching to a lighter-weight web server (e.g., nginx) or lighter-weight system (e.g., from mod_python to mod_wsgi).
182
![Page 183: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/183.jpg)
Database tuningWhole books can be written on DB performance tuning
183
![Page 184: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/184.jpg)
Using MySQL?
184
![Page 185: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/185.jpg)
Using PostgreSQL?http://www.revsys.com/writings/postgresql-performance.html
185
![Page 186: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/186.jpg)
Learn how to diagnoseIf things are slow, the cause may not be obvious.
Even if you think it’s obvious, that may not be the cause.
186
![Page 187: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/187.jpg)
Build a toolkitPython profilers: profile and cProfile
Generic “spy on a process” tools: strace, SystemTap, and dtrace.
Django debug toolbar (http://bit.ly/django-debug-toolbar)
187
![Page 188: Django in the Real World](https://reader034.vdocuments.net/reader034/viewer/2022050918/540e36418d7f72747e8b4cae/html5/thumbnails/188.jpg)
Shameless plug
http://revsys.com/
188