python developer's daily routine
TRANSCRIPT
Python Developer'sDaily Routine
Maxim Avanov (github.com/avanov)
Project structure
Project structure./chebit
|-- chebit
| -- __init__.py
-- setup.py
3
setup.pyfrom setuptools import setup, find_packages
setup(
name='Chebit',
version='0.0.1',
packages=find_packages(),
test_suite='tests',
tests_require=['pytest', 'coverage'],
install_requires=['Pyramid==1.5a2']
)
4
Bootstrapping$ python setup.py install
$ python setup.py develop
$ python setup.py test
$ python setup.py regtister sdist [bdist_egg] upload
5
Package
management
pip commands$ pip install Pyramid
$ pip install Pyramid --upgrade
$ pip install Pyramid==1.5a2 --upgrade
$ pip uninstall Pyramid
7
pip commands$ pip freeze
Babel==1.3
Chebit==0.0.1
CoffeeScript==1.0.8
Jinja2==2.7.1
Mako==0.9.0
...
8
pip-tools$ pip-review
requests==0.13.4 available (you have 0.13.2)
$ pip-review --auto
... <pip install output>
$ pip-review --interactive
requests==0.14.0 available (you have 0.13.2)
Upgrade now? [Y]es, [N]o, [A]ll, [Q]uit y
...
9
Cookiecutter• Generates initial project files from defined templates;
• File names are also templates;
• Transparently works with remote VCS.
10
Cookiecutter template./chebit-template
|-- cookiecutter.json
-- {{cookiecutter.project_name}}
|-- {{cookiecutter.project_name}}
| -- __init__.py
-- setup.py
11
cookiecutter.json{
"project_name": "myproject",
"project_version": "0.0.1"
}
12
Cookiecutter in action$ cookiecutter {project_template_path}
project_name (default is "myproject")? mynew
project_version (default is "0.0.1")? 0.1.1
$ tree ./mynew
./mynew
|-- mynew
| -- __init__.py
-- setup.py
13
We'd like to
isolate projectenvironments
virtualenv$ virtualenv ~/venv/{project_name}
$ source ~/venv/{project_name}/bin/activate
What if we need another Python version?$ virtualenv ~/venv/{project_name} -p python3
15
Still not happy
enough!
Introducing pyenv
pyenv - 5 steps to happiness$ git clone git://github.com/yyuu/pyenv.git ~/pyenv
$ echo 'export PYENV_ROOT="$HOME/pyenv"' >> ~/.bashrc
$ echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
$ echo 'eval "$(pyenv init -)"' >> ~/.bashrc
$ exec $SHELL
18
pyenv - Rise and Shine!$ pyenv install -l
$ pyenv install 2.7.6
$ pyenv install 3.3.3
$ pyenv install pypy-2.2.1
$ pyenv rehash
19
The Happiness$ pyenv versions
$ pyenv version
$ pyenv local 2.7.6 3.3.3
$ pyenv global 3.3.3
$ pyenv shell pypy-2.2.1
$ pyenv whence 2to3
20
...
$ which python/home/ghostwriter/pyenv/shims/python
$ pythonPython 2.7.3 (87aa9de10f9c, Nov 24 2013, 18:48:13)[PyPy 2.2.1 with GCC 4.6.3] on linux2>>>>
21
virtualenv meets
pyenv
pyenv-virtualenv
$ git clone git://github.com/yyuu/pyenv-virtualenv.git ~/pyenv/plugins/pyenv-virtualenv
$ pyenv virtualenv pypy-2.2.1 ~/venv/{project_name}
23
pyenv-virtualenv in action$ pyenv virtualenvs
{project_name} (created from /home/ghostwriter/pyenv/versions/pypy-2.2.1)
$ source ~/pyenv/versions/{project_name}/bin/activate
24
Well done!
Now, what about
other components?
Virtualization
Vagrant• Written in Ruby
• Relies on VirtualBox
• Runs on Linux / OS X / Win
28
Juju• Written in Python / Go
• Relies on LXC
• Linux only
29
Docker• Written in Go
• Relies on LXC and AUFS
• Linux only
30
Vagrant's Success Story...1. Put a Vagrantfile into the project root
2. $ vagrant up
3. $ vagrant ssh
4. Profit!
31
Vagrantfile
Vagrant.configure("2") do |config|
config.vm.box = "project_name"
config.vm.box_url = "http://files.vagrantup.com/precise64.box"
config.vm.network :private_network, ip: "192.168.33.100"
end
32
...Vagrant's Success Story1. Put a Vagrantfile into the project root
2. $ vagrant up
3. $ vagrant ssh
4. Profit! ..???
5. Deployment
33
Deployment toolsChef (Ruby)
Puppet (Ruby)
Ansible (Python)
Salt (Python)
34
Ansible Success Story1. Specify an inventory file
2. Specify a playbook
3. $ ansible-playbook ./playbook.yml -i ./hosts
35
Ansible inventory[develop-vm]
192.168.33.100
36
Ansible playbook---
-
hosts: develop-vm
sudo: yes
tasks:
-
name: ensure apt cache is up to date
action: apt update_cache=yes
http://www.ansibleworks.com/docs/modules.html
37
Ansible meets Vagrant
$ ansible-playbook ./playbook.yml -i ./hosts -v -u vagrant -c paramiko --private-key=$HOME/.vagrant.d/insecure_private_key
or
$ vagrant provision
38
Ansible - Vagrant Integration# Vagrantfile
Vagrant.configure("2") do |config|
# ...
config.vm.provision :ansible do |ansible|
ansible.playbook = "./playbook.yml"
ansible.inventory_path = "./hosts"
ansible.verbose = "v"
end
end
39
Python & Web
Web Server Gateway InterfaceStandardized by PEP 333, PEP 3333
def application(environ, start_response):
start_response('200 OK', [('Content-Type', 'text/plain')])
yield 'Hello World\n'
41
WSGI Servers• CherryPy WSGI Server
• gunicorn
• tornado.wsgi
• uWSGI
• Waitress
http://nichol.as/benchmark-of-python-web-servers
42
Frameworks
Heavyweights• Django
• Twisted
• web2py
• Zope
44
Middleweights• CherryPy
• Pyramid
• Tornado
• TurboGears
45
Flyweights• Bottle
• Flask
• Webpy
46
Database Access
ORM• SQLAlchemy ORM
• Django ORM (Django only)
• Pony ORM
• Peewee
48
Something else• Native DB-API 2.0 (standardized with PEP 249)
• SQLAlchemy's raw queries
• mosql
49
SQL DDL Versioning & Data Migration• South (Django only)
• Alembic (SQLAlchemy)
• Pyrseas
50
Background task
processing
Celery• Out of the box solution for complex cases;
• Multiple brokers - RabbitMQ, Redis, RDBMS, SQS, MongoDB etc;
• Tasks scheduler;
• Results backends.
52
RQ (Redis Queue)• Designed to have a low barrier to enter;
• Uses Redis' Pub/Sub feature.
53
Process managers
Process managers• Supervisor
• Circus.io
55
Sentry
Testing tools• unittest
• py.test
• nose
• WebTest
• coverage
• tox
60
Thank you!