Download - Bento lunch talk
弁当 (BENTO)A packaging solution for python software
WHAT’S PACKAGING ?•Whatever that has to happen between svn/git/etc... and the
end-user
•Different end-users, different workflows: not just tarballs
(maybe those guys don’t need packaging)
DISTUTILS
•De-facto solution since ±2000
•Works well for simple packages:
• Setuptools/distribute
• add package dependencies
DISTUTILS ISSUES
•Make one’s head hurt:
•Hard to extend
•Fragile codebase
•Reliability issues
BENTO
•Declarative description of package: metadata inspection is simple
• Flexible installation scheme: install any file anywhere
• Layered internal architecture: easy to extend
• Scale down and up: simpler for small packages, more flexible for big ones
A SIMPLE BENTO FILE
Name: fooVersion: 0.1Description: Package description.
Library: Packages:! foo, foo.core
BENTOMAKER
• Command-line interface to bento
$ bentomaker configure$ bentomaker build$ bentomaker install# Or more simply$ bentomaker install
ADDING DATA-FILES
Datafiles: pdf_documentation TargetDir: $pdfdoc Files: main.pdf
$ bentomaker configure --pdfdoc=/usr/pdfdoc
# ant-like glob format supportedDatafiles: pdf_documentation TargetDir: $pdfdoc Files: doc/**/*.pdf
CUSTOM INSTALL PATHS
Path: foodir Description: foo directory Default: $datadir/foo
Datafiles: fubar-doc TargetDir: $foodir Files: main.foo
$ bentomaker configure --foodir=/usr/pdfdoc
HOOK FILES
• Can “hook” python files into the build process
from bento.commands import [email protected] hello(context): print "hello"
• Hook are pure python: no auto-import, etc...
$ bentomaker hellohello
FITTING IN THE PYTHON ECOSYSTEM
# Convert a setup.py to bento.info$ bentomaker convert
# setup.pyimport setuptoolsimport bento.distutilsbento.distutils.monkey_patch()# grab metadata from bento.infosetuptools.setup()
Convert distutils package to bento
Distutils compatibility layer (pip-installable !)
INTERNALS
DESIGN PRINCIPLES
• Usable as a library (still a few singleton to remove...)
• Low-coupling of commands: new package format should not care about building phase
•Driven by real packages (numpy, scipy, twisted, ipython, etc...)
CODE ORGANIZATION• bento/core
• ply-based parser (bento/core/parser)
• python representation of bento.info
• bento/commands
• commands and contexts
• bento/distutils
• distutils compat layer
COMMANDS AND CONTEXTS
Context Commandself.run(context)
• Contexts encapsulate a command ‘environment’• Context-command relationship is overridable (hook)• e.g. mechanism to interact with 3rd party build tools
self.pkgself.cmd_argvself.run_node
MORE ABOUT COMMANDS
• Command options defined externally: any command can query any other’s options
• Commands dependencies resolved at runtime
@hooks.commanddef hello(context): print "hello"@hooks.commanddef goodbye(context): print "goodbye"@hooks.startupdef startup(context): context.set_before("goodbye", "hello")
BUILD MANIFEST
• Goal: decouple build and install
• format: a json file containing info about built parts
• Inspired by cabal (‘distutils’ for haskell)
• build command produces a build manifest
• install only knows about build manifest
• Install only install built files
NODE• Every file is represented internally as a node (waf concept)
• nodes are a in-memory representation of the fs
• gives reliable relative paths between files
sdir = os.getcwd()bdir = os.path.join(os.getcwd(), "build")root = create_root_with_source_tree(sdir, bdir)top_node = root.find_node(sdir)build_node = root.find_node(bdir)print top_node.path_from(build_node)
SOME NEAT FEATURES (1)
• Using metadata in the packaged software
Name: fooVersion: 1.0MetaTemplateFile: foo.py.in
# foo.py.inname = $NAMEversion = $VERSION
# foo.py.inname = "foo"version = "1.0"
SOME NEAT FEATURES (2)
• Sane way to load data files
Name: fooVersion: 1.0ConfigPy: __config.py
# __config.pyBINDIR = r”/usr/bin”...
SOME NEAT FEATURES (3)
•Out of tree build (bye bye source tree pollution)
# Convert a setup.py to bento.info$ bentomaker --bento-info=../source_dir/bento.info
A FEW NUMBERS
• bento: ± 8000 LOC (not including ± 2500 LOC for tests)
• numpy.distutils: ±10000 LOC
• distutils: ± 10000 LOC
• numpy: ±2000 LOC for distutils vs ±1000 LOC for bento
• scipy: ±1600 LOC vs ±1100 LOC
DEMO ON NUMPY
WHERE TO GET IT ?
• Getting bento on github: http://github.com/cournape/Bento.git
• Bento doc: http://cournape.github.com/Bento
• Bento mailing-list: [email protected]