dynamic code patterns: extending your applications with plugins

Dynamic Code Patterns:Extending Your

Applications with PluginsDoug Hellmann – @doughellmann – PyCon 2013

1Saturday, March 16, 13

What is a “Plugin”?

• Loaded Dynamically

• Extends Core

• Possibly Unknown Source

2Saturday, March 16, 13

Why Plugins?

• Better API Abstraction

• Reduce Core Dependencies

• Strategy Pattern

• Visitor Pattern

• Indirect Code Contributions

3Saturday, March 16, 13

• OpenStack Metering

• Measures Clouds

• Varied Billing Requirements

• Deployers Extend and Customize

Ceilometer

4Saturday, March 16, 13

Ceilometer Design

ComputeImagesVolumesNetworkObjects

NotificationMessage Bus

EventListener

ComputePollsters

CentralPollsters

Collector

Meter DataMessage Bus

5Saturday, March 16, 13

Ceilometer Design

ComputeImagesVolumesNetworkObjects

NotificationMessage Bus

EventListener

ComputePollsters

CentralPollsters

Collector

Meter DataMessage Bus

5Saturday, March 16, 13

Ceilometer Design

ComputeImagesVolumesNetworkObjects

NotificationMessage Bus

EventListener

ComputePollsters

CentralPollsters

Collector

Meter DataMessage Bus

5Saturday, March 16, 13

Ceilometer Design

ComputeImagesVolumesNetworkObjects

NotificationMessage Bus

EventListener

ComputePollsters

CentralPollsters

Collector

Meter DataMessage Bus

5Saturday, March 16, 13

Ceilometer Design

ComputeImagesVolumesNetworkObjects

NotificationMessage Bus

EventListener

ComputePollsters

CentralPollsters

Collector

Meter DataMessage Bus

5Saturday, March 16, 13

Ceilometer Design

ComputeImagesVolumesNetworkObjects

NotificationMessage Bus

EventListener

ComputePollsters

CentralPollsters

Collector

Meter DataMessage Bus

5Saturday, March 16, 13

Ceilometer Design

ComputeImagesVolumesNetworkObjects

NotificationMessage Bus

EventListener

ComputePollsters

CentralPollsters

Collector

Meter DataMessage Bus

5Saturday, March 16, 13

Ceilometer Design

ComputeImagesVolumesNetworkObjects

NotificationMessage Bus

EventListener

ComputePollsters

CentralPollsters

Collector

Meter DataMessage Bus

5Saturday, March 16, 13

Ceilometer Design

ComputeImagesVolumesNetworkObjects

NotificationMessage Bus

EventListener

ComputePollsters

CentralPollsters

Collector

Meter DataMessage Bus

5Saturday, March 16, 13

Ceilometer Plugins

• Message Bus

• Receiving Notifications

• Polling Compute

• Polling Other

• Storage

6Saturday, March 16, 13

Research• Blogofile

• Sphinx

• Mercurial

• cliff

• virtualenvwrapper

• Nose

• Trac

• Django

• Pyramid

• SQLAlchemy

• Diamond

• Nova

7Saturday, March 16, 13

Discovery

Explicit Scan

File

Import Reference

Mercurial DiamondBlogofile

Mercurial DjangoPyramidSphinxNova

TracNosecliff

virtualenvwrapper

8Saturday, March 16, 13

Enabling

Explicit Implicit

DjangoPyramid

SQLAlchemyBlogofileMercurial

TracSphinx

virtualenvwrappercliff

9Saturday, March 16, 13

Importing

Custom pkg_resources

DjangoPyramidSphinx

DiamondNova Nose

SQLAlchemyBlogofile

TracNose

SQLAlchemyBlogofile

cliffvirtualenvwrapper

10Saturday, March 16, 13

Integration

Fine Coarse

Prompt

Inspect

Nosevirtualenvwrapper

SphinxTrac

cliffDiamond

MercurialBlogofilePyramidDjango

11Saturday, March 16, 13

Integration

Fine Coarse

Prompt

Inspect

Nosevirtualenvwrapper

SphinxTrac

cliffDiamond

MercurialBlogofilePyramidDjango

12Saturday, March 16, 13

API Enforcement

Convention Base Class / Interface

BlogofileMercurialDjangoSphinx

Pyramidvirtualenvwrapper

Nose (optional)Trac (interface)

Diamondcliff (abc)

13Saturday, March 16, 13

Invocation

Driver Dispatcher Iterator

SQLAlchemyNova

MercurialDjangoPyramid

cliff

NoseDiamond

virtualenvwrapper

14Saturday, March 16, 13

Discovery / Importing

• Entry Points

• Distribute and pkg_resources

• Be Consistent

15Saturday, March 16, 13

Enabling

• Explicit disabling

• Automatic disabling

16Saturday, March 16, 13

Integration

• Fine

• Inspect

• Application Owns Relationship

17Saturday, March 16, 13

API Enforcement

• Abstract Base Classes

• Duck Typing

18Saturday, March 16, 13

Invocation

• Storage – Driver

• Notifications – Dispatcher

• Pollsters – Iterator

19Saturday, March 16, 13

stevedore

• Implements Patterns

• Wraps pkg_resources

20Saturday, March 16, 13

NamedExtensionManager

• Multiple Plugins

• Only Loads Named Plugins

• map()

21Saturday, March 16, 13

EnabledExtensionManager

• Multiple Plugins

• Checks Each with Function on Load

• map()

22Saturday, March 16, 13

DispatchExtensionManager

• Multiple Plugins

• Invokes Subset on map()

23Saturday, March 16, 13

DriverManager

• Single Plugin

• Direct Access

24Saturday, March 16, 13

Dynamic Code Patterns:Extending Your

Applications with PluginsDoug Hellmann – @doughellmann – PyCon 2013

http://packages.python.org/distribute/pkg_resources.html

https://github.com/dreamhost/stevedore

https://launchpad.net/ceilometer

http://doughellmann.com

25Saturday, March 16, 13