exceptable - exception propagation for pg

Exceptable

Tuesday, October 20, 2009

Exception Propagation, in PostgreSQL and PythonAurynn Shaw, Commandprompt, Inc.

PostgreSQL Conference West, 2009


Background

• Large, enterprise Python + Pylons application

• Simpycity ORM utilizes significant stored procedures

• Strong procedural interfaces


Stored Procedures are Rare

• Most people don’t use Stored Procedures.


The Present SystemWherein

• A Database Query

• A Stored Procedure

• Or, A Database Query

• Exceptions Galore


A Database Query

•The App queries the Database


A Database Query

• The App queries the Database

• Any variety of query


A Database Query

>>> c.Raw("SELECT * FROM auth_only")()

Traceback (most recent call last):

... <SNIP> ...

psycopg2.ProgrammingError: permission denied for relation auth_only


A Database Query

• The App queries the Database

• Any variety of query

• Or A Stored Procedure


Or a Stored Procedure

CREATE OR REPLACE FUNCTION except_test() RETURNS VOID AS $$

BEGIN

RAISE EXCEPTION 'Test!';

END;

$$ LANGUAGE PLPGSQL;

>>> c.Function("except_test")()

Traceback (most recent call last):

... <SNIP> ...

psycopg2.InternalError: Test!


Beget an Exception

• Stored procedures raise InternalErrors

• Every procedural exception becomes this

• Permissions violations raise ProgrammingErrors

• I haven’t tested if this is all PG exceptions.


Seriously?

• This is what we have to work with?


Why this is ImportantWherein

• Why?

• Defining your APIs

• Separation of Concerns

• Procedural Spaghetti


Why?

• Limited information to propagate upwards

• Stored Procedures are Harder to use

• Permissions violations are difficult to detect


API Undefined

• Exceptions are part of your API


API Undefined


• Easily overlooked


API Undefined


• Easily overlooked

• Delineations will make life easier


Separation of Concerns

• The Database is for Database Logic


Separation of Concerns

• The Database is for Database Logic

• Harder to write Data Logic


Spaghetti

• InternalErrors everywhere

Early version of our app didn’t have Exceptable - we were left catching InternalErrors and guessing at what the error was, based on timing.


Spaghetti


• Insufficiency of Information


Spaghetti


• Insufficiency of Information

• Considerable Repeated Code


A Saving Grace

• Violating Procedure Signatures -> Python DataError


A Better DealWherein

• Database API

• Easy Python implementation

• Universality

• Exceptable


Database API

• Easier to define an API


Database API


• The DB becomes part of that API


Database API



• Simple Stored Procedure interface


Database API



• Simple Stored Procedure interface

• Easily declare new Exceptions


Wonderfully Python

• A Simple Decorator


Simply Decorated

from exceptable.exceptable import Except

base = Except()

@base

def db_function():

pass


Wonderfully Python

• A Simple Decorator

• Catches and re-emits Exceptions

• The core of Exceptable

• Easy to Integrate - 2 lines, in Simpycity


Universality

• Exceptable Procedures never change

• DB logic doesn’t change

• Application support is Easy


Exceptable

• More Pythonic Database Access

• Better exceptions means better app flow


Example CodeWherein

• The DB Library

• The Application Implementation

• Catching Permissions Violations

• PostgreSQL 8.4


To Start,

CREATE TABLE exceptions (

name text primary key,

description text not null,

parent text references exceptions(name)

);

INSERT INTO exceptions VALUES ('Exception', 'Base exception',NULL);

INSERT INTO exceptions VALUES ('NotFoundException', 'Could not find specified record', 'Exception');


Which leads to

CREATE OR REPLACE FUNCTION not_found ( in_reason TEXT) RETURNS VOID as $body$

SELECT exceptaple.raise( 'NotFoundException', $1 );

$body$ LANGUAGE SQL;


Application Level

• Easy to Query the Exception tables


Application Level

• Easy to Query the Exception tables

• Easy to set up a new library

• Python took 50 lines


Our Python Example

• The Exceptable decorator is easy to set up

• Designed for DB-API integration


In the Application

base = Except(InternalError, { 'Exception': Exception, 'NotFoundException': NotFoundError,})


Our Python Example



• User-defined


User Definitions

base = Except(InternalError, {

'PermissionError': PermissionError,

'UnknownUser': UnknownUserError,

'NotFoundException': NotFoundError,

})


Our Python Example



• User-defined, and soon, table introspection


base is a decorator

@basedef db_api(query): con = db.connect(conn_string) cur = con.cursor() return cur(query)


Which leads to

try: rs = db_api(‘select * from test_api()’)except NotFoundError, e: # A hah! A usable error! pass


As Opposed To

try: rs = db_api(‘select * from test_api()’)except InternalError, e: if “NotFoundException” in str(e): raise NotFoundError(str(e)) elif “PermissionsError” in str(e): raise PermissionsError(str(e))


Our Python Example



• User-defined, and soon, table introspection

• Existing decorators can be expanded easily


Grow, my Pretties!

class NotNullError(BaseException): passclass SCE(BaseException): pass

base.add({‘NotNullException’: NotNullError,‘SufficientCoffeeException’: SCE

})


Permission Denied

• Exceptable.py also wraps the base permission denied

• DB permissions violations work at the app level


Vertically Challenged

a = c.Function(‘test_auth’);try: result = a()except PermissionDenied, e: abort(403)except NoSuchUser, e: abort(401)


AND THUSQuestions?


Get It!

https://projects.commandprompt.com/public/exceptable/






THANK YOU!


exceptable - exception propagation for pg

Documents

database api easier

database logictuesday

api undefined exceptions

database query c

dbapi integrationtuesday

database querythe app

exceptions values exception

database logic harder