yapc10 cdt world domination
DESCRIPTION
TRANSCRIPT
Catalyst, DBIC, and TT for world domination
My Background
• Lazy (Programmer)
• Often involved in sys admin (repetitive) work
• Leverage CPAN so I don’t have to figure things out others already have (Lazy)
In the beginning...
• automate maintenance
• systems checks, report by email.
• check status of various processes/disk/memory
• automate user tasks
Scripts to:
In the beginning...
• automate maintenance
• Job failures difficult to detect
Scripts to:
In the beginning...
• systems checks, report by email
• Inbox flooded.
• Still have to read and process them
• Follow ups often require system access
Scripts to:
In the beginning...
• check status of various processes/disk/memory ($> appname_health.pl)
• often requires privileged access
• output in fixed font
Scripts to:
In the beginning...
• automate user tasks (mkuser, unlock account, etc)
• user at least has to have a login
Scripts to:
DBI + CGI
CGI - Benefits
• Accessible to anyone with a web browser
• Programs easy to deploy
• No need to login to host
CGI - Bad habits
• Accessible to anyone with a web browser
• Perl compile cost every page hit
• Long running processes not easily possible
• HTML Code mixed in with perl code (HERE or qw{}) print q{<HTML>
hello world</HTML>};
DBI - Benefits
• Standardized access to back end data
• Database not perl process does hard query work
DBI - Bad habits• Vulnerable to SQL injection
• Perl code is hard coded to a specific DB type (MySQL, Postgres, Oracle, Sybase, MSSQL)
• Redundant prep/execute/extract code usually splashed all over the code
• Detection of errors usually not standardized
• SQL code (often many lines long) intermixed with perl code. (Hard to read)
• Badly formed SQL can reveal sensitive information to end user
DBI - Bad habits
• Vulnerable to SQL injection
• select * foo where bar = “$test”
• $test = q{5; insert into admin_table...};
• $test = q{“blah}
• Difficult to handle BLOB data
• Placeholders limited
• where of select, set of update, values for insert)
• From clause inaccessible via placeholders
DBI + CGI
Catalyst, DBIx::Class, TT
The Learning Curve• Moose!
• DBIx::Class
• TT
• Catalyst
• Model/View/Controller paradigm a difficult balance to maintain
• Perl Testing techniques in Catalyst
• Catalyst unit testing (how do I exclude the models in the test?)
• Model testing (build a test DB complicated enough to test the model)
• Selenium (minor changes in TT easily break tests)
Moose
• Easy to learn for if you’ve done OO programming before
• http://search.cpan.org/perldoc?Moose::Cookbook
• No more new!?!?
• Not critical to learn but useful when thinking about how Catalyst works
DBIx::Class• Best way to call SQL but read like perl
• Standard interface to returning query from submy $rs = $schema->resultset(‘CD’) ->search({ year => 2000 }, { prefetch => 'artist' });while ($rs->next) {
print $rs->artist . “\n”;}
my $dbi = DBI->connect(...) or die DBI->error;my $query = q{select artist from CD where year = 2000};my $sth = $dbi->prepare($query) or die;$sth->execute or die;while(my ($artist) = $sth->fetchrow_array) {
print “$artist\n”;}
DBIx::Class table setup
• Schema Loader Magic! (unless it’s not a well built Schema)
• Make sure fkeys,pkeys setup in db first if you can.
• Not all Pg features pull from schema loader
• DBIx::Class::Schema::Loader to load schema
• Alternatively: cd ~catapp; script/create.pl model CatalystModelName...
DBIx::Class - Caveats
• wrap insert/updates in eval {} or x; blocks
• txn_do($code_ref) (still inside eval block)
• left outer joins by default. (null rows removed)
• Understand when a query is a row or a result set. They behave differently.
• inflate/deflate only works on rows, not $rs
• relation columns can have same name as variables. causes you to have to do $row->get_column
Learning TT
• Think Cool PHP (if it was perl)
• http://template-toolkit.org/docs/manual/
• Syntax differences can be subtle and TIMTOWTDI
• Catalyst will do all the heavy lifting here.
• Mailing list is very friendly.
TT: Advice from the trenches
• “Spend a solid day or more dedicated to learning Template::Toolkit. Andy Wardley has done an amazing job at creating and maintaining this system. By spending a little time up front, it will save you a TON of time in the future and get you familiar with its capabilities. Learn how to make it sort arrays, sort hash keys, loop through keys of hashes, etc - things that are vital to the template development for web applications.”
• TT manual can be confusing to the beginner.
• “some basic concepts were buried in the middle of examples of various features, so I had to read about almost everything before I could do more than the trivial.”
• “I felt the TT language had too many syntactic variations for accomplishing the same thing, and again the examples randomly chose which version they used in order to demonstrate this breadth, but until you get to the part which explains the equivalences, you are left scratching your head trying to figure out what the semantic differences are.”
• “Aside from that, TT seems to be a powerful, stable templating system.”
TT - Trenches II
• Case matters: [% FOR foo IN chairs.keys.sort %]
• Prevent line wrap: [% author -%]
• Altering the stash is local (sometimes): [% foo = ‘HELP’ %]
Learning Catalyst (are we there yet?)
• CPAN Tutorial
• Catalyst Examples from SVN
• Catalyst Advent Calendar
• Johnathan Rockway’s Book “Catalyst”
• Initial setup is 90% of the battle
• catalyst.pl AppName
Cost Justification for the learning Curve
• Multi-developer approach easier when code is broken into multiple modules
• Automated testing setup for you
• Stupid DBI mistakes no longer a risk (Now it’s stupid DBIC mistakes but they fail more consistently)
• DB Structure separated from code in one place.
• Annoying HTML page setup for each CGI script centralized
• Model structure allows for easy re-use outside of Catalyst
• Application portable to multiple developers with self contained web server.
MVC
• Spend time thinking about and talking about the demarcation for MVC
• http://en.wikipedia.org/wiki/Model-view-controller
View
• What presents the data to the user
• HTML
• JSON
• file download
• csv
Controllers
• Air traffic control of the application
• Receives user input
• Authenticates and authorizes access with help of models
• Requests information from models based on user input
• Data cleansing of user input
• Formats data (input and output) to present back to view for display
Models• Library of calls for controllers and outside programs to:
Query/update/insert information by:
• Invoking programs
• Accessing databases
• Messaging (email, IM)
• Maintains Data integrity where pkey/fkeys stop.
• Enforces authorization based on controllers provided credentials
• Centralizes auditing of events
Lessons learned from Controller Development
• Do all authentication and authorization in root.pm via auto (Private)
• Take care not to do view or model work in the controllers. This is the easiest place to do too much
• Common Controller routines or repetitive code sections indicates you have a model routine
Lessons learned from Model Development
• Pod EVERYTHING. It takes a very short time to forget what it does or why it’s there or how to use it.
• Inheritance is probably not the right solution - Use Moose Roles instead to separate the sections of your models for better management
• Models provided us a way to standardize back end interface to on-boarding.
• Create a common library to validate hash passed to model
• my $params = validate_hash(shift, qw/date_inserted id owner on_hold/) or return toss(“reason...”);
Is it a model or controller job?
• Whether you agree with all of this doesn’t matter. What matters is you become aware of the coding issues and decide which way you’re going to go.
Model test suite
• Can be difficult to create test databases complicated enough to validate functionality or replicate errors.
• Clear /re-build database between tests
• input/output can be complex. Build helper to make life easier (test_model_func({model=>.., func => ..., args => ...., toss =>...,}))
• Controller bugs require a test to fix in model
Apache + FastCGI
• Allows app to be scalable.
• Multiple perl engines can run in the back end
• Engines can be on external hosts
• Scales down connections on idle
• Remember [% c->uri_for(‘/’) %] in all your templates
• Catalyst::View::TT->CATALYST_VAR
In the end
• Splitting up areas of responsibility between controllers and models helps enforce:
• documentation
• interface standards
• testing