Maxim Grigoriev

Fermi National Accelerator Laboratory

PXB: Perl XML Binding

Outline of the talk

● Motivations or why Yet Another thingy which pollutes XML modules

space

● Data Model

● Goodies in the bag

● Test suit, logging, style, perltidy, PBP

● SQL mapping, perfSONAR-PS project

● Problems, plans

Motivations or Who needs Yet Another XML “framework”● RelaxNG Compact Schema popularity

● Interoperable Document/literal SOAP webservices are standard these

days

● Schema development goes in parallel with API, needs agility

● XPath / DOM provides nice walking but lacks ability to assign callbacks to

specific elements in the tree

● Ability to map some element from the external data model on the XML

elements tree is required for every webservice with SQL DB backend

● XML datatypes binding is heavily supported in Java world ( xmlbeans,

CASTOR, METRO and older JAXB)

● Having the same message based, easily refactored OO API for client and

server is a very attractive idea (SOAP::Lite was doing it for years with

WSDL)

● Native XML databases are arrived a while ago but not really of production

quality

Data Model XML element represented in perl, see http://code.google.com/p/pxb/wiki/PXB for complete docs:<element-variable> = { attrs => { attributes-definition , xmlns => 'string' }, elements => [ elements-definition ], text => 'text-content', sql => {sql-mapping-definition}, }attributes-definition = (string => 'attribute-value' ) (, attributes-definition)*attribute-value = scalar | (enum: (string ( , string)*))

elements-definition = ( [ string => ( <element-variable> | [ <element-variable> ] | [ ( <element-variable> ,)+ ] | [ ([ <element-variable> ],?)+ ] ) , 'conditional-statement'? ])*

text-content = scalar | conditional-statementconditional-statement = ( unless | if ) : (variable-name (, variable-name)*)

Example: $parameter = { attrs => { name => 'enum:name1,name2', value => 'scalar', xmlns => ‘nsid1'}, elements => [], text => 'unless:value‘ };

Data Model ( continued... )

The rest of the model, SQL mapping:sql-mapping-definition = (sql-table-name => { sql-table-entry } ) ( , sql-mapping-definition)*

sql-table-entry = (sql-entry-name => { entry-mapping } ) (, sql-table-entry)*

entry-mapping = value => ( element-name | ( [ element-name (, element-name)+ ] ) ) (, 'if-condition')?

if-condition = if => attribute-name : attribute-value

sql-entry-name = string

sql-table-name = string

Example: $parameter->{sql} = {tableName => { field1 => {value => ['value' , 'text'], if => 'name:name1'}, field2 => {value => ['value' , 'text'], if => 'name:name2'}, } }

Data Model (still continued...)

What about complex types ? Lists, choice between different elements or choice

between the elements with the same local name but from the different namespaces:

For example:

elements => [parameter => [$parameter]] - defines list of parameter elements

elements => [parameter => $parameter] - defines single parameter element

elements => [parameter => [$parameter, $other_parameter]] - defines choice between two single elements with different local names (for example nmwg:parameter and nmwg:otherParameter)

elements => [parameter => [[$ns1_parameter], [$ns2_parameter]] ] - defines choice between two lists of elements with the same local name but belonged to different namespaces

DONE WITH DATA MODEL...DONE WITH DATA MODEL...

Building API

Once your model is defined its very easy to create your API: use XML::RelaxNG::Compact::PXB; use POD::Credentials; my $api_builder = XML::RelaxNG::Compact::PXB->new({ top_dir => "/home/joedoe/API", datatypes_root => "XMLTypes", nsregistry => { ’nsid1’ => ’http://some.org/nsURI’}, schema_version => "1.0", test_dir => "t", footer => POD::Credentials->new({author=> ’Joe Doe’}), }); $api_builder->buildAPI(‘myParameter’, $parameter); It will create package XMLTypes::v1_0::nsid1::MyParameter as: /home/joedoe/API/XMLTypes/v1_0/nsid1/MyParameter.pm Some helper classes and the test suit:

/home/joedoe/API/t/XMLTypes::v1_0::nsid1::MyParameter.t /home/joedoe/API/t/conf/perlcriticrc /home/joedoe/API/t/conf/perltidyrc /home/joedoe/API/test.pl

All the goodies ( aka API introspection )

Every generated class has constructor with the same interface, it accepts single hash ref as an argument and every class implements the same set of methods.

Every method in the class follows the same prototype.

Every object can be initialized from the XML fragment (scalar), DOM object or reference to the hash. Of course it can be serialized back into the DOM or XML.

It knows how to handle complex types. There are many XML schema where each element is identified by unique attribute named id. The generated API can build a map of such elements and supports addById, removeById to allow faster lookup for the multiple elements in the list.

There is a special call named registerNamespaces for returning hash of the all namespaces registered for the root object

Every element is mapped on the particular namespace by the namespace prefix.

Example of API utilization, perldoc XMLTypes/v1_0/nsid1/MyParameter to see full list of calls : use XMLTypes::v1_0::nsid1::MyParameter;my $object = XMLTypes::v1_0::nsid1::MyParameter->new({ xml => ‘<nsid1:myParameter xmlns:nsid1="http://some.org/nsURI" name=“name1” value=“newValue"/>’});print ‘Name:’ . $object->get_name . ‘ Value:’ . $object->get_value;

SQL Mapping

Supported by querySQL call, it goes recursively through the objects tree and returns ref to hash with contents of the mapped XML elements. For the previously defined parameter element:

Example: XML serilaized into $object: <nsid1:myParameter name='name1' value='100/> <nsid1:myParameter name='name2' value='200/>

call $object->querySQL($hash_ref_to_return);

$hash_ref_to_return is { tableName => { field1 => '100' , field2 => '200'} }

where it can be easily passed to any of SQL ORM frameworks. For example: in case of Class::DBI:

my @records = TableName->search(%{$hash_ref_to_return});

or with minor refactoring in Rose::DB::Object

my @records = TableName::Manager->get_tablenames( query => [ field1 => { eq => $hash_ref_to_return->{field1}}, field2 => { eq => $hash_ref_to_return->{field2}}, ] );

The rest of the story

Centralized logging is supported by Log::Log4perl module

Each module is throughly documented with pod

Test suit is built for each generated class

There are perlcritic and perltidy profiles created for the API and perlcritic parsing is an integral part of the module testing

Essentially, one can create a bunch of RelaxNG or XML schema derived CPAN modules in a matter of minutes and pollute XML:: namespace even more Orone can start schema derived API with properly formed classes and follow the same style and utilize automated tests to assure enterprise level quality of the software (and perl needs it badly)

Problems, Plans First releases were failing automated tests on CPAN due failed dependencies:

● XML::LibXML was the first one to blame for perfSONAR-PS project we even started packaging libxml2 libraries with PAR::Packer and ship perl binaries● Some people have Perl::Critic configured site-wide with set of default themes, different versions have different default themes

Its not blazing fast:● Log::Log4perl, if used as recommended,

adds “$category =~ s/::/./g;” for each method where get_logger called, switched to class member logger

● Started with Class::Fields and Class::Accessor and had to move away and provide PBP style accessors/mutators because it was slowing things tremendously to do...● Add direct parsing for the RelaxNG Compact schema files and generate API based on parsed schema DOM● Add option to choose between hash based objects or inside-out ones.● Add option to push created API into memory in the run-time rather than write it on disk

Questions ?

Oh, and yes, perl is indeed unDead !

Links:Links:

PXB project on Google code – http://code.google.com/p/pxb/w/list

perfSONAR-PS wiki - https://wiki.internet2.edu/confluence/display/PSPS