![Page 1: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/1.jpg)
Beyond the simple Pipeline - managing processes over time
XML Prague , 17/06/2007
Geert Bormans, independent [email protected]
![Page 2: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/2.jpg)
[email protected] realising the value of your content
Introduction
Who am I? Background
10 years team-lead and researcher researchgroup document architectures Catholic University Leuven
5 years VP professional services British/Canadian XML software vendor
Independent Consultant Mainly projects that surface the intrinsic value of (yet existing)
content Migrations, ‘legacy’ conversions, XML transformations Content Management, content reuse e-publishing
![Page 3: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/3.jpg)
[email protected] realising the value of your content
Introduction
What to expect? A quick introduction of the problem space Introduction of the project Introduction of the solution A word about pipelines Challenges for building pipeline solutions The solutions Architecture Wider applicability and future plans A quick demo
![Page 4: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/4.jpg)
So we all love XML pipelines by now…
Ever thought about managing them?
![Page 6: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/6.jpg)
[email protected] realising the value of your content
The project
The customer A financial institution working for the goverment Publishing yearly statements New legislation requires archiving the statements
Current Architecture Complex Monolithic Code Growing with yearly changes
![Page 9: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/9.jpg)
[email protected] realising the value of your content
Next years Example Statement
Publisher company rebrand
ID added to statement
![Page 12: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/12.jpg)
[email protected] realising the value of your content
... and version managing the pipeline
Each asset in the CMS is linked (through metadata) to a specific version of the pipeline, hence assuring the possibility to reconstruct a statement as it was printed before
![Page 13: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/13.jpg)
[email protected] realising the value of your content
Pipeline benefits
Break up complexity Simpler to develop Easier to debug and maintain
Clear assignment of responsibilities Mix technologies easily Reuse components
![Page 14: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/14.jpg)
[email protected] realising the value of your content
Pipeline solutions - challenges Execution
Fit components together and Compile Pipeline Execute with high performance in a robust way
Avoiding intermediate (de)-serialisation Allow for mixed technologies
Different technology per component possible Easy creation of pipeline Manageability
![Page 16: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/16.jpg)
[email protected] realising the value of your content
Pipeline Component Managment System Processing steps are URI-addressable resources Steps pulled together at run time to create pipeline Execution by Pipeline Execution Engine Datamodel for simple pipeline
![Page 17: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/17.jpg)
[email protected] realising the value of your content
Pipeline Execution Engine
Thin layer on top of NetKernel Resource oriented computing platform Microkernel that schedules resource requests Dynamic compilation of various scripting
languages Strong support for XML processing tasks Pipeline language: DPML
![Page 19: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/19.jpg)
[email protected] realising the value of your content
Meeting the challenges
Pipeline execution by NetKernel Unnecessary serialisation/deserialisation
avoided by notion of aspects and transreptors NetKernel supports pipeline components
developed in various technologies PCMS serves as abstraction layer between
pipeline development and execution PCMS takes care of pipeline version
managment
![Page 20: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/20.jpg)
[email protected] realising the value of your content
Wider Applicability
Architecture provides version managment of pipeline components
Most important factor of complexity of XML processing is variation in time
Applications layered on this architecture will be better protected against damage by changes
A solution worthwhile considering for any complex XML processing application
![Page 21: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/21.jpg)
[email protected] realising the value of your content
Future Plans
Non lineair pipelines (forking, looping) Dashboard for monitoring
Timings, breakpoints Import filters for
XProc Stylus Studio pipes
...
![Page 22: Beyond the simple Pipeline - managing processes over time](https://reader036.vdocuments.net/reader036/viewer/2022062519/56814e7e550346895dbc1b85/html5/thumbnails/22.jpg)
Questions and Discussion
Thanks for your attention