resource-oriented computing w/ netkerneljava.ociweb.com › javasig › knowledgebase › 2008-01...
TRANSCRIPT
Resource-Oriented Computing w/ NetKernel :
Software forthe 21st Century
Brian SlettenZepheira, [email protected]
Speaker Qualifications
Over 14 years of software development experience
Has own software consulting company for design, mentoring, training and development
Currently working in Semantic Web, AOP, Grid Computing, P2P and security consulting
Pinky Committer
Agenda
What the heck is NetKernel?
Why We Care
NetKernel Overview
NetKernel Languages
Advanced Concepts
Pinky
What the heck is NetKernel?
A micro-kernel based resource ecosystem built around the principles of REST, Unix Pipes and SOA
Why We Care: The Simple Answer
Most enterprise systems use XML and are built in Java and this is still too much of a pain
Complicated
Different Languages for different technologies
Predicting scalability
Why We Care : The Complicated Answer
This is hard because of a general mismatch at the intersection of languages, data model, processing abstractions, architectural tiers, etc.
Larger Trends
1970s 1980s 1990s 2000s
Code written
Dataformatted
NetKernel
A Ground-Up Resource-oriented computing environment
Modern, microkernel architecture
Takes the best of REST, Unix Pipes, SOA
Open Source w/ Dual License for non-open source uses
Resource-Oriented Computing
Everything is a Resource
Everything is URI-Addressable (logical reference)
“Computation” is the turning of logical resource references into physical representation
Lossless conversion between representations
REST
Representational State Transfer
Architectural style described in Roy Fielding’s Ph.D. thesis
URLs locate resources in Internet-sized address spaces
Client passes all state in request
History of Addressing
Network
-----
-----
-----
Documents
http://myserver.com/docs/reports/3rdQtr/
History of Addressing
Network
Data
http://myserver.com/data/graph/1
History of Addressing
Network
Service A
Services
http://myserver.com/service/a
URLs ! => Scalability
Addressability of the resources is important, but REST derives its properties from the architectural constraints
REST Properties
Unix Influences
Everything is a file (common interface/resource abstraction)
Small, focused, reusable tools
Tools are scripted
Processes have context from environment
SOA Influences
Loosely-coupled
Asynchronous requests
Reusable services composed through orchestration languages
Putting the Kernel in NetKernel
Internally, a microkernel manages threading resources
Maps URI requests to managed resources and schedules them
General Concept
Calling a BeanShell Script
XQuery
XSLT
Active URIs
Internally, NetKernel uses a special URI naming scheme
A proposed IETF standard by HP and 1060 Research to encode complex URIs into standard forms
General formactive:<processURI>*(+<inputName>@<inputURI>)
http://ietf.mirror.netmonic.com/draft-butterfield-active-uri-01.txt
Sample Active URIs
BeanShell
active:[email protected][email protected]
XQuery
active:[email protected][email protected]
active:[email protected][email protected]
XSLT
NetKernel Benefits
“Deep” REST environment
Appropriate abstractionsFocus on information, not code!
Layered approach to building applications
Logical connections/requests get resolved at runtime (late-binding)
NetKernel Properties
Fine-grained caching
Self-optimizing based on use
“Aspect-Oriented” architecture
Security, logging, etc. behind URI interception
Tiny Footprint
Can do “real” work in a VM w/ 10-12MB
Takes full advantage of multi-core, multi-CPU boxes
Basic NetKernel Tour
NetKernel Concepts(aka Really Cool Terms)
Accessors
Transports
Representations
Aspects
Transreptors
Accessors
URI-addressable component that provides access to NetKernel resources and services
Transport
A component that funnels requests from external sources into NetKernel URI requests (and ultimately Accessors)
ExamplesHTTP
SMTP
JMS
Representations
The ‘R’ in REST
In NetKernel, an object representing the resource
Exposed via immutable, cacheable “views”
Aspects
Not AOP “Aspects”
Immutable views of the underlying representation in a given form
ExamplesJDOM, DOM, String, BinaryStream
Transreptors
Fancy term for “transforming representations”
Transreptors return a resource in a specified form if conversion is possible
Useful for converting between Aspects
NetKernelLanguages
DPML
A simple NetKernel scripting language for specifying URI requests
Supports conditionals, loops, subroutines, etc.
DPML Sample
<idoc><seq><instr><type>copy</type><operand>ffcpl:/usr/local/NetKernel/modules/workbench/nfjs/hello.xml</operand><target>this:response</target>
</instr></seq>
</idoc>
BasicExamples
XQuery
XML Query Language
W3C Candidate Recommendation
Functional language
FLWOR (For Let Where Order By Return)
XQuery Sample
<abbeysayings>{ for $author in doc("active:xquery+operator@file:/usr/local/ NetKernel/modules/workbench/nfjs/quotations/ authors.xq")//* where $author/name/last = 'Abbey' return $author/quotation}</abbeysayings>
BeanShell
Supports embedded BSH interpreter
Java source interpreter
Interacts with other scripts, Java, etc.
Makes for very fast scripting
BeanShell Sample
void main(){ // Extract all speeches with lines // containing the parameter argument req=context.createSubRequest(); req.setURI("active:xquery"); req.addArgument("operator","abbey.xq"); output=context.issueSubRequest(req);
//Finally return response response=context.createResponseFrom(output); context.setResponse(response);}
“Real World”Examples
XSLT Input Data
XSLT Example
if( context.exists("this:param:param") ) { param=context.sourceAspect("this:param:param", IAspectNVP.class); mbid = param.getValue("mbid");} req=context.createSubRequest();req.setURI("active:httpGet");
url="http://musicbrainz.org/ws/1/artist/" +mbid+ "?type=xml&inc=sa-Album";req.addArgument("url", url);output=context.issueSubRequest(req);
sa= new StringAspect("<albums>" + mbid + "</albums>");req=context.createSubRequest();req.setURI("active:xslt");req.addArgument("operand", output); req.addArgument("operator", "albums.xsl");req.addArgument("band", sa);output=context.issueSubRequest(req);
XSLT Stylesheet
<xsl:param name="band"/> <xsl:template match="mo:metadata/mo:artist/mo:release-list"> <html> <body> <h1>Results for <xsl:value-of select="../mo:name"/></h1> <table border="1"> <xsl:apply-templates/> </table> </body> </html></xsl:template> <xsl:template match="mo:release"> <tr><xsl:apply-templates/></tr></xsl:template> <xsl:template match="mo:title"> <xsl:variable name="id" select="../@id"/> <xsl:variable name="url" select="concat('http://musicbrainz.org/ws/1/release/',$id)"/> <td><a href="{$url}?type=xml&inc=tracks"><xsl:value-of select="."/></a></td></xsl:template>
XSLTExamples
Simple Tree Manipulation
Kind of a DSL for XML
XML is tree-oriented, let’s use it that way
STM
Sets element values to the specified value
stm:set
<stm:set select=”/x/y” xpath=”/a/b[1]/c”><x><y>zebra</y>
</x></stm:set>
<a> <b> <c food="hay">horse</c> </b> <b> <c food="hay">donkey</c> </b></a>
<a> <b> <c food="hay">zebra</c> </b> <b> <c food="hay">donkey</c> </b></a>
Inserts specified value after the selected element
stm:insert-after
<stm:insert-after xpath=”/a/b[1]”><x><y>zebra</y>
</x></stm:insert-after>
<a> <b> <c food="hay">horse</c> </b> <b> <c food="hay">donkey</c> </b></a>
<a> <b> <c food="hay">horse</c> </b><x><y>zebra</y>
</x> <b>
<c food="hay">donkey</c> </b></a>
Applies the specified namespace to the selected elements
stm:apply-ns
<stm:apply-ns prefix=”ns” uri=”http://www.bosatsu.net/ns”xpath=”/a/b[2]”/>
<a> <b> <c food="hay">zebra</c> </b> <ns:b xmlns=”http://www.bosatsu.net/ns”> <ns:c food="hay">donkey</ns:c> </ns:b></a>
AdvancedConcepts
Asynchronous requests are easy in NetKernel
Asynchronous Requests
// Set up request 1req1=context.createSubRequest();req1.setURI("active:beanshell");req1.addArgument("operator", “script1.bsh” );req1.addArgument("param", param );handle1 = context.issueAsyncSubRequest(req1);
// Set up request 2req2=context.createSubRequest();req2.setURI("active:beanshell");req2.addArgument("operator", “script2.bsh” );req2.addArgument("param", param );handle2 = context.issueAsyncSubRequest(req2);
// Block until both calls finishretValue1 = handle1.join();retValue2 = handle2.join();
Throttling requests is also easy
Throttling Requests
// attempt to issue 100 concurrent requests to script1.bshfor (int i=0; i<100; i++) { req=context.createSubRequest("active:throttle"); req.addArgument("id","throttle:my_throttle"); req.addArgument("uri","active:beanshell"); req.addArgument(“operator”,”myscript.bsh”); context.issueAsyncSubRequest(req);}
<throttle> <concurrency>4</concurrency> <queue>50</queue></throttle>
Expects to find a configuration file in ffcpl:/etc/ConfigThrottle.xml
Golden Thread Pattern
Virtual Dependency Management
Powerful technique for invalidating arbitrary collections of resources
Easier to see than hear about!
Long Computation wrapped in an Accessor called from a DPML script
Scalability - Scenario 1
Shorter Computation called multiple times from DPML script
Scalability - Scenario 2
Random document chosen from DPML script
Scalability - Scenario 3
Real NetKernel Infrastructure is built around modules
Modules
Ways of segmenting Address Space and functionality
Modules export public URIs and class packages
Separately Versioned units of deployment
Modules (cont)
Modules (cont)
Can hold static resources, Java code, scripts, accessors, aspects, transreptors, transports, fragmentors and caches
Import module definitions to access another namespace
If you create a new module, you will need to at it to the deployedModules.xml
Sample Modules
Module Functionality
ext-xml-core-1.4.2.jar DOM, JDom, SAX, XDA, XMLBean
ext-dpml-1.2.2.jar DPML Language runtime
ext-kernel-1.0.6.jar System component accessors
doc-netkernel-4.0.0.jar NetKernel Core Documentation
mod-fulcrum-frontend HTTP Server FrontEnd Fulcrum
mod-fulcrum-backend HTTP Server Backend Fulcrum
tpt-http-2.1.8.jar HTTP Transport
tpt-jms-1.0.2.jar JMS Transport
app-address-book-1.1.4.jar Address Book application
Module Example
Pipes in NK (Yahoo)Module for implementing Yahoo Pipes-like functionality
Runs in your computational space
No ads
Full control
Pinky
http://sourceforge.net/projects/pinky
Union - combine one or more feeds
Truncate - keep things manageable
Unique - remove duplicate stories
Count - how many elements per feed
Filter - restrict feed elements to filter criteria
Pinky Accessors
Pinky Example
Sophisticated management consoleSystem Healthcheck
Manipulate Modules
Monitor Memory, Throttle Settings, Workload
Edit Kernel Parameters
NetKernel Administration
http://localhost:1060/ep+name@app_ext_introspect_controlpanel
Non-XML Examples
NetKernel Links
Open Source Site http://www.1060.org
Commercial Site http://www.1060research.com
Article http://tinyurl.com/ghxg3
Front-End Fulcrum http://localhost:8080
Back-End Fulcrum http://localhost:1060
Slides http://zepheira.com/presentations/2008/NetKernel/NetKernel.pdf
Tutorial http://zepheira.com/presentations/2008/NetKernel/tutorial.txt
Examples http://zepheira.com/presentations/2008/NetKernel/NetKernel-Examples.zip
