bp207 - meet the java application server you already own – ibm domino

© 2013 IBM Corporation

BP207Meet the Java Application ServerYou Already Own – IBM® Domino®

Bruce Elgort | Elguji / OpenNTF / Clark CollegeSerdar Basegmez | Developi / OpenNTF

2 © 2013 IBM Corporation

Agenda

Introduction to OSGi and DOTS

Preparing Runtime and Development Environment

Your First DOTS Tasklet

Running and Debugging Tasklets

Deploying Tasklets

Tips, Tricks and Good Practices

Q&A

3

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

4

Cast

Who are we?

5

Serdar Basegmez

IBM Collaboration Solutions Champion (2011-2013)

Owner of Developi Information Systems (Istanbul, Turkey)

Founder and Co-leader of LUGTR – Turkish Lotus User Group

Bilingual Blogger at LotusNotus.com (Turkish/English)

OpenNTF Guy in Turkey– Contributor in XSnippets and CollaborationToday.info– Member Director at OpenNTF Board

IBM Notes/Domino Design Partner

Writer at The View Magazine

Expertise on XPages, IBM Domino, IBM Connections, etc.

7

Prelude

Introduction

8

What’s all the fuss about DOTS about?

XPages is great!– We all know it...

But...– XPages are based on interaction.– Users demand, they reply back.

What if…– Processing takes too long?– You need housekeeping?– You have scheduled reports?– You need to send reminders?– Your application needs batch processing?– You require unattended tasks?

You have Agents, right?– We have a much better idea...

• Much better idea…

Some Basics

XPages ≈ Java

IBM Domino– Java Application Server (almost)– JSF Container– OSGi platform

Positioning Java on XPages– For classical Domino developers:

• Formula SSJS • Lotusscript Java (Truth hurts!)

– Performance– Reusability– Modularity– Separation of Powers (UI-Data-Logic)

OSGi is the key technology!

9

Model

View

Controller

10

What is OSGi?

Open Services Gateway Initiative– OSGi™ Alliance (IBM involved)– Developed first in 1999– Continues to be improved

Module System and Service Platform– OSGi Bundles (Applications/components)– Bundles can be remotely...

• Installed / Uninstalled• Started / Stopped• Updated

– No reboot required for operation!

OSGi within IBM Notes®/Domino®...

IBM Domino Server 8.5.2– OSGi introduced in HTTP task– XPages runtime as OSGi plugin– Extension Points

IBM Domino Server 8.5.3+– OSGi support extended to;

• XPages Extension Library,• REST APIs,• Social Enabler• DOTS (Domino OSGi Tasklet Service)• Domino Servlet Container• OpenSocial Container• ...

11

8.5.2

8.5.3

9+

12

Why OSGi matters

Extending toolbox– Designer components– Server-side implementation for components

Providing services for non-UI consumers– REST Services– Social APIs

Adapting standards– OAUTH– JDBC

Running Tasklets– DOTS

13

Meet DOTS – your new best friend

Domino OSGi Tasklet Services: Next Generation Agents for Domino

DOTS provides ability to develop and run OSGi level server Tasklets for Domino– Easy deployment.– Tooling with Eclipse IDE and its Java and plugins editors and Java debugger.– Leverage existing OSGi assets (within the company or open source).– Tasklets can be run:

• Manually (local and remote calls)• Scheduled or on server start• Triggered (Hooks through data events) Not supported on Domino 9 SE Public Beta!

OpenNTF project by IBM since April, 2011 (replaced JAVADDIN project)

Included in IBM Domino 9 Social Edition Public Beta

14

What is a Tasklet?

Component to process a step in your business logic– Lightweight addin task.– DOTS Tasklets are simply Java methods within OSGi bundles.– Declared as OSGi extension point

Tasklet is functionally less capable than a Server Task

Tasks DOTS Tasklets

Loaded once, run continuously Loaded once, run when needed

Manages own threads DOTS manages threading

Should be native (or encapsulated) Runs inside DOTS container

Unmanaged security Managed security

Hard to develop Easy to develop

15

DOTS Service Architecture

Source: Lotusphere 2012 – SHOW112 session by David Taieb, Paul Fiore, Elizabeth Sawyer

16

DOTS vs. Agents

DOTS Tasklets outperforms over Java Agents!– Why?

17

DOTS vs. Agents

DOTS Tasklets outperforms over Java Agents!

AMGR launches a

Thread

JVM Initializes

Prepare Java and JNI

interfaces

Load Agent Bytecode Run!

Run!

Java Agent:

DOTS tasklet: Everything is ready to run

> tell amgr run "test\XPagescrash.nsf" 'LongJobAgent' 09.11.2012 19:38:39 JVM: Java Virtual Machine initialized. 09.11.2012 19:38:39 AMgr: Start executing agent 'LongJobAgent' in 'test\XPagescrash.nsf' 09.11.2012 19:38:39 Agent Manager: Agent printing: 181349 09.11.2012 19:41:02 Agent Manager: Agent printing: 2227 09.11.2012 19:41:02 Agent Manager: Agent printing: Finished in 143 secs... - 09.11.2012 19:41:02 AMgr: Agent 'LongJobAgent' in 'test\XPagescrash.nsf' completed execution

> load dots > Listening for transport dt_socket at address: 8001 09.11.2012 19:42:40 Domino OSGi Tasklet Container started ( profile DOTS ) > 181349 > 2227 09.11.2012 19:43:22 [DOTS] (annotated) Finished in 41 secs...

18

DOTS vs. Agents

Amgr is old and complicated– Security model problems– Limited functionality (like issues on class loading, reflections)

Range– Agents are database-dependent– DOTS runs server-wide

More flexible scheduling– Precise timings like 45 seconds– Programmatic control over scheduling

DOTS provides wide option of database hooks Not supported on Domino 9 SE Public Beta!

19

Examples on DOTS

IBM SmartCloud™ for Social Business (a.k.a. LotusLive)– Some part of back-end processing has been implemented using DOTS

CollaborationToday.info– Feed Aggregation is implemented using DOTS

Other possible uses...– Workflow Engine implementations– Social Media Monitoring– Database Monitoring– Application Architecture Assessment

20

Episode 1

Preparation

21

Preparing Runtime Environment

DOTS should be installed into IBM Domino server– IBM Domino 9.0 Social Edition Public Beta does not need installation!– For IBM Domino 8.5.2 or 8.5.3;

• Download DOTS package from OpenNTF• You will also need some Eclipse plugins from Eclipse 3.6.2+

22


Prepare directory structure under Domino folder– [Domino Program Files]\osgi-dots\rcp\eclipse\plugins– [Domino Program Files]\osgi-dots\shared\eclipse\plugins

These plugins will be copied from Eclipse installation

[Eclipse]\plugins

23


DOTS package contains executable files and Java libraries– Copy JAR files into folders you have created– Copy executable and libraries into Domino program folder

Copy to

[Domino]\osgi-dots

Copy to

[Domino]\osgi-dots\shared\eclipse\plugins

24

Preparing Development Environment

DOTS Tasklets consist of methods in Java classes as OSGi bundles.

Eclipse IDE is needed for development and debugging.– In theory, Domino Designer can be used with lots of unsupported configuration changes.

Configuration is easy.– You will see steps for setting up Eclipse to develop OSGi plugins for Domino

environment.• Eclipse (Development Environment) OSGi on Domino (Runtime)

– Resulting Eclipse environment is useful to develop...• DOTS• XPages extensions• Custom validators, data sources, converters, etc.• Servlets• J2EE applications for Domino

25


Download and extract Eclipse IDE...– http://www.eclipse.org/downloads– Eclipse 3.6.2 (Eclipse IDE for Java EE Developers) needed at least.

Download and extract necessary OpenNTF project releases– Domino OSGi Tasklet Service : http://tiny.cc/DOTS– Domino Debug Plug-in from XPages SDK for Eclipse RCP: http://tiny.cc/XPagesSDK

Prepare IBM Domino Server 8.5.2+– DOTS should be installed properly.– A local IBM Domino Server is strongly recommended.– Use 32-bit for Eclipse and Domino Server for development/testing

http://www.eclipse.org/downloads

http://tiny.cc/DOTS

http://tiny.cc/XPagesSDK

Development Environment Basics

Target Platform:– Simulate the runtime environment (e.g. JRE settings, classpaths, plug-ins...)

Notes Java API:– DOTS Tasklets use Notes objects (e.g. Session).– Needed for development only.

OSGi Framework– Comes with the Debug Plug-in.– Contains information about OSGi runtime (e.g. workspace)

Configuring Development Environment

Launch Eclipse IDE, – Window\Open Perspective Plug-in Development

27

28


Add Target Platform– Window\Preferences Plug-in Development Target Platform Add...

29


Setup new Target Platform– Name it and add OSGi Directories from Domino Server– Mark it as Active

30


Create Notes Java API project– File New Plug-in Project

31


Import Notes.jar into the Notes API Project

32


Configure Notes Java API project– Open Manifest file, add Notes.jar to the class path...

33


Configure Notes Java API project– Add all packages to the «Exported Packages»...– Save the Manifest file (Ctrl+S or File Save)

34


Install Domino Debug Plugin from XPages SDK for Eclipse – Help Install New Software...– Add Local Point to the update site folder extracted from OpenNTF Package– Accept warnings and restart Eclipse...

35


That’s it... You are now ready to develop OSGi bundles on Domino!– To confirm what we have done...

36

You are Ready!

You are ready to develop your first Tasklet...

Here are some resources for you!– Lotusphere 2012 – Show 112 Session by David Taieb, Paul Fiore and Elizabeth Sawyer

• Available on Greenhouse – Lotusphere Community– YouTube: OSGi Tasklet Service for IBM Lotus Domino by Niklas Heidloff

• http://youtu.be/CRuGeKkddVI– NotesIn9 Episode #93 – «Introduction to DOTS» by Serdar Basegmez

• http://notesin9.com/index.php/2012/12/04/notesin9-093-introduction-to-dots/– XPages Portable Command Guide by Martin Donnelly, Maire Kehoe, Tony McGuckin,

Dan O'Connor• Chapter #3: Working with the Console

http://youtu.be/CRuGeKkddVI

http://notesin9.com/index.php/2012/12/04/notesin9-093-introduction-to-dots/

37

Episode 2

Develop your First Tasklet!

38

Tasklet Development Basics

A Tasklet project is a Plug-in Project contains a number of Java classes– Same project may contain lots of Tasklets.

Extension Points– Extension point to be defined for «com.ibm.dots.task»

Imported Packages– Notes Java API to be linked for compile-time.

OSGi Runtime Configuration– OSGi on Domino will start bundles from its own repository. – A link between Eclipse workspace and OSGi Framework is needed.

• Eclipse will tell DOTS that «I have bundles to install!»

Debug Configuration– Eclipse needs to know how it will connect to the target JVM for debug

39

Developing your first Tasklet

Create a new Plug-in Project– File New Plug-in Project

40


Import DOTS extensions– Open Manifest file and click extensions

41


Import DOTS extensions– Deselect «Show only extension points from the required plug-ins»– Find & pick «com.ibm.dots.task» extension from the list– Correct the warning in the Manifest.mf tab, then Save & Close Manifest view.

42


Import «lotus.domino» package into your project

S

43


Create a Java Class for your Tasklet

44


Welcome to your first DOTS Tasklet!

Runs when your tasklet unloaded

Runs when your tasklet runs

45


Now add some code

46


We will define how it will run!– Open Plugin.xml (Plug-in Manifest) from the Package Explorer...– These markups declare your Tasklet

Name of your class

An id for your tasklet

47

Running your first Tasklet

So far, we have created a Java class, written some code and Eclipse compiled.

DOTS task running on server will load and run our tasklet, but how?– Normal way, we should deploy the plugin into server.– For development/testing, we will tell DOTS to load my bundle directly from Eclipse PDE.

There are two ways:– Run only,– Run and Debug

48


Create a «Run Configuration»

49


Domino OSGi PDE Configuration should be made for the first time

Eclipse created a special file in DOTS workspace– So DOTS task will load bundles from our run configuration...

50


Now, you can launch DOTS task in the server– Also check for tasklist to see that our Tasklet has been loaded.– ...and run!

51

Debugging your first Tasklet

For debugging, Eclipse has to connect to DOTS task.– Eclipse will know what runs on DOTS

More settings needed now...– Close DOTS task

• «tell dots quit»– Set Notes.ini parameters for debugging DOTS:

• «set config DOTS_DEBUGADDRESS=8001»• «set config DOTS_DEBUGSUSPEND=y»

– Reload DOTS task• «load dots»• DOTS task will not start until Debugger connects.

Security Alert!These settings create a security breach and should be used with caution...

52


Server is listening. Now, Eclipse has to know where to connect– Create a new Debug Configuration– Use the port we have set before!

53


You can now set a breakpoint and start debugging.– Eclipse will switch to debug perspective as DOTS reach your breakpoint.– Your changes will be applied immediately!

54

A couple of tips on Debugging & Testing...

Suspending DOTS...– «set config DOTS_DEBUGSUSPEND=y»– This setting will prevent DOTS task starting until a debugger connects.– Suspending DOTS task would be useful to debug scheduled Tasklets or those which

starts on server start. But if your debugger cannot connect for some reason, server will be unstable!

You can also enable debugging for DOTS temporarily...– load dots -debugaddress=<port> -debugsuspend=<y/n>

Eclipse utilizes ‘hot-replace’ feature of JVM– In Debug mode, we can change the source code and bundle will be refreshed upon

compiling!

55

A couple of tips on Debugging & Testing...

Refresh bundles...– Without debug, there will be no «hot-replace».

• Meaning: Recompiling classes will have no effect until DOTS task restarted.– OSGi allows you to refresh bundles without restart, here is how:

• Find out the «bundle-id» for the plug-in• Refresh the bundle

– Doesn’t work for scheduled tasklets.

56

Scheduling Tasklets

DOTS looks for extension point configuration in «plugin.xml» to find out how to run tasklets.

<plugin> <extension point="com.ibm.dots.task"> <task class="com.ic13.samples.helloworld.HelloWorld" Tasklet class id="helloworld"> Tasket id

<run every="5" unit="second" /> Add this! </task> </extension>

</plugin>

57

Magic Word: Annotation

So far, we have declared our tasklets with an XML file.– A Java class «doRun» method A Tasklet

DOTS provides annotated declarations within Java classes– Remember, you still need to create an extension point in «plugin.xml»

@RunOnStartpublic void runOnStart( IProgressMonitor monitor ){

logMessage("Annotated onStart method");}

@Run( id="manual")public void runManual( String[] args, IProgressMonitor monitor ){logMessage("Annotated run method with id=manual");}

@RunOnStart@RunEvery( every=60, unit=RunUnit.second )@HungPossibleAfter( timeInMinutes=1 )public void runEvery60seconds( IProgressMonitor monitor ){logMessage( "Called from annotated method every 60 seconds");}

<plugin> <extension point="com.ibm.dots.task"> <task class="com.ic13.samples.helloworld.AnnotatedTasklet" id="someid" /> </extension></plugin>

58

Episode 3

Tasklet Going Production!

59

Deploying Tasklets - Basics

What we develop in Eclipse IDE is a «Plug-in»

Plug-ins can be deployed into OSGi by...– Referencing PDE tool to the Eclipse Run Configuration

• For testing & development... We have seen this...– Putting JAR file into OSGi plug-ins folder,

• Needs file-level connection to the server and (sometimes) begging to your Admin!– Referencing an update site

• DOTS provides profiles!

Before deploying Tasklets, make sure:– Delete PDE runtime configuration (pde.launch.ini)– Disable suspend feature in debugging (DOTS_DEBUGSUSPEND)

60

Export Plug-in to a JAR file– Right Click Export Deployable plug-ins and fragments...

Deploying Tasklets as Plug-in

61

Select your plug-in project and a destination

Place exported JAR file into plugins folder, then restart DOTS task– [Domino Data Folder]\domino\workspace-dots\applications\eclipse\plugins

Deploying Tasklets as Plug-in

62

DOTS Profiles

A DOTS profile is a specific DOTS container.– Each profile defines «Tasklet Container» that has different settings for;

• Remote Controller• Security Configuration• Specific update sites/folders to load plug-ins

DOTS support multiple profiles for different needs.

Profiles can be stored in XML files or Notes Databases– It can be any Notes Database like names.nsf.

Every profile has a unique name (Message Queue Name – mqName)– Profiles accept separate tell commands, like «tell profileName run XYZ»– DOTS loaded with a default profile named as «DOTS»

63

Creating Profile Repository as NSF

As a best practice, store DOTS Profiles in a Notes database– Create an empty database and secure it with ACL.

On Server, configure DOTS to use Profiles database– tell dots quit– set config OSGI_CONFIGURATION_DB=dotsprofiles.nsf– load dots– tell dots profileCreate dots

DOTS will create necessary design elements

By using DOTS profiles, we can import Tasklets from NSF-based update sites...

64

Update Sites and Features

Update Site is used to organize and export features.– So they can be installed into eclipse-based platforms.

Feature is a collection of plug-ins.– Made of plug-ins that can be installed/updated together.

Update site, therefore is a directory structure.

Plug-ins can be deployed via:– File system– Archive file– HTTP/HTTPs– Virtual file system

• e.g. Domino OSGi creates VFS for NSF databases

Update Site

(site.xml)

Features•feature#1•feature#2•...•feature#m

Plugins•plugin#1•plugin#2•plugin#3•...•plugin#n

65

Deploying Tasklets as Update Site

Create a Feature project in Eclipse


Create an Update Site project in Eclipse

66


Configure the Update Site project

67

68

Creating Update Site Database

Create an empty database from Eclipse Update Site template.

Import update site we just created.

Confirm our feature and plug-in has been imported into database.

69

Configure Profile to use our Update Site

Open OSGi profile we have created.– Add update site database into Sites section.– You may also select Features you want to

include.

Restart DOTS task and check for tasklist.

When you need to update your plug-in, load a new version to the update site database and refresh the bundle.

70

Security with Profiles

You might define additional Java security to the profile.

One reminder:– File-based update sites or plug-ins have unlimited security by default.– NSF-based update sites have a default set of permissions. Check DOTS documentation!

71

Multiple Profiles

You may create more than one profiles.– For testing– Tasklets that you use occasionally– Tasklets with different security needs– Heavily loaded Tasklets

You can create a second profile from the server console.

Second profile should be loaded separately.

72

Episode 4

Tips, Tricks and Good Practices!

73

Tips and Tricks on Programmability

Logging– Try not to use «System.out.println». (No output to log.nsf)– Instead, use «logMessage(...)» and «logException(...)»

runWhen argument helps– If you overload doRun method (run the same class for manual and scheduled operations

at the same time), runWhen will help you to identify which.

Parameterization– Manual tasks can get parameters through Console or Plug-in manifest (plugin.xml)

• «tell dots run XYZ param1 param2».• «args[n]», «getNthArgument(n)» or «getKeyedArgument(key)» can be used.

– For other tasks, you have multiple options.• «notes.ini» settings• Argument Resolver (refer to DOTS samples)• Profile parameterization extension via DXL (refer to DOTS readme.pdf)

74


Profile Configuration Document:

75


Life-cycle for a scheduled Tasklet– When a tasklet runs on schedule, DOTS class loader will create an instance from your

tasklet class. This instance will not be disposed until DOTS process stops.– This is a great advantage!


Multiple run-mode for the same Tasklet and different Java Instances– When building multi-purpose tasklets,

• Accomplish tasks with more than one functions via schedules & manual calls– Each run will create a new Java object independent from each other

• Scheduled calls persistent objects• Manual calls will be unloaded from memory afterwards

An example: Feed Reader for CollaborationToday.info– A managed queue for feeds to be consumed– Lots of tasklets

• Scheduled tasklet to read the next feed for new stories every 2 minutes• Scheduled tasklet to check if there is a new feed definition every 60 minutes• Manual tasklet to force refresh queue (when we have added a new feed)• Manual tasklet to force one or more feeds to be refreshed immediately

– All can be defined in a single class but will run on different Java objects!!!

76

77


Instantiation Problem– You can employ singleton pattern to create a global object within bundle.

• Activator class can be used to initialize your objects...• Be careful about synchronization.• Domino-based objects will be recycled after each run. Don’t persist those objects!

– In Feed Reader example;• We can design a bundle-level Queue manager,• Initialize it on bundle start (and even save it when bundle stops).

78


Feed Reader – Singleton Approach– Still, careful about lifecycles… NotesSession should be used carefully!

BundleBundle.start Initialize the Queue

Bundle.stop Save the Queue

QueueManager

(singleton object)

readNextFeed()refreshQueue()readFeed(id)

Tasklet Class

@RunEvery( every=2, unit=RunUnit.minute )Sched1() qm.readNextFeed()

@RunEvery( every=60, unit=RunUnit.minute )Sched2() qm.refreshQueue()

@Run( id="refreshQueue" )Manual1(args) qm.refreshQueue()

@Run( id="refreshFeed" )Manual21(args) qm.readFeed(args[0])

79


Remote Controller– DOTS might accept remote commands from other applications (e.g. XPages apps)– Remote commands are accepted from localhost only.– Listener should be configured from Profile document.

Source: DOTS Readme File

80


Progress Monitoring– «IProgressMonitor monitor» argument passed on tasklet methods.– Very useful for tasklets running long time.– Progress monitor can be used to...

• Inform DOTS task manager about how much tasklet has completed.• Aware of situation that might break execution.

Vector<?> views = db.getViews();

monitor.beginTask( "List the views", views.size() ); Init with # of steps

for ( Object view : views ){if ( monitor.isCanceled() ){

break; Break if canceled.}logMessage("\t" + ((View)view).getName() );try {

Thread.sleep( 1000 );} catch (InterruptedException e) {

e.printStackTrace();}monitor.worked( 1 ); One step completed

}

81

Problems you may encounter using DOTS

OSGi Context– DOTS runs in a different OSGi level than HTTP.– So there is almost no connection between DOTS and XPages

• You can start DOTS tasks from XPages, using remote controller socket• Refer to Readme documentation for configuration and a basic example

– Some possibilities• Communicate over Notes Document• Using REST API to send/receive information between DOTS and XPages

Extension Manager events– Problems with queuing– This feature has been removed in IBM Domino SE 9.0 Public Beta

Class Loading issues– Some Open Source APIs might have issues with OSGi class loading (e.g. Rome project)

82

Q&A

83

Thank you!

Twitter: @belgort

Blog: http://bruceelgort.com/

Twitter: @sbasegmez

Blog: http://lotusnotus.com

84 © 2013 IBM Corporation

Legal disclaimer© IBM Corporation 2013. All Rights Reserved.

The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

bp207 - meet the java application server you already own – ibm domino

Technology

domino dots

agents dots tasklets

osgi extension

logic osgi

osgi support

osgi matters

java agents

osgi level server tasklets