A DEVELOPER’S MIGRATION PLAN

by

REDHUAN D. OON

Boss of Open Source ERP Freedom

I D E M P I E R EOpen Source ERP on OSGi Technology

P o s s i b l e s a f e - h o u s e s : K u a l a L u m p u r - P a r i s - Vi e n n a - B a n g k o k - B a n d u n g - M a n i l a - B o g o t a - F o r t a l e z a - D h a k a

Godfather - Final ChapterSome eight years ago there were two parts of the Godfather series: http://compiere.red1.org/XML2AD.zip

Today is the day the final chapter is presented here to complete a great story. But first some nostalgia.

Open Source ERP beginning with Compiere in 2003 has inspired me deeply, providing not just the code of freedom to modify but the community of members from all around the globe to fraternise with. I got to see quite abit outside my otherwise peaceful suburb farm living conditions.

In Dortmund having a meal with Joerg Viola’s

group. Also present was Trifon Trifonov - 2009 Boarding a small pirate ship in Fortaleza, Brazil

- 2012

Twice in Libya - 2010 / 2012. This time around,

the brother leader is gone.

The Quest for Freedom

Coupled with this new cyber-world, it started with Marco Lombardo from Italy in 2004, where he first introduced me to his XML2AD which captures Application Dictionary or AD changes for porting to another Compiere instance. This avoid repetitive work to replicate similar changes from a particular module to other companies’ ERP systems that were also using Compiere at that time.

I helped broaden his circle by writing about his effort those early years that attracted Trifon Trifonov of Bulgaria and particularly Robert Klein in USA who tackled the problem from another angle with what he created as 2Pack. It is more user-friendly and it is inbuilt into the application itself allowing normal users to conduct such migration of modules without expert coder knowledge, right from the main menu itself.

With the passage of ADempiere from 2006-2011, and then the transition to iDempiere early 2011, based on the Equi-nox OSGi plugin framework, the 2Pack not only remained but evolved into something remarkably powerful. The reason is that first basic idea from Marco which is a universal idea of freedom. Every implementor’s wish is to avoid redundant work and enjoy the freedom to merge work freely from multiple sources and in the simplest terms. The OSGi adoption was first attempted by Schmidt András of Hungary in July 2008. Then Joerg Viola from Germany stepped in around March, 2009 and Low Heng Sin soon after and it is Low under sponsorship of TrekGlobal (run by another member of the family in USA formerly Idalica) that sees this OSGi jump completed in all its glory today.

Now, I have the final pleasure in this tutorial to expose step by step how you can migrate your changes in the format that is highly portable and modular. Today you can stay safe and happy in this family. But you are free to decouple. :)

Redhuan D. Oon! 8.22 pm, Thursday, November 8, 2012,! Kuang, Selangor, Malaysia

!

!

!

Sponsored by

!

!

BANGLADESH

He who is deaf, blind and silent, lives a thousand years in peace.

John Gotti, 1940-2002Boss of the Gambino crime family in New York City

DISCLAIMER:

This tutorial is written by someone with an attitude as he undergoes great mental stress to study as it involves thinking. As such, he will experience long lapses of insanity where his mind seeks refuge in ritual, escapism, denial and waste. Thus in order for him to produce (ouch!) any activity of human interest (ouch! ouch!) his work (argh!) is inevitably injected with kicks and prose (ahhh nice). Mind you, it still does not produce sufficient anesthetic effect compounded that he is at the moment far away from the green grassy countryside of Austria where he had aplenty last year in winter with his buddy Didiber there, who is another mental wreck.

Also he is using Victorian English so you just cannot pronounce them any other way.

DEVELOPER’S NOTE:

The reader should be one who comes from some basic ADempiere background. This tutorial expects some willingness, make that high degree of willingness to do some background research for project terminology that are new to newbies which you can easily google for further reading online, via a web browser, on a PC. There are ample resources already on our works particularly in www.adempiere.com and www.idempiere.org or strewn across many individual blogs including mine at www.red1.org and this tutorial is trying to put a cohesive teaching in a single place but only about a particular topic set. It may not get into other beginner’s gory details already written else-where and is written like a song. Hopefully after listening to it again and again you can sing it. The context of my work is usually to do justice to the underlying ideas and concepts that makes this pro-ject truly special. Even if you don’t understand a single idea, trust me, its less painful than without such tutorials.Even its illustrations make good cue and reference enough to be revisited again and again by an eager and open mind.

!

!

TABLE OF CONTENTS

Opening comments! 1

Migrate what?! 1

How to migrate?! 2

A module to migrate! 3

Doing a Pack Out! 4

Reviewing your code! 4

Preparing your Eclipse! 5

Create new plugin project! 5

Review your plugin! 7

Activating your Plugin! 8

Closing remarks! 10

Epilogue - Using Extension Points! 11

!

!

Opening comments

Trying to convey about something fundamental to the newcomer user implementor or developer is not easy. Like the Holy Books, it involves giving the pieces piece by piece instead of throwing a big chunk for the believers to choke on. So the first piece is very decisive and I been wondering what to tell first and the rest in succession at the right time. It can begin with the right questions and answers.

What is an Open Source ERP System?

I think this is the very first question to be asked. I would answer it that an ERP system is essentially a bunch of code and data. But for Open Source there is something extra. It has two owners. One is the primal mover or vendor or community that is responsible to maintain its codebase as given. The other are the users themselves that will modify the code to their own content and it need not be given back to the community because it is peculiar to certain users only.

Now with that simplistic explanation out of the way, we then ask the million dollar question of ‘what then is the problem?’

Why migrate?

Well, for anyone working with software the main big headache is maintenance of bugs, and upgrading the fixes or changes. What more now that the users have modified their own source and configuration model that it may conflict with the main fixes from the original owner or community behind it. An ERP is also a monster if you put everything together. It makes sense for the ERP core to be a common ground for most users and minor users find their way in using add-on modules and in iDempiere case, as plugin modules/

The challenge is to migrate safely and surely without breaking anything. The cover diagram earlier shows how there is a present day System A of end-users and they will want to get to System B which is a future system that is working with latest fixes or major upgrades and they need to migrate their own customised changes over. Or consider System B as a target system which belongs to a separate do-main that they want to send the changes over to.

Migrate what?

The next good question is what exactly is sent over? Is it just code? Yes and No. With the Compiere family which both ADempiere and iDempiere is based on, the changes can be both code and data. And the data is meta-data that configures the Application Diction-ary that determines the look and feel and model of the system such as Menus, Windows, Tabs, Tables, Processes and other settings that are not hard-coded but stored separately in such meta-data so to speak.

If we look again at the diagram but think of what we just said it will look like this on the right with two terms been highlighted - AD and code.

AD stands for Application Dictionary, the meta-data. What this diagram is saying is that the AD is sent over as 2Pack whereas the code is sent as a new plugin, developed using a developer’s tool called Eclipse.

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 1

How to migrate?

Then the next thing is to see both of that been done. The first is using 2Pack which will export the changes in the form of a 2Pack.zip that contains all the descriptions of the AD changes in XML format. The second is the Java code pack-ages formatted in a plugin structure. Both will be revealed later in exact details. In our diagram only the AD changes using 2Pack is illustrated well with those red arrow boxes. But the plugin module is not. It only has a simple box ‘Eclipse’. So we like to show that by way of a new diagram expanding more of that box:

Here we see new boxes with new terms. On the left at System A we see that the old code is wrapped in the old JVM style which is closed and you cannot intercept it through any con-sole. For System B you use the OSGi container which allows you to access its plugins via the ‘ss’ command.

The new system also has intra-plugin relation-ship defined under the MANIFEST file under the META-INF folder. In that MANIFEST file is defined its own plugin’s Activator code that will automatically detect any 2Pack.zip file in that folder and Pack In into the Database.

We shall see actual code in the action of been transferred. I have also made movies.

Meanwhile rest your tired eyes on a screenshot of the Eclipse environment that gives visual cue of things discussed.

1. The plugin module, POS Integration. 2. The Activator called by the Manifest file (3 and blue highlight). You can see the code snip is hardcoded to call the 2Pack.zip under the META-INF folder (4 and red highlight).

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 2

A module to migrate

Now we look at an example of a module to migrate. I have one here which is my work last year on the integration of Openbravo POS to ADempiere. In it I created a menu called POS Integration with 3 sub items which are 2 proc-esses called Export to Queue and Import Orders from Queue, and 1 window called Process Imported Orders.

All these are customised modification and is not part of standard ADempiere nor iDempiere. There exists a number of ways to bring over the changes here to iDempiere. One way is through the Log Migration scripts which is recorded when such customisation was done. I have done that and you can take them from the link given below.

Another way is to pack out and pack back in using 2Pack which has been around for some years within ADempiere. However there is now a break with the old as Low Heng Sin has refactored and improved 2Pack somewhat that in order to use the iDempiere Pack In, it should be on the 2Packs done using iDempiere’s PackOut.

So thus we start off with a fresh iDempiere v1.0.a and we can use the migration scripts given below, apply them using ANT build.xml and then proceed to the Pack Out.

https://sourceforge.net/p/red1/small/101/tree/trunk/POSIntegration/migration/

You will find a list of SQL scripts as shown here. The next thing will be the source code of the module. You can take at one go together with the plugin, migration scripts, the 2Pack.zip and the Openbravo POS that goes with it! Note that we are just focussing on the migration scripts and the code packages within as the customisation that we are migrating with. The link below is the whole finished end-product.

http://sourceforge.net/projects/red1/files/Software%20Packages/IntegratedPOSplugin.zip/download

Launching your iDempiere

If you are using Windows environment then you can install iDempiere easily on a fresh Windows PC or notebook that has no Java nor Postgres DB with this installer :

http://sourceforge.net/projects/red1/files/Software%20Packages/iDempiere1.0.a_Setup.exe/download

If needed, refer to the guides I written to use the installer , together with an Upgrade Assistant. If you are using non-Windows, then refer to the online resource at http://wiki.idempiere.org to get your iDempiere running. You can also jump ahead by having your iDempiere source-code running from Eclipse directly which is what we are limiting to within this tutorial.

Once your iDempiere is running, you are ready to apply the migration scripts from your module and do a Pack Out to produce a 2Pack.zip for a new plugin module. Again refer to online help on migration scripts. Let us see now how the Pack Out is done. Ensure your custom module has its own menu group such as the case is with my POS Integra-tion case above.

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 3

Doing a Pack Out

Login as System, and from the main menu or in the Search box type ‘Pack’ until you see the Pack Out item high-lighted (1). Open its window as shown below. At the main tab give the details that you want to call your 2Pack as (2).

In the second tab, select Application and Module so that the Menu selection pull-down appears (3). In it type the starting word of your menu item and it should appear for you to select it. Once that is accepted in the field, return to the parent tab and click on ‘Export Package’ button (4). IT will take a few seconds or so depending on how complex is your module structure.

Then at the bottom of the window at the status bar will display the path of the zip file created (5). We will take that file for the new plugin later. Now we can go and look at our customised code for moving them into the new plugin also.

Reviewing your code

Your code should be in packages somewhat similar to what I have here. Mine is not big for POS Integration but it covers sufficient variety of the ERP core. For example I have some org.adempiere.process classes in my own process windows, together with an org.compiere.process class which is ImportOrder.java. This one is an overloading of the original class in the Base package of ADempiere.

Previously in ADempiere we override that via a customization.jar but in OSGi, we do not use that anymore. We just have the new code in a different plugin but declaring the same package (org.compiere.process) and activated during runtime where it will be executed instead of the main base org.compiere.process package. This is basically the power of OSGi componentisation or modularity where we do not need to recompile the core or replace customization.jar

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 4

1 2

3

4

5

each time we make a change. Thus we are decoupled from the core and we merely add our plugin into the stack of other plugins. This means easy debugging and troubleshooting or maintenance as when any error occurred from our code, we merely cancel our plugin by stopping it during runtime in the OSGi console. I have tried that and it worked in my example plugin.

Preparing your Eclipse

You should again refer to the online wiki or resources to setup your Eclipse with Buckminster and Mercurial function-ality. Mercurial is needed to access the online mercurial bitbucket repository of the iDempiere sourcecode. Buckmin-ster is a maven type tool that materialises the 3rd party library jars in a managed and online way into a Target Plat-form. Once this is done as on page 2, with no red cross error marks, we are now set to create our first plugin.

Create new plugin project

Select New > Other (1) > Plug-in Project (2) and give it the name of your intended module (3).

Uncheck the Options > Generate an activator... (4) as we be using the ready-made AdempiereActivator.

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 5

1

2

3

4

Under the META-INF folder, find and open the MANIFEST.MF file (1) and at the Overview tab. check the Activator box and click on its browse button to look for AdempiereActivator class. You won’t be able to do that because you have not defined any dependency for your plugin yet. Go to the Dependencies tab (2) and add the plugin that con-tains it which is org.adempiere.plugins.utils (3). Also add the org.adempiere.base which is the core for org.adempiere.process and org.compiere.process packages which we are contributing to from our new plugin.

Note that under Imported Packages I have selected a library reference for my ActiveMQ code to work (4). Thus I need not carry my own library jars within my plugin.

Then return back to the Overview tab (1), click Browse (2) and you should be able to add the Activator class (3).

1

2

3 4

12

3

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 6

Review your plugin

Now we take a closer look at custom code and ensure they are put in proper packages (1) and its dependencies (2) resolved. Note that in an OSGi framework its Import of library jars are resolved by (a) TargetPlatform, (b) Other plug-ins which are defined in the MANIFEST file (3), (c) Internal Library jars also defined in MANIFEST. Below is how my plugin looks in the end.

But this is not all. Before your plugin can work, it has to be associated with the core plugin org.adempiere.base be-cause the bundle classpath works differently in OSGi where the core plugin is already set for the common packages calling such as org.adempiere.process. You do that by placing the Eclipse-RegisterBuddy (1) line anywhere in the file. (Hint: copy/paste from another plugin’s manifest!) You can also export your packages under the Runtime tab (2) so that it is visible to the other plugins that may use them. Note the Classpath panel on its right (3). That is when you need to use your own internal library jars. You then inform the MANIFEST there to authorise them.

2

1

3

With this done, you are quite set to go!

1

2

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 7

Demonstration Movies

By the way, here are some movies I promised.

This one is about creation of the 2Pack via Pack out: http://youtu.be/oZSnsfW2A2Y

This one is about creating a new Plugin: http://youtu.be/JjUgGJiXDD0

This one is about running the new plugin to Pack In and then function as your latest module addition!: http://youtu.be/TnqizjaCEm8

This step I will go through next.

Activating your Plugin

With everything in place and no red error Xs in your Eclipse we are going to run the new plugin from there. (Proper deployment will be covered in a future tutorial exercise).

First you do a RUN_ImportiDempiere (1) on your system so that the database is refresh as new and your module is not there yet and witness it exis-tence after this. You can alternatively go to another separate machine or instance of iDempiere to try this magic on.

Then in your Eclipse click on Run > RUN Configurations (2). Then select swingclient.product (3). Check the box on the left of your new plugin and set its Startup to default (4) as you will start this plugin via the console manually.

1

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 8

2

34

When the Java Client login box appears do not login as the 2Pack needs to Pack In first. If you login it will obstruct the Pack In process. Go to the console tab and type in ‘ss’ (1). You will see a stack of plugins with their status dis-played. ‘ss’ stands for short status.

Scroll up to look for your plugin and note its id number. Mine is 2 and so I type ‘start 2’ (2) next to get it activated. Then you will notice the Pack In working and you wait till it finishes with the OSGi prompt appearing again at the last line (3).

1

2

3

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 9

Now you can go to your login box and enter the application. But you need to do a Role Access Update first, and then relogin to see that your new module is now there.

You do not need to worry about your code as it is on standby in your new plugin! I tested mine in the movie and it does work. My overriding org.compiere.process.ImportOrder which is an enhancement of the old code as it handles Lo-cator Name instead of ID. When I stop the bundle by typing in the console ‘stop 2’ and run my ImportOrder again, it revert to the base code. This is so amazing as I did not stop the application but merely play around with start and stop commands. This proves that our ERP application has come out of the stone closet of Java and become truly dy-namic, componentised and modular.

Closing remarks

Of course developing more sophisticated plugins is possible with this sparkling new OSGi framework. There is tre-mendous power from this point on for the iDempiere project. Among them can be (a) further refactoring into more granular plugins for the whole ERP such as the base plugin be broken up further into finer pieces like what is done with callout and process plugins as well as the ui plugins, (b) functional wise as what is done with my own POS In-tegration, we could have done the same with Manufacturing and Fixed Assets or even core functions such as Invoic-ing, Payments and Reports, (c) enormous tools and utilities from Equinox and other community projects experiences. Been part of an open source world for OSGi, iDempiere will assimilate more each day as it has done the last years.

Whatever the future path maybe it is not letting up nor slowing down. As I was writing this tutorial, Carlos Ruiz via my Skype flashed me of more new changes to the iDempiere application as can be followed here: http://wiki.idempiere.org/en/Category:New_Features_v0.01. And he just released iDempiere alpha’s version only days ago!Things are becoming fast and furious with this magic cauldron, a phrase I borrowed from Eric Raymond’s writing: http://www.catb.org/~esr/writings/cathedral-bazaar/magic-cauldron/index.html, a must-read for the perplexed.

This tutorial is only meant as a can opener and a starter guide to many developers out there who can now avoid a steep learning curve or at least shorten it greatly. This has always been my signature in my tutorials. It is both my pleasure and itch to get what I learn to be out there for others to experience. It is also a legacy I wish to leave behind in my closing years, in years to come, God willing.

If you wish to make suggestions or just a word of thanks you can email me personally at red1@red1.org.

Thank you for been here, reading this, and hopefully become part of us. Peace and blessings of Allah be upon you.8.21 am, Saturday, November 10, 2012

- THE END -

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 10

Epilogue - Using Extension Points

Our don, Low Heng Sin remarked that my use of Eclipse-RegisterBuddy is not recommended and to use Extensions instead to define the processes.

So we get back to our Eclipse IDE below and call up the MANIFEST file. Kill the buddy (delete it) (1).

In the Overview (2) panel, click on Extensions (3) for your tab to appear. Call up the Extensions tab (4).

Click on the Add button and a selection win-dow (5) for Extensions Points will pop-up. Click on org.adempiere.base.Process (6) and it will return back to the window. Right click on the new line and select New and Process (7).

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 11

1

2

3

4

5

6

7

A new class is defined for you (1) but we do not want to code anew as we already have the classes. Click on the browse link (2) and a dialog box appears allowing you to choose your class (3).

Then you have to give an ID (5) to your class which will register that for the bundle classpath. Click back on the main line (4) if you do not see the ID input field. You can give it a label name too (6).

An additional movie showing all this is uploaded: http://youtu.be/HXTy9GvCgu8

~ See you at Buddy’s funeral ~

i D e m p i e r e O p e n S o u r c e E R P o n O S G i Te c h n o l o g y! A D e v e l o p e r ’ s M i g r a t i o n P l a n

!

C o p y l e f t 2 0 1 2 R e d h u a n D . O o n S Y S N O VA , B a n g l a d e s h P a g e 12

12

3

4

5

6