let me begin by introducing myself. have been a progress ... · pdf filei have been a progress...

Let me begin by introducing myself. I have been a Progress Application Partner since 1986 and for many years I was the architect and chief developer for our ERP application. In recent years I have refocused on the problems of transforming and modernizing legacy ABL applications. A key part of transformation is creating patterns and infrastructure for the target architecture. Since the introduction of 10.1A, I have been paying a lot of attention to best practice patterns for writing OO ABL.

1

What is it that I mean by “Patterns for managing relational data in OOABL”? What problem does this relate to?

As we all know, data is most often stored in relational databases. This is certainly the case for an OpenEdge application. In our pre‐OO ABL experience, we are all used to the facilities in ABL for dealing directly with retrieving and storing data from and to the database and with manipulating it in local analogs such as temp‐tables and ProDataSets. Not everyone may be using ProDataSets yet in their own application but I’m pretty sure that most of youmay be using ProDataSets yet in their own application, but I m pretty sure that most of you have at least been reading about how wonderful they are. ABL was designed to be a language for developing On‐Line Transaction Processing (OLTP) applications, so ABL is very good at this kind of manipulation.

4

But, OO applications are not structured on tables and tuples like a relational data base is, but rather is structured on objects and collections. Objects connect directly to other objects by relations. When that relation is one to many, then one uses a collection to hold the objects. The whole notion of linking things with keys or having data arranged in columns and rows is just not a part of the OO paradigm.

So, if we are storing data in a relational database, but trying to work with the data in an OO fashion how are we going to do that and how are we going to exploit the features of ABLfashion, how are we going to do that and how are we going to exploit the features of ABL for handling such data.

5

As I have said, in a relational data model or RDM, data is arranged in tuples and tuples are arranged in tables. A tuple in one table is linked to tuples in other tables, or even the same table, by foreign keys. Access by these keys is enhanced by indexes so that we don’t have to search the entire table looking for matching values. This RDM applies not only to the way data is stored, but to the way that we handle data in our non‐OO ABL applications where much of the data is in buffers, temp‐tables, and ProDataSets.

7

Contrast this with the OO model where both data and behavior are encapsulated into an object and the objects are connected by direct relationships. In traditional ABL we think in terms of “looking up” some data. In OO we think in terms of “navigating a relationship” to get to another object for its data and behavior.

8

Some relationships are one to one or zero to one and are a simple link. When a relationship is one to many, conceptually there is a still a simple direct link which one can navigate, but under the covers one uses a collection. Collections do come in a variety of forms, but most are very simple and are sequentially accessed because of the expectation that every member of the collection will receive the same processing.

9

In ABL, the closest thing to a 3GL collection is the Work‐Table, a long deprecated feature. It has order, but no index and is only efficient if used for sequential access. Order is simply the order in which records are added unless one takes care to position the cursor in the appropriate place prior to each add. Work‐tables do not support objects, however.

10

In a 3GL OO model, everything that happens is about interactions between currently activeinstances of objects without reference to how or whether any data will be persisted. Compare that with traditional non‐OO ABL where much of the processing is directly referencing a database buffer or something that acts like a database buffer. In the 3GL model, persistence is something that happens off at the very edge of the application in a separate layer. In the active instances, all of the object to object relationships are direct links, not something searched by keys and indexes as one does with stored data or local RDM artifacts like temp‐tables and ProDataSetsRDM artifacts like temp tables and ProDataSets.

11

Thus, collections seems very far from the kinds of features we are used to using in ABL. While simple collections are likely to play some role in OO ABL … which we will explore further in a few minutes, slavishly imitating 3GL OO seems like we would be ignoring all the advanced data handling features in ABL. So, what patterns should we be using?

12

Before exploring some patterns for handling relational data in OO ABL, it might be a good idea to briefly review the standards we are going to use to evaluate which of these is best. We have both prevailing standards, i.e., ones which should already apply to our traditional ABL development, and OO standards, i.e., ones which relate specifically to what is considered good OO practice.

Prevailing standards include:•Ease of maintenance;Ease of maintenance;•Clarity;•Nimbleness, i.e., the ease and speed of making changes;•Rapid Development;•Good performance; and•Adherence to normalization standards.

14

Standards which come from OO best practice include:• Encapsulation ‐‐ All of an object’s data and behavior is hidden within an object and the

implementation is not exposed to other objects;• Single Responsibility Principle ‐‐ One object should only have a single “responsibility” or

role;• Abstraction – Visible data and behavior should be separated from the details of its

implementation;• Inheritance and Generalization – Common data and behavior of related objects shouldInheritance and Generalization Common data and behavior of related objects should

be removed to a common ancestor; and• Polymorphism – Multiple types can be substituted one for the other and exhibit

related, but different behavior by type.

15

Given the pride and joy with which many people regard ProDataSets, it is not surprising that there have been multiple patterns advanced for handling relational data in OO which revolve strongly around ProDataSets. Indeed, one can regard a ProDataSet as a kind of proto‐object, since it encompasses some collection of data and has a limited number of methods and events which have both built‐in behavior and the ability to override that behavior.

The three PDS‐based patterns we will consider today are:The three PDS based patterns we will consider today are:• The pattern which John Sadd has written about in various whitepapers on PSDN;• A pattern called Model‐Set‐Entity which is a part of the Progress Professional Services

Cloudpoint offering; and• A pattern called NSRA for NomadeSoft Reference Architecture from the Argentinean

company Nomadesoft.

I should note that the Cloudpoint offering is used by PPS in the Americas and I would be very interested in hearing from someone in the EMEA PPS organization and or from iMoabout their patterns to add to this discussion. I would like to particularly thank Phil Magnay for his choosing to share many details of the M‐S‐E pattern with me and David Abdala of Nomadesoft for helping to clarify their pattern for me. I should not that John Sadd and I have been arguing about this subject in a very friendly way for years and some of you may have caught our joint session at the on‐line Exchange this year. That sessionof you may have caught our joint session at the on line Exchange this year. That session was the trigger for my deciding to develop this material into a more formal review.

17

Let’s start with the “elephant in the room”, i.e., the PSC “Whitepaper” pattern. I call this h l h i h b i i hi h h b i hthe Elephant in the Room because it is a pattern which has been very conspicuous as the only published pattern, other than my dissenting whitepaper. Coming from PSC’s primary spokesperson in the area, many people have regarded this as an official PSC position, despite John’s effort to characterize it as something more on the order of “here are some thoughts”.

This pattern is very similar in some ways to what one might do in non‐OO programming,i e the object responsible for getting the data from the database builds the PDS mosti.e., the object responsible for getting the data from the database builds the PDS, most often using some form of the PDS’s FILL() method. That PDS is passed to a Business Entity object in the Business Logic layer. The BE is essentially a stateless set of logic which acquires state when it receives the PDS.

Any one PDS may contain one or many instances of the top level entity. If there are multiple entities involved, e.g., an Order and its Lines, then all of that data is in a single PDS, not separated out into different object types as one would expect in traditional OO.PDS, not separated out into different object types as one would expect in traditional OO. The BE has the notion of the Current instance and all operations apply to that Current instance. If there are multiple top‐level entities, one can navigate among them. Thus, the BE and its PDS actually fulfill the function of a collection, but there is only one collection available at a time. E.g., if the OrderLines had a property called ProductGroup, there is nothing in this mechanism that would directly support multiple collections by ProductGroup. Since ProductGroup is simply one of the attributes in a temp‐table, any ProductGroup processing would simply be implemented by internal logic in the BE directly on the temp‐tables.

18

On the pro side, this pattern is one that is very similar to how one might use PDSs in a procedural environment, so it should be easy to learn for existing ABL programmers who have used PDS. It certainly makes full use of PDS capabilities and it should be easy to interface with any pre‐existing non‐OO code.

19

But, if John and I have been arguing about this for years, there must be something that I see as flawed. I see it as poorly encapsulated because the DAO and BE have to share the PDS definition. I abhor include files. It is also poorly encapsulated from an OO perspective since the data is separate from the logic instead of their being in the same object together. And, it violates the Single Responsibility Rule since a BE can be both a single instance or a collection. If one needs to deal with a collection as a set, not just traverse the individual members one by one, then one has to have a second BE for the set and it also has to share the PDS definitionthe PDS definition.

I should note that the idea of sending a data packet message between layers is often considered good OO programming practice, but that using a PDS for that message implies that each object knows more about the implementation that is considered desireable.

I also find the idea of a full‐fledged PDS to send the data for a single instance of a simple object to be rather heavyweight.

You can probably tell that I am not going to recommend this pattern and I suspect that PSC will not be defending it much longer.

20

I have recently had an extended interaction with Phil Magnay of the North American Progress Professional Services organization about the Model‐Set‐Entity pattern which is part of the CloudPoint framework used by them in their consulting practice throughout the Americas. Phil has also recently started a thread on the OO forum on PSDN to talk about this pattern. I am calling it the “Elephant Not in the Room” because it is a pattern used by Progress in their consulting business, but it has not until this recent shift been one which has been visible to the general public. Unlike the Whitepaper pattern, which was never intended to be a proscription for direct production implementation M‐S‐E is used inintended to be a proscription for direct production implementation, M S E is used in creating production systems.

21

Model‐Set‐Entity encapsulates all of the data for one or more instances of an entity and its dependents, e.g., Orders and Lines in a PDS which is fully encapsulated by an object of type Model. The Model has no direct database access. To access a specific instance of the object type in the Model, one requests the Model to construct a Business Entity class. The Business Entity class contains the logic of that entity and exposes properties appropriate to the interface for that entity. They properties use Value Accessor method on the Model which reference the data in the ProDataSet based on an identifier for the Entity. Thus, with respect to the data of the Entity the BE is functioning as a façade to the data in the PDSrespect to the data of the Entity, the BE is functioning as a façade to the data in the PDS. When a set or collection is needed, one requests the Model to create an EntitySet object which contains an identifier for a query in the Model whose WHERE clause identifies the members of the set. Thus, unlike a 3GL collection, the EntitySet contains no actual objects, but is just a pointer to data in the Model which can be turned into Business Entity objects when processing is needed.

22

In sequential access of the instances in a collection, it is typical to create, process, and delete Business Entity objects in sequence and only one actual Business Entity object will exist at any one time. This works because the data is in the Model, not the Business Entity. If one is dealing with the dependent entity, e.g., the lines of the Order, one would have an EntitySet object corresponding to the lines of the individual order and one would create, process, and delete the Business Entity objects for each line in a similar fashion.

23

To fill or persist data in the Model, a reference to it is sent to a Data Access Controller (DAC) object. This DAC object is generic and not specific to the Model. Using a SuperClass of the Model, the DAC uses the Decorator pattern to attach logic to the Model to accomplish the filling or saving of data in the Model. These Decorations are detached when the Model is returned to the Business Logic layer so the Model itself knows nothing about persistence of the data. Models may be sent to the DAC for persistence of an individual instance within the set or for the set as a whole.

I should note that there is a LOT of detail behind this quick, basic review, but there is currently no whitepaper or other document which you can read. There is the recent thread on the OO community on PSDN, although you should expect it to take a while before a very comprehensive picture develops. Subsequent to this meeting, I will be preparing one or more whitepapers to go into this analysis more deeply and you can expect a reasonably detailed description there, but, naturally, from my perspective, not PPS’s. Hopefully, they will produce a whitepaper at some point, but that may take a while.

24

I will say that, while there are aspects of this pattern which I do not like, and I will get to those in a moment, it certainly is a very thoroughly thought out pattern which has received a great deal of attention and experimentation. There is no question that this is a production ready pattern. Unlike the Whitepaper pattern, the PDS is strongly encapsulated in a single object and there is no question that they are getting full use of the facilities of the PDS. With some qualification, there is good layer separation between the Data and Business Logic layers. In many respects, while the actual implementation of the BE and ES objects is very different from their 3GL counterparts their behavior and characteristics areobjects is very different from their 3GL counterparts, their behavior and characteristics are in many ways very similar … one just needs to forget about the “man behind the curtain” of the Model.

25

I should note up front that many of the objections I have to this pattern are ones that not everyone will agree with In particular any of my remarks about not conforming to what I consider to be OO best practice thewith. In particular, any of my remarks about not conforming to what I consider to be OO best practice, the architects of this would say either are perfectly good OO practice and/or are justified in the context of the pattern. You will have to make your own judgments.

Most of my issues about this pattern relate to issues of OO best practice, including:• The value accessor methods on the Model duplicate the corresponding properties on the BE. This

violates normalization because the same value can be reached by two ways. Their argument is that in practice the Value Accessor methods can only be used by the appropriate BE because the internal identifier which connects the Value Accessor methods to the specific instance are known only by the Model and the BE.

• Queries in the Model to support the ES objects violate encapsulation because it is part of the internal• Queries in the Model to support the ES objects violate encapsulation because it is part of the internal implementation.

• While PPS describes this pattern as supporting generalization and inheritance, the implementation used to provide this in the dependent objects is to use the Decorator pattern to attach subtype specific data and behavior to the base class. This was done to avoid the complex inheritance structures which can result from a “cross‐tab” of subtypes with objects like OrderLine, but I feel it is far better to decompose the complex object into a base and Delegates and then provide true inheritance within each Delegate. The Decorator pattern was primarily intended to provide transient additional functionality to an existing object, e.g., a scroll bar on a window, but in this pattern is used both as a substitute for Delegation and Generalization and is the basis of attaching persistence code to the Model, rather than truly separating the Model from the data so rcethe Model from the data source.

• While the description sounds elegant when considering a complex case like Orders and OrderLines, it seems like a very heavyweight architecture for a single instance of a simple object.

There are more specifics which I will get into in my whitepaper and discussions on PSDN, but the overall pattern here is that if one takes the pattern as presented it comes across as very thorough and well‐developed, but as one looks at implementation details, I find myself thinking that I wish it didn’t do things in the way it does … although in many cases there are no other good choices if one assumes the basic structure … and generally feeling like there is much more weight and complexity than needs to be there, a complexity which to me runs counter to the desired simplicity and encapsulation that should go with OO development.

26

Finally, in our consideration of PDS‐centric patterns, let’s take a look at NSRA which emerged in a recent discussion on the PEG. NSRA was developed by Nomadesoft with the goal of taking general OERA principles and producing a practical model for production implementation.

27

In some way NSRA seems similar to M‐S‐E, but in other ways it is very different. Like M‐S‐E, all data is contained in single base class, which they currently call the “Planet”, and there are classes which are somewhat like Business Entity classes which are used to access dependent objects. They call these objects “Satellites”. The Planet can have data for one or more instances of the top level entity and has a concept of Current, i.e., it also functions as a collection which can be serially processed one instance at a time. Properties of the top level entity for the current instance come directly from the Planet.

F th li f th d i il d l l titi th Pl t S t llit hi h hFor the lines of the order or similar second level entities, the Planet spawns a Satellite which has a buffer to the corresponding temp‐table in the Planet. Thus, the Satellite also has a concept of Current and functions like a collection for the second level entity. The use of the buffer avoids the need for the Value Accessor methods of M‐S‐E, but means that the Planet and Satellite must share the temp‐table definition. They consider this acceptable since the Satellite is entirely dependent on the Planet and is effectively a façade for the data in it.

The Planet is a subset of a Data Access Object and thus has the logic to fill itself NomadesoftThe Planet is a subset of a Data Access Object and thus has the logic to fill itself. Nomadesoftconceives this as the Planet living on the Data Layer side but the Satellites living on the Business Logic side. Separation from the actual data source is provided by the use of Data Source objects in a fashion similar to some of the PSC whitepaper materials.

Actual collection objects are only created for special purposes because the Planet and Satellite fill most functions.

28

This pattern also makes strong use of PDS capabilities. Both the database access aspects and some other aspects seem less complex than with the CloudPoint M‐S‐E pattern and there is a more traditional pattern of generalization. The use of the buffer over the Value Accessors is simpler and more private, but raises its own concerns. I should note that PPS also tried using buffers and has generator code to use them instead, but decided in the end that the Value Accessor approach was nearly as performant and provided better isolation.

29

However, there are also things to object to in this pattern. The ambiguity of a single object behaving as both an instance and a set violates the Single Responsibility rule and the use of the buffer to join Planet and Satellite violates encapsulation. The separation of Data layer and Business Logic layer seems much murkier here compared with CloudPoint.

In general, it shares many of the questions that I have about all of the PDS‐centric implementations. I.e., while it is friendly to an ABL programmer experienced with PDSs, it doesn’t seem very OO‐like overalldoesn t seem very OO like overall.

30

Overall, I think the PSC Whitepaper pattern is the poor cousin here and I suspect that after John has the chance look over CloudPoint that he will be one of the first to agree. John needed to write something to get us started thinking on this subject, but he never pretended what he did to be a finished work or ready for production deployment. So, let’s move on with that in mind.

The Model‐Set‐Entity pattern is certainly very impressive and likely to appeal to a lot of people There are parts of it which have a nice OO feel It may be a little complex topeople. There are parts of it which have a nice OO feel. It may be a little complex to implement by hand, without PPS’ MDA generation tools, but once documented and provided with examples, it certainly has some appeal. However, I also feel that there are some fairly serious objections on OO principles that should make us, at the least, suspicious and continue to look around.

NSRA is a somewhat simpler pattern than M‐S‐E in some respects, while sharing some of its qualities, but I don’t think it is yet as fully fleshed out as M‐S‐E and it has its own OO issues of concern.

31

Bottom line:• The strength of all three of these comes in use of PDS, a notable ABL feature;• The weakness of all, perhaps, comes also from use of PDS which, at bottom, is not

very OO and requires us to do non‐OO things;• If committed to the use of PDS, both CloudPoint and NSRA have good ideas to start

from.

32

Back when OO was first added to ABL, all we had for guidance were PSC whitepapers and those, frankly, were not based on production development. Interesting ideas to think about, but not really finished patterns.

Because of features like WRITE‐XML and READ‐XML, some people jumped on the idea of using temp‐tables everywhere, but that lead to issues with the Too Many Temp‐Tables problem.

Sharing of temp‐table and PDS definitions, while common in procedural ABL code, has always seemed to me to be a bad idea in terms of encapsulation … not quite up there with shared variables, but certainly not the sort of thing which we should be looking at as the basis for a new OO paradigm.

So, let’s set temp‐tables and ProDataSets aside for the moment and start from the ground up to consider what we might want in an OO pattern.

34

All this focus on temp‐tables and ProDataSets has us thinking about sets of things or things i h f d d h hi h i f l i f b hwith sets of dependent other things. That is a part of our solution, of course, but at the

core, our starting point should be focusing on an individual, isolated business entity.

OO principles suggest that we should encapsulate behavior and data in the same object. Yes, there are the Façade and Delegation patterns where that isn’t always the case, but those are for special cases. Your basic everyday Business Entity ought to have both parts encapsulated.

If we implement data as properties, that is very simple and clean and we can put all the behavior in there we need as well. What we get is the ABL equivalent of a POJO, Plain Old Java Object. Perhaps we should call it a PABLO, Plain ABL Object.

We do have to provide our own serialization, at least until PSC gets around to extending WRITE‐XML functionality to classes, but that isn’t difficult or very time‐consuming, particularly if we use a generator to do it.particularly if we use a generator to do it.

This “PABLO” gives us a focus for clear Single Responsibility and a nicely encapsulated focus for maintenance and separation of responsibilities.

OK, so now what about collections, sets, and structured complex things like orders with lines…

35

Let’s think about collections a little. We know that a collection in the 3GL world is the fundamental infrastructure used to represent a one to many relationship such as we have with things like Order and OrderLines.

Some people have proposed embedding the OrderLine as a temp‐table in the Orders object, but then, if we have a lot of orders we have a lot of temp‐tables and we are back with the TMTT problem.

Besides, Order and OrderLine are really two different things with a relationship, not something which is an internal aspect of the other.

Very often there isn’t just one relationship between such thing, but many. I.e., we have the lines for a particular order and we have the lines in each product group on that order. If there are three product groups in the order, that is at least four different collections of lines.

36

So, let’s consider how we might implement collections. I mentioned earlier that an ABL Work‐Table shares a number of characteristics of lightweight 3GL collections, but, unfortunately, since it is a deprecated feature, it does not support fields of objects.

We could create a collection object which contained a temp‐table of the data in the objects and then create an individual BE object per row on demand.

But what if we put the actual BE objects in the temp‐table as Progress Lang Objects?But, what if we put the actual BE objects in the temp table as Progress.Lang.Objects? Then, the same object can be in as many collections as we want because what is actually in the temp‐table is a reference to the object.

37

So, how do we decide between these two approaches, data or actual objects in the temp‐table? Data is familiar and comfortable, but we have to create the BE object anytime we want to do anything. Touching the data outside the object, while tempting, would need to be forbidden. Using data in the temp‐table, we have two copies of the same data when an object has been created and this presents a synchronization issue. Yes, we can sync at check‐in, but if we ask any other questions of the data while the object still exists, we may not have an accurate view. Moreover, using data means that we have a problem putting the object in two or more collections at the same time However it is also very easy tothe object in two or more collections at the same time. However, it is also very easy to provide multiple indexes on the data to provide different groups and sort orders, although that is not very generic.

38

Using objects in the temp‐table would seem to be indicated by the OO tradition. Then, they already have behavior built in and don’t need to be constructed when they are to be accessed. There is no issue about the same object being in as many collections as we want. But, if we want indexing, be have to add data columns for the index data since we can’t index on object properties.

39

So, on balance, it seems like there is a strong argument in favor of using actual objects in the temp‐tables for collections. This is the approach I took for a set of generic collection and map classes that one can find at the URL shown. These are derived from the Java collection and map class hierarchies and parallel them closely. The accompanying documentation covers the relationship in some depth.

A traditional 3GL OO person would consider our job done at this point since all they want is simple generic collections As ABLers though we tend to think that we can do better andsimple generic collections. As ABLers, though, we tend to think that we can do better and take advantage of the multiple index and sort options of temp‐tables.

40

We can, of course, simply create specialized objects containing temp‐tables and add all the fields and indexes we want, but we still have a potential problem of syncing the current state of the object with any of that indexed data. And, we really would like to keep the collection objects generic if we can.

Enter the SuperMap, so named because it is like the Map class in my generic set, but instead of having a single key ‐ value pair, it has multiple keys. My current concept is to define both a completely generic version and a pseudo‐generic version I will talk aboutdefine both a completely generic version and a pseudo generic version. I will talk about this more in a moment.

The business entities will be stored in the temp‐table as a Progress.Lang.Object and data fields will be provided for indexing. Standard methods will be provided for navigation and retrieval like the existing Map class. Additional subset method will return simple collections of subsets.

To handle the sync of object state and the data in the indexes, named events, new with 10.2B, will be fired by the object when the data value changes and will be subscribed to by the SuperMap. It seems likely that these events will be useful elsewhere as well.

41

The pseudo‐generic version will be a template pattern for creating individual instances of a SuperMap for a specific use, such as one might use with a program generator. It will be simpler than the generic version and have a more intuitive interface, but at the expense of being no longer truly generic. One SuperMap can cover all of the relations between any two object types.

42

The completely generic version will be added to the other generic classes so it can just be used and you don’t have to worry about its implementation. The downside of this is that the interface to it will also have to be generic and thus less intuitive in terms of the problem space.

43

So, we now have simple PABLO business entity objects, generic collections, and a SuperMap for complex sets of relations. What else do we need?

Data access is completely separate from any of these business layer objects and this provides a good context for using a PDS close to the database since it is an efficient and simple way to obtain a local copy of data with which to build the objects and a simple way to persist it when done.

I will be experimenting with a couple of different models, including binding the object factory to the DAO or separating them so that the factory can live in the Business Logic layer.

Objects are simply instantiated directly from the persisted data using the constructor or an init method as preferred. Collections are created by simply adding the objects to the appropriate collection type.

All extremely simple.

44

This overall pattern produces a model which is extremely close to what has been understood as OO best practice for years. The one funny part about it is that it uses a temp‐table as the internal implementation of a collection in cases where the power of a temp‐table isn’t really needed, but that is an implementation detail and only should worry us if there is a performance impact.

I believe this approach provides an improvement in maintenance and comprehension because of the simple encapsulation and clear responsibilitybecause of the simple encapsulation and clear responsibility.

It does everything one wants an OO implementation to do – polymorphism, delegation, whatever you want for the problem, without using odd workarounds.

45

It might be disappointing that we have pushed the PDS to the edge of the application, but at the same time we are using the PDS to do what the PDS is good at, i.e., being a local surrogate for the database. When we are thinking relational, we use it. When we are thinking object, we don’t. There are some performance questions which need testing here, but my expectation is that there is no problem. And, as mentioned, creating the truly generic SuperMap might be complex, but it is a problem I will have to solve only once and then it will simply be a tool for people to use.

46

let me begin by introducing myself. have been a progress ... · pdf filei have been a progress...

Documents