incredible odi tips to work with hyperion tools that you ever wanted to know

Incredible ODI tips to work with Hyperion tools that you ever

wanted to know Ricardo Giampaoli

Rodrigo Radtke

About the Speakers

Giampaoli, Ricardo• Oracle Ace• Master in Business

Administration and IT management

• EPM training instructor• Essbase/Planning/OBIEE/ODI

Certified Specialist• Blogger @ devepm.com

Radtke, Rodrigo• Oracle Ace• Graduated in Computer

Engineering• Software Developer Sr.

Advisor at Dell• ODI, Oracle and Java

Certified• Blogger @ devepm.com

DevEpm.com

@RZGiampaoli@RodrigoRadtke@DEVEPMWhat we'll learn

• Looping ODI Objects• Dynamic “OS” commands• Powerful “Dynamic Variables”• Substitution Tags and how useful they are• Dynamic Topology• “Third Command Tab“ Technic• ODI custom execution information• Stop, Start and Restart ODI priority jobs automatically

DevEpm.com

@RZGiampaoli@RodrigoRadtke@DEVEPMLooping ODI Objects (Regular Way)

• Too much work to control complex loop order and parameter variables

• More ODI Objects to maintain• Harder to make it dynamic

DevEpm.com

@RZGiampaoli@RodrigoRadtke@DEVEPMLooping ODI Objects (Champion Way)

• Easy to control complex loop order and parameter variables

• Less ODI Objects to maintain• Easier to make it dynamic• Be careful about the Log Counter and parallel executions

DevEpm.com


• Command on Source• Always a Select command

• Command on Target• DML commands

• Inserts, Update, delete, truncate, create…• ODI Commands

• odiStartScen, odiFileXxxx, OS Command...• Programming languages

• Java/groove/jython…

DevEpm.com


• Loop virtually anything• Can “Generate” dynamic code

• Using ALL_TAB_COLUMNS we can change the code in the “command on target” dynamically

• Change the columns dynamically• Change the code in the WHERE/AND statement• Creating SQL code (regular expression depending on

the amount of columns of the source table/repository)

• Using LISTAGG we can build models dynamically

DevEpm.com

@RZGiampaoli@RodrigoRadtke@DEVEPMDynamic "OS" Commands

• Use the same loop approach to get dynamic information from the source and apply on the target tab

• Any type of OS command• Great to automate EPM environments

• Stop, Start and restart services• Log windows events• Execute Esscmd and Essmsh commands (needs to configure

system variables in the Agent server to enable the execution of the commands anywhere in the system)

• Execute EPMAutomate for PBCS

DevEpm.com

@RZGiampaoli@RodrigoRadtke@DEVEPM

Dynamic "OS" Commands (Essmsh/Esscmd)

• For Essmsh/Esscmd commands we need to use the DOS command “echo” with the option -i

• “echo” will echo all the command until the | to the next command

• “echo” allow us to connect and then execute a maxl command in Essbase

DevEpm.com


Dynamic "OS" Commands (Essmsh/Esscmd)

• Command on Source• None. Set just the connection information to be used in

the target tab• Command on Target

• Essmsh• Alter, Create, Display, Drop, Execute, Import, Export, Refresh

and Query• Esscmd

• Get, Create, Delete, List, Rename, Set, Update, Validate, Reset, Remove….

DevEpm.com


Powerful “Global Variables“ (Regular way)

• Parameter table to store parameter values

• One column is the Session Name to get the parameter dynamically depending of the Session being executed

• Three more columns to store the parameter type, parameter name and parameter value

• All Scenarios will start with the SESSION_NM Variable

• This technic allow us to have just one variable to get different “LOG_PATH” for different scenarios

DevEpm.com


Powerful “Global Variables“ (Champion way)• The only thing that changes is the

select inside the variables:• We use the previous query as a sub

query to test if the query will return any rows from the parameter table for that specific Session Name

• If the results in empty the NVL command will return “GLOBAL”

• This allow us to create just one Global parameter for all interfaces

• Also if anything changes, we just need to insert a new value for any session name and the variable will bring that value just for that scenario

DevEpm.com


Powerful “Global Variables“ (Champion way)• Less variables to maintain and easier for deployment

• In most part the value of the variables are the same for all interfaces

• Servers name, paths, emails, application name…• We can leverage this technic using ODI Global variables

• No need to create the same variable in all projects• More flexible

• If a test needs to be performed or something changes the only thing needed is an insert for that specific scenario

• To rollback we just need to delete the Session specific row

DevEpm.com


Substitution Tags and how useful they are

• Substitution tags are part of the java substitution API• The API methods are java methods that return a string value• 4 type of Substitution Tags

• <%>, <?>, <@> and <$>• All of them does the same thing but each one has its own parse phase

• <%>: generates the command when it is sent to the agent, but BEFORE it gets to the agent;

• <?>: generates the code in the agent server, BEFORE the command is sent to Operator and BEFORE ODI variables are substituted

• <$> (available from 11.1.1.6 on): generates the code in the agent server, BEFORE the command is sent to Operator but AFTER ODI variables are substituted

• <@>: generates the code in the agent server, AFTER the command appears in the Operator and AFTER ODI variables are substituted

DevEpm.com


Substitution Tags and how useful they are• Create an ODI variable and write ‘TEST’ on it• Execute in Jython technology:

• Raise ‘<%= System.getProperty("os.name") %> - <%="#ODI_VAR"%>’

• Repeat for each substitution tag• Agent is Windows server 2008• Client is Windows server 2003

Subs Tag

Code generated Results

<%> raise 'Windows 2003 - #KSCOPE.ODI_VAR' Windows 2003 - TEST

<?> raise 'Windows Server 2008 R2 - #KSCOPE.ODI_VAR' Windows Server 2008 R2 - TEST

<$> raise 'Windows Server 2008 R2 - TEST' Windows Server 2008 R2 - TEST

<@> raise '<@= System.getProperty("os.name") @> - <@="#KSCOPE.ODI_VAR"@>'

Windows Server 2008 R2 - TEST

DevEpm.com


Substitution Tags and how useful they are• Different parse phases are useful to create dynamic

code• Heavily used in ODI KMs

• We may “loop” tags within tags• We can have a code in % phase passing information to a ?

Phase

DevEpm.com

@RZGiampaoli@RodrigoRadtke@DEVEPMDynamic Topology (Regular Way)

• Featured by multiple contexts• One Physical for each DB• One Logic For All Physics• One Context for each Physical

• Context nightmare• Tons of context to manage• If users executes scenarios in

operator, multiple context make it difficult

• In multiple projects one context could mean nothing

Amer

Apj

Emea

N DB

Amer

Apj

Emea

N DB

Target DWGloba

l

PhysicalContextLogical

DevEpm.com

@RZGiampaoli@RodrigoRadtke@DEVEPMDynamic Topology (Champion Way)

• Featured by dynamic topology• One Physical for each DB + one

physical with variables (Dynamic)• One Logical for each Physical• One Context for all

• Context heaven• One context to manage• Easier for users to execute• One context for all projects

• Works in ODI 11 and ODI 12 with patch 21156142

Amer Apj Emea N DB

Global DWDynamic

Amer Apj Emea N DB

Dynamic

PhysicalContextLogical

DevEpm.com


• Create one variable for each field in the physical topology

• Data Server:• User: #ODI_PROJECT.CONN_USER• Password: #ODI_PROJECT.CONN_PASS• JDBC URL: #ODI_PROJECT.CONN_URL

• Physical Schema:• Schema: #ODI_PROJECT.CONN_DATA_USER• Work Schema: #ODI_PROJECT.CONN_WORK_USER

• Need to inform the project_code.variable_name

DevEpm.com


• Create one procedure• One step for each database

to get the connection information

• The connection information will be retrieved from the “Command on Source” tab

• “Command on Target” to execute a scenario passing all connection information needed

DevEpm.com

@RZGiampaoli@RodrigoRadtke@DEVEPM“Third Command Tab“ Technic

• Sometimes we need more than a command on source and target

• Executions of Maxl in multiple Essbase applications

• Select on source with application information (Loop)

• Command on target (to execute what we want)

• Since we are using the Oracle technology in the command on source we cannot get Essbase login information

• For this example we need a “Third Command Tab”

DevEpm.com


• Create a new step to get the connection information

• On the command on source set the Essbase technology and schema

• On the command on target use Java technology and create 3 java variables that will hold the connection information

• Use the Java Variable in the next Steps

DevEpm.com


• Loop Maxl/Cmd commands• Get the return of a SQL into a Java variable to use in

other steps• This is also useful because the java variable is replaced for his

value in operator• Anything you want to store and use later within the

scenario execution• If you want to use inside a children scenario you need to pass

it to a variable inside the scenario

DevEpm.com

@RZGiampaoli@RodrigoRadtke@DEVEPMODI Custom Execution Information

• ODI repositories contains all kind of useful information that we can retrieve at run time

• Execution information, current variable values, jobs in execution…

• We can retrieve all this information using the previews technics

• Inside a procedure in the command on source set the ODI repository connection

• In the command on target create a SQL and add it to a java code that will connect in the database to retrieve the information

DevEpm.com

@RZGiampaoli@RodrigoRadtke@DEVEPMODI custom execution information

• In another step we just need to “Raise” the previous variable

• Now you can use anywhere in the package

• This will show in the operator the value that all existing variables until that point has

• Works with all kind of variables: refresh, set, declare, global, project only…

DevEpm.com


Stop, Start and Restart ODI priority jobs automatically

• There’re some commands that cannot be executed within ODI tools:

• Restart load plan, Restart session, Stop load plan, Stop session, Start load plan, Start scen

• But all these commands can be executed outside ODI using some .bat/.sh files located in the agent server:

• oracledi\agent\bin (Only exists in the agent)• The command to ran is:

• stopsession <session_number> “-AGENT_URL=<agent_url>” [“-STOP_LEVEL=<normal(default)|immediate>”]

DevEpm.com


Stop, Start and Restart ODI priority jobs automatically

• If we want to stop all jobs before running a priority job, we could get all running jobs from ODI repository and stop them

• To do so we need to get the Agent URL from the master repository

• Get the job information from the work repository

• And stop all the sessions

DevEpm.com


Stop, Start and Restart ODI priority jobs automatically • Useful for priority job

• Check if other jobs are running and stop them automatically

• Executing jobs respecting a time window• We can stop jobs that executes longer than a specific time

window• Depending of the error restart automatically a specific job

DevEpm.com


Ricardo Giampaoli – TeraCorpRodrigo Radtke de Souza - Dell

Thank you!

incredible odi tips to work with hyperion tools that you ever wanted to know

Data & Analytics