designer and workflow
TRANSCRIPT
-
8/6/2019 Designer and Workflow
1/140
Informatica Training
DES IGNER
-
8/6/2019 Designer and Workflow
2/140
:
The transfer of data is called transformation
A transformation is a repository object that generates, modifies, orpasses data
The Designer provides a set of transformations that perform specificfunctions
Transformations in a mapping represent the operations theInformatica Server performs on the data.
Data passes into and out of transformations through ports that youlink in a mapping or mapplet.
Transformation
-
8/6/2019 Designer and Workflow
3/140
:
Transformations can be active or passive
A passive transformation does not change the number of rows thatpass through it,
An active transformation can change the number of rows that passthrough it
Transformations can be connected to the data flow
An unconnected transformation is not connected to othertransformations in the mapping
It is called within another transformation, and returns a value to thattransformation.
Transformation
-
8/6/2019 Designer and Workflow
4/140
.
Joiner Transformation
The Joiner transformation joins two related heterogeneoussources .
.The combination of sources can be varied
The Joiner transformation uses a condition that matches one or more
pairs of ports between the two sources.
-
8/6/2019 Designer and Workflow
5/140
.
Joiner transformation
You can use the following sources:
Two relational tables existing in separate databases
Two flat files in potentially different file systems
Two different ODBC sources
A relational table and an XML source
A relational table and a flat file source
Two instances of the same XML source
-
8/6/2019 Designer and Workflow
6/140
:
Creating a Joiner Transformation
In the Mapping Designer, choose Transformation-Create. Select theJoiner transformation
Drag all the desired input/output ports from the first source into theJoiner transformation
-
8/6/2019 Designer and Workflow
7/140
:
Edit transformation
Double-click the title bar of the Joiner transformation to open theEdit Transformations dialog box
Select the Ports tab.
-
8/6/2019 Designer and Workflow
8/140
:
Port tab
Add default values for specific ports as necessary.
-
8/6/2019 Designer and Workflow
9/140
:
Setting the condition
Select the Condition tab and set the condition
Click the Add button to add a condition.
-
8/6/2019 Designer and Workflow
10/140
:
Defining the Join Type
join is a relational operator that combines data from multiple tablesinto a single result set
You define the join type on the Properties tab in the transformation
The Joiner transformation supports the following types of joins:
Normal
Master Outer
Detail Outer
Full Outer
-
8/6/2019 Designer and Workflow
11/140
The Joiner transformation requires two input transformations from two separate
pipelines.An input transformation is any transformation connected to the input ports of thecurrent transformation.
In the example, the Aggregator transformation and the Source Qualifiertransformation are the input transformations for the Joiner transformation
Joiner in Mapping
-
8/6/2019 Designer and Workflow
12/140
You cannot use a Joiner transformation in the following situations:
Both input pipelines originate from the same Source Qualifier transformation
Both input pipelines originate from the same Normalizer transformation.
Both input pipelines originate from the same Joiner transformation
Either
You connect a Sequence Generator transformation directly before the Joinertransformation
Configuring the Joiner Transformation
Master and detail source
Type of join
Condition of the join
input pipeline contains an Update Strategy transformation
Condition of joiner Transformation
-
8/6/2019 Designer and Workflow
13/140
The Joiner transformation supports the following join types
Normal (Default)
Master Outer
Detail Outer
Full Outer
An example of a partial mapping with two Joiner transformations in the sametarget load order group and three pipelines to process the data in the correct order
The join Types In Joiner Transformation
-
8/6/2019 Designer and Workflow
14/140
Cache Directory $PMCacheDir
Join Type Normal
Null Ordering in Master Null Highest value
Null Ordering in Detail Null Highest value
Tracing Level Normal
Joiner Data Cache Size. 2,000,000 bytes
Joiner Index Cache Size 1,000,000 bytes.
Click OK.
Choose Repository-Save to save changes to the mapping
The Condition appears in the join condition row
-
8/6/2019 Designer and Workflow
15/140
Source Qualifier transformation
The Source Qualifier represents the rows that the Informatica Server readswhen it executes a session.
When you add a relational or a flat file source definition to a mapping, you need tconnect it to a Source Qualifier transformation
-
8/6/2019 Designer and Workflow
16/140
Transformation type: Active Connected
We can use the Source Qualifier to perform the following tasks:
Join data originating from the same source database .
Filter records when the Informatica Server reads source data .
Specify an outer join rather than the default inner join .
Specify sorted ports.
Select only distinct values from the source.
Create a custom query to issue a special SELECT statement for theInformatica Server to read source data.
Task of Source Qualifier transformation
-
8/6/2019 Designer and Workflow
17/140
Default Query of source qualifier
For relational sources, the Informatica Server generates a query for eachSource Qualifier when it runs a session.
The default query is a SELECT statement for each source column used inthe mapping
The Informatica Server reads only the columns in Source Qualifier that areconnected to another transformation.
Source Definition Connected to a Source Qualifier Transformation
-
8/6/2019 Designer and Workflow
18/140
To view the default query
To view the default query:
From the Properties tab, select SQL Query .
Click Generate SQL.
Click Cancel to exit
Note: If you do not cancel the SQL query, the Informatica Server overrides thedefault query with the custom SQL query
-
8/6/2019 Designer and Workflow
19/140
Example of source qualifier transformation
you might see all the orders for the month, including order number, order amount,and customer name.
The ORDERS table includes the order number and amount of each order, but notThe customer name. To include the customer name, you need to join the ORDERSand CUSTOMERS tables.
-
8/6/2019 Designer and Workflow
20/140
Setting the properties to source qualifier
Edit Transformation Tab
Tracing Level
Sorted port
Filter Condition
Pre sql
Post sql
Select distinct
-
8/6/2019 Designer and Workflow
21/140
SQL Query
You can give query in the Source Qualifier transformation
Sql query
From the Properties tab, select SQL Query The SQL Editor displays.Click Generate SQL.
-
8/6/2019 Designer and Workflow
22/140
Joining source data
You can use one Source Qualifier transformation to join data from multiplerelational tables. These tables must be accessible from the same instance ordatabase server.
Use the Joiner transformation for heterogeneous sources and to join flat files
User define join Join condition
-
8/6/2019 Designer and Workflow
23/140
Entering a Source Filter
You can enter a source filter to reduce the number of rows the Informatica Serverqueries. Do not include WHERE in the source filter.
Filter condition
-
8/6/2019 Designer and Workflow
24/140
Sorted Ports
For information about using Sorted Ports to optimize join performance,
Sorted Ports
-
8/6/2019 Designer and Workflow
25/140
Aggregator Transformation
The Aggregator transformation allows you to perform aggregate calculations,such as averages and sums.
The Aggregator transformation is unlike the Expression transformation, in thatYou can use the Aggregator transformation to perform calculations on groups
The Expression transformation permits you to perform calculations on arow-by-row basis only.
you can use conditional clauses to filter rows, providing more flexibility than SQLlanguage.
The Informatica Server performs aggregate calculations as it reads, and storesnecessary data group and row data in an aggregate cache.
The Aggregator is an active transformation
-
8/6/2019 Designer and Workflow
26/140
Aggregator TransformationEdit transformation
On the aggregator transformation double clik edit transfromation
Aggregator Transformation
-
8/6/2019 Designer and Workflow
27/140
Components of the Aggregator Transformation
Aggregate expression
Group by port
Sorted input
Aggregate cache
-
8/6/2019 Designer and Workflow
28/140
Aggregate expression
An aggregate expression can include conditional clauses and non-aggregatefunctions. It can also include one aggregate function nested within anotheraggregate function, such as:
MAX( COUNT( ITEM )
Aggregate Functions:
The aggregate functions can be used within an Aggregator transformation
You can nest one aggregate function within another aggregate function.
AVG
COUNT
-
8/6/2019 Designer and Workflow
29/140
Aggregate Functions
FIRST
LAST
MAX
MEDIAN
MIN
PERCENTILE
STDDEV
SUM
VARIANCE
-
8/6/2019 Designer and Workflow
30/140
Conditional Clauses
You can use conditional clauses in the aggregate expression to reduce thenumber of rows used in the aggregation.
Setting condition to aggregator transformation is as follows :
Double click to aggregator transformation edit transformation box opens
Edit transformation
-
8/6/2019 Designer and Workflow
31/140
Click to port tab edit transformation box opens
Port tab in aggregator transformation
t t t f diti l
-
8/6/2019 Designer and Workflow
32/140
output port for condition clauses
Enable the output port for condition clauses
Output port
-
8/6/2019 Designer and Workflow
33/140
Click on expression tab expression edit transformation opens forgive any condition clauses .
Expression tab in aggregator transformation
-
8/6/2019 Designer and Workflow
34/140
Click the group by option for each column you want the Aggregator to use increating groups.
Group port
Ports in aggregator transformation
-
8/6/2019 Designer and Workflow
35/140
Creating an Aggregator Transformation
In the Mapping Designer, choose Transformation-Create. Select the Aggregatortransformation.
Enter a name for the Aggregator, click Create. Then click Done
Drag the desired ports to the Aggregator transformation
Double-click the title bar of the transformation to open the Edit Transformationsdialog box
Select the Ports tab.
Click the group by option for each column you want the Aggregator to use increating groups.
-
8/6/2019 Designer and Workflow
36/140
Click Add and enter a name and data type for the aggregate expression port.Make the port an output port by clearing Input (I).
Add default values for specific ports as necessary
Select the Properties tab
Creating an Aggregator Transformation
-
8/6/2019 Designer and Workflow
37/140
Properties tab
Cache Directory
Tracing Level
Sorted Input
Aggregator Data Cache Size
Aggregator Index Cache Size
-
8/6/2019 Designer and Workflow
38/140
Aggregator Transformation Tips
Use sorted input to decrease the use of aggregate caches.
Limit connected input/output or output ports
Filter before aggregating.
f
-
8/6/2019 Designer and Workflow
39/140
The Rank transformation allows you to select only the top or bottomrank of data.
The Rank transformation differs from the transformation functions MAXand MIN, to select a group of top or bottom values, not just one value.
For example, you can use Rank to select the top 10 salespersons
in a given territory
Sample Mapping with a Rank Transformation
The Rank transformation is ActiveTransforamtion
Rank transformation
-
8/6/2019 Designer and Workflow
40/140
Transformation-Create. Select the Rank transformation
Click OK, and then click Done.
Link columns from an input transformation to the Rank transformation.
Click the Ports tab, and then select the Rank (R) option for the portused to measure ranks.
to create groups for ranked rows, select Group By for the port thatdefines the group
How To create rank transformation
-
8/6/2019 Designer and Workflow
41/140
Rank Transformation Properties :
Enter a cache directory.
Select the top or bottom rank.
Select the input/output port that contains values used to determine
the rank
Select the number of rows falling within a rank.
Define groups for ranks, such as the 10 least expensive products for
each manufacturer.
The Properties of Rank Transformation
-
8/6/2019 Designer and Workflow
42/140
Click the Properties tab and select whether you want the top orbottom rank
For the Number of Ranks option, enter the number of rows you wantto select for the rank. Change the following properties, if necessary:
Cache directory
Top/Bottom
Tracing level Normal
Rank Data Cache Size Default is 2,000,000 bytes.
Setting Properties to Rank Transformation
-
8/6/2019 Designer and Workflow
43/140
Edit Transformation tab for the rank transformation
Edit Transformation tab :
Port tabProperties tabMetadata expression tab
Port tab
Data type
-
8/6/2019 Designer and Workflow
44/140
Properties tab function
Top/Bottom
Number of Ranks
Tracing level Normal
Rank Data Cache Size
-
8/6/2019 Designer and Workflow
45/140
The Properties of rank transformation
Top
Number of Ranks 1
Tracing level NormalRank Data Cache Size 4000,000
Rank Index Cache Size 2,000,000
-
8/6/2019 Designer and Workflow
46/140
Example of rank transformation
-
8/6/2019 Designer and Workflow
47/140
Setting the properties to rank transformation
.
Click to port tab
The Rank transformation includes input or input/output ports connected to
another transformation in the mapping. It also includes variable ports and arank port.
Port tab
-
8/6/2019 Designer and Workflow
48/140
Ports in a Rank Transformation
Give the field name as sal1
-
8/6/2019 Designer and Workflow
49/140
Ports in a Rank Transformation
Rank Transformation Ports
-
8/6/2019 Designer and Workflow
50/140
Properties in rank transformation
click on the properties tab to set properties to rank transformation
-
8/6/2019 Designer and Workflow
51/140
Sequence Generator Transformation
The Sequence Generator transformation generates numeric values
You can use the Sequence Generator to create unique primary key values,cycle through a sequential range of numbers
The Sequence Generator transformation is a connected transformation.
The Informatica Server generates a value each time a row enters aconnected transformation, even if that value is not used
When NEXTVAL is connected to the input port of another transformation,the Informatica Server generates a sequence of numbers
When CURRVAL is connected to the input port of another transformation,the Informatica Server generates the NEXTVAL value plus one.
-
8/6/2019 Designer and Workflow
52/140
.
You can make a Sequence Generator reusable, and use it in multiplemappings
You might reuse a Sequence Generator when you perform multiple loads to
a single target.
If you have a large input file that you separate into three sessions running inparallel, you can use a Sequence Generator to generate primary key values.
If you use different Sequence Generators, the Informatica Server mightaccidentally generate duplicate key values
Instead, you can use the reusable Sequence Generator for all three sessionsto provide a unique value for each target row.
Sequence Generator Transformation
-
8/6/2019 Designer and Workflow
53/140
.
Tasks with a Sequence Generator transformation
Replace missing values.
Create keys.
Cycle through a sequential range of numbers.
-
8/6/2019 Designer and Workflow
54/140
.
Creating a Sequence Generator Transformation
In the Mapping Designer, select Transformation-Create. Select theSequence Generator transformation.
-
8/6/2019 Designer and Workflow
55/140
.
Edit transformation
Double-click the title bar of the transformation to open the EditTransformations dialog box
-
8/6/2019 Designer and Workflow
56/140
.
Properties tab
Select the Properties tab. Enter settings as necessary.
Click OK.
To generate new sequences during a session, connect the NEXTVAL port toat least one transformation in the mapping.
Choose Repository-Save.
-
8/6/2019 Designer and Workflow
57/140
.
Click OK.
To generate new sequences during a session, connect the NEXTVAL port toat least one transformation in the mapping
Choose Repository-Save.
Properties tab
-
8/6/2019 Designer and Workflow
58/140
.
Sequence Generator Ports
The Sequence Generator provides two output ports: NEXTVAL andCURRVAL.
Use the NEXTVAL port to generate a sequence of numbers by connectingit to a transformation or target.
You connect the NEXTVAL port to a downstream transformation to generatethe sequence based on the Current Value and Increment By properties
Connect NEXTVAL to multiple transformations to generate unique values foreach row in each transformation
you might connect NEXTVAL to two target tables in a mapping to generateunique primary key values.
NEXTVAL to Two Target Tables in a Mapping
-
8/6/2019 Designer and Workflow
59/140
.
NEXTVAL to Two Target Tables in a Mapping
you configure the Sequence Generator transformation as follows:
Current Value = 1, Increment By = 1.
When you run the workflow, the Informatica Server generates the followingprimary key values for the T_ORDERS_PRIMARY andT_ORDERS_FOREIGNtarget tables:
-
8/6/2019 Designer and Workflow
60/140
.
Output for NEXTVAL in the sequence generator
T_ORDERS_PRIMARY
TABLE:
PRIMARY KEY
T_ORDERS_FOREIGN TABLE:
PRIMARY KEY
1 2
3 4
5 6
7 8
9 10
-
8/6/2019 Designer and Workflow
61/140
.
Sequence Generator and Expression Transformation
you configure the Sequence Generator transformation
as follows: Current Value = 1, Increment By = 1.
O t t
-
8/6/2019 Designer and Workflow
62/140
.
Output
key values for the T_ORDERS_PRIMARYandT_ORDERS_FOREIGN target tables:
T_ORDERS_PRIMARYTABLE:
PRIMARY KEY
T_ORDERS_FOREIGNTABLE:
PRIMARY KEY1 1
2 2
3 3
4 4
5 5
CURRVAL
-
8/6/2019 Designer and Workflow
63/140
.
CURRVAL
CURRVAL is the NEXTVAL value plus one or NEXTVAL plus theIncrement By value
You typically only connect the CURRVAL port when theNEXTVAL port is already connected to a downstream transformation
When a row enters the transformation connected to the CURRVALport, the Informatica Server passes the last-created NEXTVAL
value plus one.
Connecting CURRVAL and NEXTVAL Ports to a Target
-
8/6/2019 Designer and Workflow
64/140
.
Connecting CURRVAL and NEXTVAL Ports to a Target
you configure the Sequence Generator transformation asfollows: Current Value = 1, Increment By = 1.
When you run the workflow, the Informatica Servergenerates the following values forNEXTVAL and CURRVAL:
OUT PUT
-
8/6/2019 Designer and Workflow
65/140
.
OUT PUT
When you run the workflow, the Informatica Servergenerates the following values forNEXTVAL and CURRVAL:
NEXTVAL CURRVAL
1 2
2 3
3 4
4 5
5 6
If you connect the CURRVAL port without connecting theNEXTVAL port, the Informatica Server passes a constantvalue for each row.
O l th CURRVAL P t t T t
-
8/6/2019 Designer and Workflow
66/140
.
Only the CURRVAL Port to a Target
For example, you configure the Sequence Generator
transformation as follows
OUTPUT
-
8/6/2019 Designer and Workflow
67/140
.
OUTPUT
Current Value = 1, Increment By = 1.
CURRVAL
1
1
1
1
1
When you run the workflow, the Informatica Servergenerates the following constant values forCURRVAL:
-
8/6/2019 Designer and Workflow
68/140
.
Sequence Generator Transformation Properties
The Sequence Generator is unique among allbecause you cannot add, edit, or delete its default ports
(NEXTVAL and CURRVAL).
Properties tab
-
8/6/2019 Designer and Workflow
69/140
.
Properties tab
-
8/6/2019 Designer and Workflow
70/140
Normalizer Transformation
It is an Active and Connected Transformation
It normalizes COBOL sources and relational sourcesallowing the user to organize the data as per the
requirement
Primarily used for a COBOL source
-
8/6/2019 Designer and Workflow
71/140
Tasks performed by Normalizer
Break out repeated data within a record into separate records
For each new record it creates a unique identifier
Can be used with relational sources to create multiple rowsfrom a single row of data
It generates the following columns when we create thetransformation 1) Generated key 2)Generated Column Id
Designer generates a port for each REDEFINES clause tospecify the generated key which can be used as the primarykey column in the target table
Designer generates a port for each OCCURS clause tospecify the positional index within an OCCURS clause and
can be used as the generated column ID to create a primary-foreign key relationship
-
8/6/2019 Designer and Workflow
72/140
-
8/6/2019 Designer and Workflow
73/140
Workflow
Workflow is a set of instruction thatdescribes how and when to run tasksrelated to extraction, transformation and
loading of data.
-
8/6/2019 Designer and Workflow
74/140
Architecture
sourcePowercentreserver
target
repository
Instructions frommetadata
Sourcedata
Transformeddata
-
8/6/2019 Designer and Workflow
75/140
Workflow Manager
TasksTasks
Variables/Variables/EventsEvents
WorkWorkFoldersFolders
LinkLink
N C ti F t
-
8/6/2019 Designer and Workflow
76/140
New Connection Features
Connection Dialogs
Replace Connections Register Server
Assign to Workflows
-
8/6/2019 Designer and Workflow
77/140
D.T.MD.T.ML.ML.M
Work
flow
Work
flow
Load manager
-
8/6/2019 Designer and Workflow
78/140
Load manager
1. Locks the workflow and reads workflow properties. 2. Reads the parameter file and expands workflow
variables. 3. Creates the workflow log file.
4. Runs workflow tasks. 5. Distributes sessions to worker servers. 6. Starts the DTM to run sessions. 7. Runs sessions from master servers. 8. Sends post-session email if the DTM terminates
abnormally.
Data Transformation manager
-
8/6/2019 Designer and Workflow
79/140
Data Transformation manager
1. Fetches session and mapping metadata from the repository. 2. Creates and expands session variables. 3. Creates the session log file. 4. Verifies connection object permissions. 5. Runs pre-session shell commands.
6. Runs pre-session stored procedures and SQL. 7. Creates and runs mapping, reader, writer, and
transformation threads to extract, transform, and load data. 8. Runs post-session stored procedures and SQL. 9. Runs post-session shell commands.
10. Sends post-session email.
Create a Workflow
-
8/6/2019 Designer and Workflow
80/140
Create a Workflow
Create a Workflow
-
8/6/2019 Designer and Workflow
81/140
Create a Workflow
Create a Workflow
-
8/6/2019 Designer and Workflow
82/140
Create a Workflow
Create a Workflow
-
8/6/2019 Designer and Workflow
83/140
Create a Workflow
Create a Workflow
-
8/6/2019 Designer and Workflow
84/140
Create a Workflow
New Objects (Tasks)
-
8/6/2019 Designer and Workflow
85/140
New Objects (Tasks)
Local Tasks Sessions Commands Email Decision Assignment Timer Control Event Raise Event Wait Worklet
Global Tasks
Sessions
Commands
Email
Tasks are now the default units of work for building the workflow.Global tasks are reusable across workflows. Local tasks areindependent and self-contained within workflows.
Sessions (Updates)
-
8/6/2019 Designer and Workflow
86/140
Sessions (Updates)
GeneralGeneral
First page allows for options to treat conditional links attached to the object as AND/ORfunctionality. Also control option to fail the parent (container) if task fails or does not run.Disabling a task in a workflow allows the task to be skipped instead of having to remove it.
UpdatedUpdatedparametersparameters
-
8/6/2019 Designer and Workflow
87/140
-
8/6/2019 Designer and Workflow
88/140
-
8/6/2019 Designer and Workflow
89/140
Sessions (Updates)
-
8/6/2019 Designer and Workflow
90/140
Sessions (Updates)
Config ObjectConfig Object
This area is where typical allocation of memory occurs, log file options and error handlingThis area is where typical allocation of memory occurs, log file options and error handlingstrategies. The upgrade here is with the concept of maintaining common setting in a configstrategies. The upgrade here is with the concept of maintaining common setting in a configobject which is setup separately and chosen from a drop-down list.object which is setup separately and chosen from a drop-down list.
Drop-down forDrop-down for
commoncommonsettingssettings
Config Object
-
8/6/2019 Designer and Workflow
91/140
g j
Config ObjectConfig Object
By creating a session configuration object you can cut down on repetitively assigning memoryBy creating a session configuration object you can cut down on repetitively assigning memoryand log options. When migrating sessions you can optionally migrate configurations or maintainand log options. When migrating sessions you can optionally migrate configurations or maintainenvironment specific settings to be reassigned.environment specific settings to be reassigned.
Setup for reusableSetup for reusableConfiguration objectConfiguration object
Sessions (Updates)
-
8/6/2019 Designer and Workflow
92/140
( p )
SourcesSources
This area has primarily just been subdivided into sub tabs. The main addition is the filter drop-This area has primarily just been subdivided into sub tabs. The main addition is the filter drop-down that allows for either viewing all source object settings or by individual source object.down that allows for either viewing all source object settings or by individual source object.Instead of filelist option it is either direct or indirect (filelist)Instead of filelist option it is either direct or indirect (filelist)
FilterFilter
Sessions (Updates)
-
8/6/2019 Designer and Workflow
93/140
Sessions (Updates)
TargetsTargets
Same subdivisions as sources. However in the Properties sub tab is where the Target OptionsSame subdivisions as sources. However in the Properties sub tab is where the Target Optionsare defined. The writers sub tab preserves the legacy operation of writing to a flat-file whileare defined. The writers sub tab preserves the legacy operation of writing to a flat-file whileusing a database specific target (as opposed to the new flat-file target).using a database specific target (as opposed to the new flat-file target).
FilterFilter
Source Target Connections
-
8/6/2019 Designer and Workflow
94/140
g
ConnectionsConnections
The drop-down list allows to select any available connection as well as specifically define the useThe drop-down list allows to select any available connection as well as specifically define the useof a Connection Variableof a Connection Variable
ConnectionConnectiondialogdialog
Sessions (Updates)
-
8/6/2019 Designer and Workflow
95/140
( p )
ComponentsComponents
The area where commands or email unique to this object can be defined. You can alternatelyThe area where commands or email unique to this object can be defined. You can alternatelyselect a reusable task to use as well.select a reusable task to use as well.
Choice ofChoice ofreusable orreusable orlocal commandlocal command
Non-Reusable Commands
-
8/6/2019 Designer and Workflow
96/140
ComponentsComponents
Regardless of reusable or non-reusable it is necessary to name the object since there isRegardless of reusable or non-reusable it is necessary to name the object since there ispotential to promote it.potential to promote it.
Option for localOption for local
or reusableor reusable
Name ofName ofcommandcommand
objectobject
Non Reusable Commands
-
8/6/2019 Designer and Workflow
97/140
Non-Reusable Commands
ComponentsComponents
The properties tab allows for error control for commands/tasksThe properties tab allows for error control for commands/tasks
Error Control forError Control for
multiplemultiplecommands/taskscommands/tasks
Non-Reusable Commands
-
8/6/2019 Designer and Workflow
98/140
Non Reusable Commands
ComponentsComponents
The Command tab allows for editing of commands (obviously)The Command tab allows for editing of commands (obviously)
Command editorCommand editor
Reusable Commands
-
8/6/2019 Designer and Workflow
99/140
Reusable Commands
ComponentsComponents
A non-reusable command can be promoted to reusable or else you can select one from availableA non-reusable command can be promoted to reusable or else you can select one from availabletasks.tasks.
ReusableReusableCommandCommand
Selection DialogSelection Dialog
-
8/6/2019 Designer and Workflow
100/140
Sessions (Updates)
-
8/6/2019 Designer and Workflow
101/140
Sessions (Updates)
TransformationsTransformations
The transformations area preserves all the previous capabilities. The update is the addition ofThe transformations area preserves all the previous capabilities. The update is the addition ofthe main instance filter and another filter button that only displays transformation where overridesthe main instance filter and another filter button that only displays transformation where overridescan be performed. Also this is where you can input the NEW pre/post SQL commands.can be performed. Also this is where you can input the NEW pre/post SQL commands.
Filter for modifiableFilter for modifiable
transformationstransformations
Instance FilterInstance Filter
Sessions (Updates)
-
8/6/2019 Designer and Workflow
102/140
Metadata ExtensionsMetadata Extensions
Metadata extensions are available to individual object/tasks or can be made reusable and global.Metadata extensions are available to individual object/tasks or can be made reusable and global.These columns can not be auto-populated with functions but they do allow the extendibility forThese columns can not be auto-populated with functions but they do allow the extendibility for
more reporting options (e.g., Developer Name, Version, Creation Date, etc.)more reporting options (e.g., Developer Name, Version, Creation Date, etc.)
Metadata columnMetadata column
Promote toPromote toreusablereusable
Target
-
8/6/2019 Designer and Workflow
103/140
Target
Target Properties
-
8/6/2019 Designer and Workflow
104/140
Target Properties
Target database connections: specify the location oftarget
Truncate Target Tables: can truncate the tables beforeloading
Deadlock Retry : Configure server to retry when writing to
targets Drop and Recreate Indexes: Pre and Post session
commands to optimize the query
Bilk Loading : can specify bulk loading when loading toDB2, microsoft sql server ,oracle sysbase database
-
8/6/2019 Designer and Workflow
105/140
-
8/6/2019 Designer and Workflow
106/140
Command Tasks
-
8/6/2019 Designer and Workflow
107/140
CommandCommand
The command object can be created globally under the Task Developer. It can also beThe command object can be created globally under the Task Developer. It can also bepromoted here from within a mapping.promoted here from within a mapping.
Command Tasks
-
8/6/2019 Designer and Workflow
108/140
Command Tasks
-
8/6/2019 Designer and Workflow
109/140
-
8/6/2019 Designer and Workflow
110/140
Event Wait
-
8/6/2019 Designer and Workflow
111/140
Event Wait
Assignment
-
8/6/2019 Designer and Workflow
112/140
Assignment
-
8/6/2019 Designer and Workflow
113/140
E il T k
-
8/6/2019 Designer and Workflow
114/140
Email Tasks
EmailEmail
Email task is very similar to the command task since it can be either created in the TaskEmail task is very similar to the command task since it can be either created in the TaskDeveloper or promoted from a mapping. The properties tab allows for an expression editor forDeveloper or promoted from a mapping. The properties tab allows for an expression editor fortext creation utilizing the built-in variables.text creation utilizing the built-in variables.
Email textEmail textcreation dialogcreation dialog
Built-inBuilt-in
VariablesVariables
Links and Conditions
-
8/6/2019 Designer and Workflow
115/140
DefiinitionDefiinition
Links and their underlying conditions are what provide process control to the workflow. When anLinks and their underlying conditions are what provide process control to the workflow. When anattached link condition resolves to TRUE then the attached object may begin processing. Thereattached link condition resolves to TRUE then the attached object may begin processing. Therecan be no looping and links can only execute once per workflow. However more complexcan be no looping and links can only execute once per workflow. However more complexbranching and decisions can be made by combining multiple links to a single object or branchingbranching and decisions can be made by combining multiple links to a single object or branchinginto decision type paths. Each link has its own expression editor and can utilize upstreaminto decision type paths. Each link has its own expression editor and can utilize upstreamresolved object variables or user-defined variables for its own evaluation.resolved object variables or user-defined variables for its own evaluation.
Link conditionLink condition
Links and Conditions
-
8/6/2019 Designer and Workflow
116/140
Object VariablesObject Variables
The default set of object variables from a session can provide more information than just a statusThe default set of object variables from a session can provide more information than just a statusof Completed. More complex evaluation can be done for ErrorCode, StartTime,of Completed. More complex evaluation can be done for ErrorCode, StartTime,SrcSuccessRows, etc. In addition to the default object variables, User Defined variables can beSrcSuccessRows, etc. In addition to the default object variables, User Defined variables can becreated and populated via parameter files or changed in the workflow via Assignment tasks.created and populated via parameter files or changed in the workflow via Assignment tasks.Also any upstream task that has completed can have its variables utilized in downstream linkAlso any upstream task that has completed can have its variables utilized in downstream linkconditions.conditions.
ObjectObjectVariablesVariables
Workflow Variables
-
8/6/2019 Designer and Workflow
117/140
Variable Task Type Datatype ** SupportedStatus Returns
ABORTEDDISABLEDFAILEDNOTSTARTEDSTARTEDSTOPPEDSUCCEEDED
Condition Decision Task Integer
EndTime All tasks Date/timeErrorCode All tasks Integer
ErrorMsg All tasks Nstring*
FirstErrorCode Session task Integer
FirstErrorMsg Session task Nstring*
PrevTaskStatus All tasks Integer
SrcFailedRows Session task Integer
SrcSuccessRows Session task Integer
StartTime All tasks Date/time
Status** All tasks Integer
TgtFailedRows Session tasks Integer
Workflow Variables
-
8/6/2019 Designer and Workflow
118/140
User-defined VariablesUser-defined Variables
Variables are created at the container level much like the mappings. (Workflows=Mappings,Variables are created at the container level much like the mappings. (Workflows=Mappings,Worklets=Mapplets). Once created values can be passed to objects within the same containerWorklets=Mapplets). Once created values can be passed to objects within the same containerfor evaluation. (Assignment Task can modify/calculate variables)for evaluation. (Assignment Task can modify/calculate variables)
Edit VariablesEdit Variables
Workflow Variables
-
8/6/2019 Designer and Workflow
119/140
User-defined VariablesUser-defined Variables
A user-defined variable can assist in more complex evaluations. In the above example, anA user-defined variable can assist in more complex evaluations. In the above example, anexternal parameter file contains the number of expected rows. This in turn is evaluated againstexternal parameter file contains the number of expected rows. This in turn is evaluated againstthe actual rows successfully read from an upstream session. $ signifies and is reserved for pre-the actual rows successfully read from an upstream session. $ signifies and is reserved for pre-defined variables. User defined variables should maintain $$ naming.defined variables. User defined variables should maintain $$ naming.
User DefinedUser DefinedVariablesVariables
Pre-DefinedPre-Defined
VariableVariable
Link decision
-
8/6/2019 Designer and Workflow
120/140
Link decision
Parameter File
-
8/6/2019 Designer and Workflow
121/140
FormatFormat
The format is slightly different going forward for declaring variables but legacy naming isThe format is slightly different going forward for declaring variables but legacy naming ispreserved with the foldername.sessionpreserved with the foldername.session
Workflow variables:
[folder name.WF:workflow name]Worklet variables:
[folder name.WF:workflow name.WT:worklet name]
Worklet variables in nested worklets:[folder name.WF:workflow name.WT:worklet name.WT:worklet name...]
Session parameters, plus mapping parameters and variables:[folder name.WF:workflow name.ST:session name]
or[folder name.session name]
or[session name]
Assignment Task
-
8/6/2019 Designer and Workflow
122/140
UsageUsage
The assignment task allows for workflow variables to be manipulated, assigned or calculated andThe assignment task allows for workflow variables to be manipulated, assigned or calculated andstored (either for the session run or as persistent) for downstream evaluation and conditionstored (either for the session run or as persistent) for downstream evaluation and conditionresolution. The Expression Section allows for the selection of the appropriate variable toresolution. The Expression Section allows for the selection of the appropriate variable tomanipulate along with the assignment through an Expression Editor. In order to use it ismanipulate along with the assignment through an Expression Editor. In order to use it isnecessary to first create a variable through the Edit Variables dialognecessary to first create a variable through the Edit Variables dialog
Edit VariablesEdit Variables
Event Task
-
8/6/2019 Designer and Workflow
123/140
UsageUsage
Events are similar to variables. They are repository tags that are a little more flexible (cleaner)Events are similar to variables. They are repository tags that are a little more flexible (cleaner)than dealing with indicator files (although event watch can be used with indicator files). Beforethan dealing with indicator files (although event watch can be used with indicator files). Beforeutilizing the new functionality it is necessary to create these Event tags within the container thatutilizing the new functionality it is necessary to create these Event tags within the container thatthey will be used. Events can not be monitored across workflows or worklets (even if a worklet isthey will be used. Events can not be monitored across workflows or worklets (even if a worklet ispart of the workflow).part of the workflow).
Edit EventsEdit Events
Event Task
-
8/6/2019 Designer and Workflow
124/140
UsageUsage
If using Event tags then an Event Raise is used in conjunction with an Event Wait. In the aboveIf using Event tags then an Event Raise is used in conjunction with an Event Wait. In the aboveexample two branches are executed in parallel. The second session of the lower branch willexample two branches are executed in parallel. The second session of the lower branch willremain in stasis until the upper branch completes triggering the event. The lower branchesremain in stasis until the upper branch completes triggering the event. The lower branchesevent wait task recognizes the event and allows for the second session to start.event wait task recognizes the event and allows for the second session to start.
Event RaiseEvent Raise
Event WaitEvent Wait
Event Raise
-
8/6/2019 Designer and Workflow
125/140
UsageUsage
To configure the Event Raise task the drop-down box allows for selection of the appropriateTo configure the Event Raise task the drop-down box allows for selection of the appropriateuser-defined Event tag. This will create an entry in the repository for a matching event wait touser-defined Event tag. This will create an entry in the repository for a matching event wait tolook for.look for.
Event Wait
-
8/6/2019 Designer and Workflow
126/140
UsageUsage
The event wait allows for configuration for an Event Raise (user-defined event) or existenceThe event wait allows for configuration for an Event Raise (user-defined event) or existencecheck for an indicator file.check for an indicator file.
User DefinedUser Defined
EventEvent
Indicator FileIndicator File
Event Wait
-
8/6/2019 Designer and Workflow
127/140
UsageUsage
The properties section of the Event Wait task allows for further definition of behavior. If yourThe properties section of the Event Wait task allows for further definition of behavior. If yourworkflow has failed/suspended after Event Raise but before the Event Wait has resolved, thenworkflow has failed/suspended after Event Raise but before the Event Wait has resolved, then
the Enable Past Events is able to recognize that the Event has happened already. If workingthe Enable Past Events is able to recognize that the Event has happened already. If workingwith indicator files you have to ability to either delete the file or allow it to stay in case somewith indicator files you have to ability to either delete the file or allow it to stay in case somedownstream Event Waits are also keying on that file.downstream Event Waits are also keying on that file.
Resume/RestartResume/RestartSupportSupport
Flat-file CleanupFlat-file Cleanup
Decision Task
-
8/6/2019 Designer and Workflow
128/140
UsageUsage
The decision task allows for True/False based branching of process ordering. The Decision taskThe decision task allows for True/False based branching of process ordering. The Decision taskcan home multiple conditions and therefore downstream links can be evaluated simply upon thecan home multiple conditions and therefore downstream links can be evaluated simply upon the
Decision being True or False.Decision being True or False.
**Note it is possible to have the decision based on SUCCEEDED or FAILED of previous task,**Note it is possible to have the decision based on SUCCEEDED or FAILED of previous task,however if workflow is set to suspend on error than that branch is suspended and the decisionhowever if workflow is set to suspend on error than that branch is suspended and the decisionwont trigger on a FAILED conditionwont trigger on a FAILED condition
Control Task
-
8/6/2019 Designer and Workflow
129/140
UsageUsage
The control task is utilized in a branching manner to present a level of stoppage during theThe control task is utilized in a branching manner to present a level of stoppage during theworkflow. Consider if too many sessions have too many failed rows. The options allow forworkflow. Consider if too many sessions have too many failed rows. The options allow fordifferent levels such as failing at the object level to Aborting the whole workflow.different levels such as failing at the object level to Aborting the whole workflow.
Timer Task
-
8/6/2019 Designer and Workflow
130/140
Timer Task
UsageUsage
The timer task has two main ways to be utilized. The first way is by absolute time that is timeThe timer task has two main ways to be utilized. The first way is by absolute time that is timeevaluated by server time or a user-defined variable (that contains the date/time stamp to start).evaluated by server time or a user-defined variable (that contains the date/time stamp to start).
Timer Task
-
8/6/2019 Designer and Workflow
131/140
UsageUsage
The second usage is by Relative time that offers options of time calculated from when theThe second usage is by Relative time that offers options of time calculated from when theprocess reached this (Timer) task, from the start of the container this task, or from the start of theprocess reached this (Timer) task, from the start of the container this task, or from the start of theabsolute top-level workflow.absolute top-level workflow.
WORKLET
-
8/6/2019 Designer and Workflow
132/140
Object that represents a set of tasks It can contain any task available in the
Workflow Manager
You can run worklets inside a workflow. The workflow
that contains the worklet is called the parent workflow
Create a worklet when you want to reuse a set of
workflow logic in several workflows.
Creating a Reusable Worklet
-
8/6/2019 Designer and Workflow
133/140
Creating a Non-ReusableWorklet
-
8/6/2019 Designer and Workflow
134/140
To create a non-reusable worklet:1. In the Workflow Designer, open a workflow.
2. Choose Tasks-Create.
3. Select Worklet for the Task type.
4. Enter a name for the worklet.
5. Click Create.
The Workflow Designer creates the worklet and
adds it to the workspace.6. Click Done.
Worklet
Configuring Worklet Properties
-
8/6/2019 Designer and Workflow
135/140
Worklet variables. Use worklet variables toreference values and record information. You useworklet variables the same way you use workflowvariables. You can assign a workflow variable toa worklet variable to override its initial value.
Events. To use the Event-Wait and Event-Raisetasks in the worklet, you must first declare anevent in the worklet properties.
Metadata extension. Extend the metadata
stored in the repository by associatinginformation with repository objects.
Adding Tasks in WorkletsAft t kl t dd t k b i th
-
8/6/2019 Designer and Workflow
136/140
After you create a new worklet, add tasks by opening theworklet in the Worklet Designer. A worklet must contain
a Start task. The Start task represents the beginning of aworklet. When you create a worklet, the WorkletDesigner automatically creates a Start task for you.
To add tasks to a non-reusable worklet:1. Create a non-reusable worklet in the Workflow
Designer workspace.2. Right-click the worklet and choose Open Worklet.
3. The Worklet Designer opens so you can add tasks inthe worklet.4. Add tasks in the worklet by using the Tasks toolbar or
choose Tasks-Create in the Worklet Designer.5. Connect tasks with links.
-
8/6/2019 Designer and Workflow
137/140
Workflow Monitor
-
8/6/2019 Designer and Workflow
138/140
Workflow Monitor
Workflow Monitor Task View
-
8/6/2019 Designer and Workflow
139/140
Workflow Monitor Task View
-
8/6/2019 Designer and Workflow
140/140