infa notes .doc

8/10/2019 INFA Notes .doc

1/161

1. GETTING STARTED

Metadata Extensions tab

Execute SQL to create table

The source qualifer represents the rows that the

Integration Service reads rom the source when it

runs a session

session is a set o instructions that tells the

Integration Service how to move data rom sources

to targets!

session is a tas"# similar to other tas"s available in

the $or"%ow Manager!

&ou create a session or each mapping that 'ou want

the Integration Service to run!

The Integration Service uses the instructions

confgured in the session and mapping to move

data rom sources to targets!

wor"%ow is a set o instructions that tells the

Integration Service how to execute tas"s# such assessions# email notifcations# and shell commands!

&ou create a wor"%ow or sessions 'ou want the

Integration Service to run!

&ou can include multiple sessions in a wor"%ow to

run sessions in parallel or sequentiall'!

The Integration Service uses the instructions

confgured in the wor"%ow to run sessions and

other tas"s!

((Select a code page or the database connection

)*+ ,onnection -b.ects/The source code page must be a subset o the target

code page!

The target code page must be a superset o the

source code page!

&ou can create reusable or non0reusable sessions in

the $or"%ow Manager

$hen 'ou create a wor"%ow# 'ou can include

reusable tas"s that 'ou create in the Tas"

*eveloper!&ou can also include non0reusable tas"s that 'ou

create in the $or"%ow *esigner!

+' deault# the wor"%ow is scheduled to run on

demand! The Integration Service onl' runs the

wor"%ow when 'ou manuall' start the wor"%ow!

&ou can confgure wor"%ows to run on a schedule

ll wor"%ows begin with the Start tas"# but 'ou ne

to instruct the Integration Service which tas" to r

next!

To do this# 'ou lin" tas"s in the $or"%ow Manager

$hen the 1ower,enter Integration Service runs

wor"%ows# 'ou can monitor wor"%ow progress in

the $or"%ow Monitor!

&ou can view details about a wor"%ow or tas" in

either a 2antt chart view or a Tas" view! &ou canstart# stop# and abort wor"%ows rom the $or"%o

Monitor!

The $or"%ow Monitor displa's wor"%ows that hav

run at least once!

transormation is a part o a mapping that

generates or modifes data!

Ever' mapping includes a Source Qualifer

transormation# representing all data read rom a

source and temporaril' stored b' the Integration

Service!

The dvanced Transormation toolbar contains

transormations such as 3ava# SQL# and 4ML 1ars

transormations

,reating Target 0

&ou can also manuall' create a target defnition#

import the defnition or an existing target rom a

database# or create a relational target rom a

transormation in the *esigner

,lic" the Indexes tab to add an index to the targe

table!

I the target database is -racle# s"ip to the fnal

step! &ou cannot add an index to a column that

alread' has the 15IM5& 6E& constraint added to

,lic" La'out 7 Lin" ,olumns!

$hen 'ou drag ports rom one transormation to

another# the *esigner copies the port description

and lin"s the original port to its cop'!

I 'ou clic" La'out 7 ,op' ,olumns# ever' port 'o

drag is copied# but not lin"ed!

Lin" ,olumns vs! ,op' columns88

+' deault# the Loo"up transormation queries and

stores the contents o the loo"up table beore the

rest o the transormation runs# so it perorms the

.oin through a local cop' o the table that it has

cached

I the data t'pes# including precision and scale# o

these two columns do not match# the *esigner

displa's a message and mar"s the mapping inva

9


2/161

((-verview window

$hen 'ou run the wor"%ow# the Integration Service

runs all sessions in the wor"%ow# either

simultaneousl' or in sequence# depending on how

'ou arrange the sessions in the wor"%ow!

,lic" the 2antt ,hart tab at the bottom o the Time

window to veri' the $or"%ow Monitor is in 2antt

,hart view!

:ote; &ou can also clic" the Tas"


3/161

2. INFA ADMIN GUIDE

domain is a collection o nodes and services that

'ou can group in olders based on administration

ownership

-ne node in the domain acts as a gatewa' to

receive service requests rom clients and route

them to the appropriate service and node!Services and processes run on nodes in a domain!

Service Manager 0 service that manages all

domain operations

Application Services 0 Services that represent

server0based unctionalit'# such as the Model

5epositor' Service and the *ata Integration Service!

The Service Manager and application services

control securit'!

The Service Manager manages users and groups

that can log in to application clients andauthenticates the users who log in to the

application clients!

The Service Manager and application services

authoriCe user requests rom application clients!

Inormatica dministrator )the dministrator tool/#

consolidates the administrative tas"s or domain

ob.ects such as services# nodes# licenses# and grids!

&ou manage the domain and the securit' o the

domain through the dministrator tool

Nodes -

Each node in the domain runs a Service Manager

that manages domain operations on that node

The operations that the Service Manager perorms

depend on the t'pe o node!

node can be a gatewa' node or a wor"er node

&ou can subscribe to alerts to receive notifcation

about node events such as node ailure or a master

gatewa' election!

&ou can also generate and upload node diagnostics

to the ,onfguration Support Manager and review

inormation such as available E+=s and Inormat

recommendations

Gatea! Nodes -

-ne node acts as the gatewa' at an' given time!

That node is called the master gatewa'!

gatewa' node can run application services# an

can serve as a master gatewa' node!

The master gatewa' node is the entr' point to

domain

&ou can confgure more than one node to serve a

gatewa'!

I the master gatewa' node becomes unavailab

the Service Manager on other gatewa' nodes ele

another master gatewa' node!

I 'ou confgure one node to serve as the gatew

and the node becomes unavailable# the dom

cannot accept service requests

"or#er Nodes - wor"er node is an' node not confgured to se

as a gatewa'!

Service Manager -

It runs as a service on $indows and as a daemon

?:I4!

$hen 'ou start Inormatica services# 'ou start t

Service Manager!

The Service Manager runs on each node!

I the Service Manager is not running# the node

not available!

Application service s$pport -

It starts and stops services and service proces

based on requests rom clients!

It also directs service requests to applicat

services!

The Service Manager uses T,1@I1 to communic

with the application services

Do%ain s$pport -

The unctions that the Service Manager perorms

a node depend on the t'pe o node!

=or example# the Service Manager running on

master gatewa' node perorms all dom

unctions on that node!

The Service Manager running on an' other no

perorms some domain unctions on that node

D


4/161

pplication services represent server0based

unctionalit'! pplication services include the

ollowing services;

0 nal'st Service

0 ,ontent Management Service

0 *ata Integration Service

0 Metadata Manager Service

0 Model 5epositor' Service

0 1ower,enter Integration Service

0 1ower,enter 5epositor' Service

0 1owerExchange Listener Service

0 1owerExchange Logger Service0 5eporting Service

0 S1 +$ Service

0 $eb Services ub

&ig' Availa(ilit!

igh availabilit' consists o the ollowing

components;

Resilience - The abilit' o application services to

tolerate transient networ" ailures until either the

resilience timeout expires or the external s'stem

ailure is fxed!Failover0 The migration o an application service or

tas" to another node when the node running the

service process becomes unavailable!

Recover!0 The automatic completion o tas"s ater

a service is interrupted! utomatic recover' is

available or 1ower,enter Integration Service and

1ower,enter 5epositor' Service tas"s! &ou can also

manuall' recover 1ower,enter Integration Service

wor"%ows and sessions! Manual recover' is not part

o high availabilit'

In)or%atica Ad%inistrator

?se the dministrator tool to complete the ollow

t'pes o tas"s;

Do%ain ad%inistrative tas#s0

Manage logs# domain ob.ects# user permissions# a

domain reports!

2enerate and upload node diagnostics!

Monitor .obs and applications that run on the *Integration Service!

*omain ob.ects include application services# nod

grids# olders# database connections# operat

s'stem profles# and licenses!

Sec$rit! ad%inistrative tas#s -

Manage users# groups# roles# and privileges

The dministrator tool has the ollowing tabs;

Do%ain -


5/161

Do%ain Ad%inistrator -

domain administrator can create and manage

ob.ects in the domain# including user accounts#

nodes# grids# licenses# and application services

owever# b' deault# the domain administrator

cannot log in to application clients!

The deault administrator must explicitl' give a

domain administrator ull permissions and

privileges to the application services

Application +lient Ad%inistrator -

n application client administrator can create and

manage ob.ects in an application client!

&ou must create administrator accounts or the

application clients

+' deault# the application client administrator does

not have permissions or privileges on the domain!

User -

user with an account in the Inormatica domain

can perorm tas"s in the application clients!

T'picall'# the deault administrator or a domain

administrator creates and manages user accounts

and assigns roles# permissions# and privileges in the

Inormatica domain!

owever# an' user with the required domain

privileges and permissions can create a user

account and assign roles# permissions# and

privileges

Understanding A$t'entication and Sec$rit!Do%ains -

$hen a user logs in to an application client# the

Service Manager authenticates the user account in

the Inormatica domain and verifes that the user

can use the application client!

The Service Manager uses native and L*1

authentication to authenticate users logging in to

the Inormatica domain!

&ou can use more than one t'pe o authentication in

an Inormatica domain!

+' deault# the Inormatica domain uses nativeauthentication!

&ou can confgure the Inormatica domain to use

L*1 authentication in addition to native

authentication!

,rivileges

Inormatica includes the ollowing privileges;

Do%ain privileges 0 *etermine actions on the

Inormatica domain that users can perorm using the

dministrator tool and the inacmd and pm

command line programs!

Anal!st Service privilege 0 *etermines acti

that users can perorm using Inormatica nal'st!

Data Integration Service privilege 0 *etermi

actions on applications that users can perorm us

the

Ad%inistrator tool and t'e in)ac%d co%%a

line progra% 0 This privilege also determi

whether users can drill down and export proresults!

Metadata Manager Service privileges

*etermine actions that users can perorm us

Metadata Manager!

Model Repositor! Service privilege 0 *etermi

actions on pro.ects that users can perorm us

Inormatica nal'st and Inormatica *eveloper!

,oer+enter Repositor! Service privilege

*etermine 1ower,enter repositor' actions t

users can perorm using the 5epositor' Manag

*esigner# $or"%ow Manager# $or"%ow Monitor# a

the pmrep and pmcmd command line programs!,oerEc'ange application service privilege

*etermine actions that users can perorm on

1owerExchange Listener Service a

1owerExchange Logger Service using the inac

pwx commands!

Reporting Service privileges 0 *eterm

reporting actions that users can perorm using *

nal'Cer!

&ou assign privileges to users and groups and

application services! &ou assign privileges to users and groups on

Securit' tab o the dministrator tool

&ig' Availa(ilit!

I 'ou have the high availabilit' option# 'ou c

achieve ull high availabilit' o internal Inormat

components!

&ou can achieve high availabilit' with exter

components based on the availabilit' o th

components!

If you do not have the high availability optio

you can achieve some high availability

internal components

Example

$hile 'ou are etching a mapping into

1ower,enter *esigner wor"space# the 1ower,en

5epositor' Service becomes unavailable# and

request ails! The 1ower,enter 5epositor' Serv

ails over to another node because it cannot rest

on the same node!

G


6/161

The 1ower,enter *esigner is resilient to temporar'

ailures and tries to establish a connection to the

1ower,enter 5epositor' Service!

The 1ower,enter 5epositor' Service starts within

the resilience timeout period# and the 1ower,enter

*esigner reestablishes the connection!

ter the 1ower,enter *esigner reestablishes the

connection# the 1ower,enter 5epositor' Service

recovers rom the ailed operation and etches the

mapping into the 1ower,enter *esigner wor"space!

Resilience -

All clients of PowerCenter components are

resilient to service failures!

1ower,enter services ma' also be resilient to

temporar' ailures o external s'stems# such as

database s'stems# =T1 servers# and message

queue sources!

=or this t'pe o resilience to wor"# the external

s'stems must be highl' available

Internal Resilience

&ou can confgure internal resilience at the ollowing

levels;

0 *omain

0 pplication Services

0 2atewa'

The Model Repository, Data Integration

ervice, and Analyst ervice do not have

internal resilience!

I the master gatewa' node becomes unavailable

and ails over to another gatewa' node# 'ou mustrestart these services!

ter the restart# the services do not restore the

state o operation and do not recover rom the

point o interruption!

&ou must restart .obs that were previousl' running

during the interruption

&ig' Availa(ilit! in t'e ase ,rod$ct

Inormatica provides some high availabilit'

unctionalit' that does not require the high

availabilit' option!The base product provides the ollowing high

availabilit' unctionalit';

Internal ,oer+enter resilience 0 The Service

Manager# application services# 1ower,enter ,lient#

and command line programs are resilient to

temporar' unavailabilit' o other 1ower,enter

internal components!

,oer+enter Repositor! data(ase resilience 0

The 1ower,enter 5epositor' Service is resilient to

temporar' unavailabilit' o the repositor' database!

Restart services -The Service Manager can rest

application services ater a ailure!

Man$al recover! o) ,oer+enter or#/o

and sessions 0 &ou can manuall' reco

1ower,enter wor"%ows and sessions!

M$ltiple gatea! nodes 0 &ou can confg

multiple nodes as gatewa'!

:ote; &ou must have the high availabilit' option

ailover and automatic recover'

&ou can confgure the ollowing resilience propert

or the domain# application services# and comma

line programs;

Resilience ti%eo$t0 The amount o time a cli

tries to connect or reconnect to a service! limit

resilience timeouts can override the timeout!

*i%it on resilience ti%eo$t 0 The amount o ti

a service waits or a client to connect or reconn

to the service! This limit can override the cli

resilience timeouts confgured or a connect

client! This is available or the domain a

application services

+on0g$ring Service Resilience )or t'e Do%a

The domain resilience timeout determines how lo

services tr' to connect as clients to other service

The de)a$lt val$e is seconds.

The limit on resilience timeout is the maxim

amount o time that a service allows anot

service to connect as a client!

This limit overrides the resilience timeout or

connecting service i the resilience timeout igreater value!

The deault value is 9H seconds!

The 1ower,enter ,lient resilience timeout is 9

seconds and is not confgurable!

This resilience timeout is bound b' the service li

on resilience timeout!

$hen 'ou use a command line program to conn

to the domain or an application service#

resilience timeout is determined b' one o ollowing values;

+o%%and line option 0 &ou can determine

resilience timeout or command line programs

using a command line option# 0timeout or 0t# ea

time 'ou run a command

Environ%ent varia(le 0 I 'ou do not use

timeout option in the command line s'ntax#

command line program uses the value o

environment varia

J


7/161

I:=>,LIE:T>5ESILIE:,E>TIME-?T that is

confgured on the client machine!

De)a$lt val$e0 I 'ou do not use the command line

option or the environment variable# the command

line program uses the deault resilience timeout o

9H seconds!

*i%it on ti%eo$t0 I the limit on resilience timeout

or the service is smaller than the command line

resilience timeout# the command line program uses

the limit as the resilience timeout

,oer+enter Integration Service

1ower,enter Integration Service fles include run0

time fles# state o operation fles# and session logfles!

The 1ower,enter Integration Service creates fles to

store the state o operations or the service!

The state o operations includes inormation such as

the active service requests# scheduled tas"s# and

completed and running processes!

I the service ails# the 1ower,enter Integration

Service can restore the state and recover

operations rom the point o interruption!

To move data rom sources to targets#

1ower,enter Integration Service uses the ollow

components;

,oer+enter Integration Service process 0 T

1ower,enter Integration Service starts one or m

1ower,enter Integration Service processes to

and monitor wor"%ows! $hen 'ou run a wor"%

the 1ower,enter Integration Service process sta

and loc"s the wor"%ow# runs the wor"%ow tas"s# a

starts the process to run sessions!*oad alancer 0 The 1ower,enter Integrat

Service uses the Load +alancer to dispatch tas

The Load +alancer dispatches tas"s to achie

optimal perormance! It ma' dispatch tas"s to

single node or across the nodes in a grid!

Data Trans)or%ation Manager 3DTM4 proces

The 1ower,enter Integration Service starts a *

process to run each Session and ,ommand t

within a wor"%ow! The *TM process peror

session validations# creates threads to initialiCe

session# read# write# and transorm data# a

handles pre0 and post0 session operations!

1ower,enter Integration Service process acce

requests rom the 1ower,enter ,lient and r

pmcmd and perorms the ollowing tas"s;

0 Manage wor"%ow scheduling!

0 Loc" and read the wor"%ow!

0 5ead the parameter fle!

0 ,reate the wor"%ow log!

0 5un wor"%ow tas"s and evaluates the conditio

lin"s connecting tas"s!

0 Start the *TM process or processes to run session!

0 $rite historical run inormation to the repositor'!

0 Send post0session email in the event o a *

ailure!

T'read T!pes

The t'pes o threads the master thread crea

depend on the pre0 and post0session properties#

well as the t'pes o transormations in

mapping!

The master thread can create the ollowing t'pes

threads;

Mapping t'reads -

The master thread creates one mapping thread

each session!

The mapping thread etches session and mapp

inormation# compiles the mapping# and cleans

ater session execution

,re- and post-session t'reads -

K


8/161

The master thread creates one pre0session and one

post0session thread to perorm pre0 and post0session

operations!

Reader t'reads

Trans)or%ation t'reads

The number o transormation threads depends on

the partitioning inormation or each pipeline

"riter t'reads

Re5ect Files

+' deault# the 1ower,enter Integration Service

process creates a re.ect fle or each target in the

session!

The writer ma' re.ect a row in the ollowing

circumstances;

0 It is %agged or re.ect b' an ?pdate Strateg' or

,ustom transormation!

0 It violates a database constraint such as primar'

"e' constraint!

0 feld in the row was truncated or over%owed# and

the target database is confgured to re.ect truncatedor over%owed data!

Note6 I 'ou enable row error logging# the

1ower,enter Integration Service process does not

create a re.ect fle!

Recover! Ta(les Files

The 1ower,enter Integration Service process creates

recover' tables on the target database s'stem

when it runs a session enabled or recover'!

+ontrol File$hen 'ou run a session that uses an external

loader# the 1ower,enter Integration Service process

creates a control fle and a target %at fle!

The control fle contains inormation about the

target %at fle such as data ormat and loading

instructions or the external loader!

The control fle has an extension o !ctl!

The 1ower,enter Integration Service process creates

the control fle and the target %at fle in the

1ower,enter Integration Service variable director'#

1MTarget=ile*ir# b' deault!

Indicator File

I 'ou use a %at fle as a target# 'ou can confgure

the 1ower,enter Integration Service to create an

indicator fle or target row t'pe inormation!

=or each target row# the indicator fle contains a

number to indicate whether the row was mar"ed or

insert# update# delete# or re.ect!

The 1ower,enter Integration Service process names

this fle target>name!ind and stores it in the

1ower,enter Integration Service variable directo

1MTarget=ile*ir# b' deault

+ac'e Files

$hen the 1ower,enter Integration Service proc

creates memor' cache# it also creates cache fles

The 1ower,enter Integration Service process crea

cache fles or the ollowing mapping ob.ects;

0 ggregator transormation

0 3oiner transormation0 5an" transormation

0 Loo"up transormation

0 Sorter transormation

0 4ML target

+' deault# the *TM creates the index and data f

or ggregator# 5an"# 3oiner# Loo

transormations and 4ML targets in the direct

confgured or the 1M,ache*ir service proc

variable!

The 1ower,enter Integration Service process nam

the index fle 1M(!idx# and the data fle 1M(!dat!

The 1ower,enter Integration Service process crea

the cache fle or a Sorter transormation in

1MTemp*ir service process variable director'!

Incre%ental Aggregation Files

I the session perorms incremental aggregation#

1ower,enter Integration Service process sa

index and data cache inormation to dis" when

session fnished!

The next time the session runs# the 1ower,en

Integration Service process uses this historinormation to perorm the incremen

aggregation!


in the director' confgured or the 1M,ache

service process variable

,ersistent *oo#$p +ac'e


in the director' confgured or the 1M,ache

service process variable

*oad alancer

&ou confgure the ollowing settings or the dom

to determine how the Load +alancer dispatc

tas"s;

Dispatc' %ode 0 The dispatch mode determi

how the Load +alancer dispatches tas"s! &ou

confgure the Load +alancer to dispatch tas"s i

simple round0robin ashion# in a round0robin ash

using node load metrics# or to the node with t

most available computing resources!

H


9/161

Service level 0 Service levels establish dispatch

priorit' among tas"s that are waiting to be

dispatched! &ou can create dierent service levels

that a wor"%ow developer can assign to wor"%ows

&ou confgure the ollowing Load +alancer settings

or each node;

Reso$rces 0 $hen the 1ower,enter Integration

Service runs on a grid# the Load +alancer can

compare the resources required b' a tas" with theresources available on each node! The Load

+alancer dispatches tas"s to nodes that have the

required resources! &ou assign required resources in

the tas" properties! &ou confgure available

resources using the dministrator tool or inacmd!

+,U pro0le0 In adaptive dispatch mode# the Load

+alancer uses the ,1? profle to ran" the computing

throughput o each ,1? and bus architecture in a

grid! It uses this value to ensure that more powerul

nodes get precedence or dispatch!

Reso$rce provision t'res'olds 0 The Load

+alancer chec"s one or more resource provisionthresholds to determine i it can dispatch a tas"! The

Load +alancer chec"s dierent thresholds depending

on the dispatch mode

The Load +alancer uses the ollowing dispatch

modes;

Ro$nd-ro(in0 The Load +alancer dispatches tas"s

to available nodes in a round0robin ashion! It chec"s

the Maximum 1rocesses threshold on each available

node and excludes a node i dispatching a tas"

causes the threshold to be exceeded! This mode isthe least compute0intensive and is useul when the

load on the grid is even and the tas"s to dispatch

have similar computing requirements!

Metric-(ased 0 The Load +alancer evaluates nodes

in a round0robin ashion! It chec"s all resource

provision thresholds on each available node and

excludes a node i dispatching a tas" causes the

thresholds to be exceeded! The Load +alancer

continues to evaluate nodes until it fnds a node that

can accept the tas"! This mode prevents overloading

nodes when tas"s have uneven computing

requirements!

Adaptive 0 The Load +alancer ran"s nodes

according to current ,1? availabilit'! It chec"s all

resource provision thresholds on each available

node and excludes a node i dispatching a tas"

causes the thresholds to be exceeded! This mode

prevents overloading nodes and ensures the best

perormance on a grid that is not heavil' loaded!

USING ,M+MD

pmcmd is a program 'ou use to communicate w

the Integration Service!

$ith pmcmd# 'ou can perorm some o the ta

that 'ou can also perorm in the $or"%ow Manag

such as starting and stopping wor"%ows a

sessions!

?se pmcmd in the ollowing modes;

+o%%and line %ode 0 &ou invo"e and exit pmc

each time 'ou issue a command! &ou can wscripts to schedule wor"%ows with the command l

s'ntax! Each command 'ou write in command l

mode must include connection inormation to

Integration Service!

Interactive %ode 0 &ou establish and maintain

active connection to the Integration Service! T

lets 'ou issue a series o commands!

&ou can use environment variables or user nam

and passwords with pmcmd!

&ou can also use environment variables to custom

the wa' pmcmd displa's the date and time on

machine running the Integration Service process

+eore 'ou use pmcmd# confgure these variables

the machine running the Integration Serv

process!

The environment variables appl' to pmc

commands that run on the node!

Note6I the domain is a mixed0version domain#

pmcmd rom the installation director' o

Integration Service version

N


10/161

5unning ,ommands in ,ommand Line Mode

$hen 'ou run pmcmd in command line mode# 'ou

enter connection inormation such as domain

name# Integration Service name# user name and

password in each command!

=or example# to start the wor"%ow Ow>SalesvgP in

older OSalesEast#P use the ollowing s'ntax;p%c%d startor#/o -sv M!IntService -d

M!Do%ain -$ seller -p 5ac#son -) SalesEast

)7SalesAvg

5unning ,ommands in Interactive Mode

?se pmcmd in interactive mode to start and stop

wor"%ows and sessions without writing a script!

$hen 'ou use the interactive mode# 'ou enter

connection inormation such as domain name#

Integration Service name# user name# and

password! &ou can run subsequent commands

without entering the connection inormation or

each command!

=or example# the ollowing commands invo"e theinteractive mode# establish a connection to

Integration Service OM'IntService#P and start

wor"%ows Ow>SalesvgP and Ow>SalesTotalP in

older OSalesEastP;

pmcmd

pmcmd7 connect 0sv M'IntService 0d M'*omain 0u

sellerD 0p .ac"son

pmcmd7 setolder SalesEast

pmcmd7 startwor"%ow w>Salesvg

pmcmd7 startwor"%ow w>SalesTotal

USING ,MRE,

pmrep is a command line program that 'ou use to

update repositor' inormation and perorm

repositor' unctions!

pmrep is installed in the 1ower,enter ,lient and

1ower,enter Services bin directories!

?se pmrep to perorm repositor' administration

tas"s such as listing repositor' ob.ects# creating

and editing groups# restoring and deleting

repositories# and updating session0related

parameters and securit' inormation in the

1ower,enter repositor'!

$hen 'ou use pmrep# 'ou can enter commands in

the ollowing modes;

+o%%and line %ode 0 &ou can issue pmrep

commands directl' rom the s'stem command line!

?se command line mode to script pmrep commands!

Interactive %ode 0 &ou can issue pm

commands rom an interactive prompt! pmrep do

not exit ater it completes a command!

&ou can use environment variables to set u

names and passwords or pmrep! +eore 'ou u

pmrep# confgure these variables!

The environment variables appl' to pm

commands that run on the node!

ll pmrep commands require a connection to

repositor' except or the ollowing commands;

0 elp

0 Listll1rivileges

?se the pmrep ,onnect command to connect to t

repositor' beore using other pmrep commands!

Note6 I the domain is a mixed0version domain#

pmrep rom the installation director' o

5epositor' Service version!

. DESIGNER GUIDE

The *esigner provides the ollowing tools;

So$rce Anal!8er - Import or create source defnitio

or %at fle# 4ML# ,-+-L# pplication# and relational

sources!

Target Designer - Import or create target defnition

Trans)or%ation Developer - ,reate reusable

transormations!

Mapplet Designer - ,reate mapplets!

Mapping Designer - ,reate mappings!

The *esigner consists o the ollowing windows;

Navigator - ,onnect to multiple repositories and olders! &oucan also cop' and delete ob.ects and create shortcuts using t

:avigator!"or#space -


11/161

ervie -


12/161

versions o a non0composite ob.ect in the

wor"space

verting to a 1revious -b.ect


13/161

ng +usiness :ames

&ou can add business names to sources# targets# and

columns!

+usiness names are descriptive names that 'ou give

to a source# target# or column!

The' appear in the :avigator in the +usiness

,omponents source node and in the source and

target nodes!

+usiness names can also appear as column nameso the source and target defnition in the

wor"space!

&ou can also create source qualifers to displa'

business names as column names in the Mapping

and Mapplet *esigners!

ng +usiness *ocumentation

+usiness documentation provides details about a

repositor' ob.ect or transormation expression!

&ou can create and edit lin"s to business

documentation that 'ou have developed or

repositor' ob.ects through the *esigner!

The documentation must reside on a local machine#

networ" server# or compan' intranet or internet

web site in a $indows environment!

&ou can develop business documentation in TML#

1*=# or an' text ormat# or the ollowing repositor'

ob.ects;

0 Source and target tables and table instances

0 ll transormations and transormation instances

0 Mapplets

0 Mappings

0 +usiness component directories

To access business documentation# 'ou need to

complete the ollowing tas"s;

0 Speci' the documentation path in the *esigner!

0 ,reate a lin" in the repositor' ob.ect!

0 ,lic" the lin" to view the documentation


14/161

Metadata 1(change for Data Models /an add&

on product/

&ou can import sources that use multib'te character

sets!

Source code pages must be a superset o the target

code pages!

Source defnitions can be single0 or multi0group!

single0group source has a single group in the

source defnition!5elational sources use a single0group source

defnition!

multi0group source has multiple groups in the

source defnition!

:on0relational sources such as 4ML sources use

multi0group source defnitions!

ting 5elational Source *efnitions

&ou might want to manuall' edit a source defnition

to record properties that 'ou cannot import rom

the source!&ou can edit a relational source defnition to create

"e' columns and "e' relationships!

These relationships can be logical relationships!

The' do not have to exist in the database!

or"ing with ,-+-L Sources

To provide support or mainrame source data# 'ou

can import a ,-+-L fle as a source defnition in

the *esigner!

,-+-L fles are fxed0width fles that ma' contain

text and binar' data!

1ower,enter supports the ollowing code pages or

,-+-L fles;

0 K0bit S,II

0 E+,*I,0?S

0 H0bit S,II

0 H0bit E+,*I,

0 S,II0based M+,S

0 E+,*I,0based M+,S

&ou can import shit0sensitive ,-+-L fles that do

not contain shit "e's!

*efne the shit states or each column in the ,-+-L

source defnition!

,-+-L sources oten de0normaliCe data and

compact the equivalent o separate table records

into a single record!

&ou use the :ormaliCer transormation to normaliCe

these records in the mapping!

,-+-L fles oten represent the unctional

equivalent o multiple source tables within the

same set o records!

$hen 'ou review the structure o the ,-+-L fle#

'ou can ad.ust the description to identi' which

groups o felds constitute a single pseudo0table

$or"ing with ,-+-L ,op'boo"s

The *esigner cannot recogniCe a ,-+-L cop'boo

)!cp' fle/ as a ,-+-L fle )!cbl fle/ because it lac

the proper ormat!

To import a ,-+-L cop'boo" in the *esigner# 'ou

can insert it into a ,-+-L fle template b' using ,-+-L statement Ocop'!P

ter 'ou insert the cop'boo" fle into the ,-+-L

template# 'ou can save the fle as a !cbl fle and

import it in the *esigner!

I the !cbl fle and the !cp' fle are not in the same

local director'# the *esigner prompts or the

location o the !cp' fle!

$hen the ,-+-L cop'boo" fle contains tabs# the

*esigner expands tabs into spaces!

+' deault# the *esigner expands a tab character

into eight spaces!

&ou can change this deault setting in powrmart!in

&ou can fnd powrmart!ini in the root director' o t

1ower,enter ,lient installation!

To change the deault setting# add the ollowing te

to powrmart!ini;

nal'Cer-ptionsR

TabSiCen

where n is the number o spaces the *esigner

reads or ever' tab character!

To appl' changes# restart the *esigner!

=or example# the ,-+-L cop'boo" fle is called

sample!cp'! The ,-+-L fle below shows how to

use the cop' statement to insert the sample

cop'boo" into a ,-+-L fle template;

identifcation division!

program0id! mead!

environment division!

select fle0one assign to name!

data division!

fle section!d =ILE0-:E!

cop' Osample!cp'P!

wor"ing0storage section!

procedure division!

stop run!

,omponents in a ,-+-L Source =ile

$hen 'ou import a ,-+-L source# the *esigner

scans the fle or the ollowing components;

0 =* Section

9F


15/161

0 =ields

0 -,,?5S

0 5E*E=I:ES

=* Section

The *esigner assumes that each =* entr' defnes

the equivalent o a source table in a relational

source and creates a dierent ,-+-L source

defnition or each such entr'!

=or example# i the ,-+-L fle has two =* entries#,?ST-ME5S and -5*E5S# the *esigner creates

one ,-+-L source defnition containing the felds

attributed to ,?ST-ME5S# and another with the

felds that belong to -5*E5S

lds

The *esigner identifes each feld defnition# reads

the datat'pe# and assigns it to the appropriate

source defnition!

,?5S

,-+-L fles oten contain multiple instances o the

same t'pe o data within the same record!

=or example# a ,-+-L fle ma' include data about

our dierent fnancial quarters# each stored in the

same record!

$hen the *esigner anal'Ces the fle# it creates a

dierent column or each -,,?5S statement in the

,-+-L fle!

These -,,?5S statements defne repeated

inormation in the same record! ?se the :ormaliCer

transormation to normaliCe this inormation!

=or each -,,?5S statement# the *esigner creates

the ollowing items;

0 -ne target table when 'ou drag the ,-+-L source

defnition into the Target *esigner!

0 primar'0oreign "e' relationship

0 generated column I* )2,I*/

5E*E=I:ES

,-+-L uses 5E*E=I:ES statements to build the

description o one record based on the defnition o

another record!$hen 'ou import the ,-+-L source# the *esigner

creates a single source that includes 5E*E=I:ES!

The 5E*E=I:ES statement lets 'ou speci' multiple

1I,T?5E clauses or the sample ph'sical data

location!

Thereore# 'ou need to use =ilter transormations to

separate the data into the tables created b'

5E*E=I:ES!

=or each 5E*E=I:ES;

0 The *esigner creates one target table when 'ou

drag the ,-+-L source defnition into the Target

*esigner!

0 The *esigner creates one primar'0oreign "e'

relationship!

0 The *esigner creates a generated "e' )26/!

0 &ou need a separate =ilter transormation in the

mapping!

5ules and 2uidelines or *elimited =ile Settings*elimited fles are character0oriented and line

sequential! ?se the ollowing rules and guidelines wh

'ou confgure delimited fles;

0 The column and row delimiter character# quote

character# and escape character must all be dieren

or a source defnition! These properties must also be

contained in the source or target fle code page!

0 The escape character and delimiters must be valid

the code page o the source or target fle! ?se the

ollowing rules and guidelines when 'ou confgure

delimited fle sources;

0 In a quoted string# use the escape character toescape the quote character! I the escape character

does not immediatel' precede a quote character# the

Integration Service reads the escape character as an

ordinar' character!

0 ?se an escape character to escape the column

delimiter! owever# in a quoted string# 'ou do not ne

to use an escape character to escape the delimiter

since the quotes serve this purpose! I the escape

character does not immediatel' precede a delimiter

character# the Integration Service reads the escape

character as an ordinar' character!0 $hen two consecutive quote characters appear

within a quoted string# the Integration Service reads

them as one quote character! =or example# the

Integration Service reads the ollowing quoted string

IUm going tomorrow; BDGD#VIUUm going tomorrowUM*

0 The Integration Service reads a string as a quoted

string onl' i the quote character 'ou select is the fr

character o the feld!

0 I the feld length exceeds the column siCe defned

the Source Qualifer transormation# the Integration

Service truncates the feld!

0 I the row o data exceeds the larger o the line

sequential buer length or the total row siCe defned

the Source Qualifer transormation# the Integration

Service drops the row and writes it to the session log

fle! To determine the row siCe defned in the Source

Qualifer transormation# add the column precision a

the delimiters# and then multipl' the total b' the

maximum b'tes per character!

$or"ing with =ile Lists

9G


16/161

fle list is a fle that contains the names and

directories o each source fle 'ou want the

Integration Service to use!

$hen 'ou confgure a session to read a fle list# the

Integration Service reads rows o data rom the

dierent source fles in the fle list!

To con#gure the mapping to write the source

#le name to each target row, add the

CurrentlyProcessed'ile2ame port to the )at

#le source de#nition!The Integration Service uses this port to return the

source fle name!

eating Target *efnitions

&ou can create the ollowing t'pes o target

defnitions in the Target *esigner;

Relational -,reate a relational target or a

particular database platorm! ,reate a relational

target defnition when 'ou want to use an external

loader to the target database!

Flat 0le0 ,reate fxed0width and delimited %at fle

target defnitions!

:M* 0le -,reate an 4ML target defnition to

output data to an 4ML fle!

&ou can create target defnitions in the ollowing

wa's;

0 Import the defnition or an existing target! Import

the target defnition rom a relational target or a

%at fle! The Target *esigner uses a =lat =ile $iCard

to import %at fles!

0 ,reate a target defnition based on a source

defnition! *rag a source defnition into the Target

*esigner to ma"e a target defnition!0 ,reate a target defnition based on a

transormation or mapplet! *rag a transormation

into the Target *esigner to ma"e a target defnition!

0 Manuall' create a target defnition! ,reate a

target defnition in the Target *esigner!

0 *esign several related target defnitions! ,reate

several related target defnitions at the same time!

&ou can create the overall relationship# called a

schema# and the target defnitions# through wiCards

in the *esigner!

The ,ubes and *imensions $iCards ollow commonprinciples o data warehouse design to simpli' the

process o designing related targets!

eating a Target *efnition rom a Transormation

To create a relational target defnition that closel'

matches a transormation in the repositor'W 'ou can

create the target rom the transormation!

*rag a transormation rom the :avigator to the

Target *esigner# or create a target rom a

transormation in the Mapping *esigner wor"space!

,reate target defnitions rom the ollowing t'pes

transormations;

0 Single0group transormations! ,reate a single

target defnition rom a transormation with one

output group!

0 Multiple0group transormations! ,reate multiple

target defnitions rom a transormation with

multiple output groups!

0 :ormaliCer transormations! ,reate a target

defnition rom a source qualifer or pipeline:ormaliCer transormation!

0 Mapplets! ,reate one or more target defnitions

rom a mapplet instance in a mapping!

$hen 'ou create a target defnition rom a

transormation# the target database t'pe is the

same as the repositor' database b' deault!

ter 'ou create the target defnition in the

repositor'# 'ou can edit it! =or example# 'ou migh

want to change the target t'pe!

,reating a Target rom a Transormation with -ne

-utput 2roup

$hen 'ou create a target rom a transormation w

one output group# the *esigner creates one targe

ll the output ports become input ports in the

target! The name o the target is the same as the

transormation name!

,reating a Target rom a Transormation with Multiple

-utput 2roups

$hen 'ou create targets rom a transormation wi

more than one output group# the *esigner createone target or each output group in the

transormation!

$hen the transormation is a plug0in or ,ustom

transormation# the *esigner retains the primar'

"e'0oreign "e' relationships between the groups

the target defnitions!

,reating a Target rom a :ormaliCer Transormation

&ou can create a target rom a source qualifer or

pipeline :ormaliCer transormation!

.hen you create a target from a 2ormali$ertransformation, the Designer creates one

target and includes all the columns from th

2ormali$er!

It does not create separate targets to represent th

record hierarchies or multiple0occurring felds in t

:ormaliCer transormation

,reating a Target rom a Mapplet

&ou can create a target rom a mapplet that is und

a mapping Transormation Instances node!

9J


17/161

$hen 'ou drag the mapplet instance to the Target

*esigner# the *esigner creates a target or each

output group in the mapplet!

2ote% "ou cannot create a target when you drag a

transformation instance from a mapplet to the

Target Designer

Data Trans)or%ation So$rce and Target

?se a *ata Transormation source or target to

process data in an' fle ormat such as Excel

spreadsheets or 1*= documents!

&ou can also transorm data in ormats such as LK#

E*I049B# E*I=,T# S$I=T# :,# =I4+IB# and

*T,,!

*ata Transormation source or target calls a *ata

Transormation service rom a 1ower,enter session!

*ata transormation is the application that

transorms the fle ormats!

The *ata Transormation service is a *ata

Transormation service that is deplo'ed to the *ata

Transormation repositor' and is read' to run

Data Trans)or%ation Service T!pes

$hen 'ou create a pro.ect in *ata Transormation

Studio# 'ou choose a *ata Transormation service

t'pe to defne the pro.ect! *ata Transormation has

the ollowing t'pes o services that transorm data;

,arser 0 ,onverts source documents to 4ML! The

input can have an' ormat! The output o a parser

is alwa's 4ML!

Seriali8er0 ,onverts an 4ML fle to another

document! The input is 4ML! The output can be an'

ormat!Mapper - ,onverts an 4ML source document to

another 4ML structure or schema! The input is 4ML!

The output is 4ML !

Trans)or%er0 Modifes the data in an' ormat!

dds# removes# converts# or changes text! ?se

transormers with a parser# mapper# or serialiCer!

&ou can also run a transormer as standalone

component!

Strea%er -Splits large input documents# such as

multiple gigab'te data streams# into segments! The

Streamer splits documents that have multiple

messages or multiple records in them!

MA,,INGS -

-b.ect *ependenc'

Some ob.ects in a mapping are also stored as

independent ob.ects in the repositor';

0 Sources

0 Targets

0 5eusable transormations

0 Mapplets

The mapping is dependent on these ob.ects!

$hen this metadata changes# the *esigner and

other 1ower,enter ,lient applications trac" the

eects o these changes on mappings!

In these cases# 'ou ma' fnd that mappings becom

invalid even though 'ou do not edit the mapping

$hen a mapping becomes invalid# the Integration

Service cannot run it properl'# and the $or"%ow

Manager invalidates the session! The only ob3ects in a mapping that are not

stored as independent repository ob3ects ar

the non&reusable transformations that you

build within the mapping!

These non&reusable transformations are

stored within the mapping only!

Exporting and Importing a Mapping

&ou export a mapping to an 4ML fle and import a

mapping rom an 4ML fle through the *esigner!

&ou might want to use the export and import eatu

to cop' a mapping to the same repositor'# a

connected repositor'# or a repositor' to which 'o

cannot connect

Invalidating Sessions

$hen 'ou edit and save a mapping# some change

cause the session to be invalid even though the

mapping remains valid!

The Integration Service does not run invalid

sessions! I 'ou edit a mapping# the *esigner

invalidates sessions when 'ou perorm the

ollowing actions;

0 dd or remove sources or targets!

0 5emove mapplets or transormations!

0 5eplace a source# target# mapplet# or

transormation when importing or cop'ing ob.ect

0 dd or remove Source Qualifers or ,-+-L

:ormaliCers# or change the list o associated

sources or these transormations!

0 dd or remove a 3oiner or ?pdate Strateg'

transormation!

0 dd or remove transormations rom a mapplet

the mapping!0 ,hange the database t'pe or a source or targe

*eleting a Mapping

&ou ma' delete mappings that 'ou no longer use!

$hen 'ou delete a mapping# 'ou do not delete a

sources# targets# mapplets# or reusable

transormations defned outside the mapping!

2ote% If you enable version control, deleted

mapping remains chec4ed out until you che

it in! To chec4 in a deleted mapping, clic4

9K


18/161

5ersioning 6 'ind Chec4outs! elect the

deleted mapping and clic4 Tools 6 Chec4 In.

ewing Lin" 1aths to a 1ort

$hen displa'ing both lin" paths# the *esigner traces

the %ow o data rom one column in the source# in

and out o each transormation# and into a single

port in the target!

For unconnected transormations the Designer does

not display a link path"=or connected Loo"up transormations# the *esigner

shows each output port dependent upon the input

ports involved in the loo"up condition!

For *ustom transormations the Designer sho!s

that an output port depends on all input ports by

deault" +o!ever i you defne port relationships in

a *ustom transormation the Designer sho!s the

dependent ports you defne"

:ote; &ou can confgure the color the *esigner uses

to displa' connectors in a lin" path! $hen

confguring the ormat options# choose the Lin"

Selection option!

ewing Source ,olumn *ependencies

To view column dependencies# right0clic" a target

column in a mapping and choose Show =ield

*ependencies!

The *esigner displa's the =ield *ependencies dialog

box which lists all source columns connected to the

target column!

$hen 'ou defne a port expression that perorms a

calculation using multiple source columns# and then

connect that port to a target column# the =ield*ependencies dialog box lists all source columns

'ou use in the expression

tions or Lin"ing 1orts

$hen 'ou lin" transormations# 'ou can lin" with

one o the ollowing options;

0 -ne to one! Lin" one transormation or output

group to one transormation# input group# or target

onl'!

0 -ne to man'!

0 Lin" one port to multiple transormations# inputgroups# or targets!

0 Lin" multiple ports in one transormation or

output group to multiple transormations# input

groups# or targets!

0 Man' to one! Lin" man' transormations to one

transormation# input group# or target!

5ules and 2uidelines or ,onnecting Mapping

-b.ects

0 I the *esigner detects an error when 'ou tr' to

lin" ports between two mapping ob.ects# it displa

a s'mbol indicating that 'ou cannot lin" the port

0 =ollow logic o data %ow in the mapping! &ou ca

lin" the ollowing t'pes o ports;

0 The receiving port must be an input or

input@output port!

0 The originating port must be an output or

input@output port!

0 &ou cannot lin" input ports to input ports or outports to output ports!

& "ou must lin4 at least one port of an input

group to an upstream transformation!

& "ou must lin4 at least one port of an outpu

group to a downstream transformation.

0 &ou can lin" ports rom one active transormatio

or one output group o an active transormation t

an input group o another transormation!

- "ou cannot connect an active

transformation and a passive transformatio

to the same downstream transformation or

transformation input group.- "ou cannot connect more than one active

transformation to the same downstream

transformation or transformation input

group.

- "ou can connect any number of passive

transformations to the same downstream

transformation, transformation input group

or target

& "ou can lin4 ports from two output groups

the same transformation to one 7oiner

transformation con#gured for sorted data ithe data from both output groups is sorted

- "ou can only lin4 ports with compatible

datatypes.The *esigner verifes that it can ma

between the two datat'pes beore lin"ing them!

The Integration Service cannot transorm data

between ports with incompatible datat'pes! .hi

the datatypes do not have to be identical,

they do have to be compatible, such as Cha

and 5archar!

0 &ou must connect a source defnition to a sourc

qualifer onl'! &ou then lin" the source qualifer to

targets or other transormations!

& "ou can lin4 columns to a target de#nition

in a mapping, but you cannot copy columns

into a target de#nition in a mapping! 8se th

Target Designer to add columns to a target

de#nition!

0 The *esigner mar"s some mappings invalid i th

mapping violates data %ow validation!

1ropagating 1orts and attributes

9H


19/161

e *esigner does not propagate changes to the

owing mapping ob.ects;

nconnected transormations

eusable transormations

applets

ource and target instances

*6 Source Qualifer

les and 2uidelines or 1ropagating 1orts and

ributeshe *esigner does not propagate to implicit

pendencies within the same transormation!

$hen 'ou propagate a port description# the *esigner

erwrites the description or the port in the other

nsormations in the mapping

When you propagate back!ard along the link path

riy that the change does not cause the Integration

rvice to ail the session! =or example# i 'ou

opagate changes to a source qualifer# the

egration Service might generate invalid SQL when it

ns the session! I 'ou change the port name

?ST>I*P to O,?ST-ME5>I*#P the Integration Serviceght generate SQL to select the wrong column name

he source table uses O,?ST>I*!P

When you propagate port attributes veriy that the

ange does not cause the Designer to invalidate the

apping! =or example# when 'ou change the datat'pe

a port rom integer to string and propagate the

tat'pe to other transormations# the *esigner

alidates the mapping i a calculation uses one o the

anged ports!


20/161

0 &ou cannot override the deault ?1*TE statement

i the target column name contains an' o the

ollowing characters;

Y # ) / Z 7 [ 0 ( @ \ t \ n \ Zspace7

0 &ou can use parameters and variables in the

target update quer'! ?se an' parameter or variable

t'pe that 'ou can defne in the parameter fle!

&ou can enter a parameter or variable within the

?1*TE statement# or 'ou can use a parameter or

variable as the update quer'!=or example# 'ou can enter a session parameter#

1aramM'-verride# as the update quer'# and set

1aramM'-verride to the ?1*TE statement in a

parameter fle!

& .hen you save a mapping, the Designer

veri#es that you have referenced valid port

names! It does not validate the 9!

0 I 'ou update an individual row in the target table

more than once# the database onl' has data rom

the last update!

I the mapping does not defne an order or the

result data# dierent runs o the mapping on

identical input data ma' result in dierent data in

the target table!

0 $E5E clause that does not contain an' column

reerences updates all rows in the target table# or

no rows in the target table# depending on the

$E5E clause and the data rom the mapping!

=or example# the ollowing quer' sets the

EM1>:ME to OMI6E SMITP or all rows in the

target table i an' row o the transormation has

EM1>I* 7 9;

?1*TE T>SLES set EM1>:ME YMI6E SMITY$E5E ;T?!EM1>I* 7 9

0 I the $E5E clause contains no port reerences#

the mapping updates the same set o rows or each

row o the mapping!

=or example# the ollowing quer' updates all

emplo'ees with EM1>I* 7 9 to have the

EM1>:ME rom the last row in the mapping;

?1*TE T>SLES set EM1>:ME ;T?!EM1>:ME

$E5E EM1>I* 7 9

0 I the mapping includes an ?pdate Strateg' or

,ustom transormation# the Target ?pdatestatement onl' aects records mar"ed or update!

0 I 'ou use the Target ?pdate option# confgure the

session to mar" all source records as update!

es and 2uidelines or dding 1re0 and 1ost0Session

L ,ommands

0 ?se an' command that is valid or the database

t'pe! owever# the Integration Service does not

allow nested comments# even though the database

might!

0 &ou can use parameters and variables in the

target pre0 and post0session SQL commands!

=or example# 'ou can enter a parameter or varia

within the command! -r# 'ou can use a session

parameter# 1aramM',ommand# as the SQL

command# and set 1aramM',ommand to the SQ

statement in a parameter fle!

, -se a semicolon '.) to separate multiple

statements" /he Integration Service issues a

commit ater each statement!0 The Integration Service ignores semicolons

within @( !!!(@!

0 I 'ou need to use a semicolon outside o

comments# 'ou can escape it with a bac"slash )\/

0/he Designer does not validate the SQ0"

1ote2 %ou can also enter pre, and post,session SQ

commands on the 3roperties tab o the Source

Qualifer transormation!


21/161

mapping! I the mapplet includes a source qualifer

that uses an SQL override# the *esigner prompts

'ou to connect all mapplet output ports to the

mapping!

0 *atat'pes between ports must be compatible! I

'ou change a port datat'pe to one that is

incompatible with the port it is connected to# the

*esigner generates an error and invalidates the

mapping! =or example# 'ou have two *ate@Time

ports connected# and 'ou change one port to a*ecimal! The *esigner invalidates the mapping! &ou

can however# change the datat'pe i it remains

compatible with the connected ports# such as ,har

and


22/161

&ou can validate multiple mappings without etching

them into the wor"space!

To validate multiple mappings 'ou must select and

validate the mappings rom either a quer' results

view or a view ob.ect dependencies list!

:ote; I 'ou use the 5epositor' Manager# 'ou can

select and validate multiple mappings rom the

:avigator!

&ou can save and optionall' chec" in mappings that

change rom invalid to valid status as a result o thevalidation!

To validate multiple mappings;

Select mappings rom either a quer' or a view

pendencies list!

5ight0clic" one o the selected mappings and choose

idate! The


23/161

Each -utput transormation in a mapplet displa's as

an output group in a mapping!

n output group can pass data to multiple pipelines

in a mapping!

ewing Mapplet Input and -utput

Mapplets and mapplet ports displa' dierentl' in the

Mapplet *esigner and the Mapping *esigner

The ollowing fgure shows a mapplet with both an

Input transormation and an -utput transormation;

$hen 'ou use the mapplet in a mapping# the

mapplet ob.ect displa's onl' the ports rom the

Input and -utput transormations! These are

reerred to as the mapplet input and mapplet

output ports!

The ollowing fgure shows the same mapplet in the

Mapping *esigner;

&ou can expand the mapplet in the Mapping

*esigner b' selecting it and clic"ing Mappings 7

Expand!

This expands the mapplet within the mapping or

view! Transormation icons within an expanded

mapplet displa' as shaded!

&ou can open or iconiCe all the transormations in

the mapplet and mapping!

&ou cannot edit an' o the properties# navigate to

other olders# or save the repositor' while themapplet is expanded!

The ollowing fgure shows an expanded mapplet in

the Mapping *esigner;

In an e(panded mapping, you do not see the

Input and *utput transformations!

,reating a Mapplet

mapplet can be active or passive depending on

the transormations in the mapplet!

ctive mapplets contain one or more active

transormations!

1assive mapplets contain onl' passive

transormations!

$hen 'ou use a mapplet in a mapping# all

transormation rules appl' to the mapplet

depending on the mapplet t'pe! =or example# as with an active transormation# 'o

cannot concatenate data rom an active mapplet

with a dierent pipeline!

?se the ollowing rules and guidelines when 'ou a

transormations to a mapplet;

& If you use a e;uence


24/161

The *esigner validates a mapplet when 'ou save it!

&ou can also validate a mapplet using the Mapplets

7


25/161


26/161

Setting the Target Load 1lan

$hen 'ou use a mapplet in a mapping# the Mapping

*esigner lets 'ou set the target load plan or

sources within the mapplet!

1ipeline 1artitioning

I 'ou have the partitioning option# 'ou can increase

the number o partitions in a pipeline to improve

session perormance! Increasing the number opartitions allows the Integration Service to create

multiple connections to sources and process

partitions o source data concurrentl'!

$hen 'ou create a session# the $or"%ow Manager

validates each pipeline in the mapping or

partitioning! &ou can speci' multiple partitions in a

pipeline i the Integration Service can maintain data

consistenc' when it processes the partitioned data!

Some partitioning restrictions appl' to mapplets!

5ules and 2uidelines or Mapplets

The ollowing list summariCes the rules and

guidelines that appear throughout this chapter;

0 &ou can connect an Input transormation to

multiple transormations in a mapplet! owever# 'ou

cannot connect a single port in the Input

transormation to multiple transormations in the

mapplet!

0 n Input transormation must receive data rom a

single active source!

0 mapplet must contain at least one Input

transormation or source defnition with at least one

port connected to a transormation in the mapplet!0 mapplet must contain at least one -utput

transormation with at least one port connected to

another transormation in the mapping!

0 When a mapplet contains a source #ualifer that

has an override or the deault SQ0 #uery you must

connect all o the source #ualifer output ports to

the ne$t transormation !ithin the mapplet!

0 I the mapplet contains more than one source

qualifer# use a 3oiner transormation to .oin the

output into one pipeline! I the mapplet contains

onl' one source qualifer# 'ou must connect the

mapplet output ports to separate pipelines! &ou

cannot use a 3oiner transormation to .oin the

output!

0 $hen 'ou edit a mapplet# 'ou might invalidate

mappings i 'ou change the mapplet t'pe rom

passive to active!

0 I 'ou delete ports in the mapplet when the

mapplet is used in a mapping# 'ou can invalidate

the mapping!

0 *o not change the datat'pe# precision# or scale o

a mapplet port when the mapplet is used b' a

mapping!

0 I 'ou use a Sequence 2enerator transormation

'ou must use a reusable Sequence 2enerator

transormation!

0 I 'ou use a Stored 1rocedure transormation# 'o

must confgure the Stored 1rocedure T'pe to be

:ormal!

0 &ou cannot include 1owerMart D!G0st'le L--6?1unctions in a mapplet!

, %ou cannot include the ollo!ing objects in a

mapplet2

, 1ormali4er transormations

, *obol sources

, 560 Source Qualifer transormations

, 560 sources

, /arget defnitions

, 3re, and post, session stored procedures

, 7ther mapplets

Mapping ,ara%eters and ;aria(les

Mapping 1arameters

mapping parameter represents a constant value

that 'ou can defne beore running a session!

mapping parameter retains the same value

throughout the entire session!

$hen 'ou use a mapping parameter# 'ou declare

and use the parameter in a mapping or mapplet!

Then defne the value o the parameter in a

parameter fle!

The Integration Service evaluates all reerences tothe parameter to that value!

$hen 'ou want to use the same value or a mapp

parameter each time 'ou run the session# use th

same parameter fle or each session run!

$hen 'ou want to change the value o a mapping

parameter between sessions 'ou can perorm on

o the ollowing tas"s;

0 ?pdate the parameter fle between sessions!

0 ,reate a dierent parameter fle and confgure t

session to use the new fle!

0 5emove the parameter fle rom the sessionproperties! The Integration Service uses the

parameter value in the pre0session variable

assignment! I there is no pre0session variable

assignment# the Integration Service uses the

confgured initial value o the parameter in the

mapping!

Mapping


27/161

The Integration Service saves the value o a

mapping variable to the repositor' at the end o

each successul session run and uses that value the

next time 'ou run the session!

$hen 'ou use a mapping variable# 'ou declare the

variable in the mapping or mapplet# and then use a

variable unction in the mapping to change the

value o the variable!

t the beginning o a session# the Integration

Service evaluates reerences to a variable todetermine the start value!

t the end o a successul session# the Integration

Service saves the fnal value o the variable to the

repositor'!

The next time 'ou run the session# the Integration

Service evaluates reerences to the variable to the

saved value!

To override the saved value# defne the start value o

the variable in a parameter fle or assign a value in

the pre0session variable assignment in the session

properties!

?sing Mapping 1arameters and


28/161

/he ollo!ing table describes the available variable

unctions and the aggregation types and datatypes

you use !ith each unction2

riable =unctions


29/161

Although you can place a user&de#ned

function in another user&de#ned function, a

function cannot refer to itself!

=or example# the user0defned unction

5emoveSpaces includes a user0defned unction

TrimLeadingandTrailingSpaces!

TrimLeadingandTrailingSpaces cannot include

5emoveSpaces! -therwise# 5emoveSpaces is

invalid!

nfguring 1ublic =unctions that ,ontain 1rivatenctions

$hen 'ou include ports as arguments in a private

user0defned unction# 'ou must also include the

ports as arguments in an' public unction that

contains the private unction! ?se the same

datat'pe and precision or the arguments in the

private and public unction!

=or example# 'ou defne a unction to modi' order

I*s to include VI:=U and the customer I*! &ou frst

create the ollowing private unction called

,oncat,ust that concatenates VI:=U with the port

,?ST>I*; ,-:,T )VI:=U# ,?ST>I*/

ter 'ou create the private unction# 'ou create a

public unction called ,oncat-rder that contains

,oncat,ust;

,-:,T );?*=!,-:,T,?ST) ,?ST>I*/# -5*E5>I*/

$hen 'ou add ,oncat,ust to ,oncat-rder# 'ou add

the argument ,?ST>I* with the same datat'pe and

precision to the public unction!

:ote; I 'ou enter a user0defned unction when 'ou

manuall' defne the public unction s'ntax# 'ou

must prefx the user0defned unction with ;?*=The ollowing table describes the user0defned

unction management tas"s and lists where 'ou

can perorm each tas";


30/161

2ote% "ou cannot create brea4points for

mapplet Input and *utput transformations!

,reating Error +rea"points

When you create an error breakpoint the Debugger

pauses !hen the Integration Service encounters

error conditions such as a transormation error or

calls to the &((7( unction!

&ou also set the number o errors to s"ip or each

brea"point beore the *ebugger pauses;

0 I 'ou want the *ebugger to pause at ever' error#set the number o errors to Cero!

0 I 'ou want the *ebugger to pause ater a specifed

number o errors# set the number o errors greater

than Cero! =or example# i 'ou set the number o

errors to fve# the *ebugger s"ips fve errors and

pauses at ever' sixth error!

?sing IS:?LL and IS*E=?LT

&ou can create IS:?LL and IS*E=?LT conditions in

transormation and global data brea"points!

$hen 'ou use the IS:?LL or IS*E=?LT operator#

'ou cannot use the t'pe or value in the condition!.hen you create an I28 condition, the

Debugger pauses when the Integration

ervice encounters a null input value, and the

port contains the system default value!

.hen you create an ID1'A8T condition, the

Debugger pauses in the following

circumstances%

& The Integration ervice encounters an

output transformation error, and the port

contains a user&de#ned default value of a

constant value or constant e(pression!

& The Integration ervice encounters a null

input value, and the port contains a user&

de#ned default value of a constant value or

constant e(pression!

5unning an Existing Session in *ebug Mode

I 'ou choose to run an existing session in debug

mode# the *ebugger $iCard displa's a list o allsessions in the current older that use the mappi

Select the session 'ou want to use!

"ou cannot run the Debugger against a

session con#gured with multiple partitions

a session con#gured to run on a grid! "ou

must either change the properties of the

session or choose to create a debug session

for the mapping!

Set Target -ptions

-n the last page o the *ebugger $iCard# 'ou can

select the ollowing target options;

Discard target data6&ou can choose to load or

discard target data when 'ou run the *ebugger!

you discard target data, the Integration

ervice does not connect to the target!

Displa! target data; &ou can select the target

instances 'ou want to displa' in the Target windo

while 'ou run a debug session!

$hen 'ou clic" =inish# i the mapping includes

mapplets# the *ebugger displa's the mapplet

instance dialog box!

Select the mapplets rom this dialog box that 'ou

want to debug! To clear a selected mapplet# pres

the ,trl "e' and select the mapplet!

$hen 'ou select a mapplet to debug# the *esigne

expands it to displa' the individual transormatio

when the *ebugger runs!

$hen 'ou do not select a mapplet to debug# the

*esigner does not expand it in the wor"space!

D


31/161

"ou cannot complete the following tas4s for

transformations in the mapplet%

& Monitor or modify transformation data!

& 1valuate e(pressions!

& 1dit brea4points!

& tep to a transformation instance!

/he Debugger can be in one o the ollo!ing states2

Initiali$ing%/he Designer connects to the

Integration Service"Running%/he Integration Service processes the

data"

Paused%the Integration Service encounters a

break and pauses the Debugger!

:ote; To enable multiple users to debug the same

mapping at the same time# each user must

confgure dierent port numbers in the Tools 7

-ptions 7 *ebug tab!

The Debugger does not use the high

availability functionality!

The ollowing table describes the dierent tas"s 'ou

can perorm in each o the *ebugger states;

$or"ing with 1ersisted


32/161

*esigner +ehavior

.hen the Debugger starts, you cannot

perform the following tas4s%

& Close the folder or open another folder!

& 8se the 2avigator!

& Perform repository functions, such as ave!

& 1dit or close the mapping.

- witch to another tool in the Designer, such

as Target Designer!& Close the Designer"

:ote; *'namic partitioning is disabled during

debugging!

Monitoring the *ebugger

$hen 'ou run the *ebugger# 'ou can monitor the

ollowing inormation;

Session stat$s6Monitor the status o the session!

Data %ove%ent6Monitor data as it moves through

transormations!

rea#points6Monitor data that meets brea"point

conditions!Target data6 Monitor target data on a row0b'0row

basis!

The Mapping *esigner displa's windows and debug

indicators that help 'ou monitor the session;

De($g indicators; *ebug indicators on

transormations help 'ou ollow brea"points and

data %ow!

Instance indo6$hen the *ebugger pauses# 'ou

can view transormation data and row inormation in

the Instance window!Target indo6


33/161

*oo#$p trans)or%ation6:ewLoo"up5ow port or a

Loo"up transormation confgured to use a d'namic

cache!

+$sto% trans)or%ation61orts in output groups

other than the current output group!

>ava trans)or%ation61orts in output groups other

than the current output group!

Additionally, you cannot change data associated

with the following%

& Mapplets that are not selected for debugging

& Input or inputoutput ports

& *utput ports when the Debugger pauses on an

error brea4point

?. "9R@F*9" ASI+S GUIDE

$or"%ow Tas"s

&ou can create tas"s in Tas" *eveloper# $or"%ow

designer or $or"let designer

Tas"s created in Tas" *ev are reusable but those

created in $or"%ow@wor"let designer are not!

&ou can create the ollowing t'pes o tas"s in the

$or"%ow Manager;

Assign%ent6ssigns a value to a wor"%ow

variable!

+o%%and6Specifes a shell command to run during

the wor"%ow!

+ontrol6Stops or aborts the wor"%ow!

Decision6Specifes a condition to evaluate!

E%ail6Sends email during the wor"%ow!

Event-Raise6:otifes the Event0$ait tas" that an

event has occurred!

Event-"ait6$aits or an event to occur beoreexecuting the next tas"!

Session65uns a mapping 'ou create in the

*esigner!

Ti%er6$aits or a timed event to trigger!

$or"%ow Manager $indows

The $or"%ow Manager displa's the ollowing

windows to help 'ou create and organiCe

wor"%ows;

0 :avigator

0 $or"space0 -utput

0 -verview

Enhanced Securit'

The $or"%ow Manager has an enhanced securit'

option to speci' a deault set o permissions or

connection ob.ects!

.hen you enable enhanced security, the.or4)ow Manager assigns default

permissions on connection ob3ects for users

groups, and others.

$hen 'ou disable enable enhanced securit'# the

$or"%ow Manager ssigns read# write# and execu

permissions to all users that would otherwise

receive permissions o the deault group!

If you delete the owner from the repository,

the .or4)ow Manager assigns ownership o

the ob3ect to the administrator!


34/161

originall' included version D o a session# but version

D o the session is purged rom the repositor'! $hen

'ou view version G o the wor"%ow# version B o the

session appears as part o the wor"%ow!

& "ou cannot view older versions of sessions if

they reference deleted or invalid mappings, or

if they do not have a session con#guration.

Searching or


35/161

?se the ollowing guidelines when creating the SQL

statements;

0 ?se an' command that is valid or the database

t'pe! owever# the Integration Service does not

allow nested comments# even though the database

might!

0 ?se a semicolon )W/ to separate multiple

statements! The Integration Service issues a commit

ater each statement!

0 The Integration Service ignores semicolonswithin @( !!!(@!

0 I 'ou need to use a semicolon outside o

comments# 'ou can escape it with a bac"slash )\/!

0 The .or4)ow Manager does not validate the

9!

Error andling 0

&ou can confgure error handling on the ,onfg

-b.ect tab!

&ou can choose to stop or continue the session i the

Integration Service encounters an error issuing the

pre0 or post0 session SQL command

The $or"%ow Manager provides the ollowing t'pes

o shell commands or each Session tas";

,re-session co%%and -The Integration Service

perorms pre0session shell commands at the

beginning o a session! &ou can confgure a session

to stop or continue i a pre0session shell command

ails!

,ost-session s$ccess co%%and 0 The Integration

Service perorms post0session success commands

onl' i the session completed successull'!,ost-session )ail$re co%%and 0 The Integration

Service perorms post0session ailure commands

onl' i the session ailed to complete!

1re0Session Shell ,ommand Errors

I 'ou select stop# the Integration Service stops the

session# but continues with the rest o the wor"%ow!

I 'ou select ,ontinue# the Integration Service

ignores the errors and continues the session!

+' deault the Integration Service stops the session

upon shell command errors!

Session +on0g$ration 9(5ect

,onfguration -b.ect and ,onfg -b.ect Tab Settings

&ou can confgure the ollowing settings in a session

confguration ob.ect or on the ,onfg -b.ect tab in

session properties;

Advanced0 dvanced settings allow 'ou to

confgure constraint0based loading# loo"up caches#

and buer siCes!

*og options0 Log options allow 'ou to confgure

how 'ou want to save the session log! +y default

the og Manager saves only the current

session log!

Error 'andling0 Error andling settings allow 'o

to determine i the session ails or continues when

encounters pre0session command errors# stored

procedure errors# or a specifed number o session

errors!

,artitioning options0 1artitioning options allowthe Integration Service to determine the number o

partitions to create at run time!

Session on grid0 $hen Session on 2rid is enabl

the Integration Service distributes session threads

the nodes in a grid to increase perormance and

scalabilit'!

DG


36/161


37/161

*ecision Tas"

&ou can enter a condition that determines the

execution o the wor"%ow# similar to a lin" condition

with the *ecision tas"!

The *ecision tas" has a predefned variable called

*ecision>tas">name!condition that represents the

result o the decision condition!

The Integration Service evaluates the condition in

the *ecision tas" and sets the predefned condition

variable to True )9/ or =alse )/!

"ou can specify one decision condition per

Decision tas4!

ter the Integration Service evaluates the *ecision

tas"# use the predefned condition variable in other

expressions in the wor"%ow to help 'ou develop the

wor"%ow!

*epending on the wor"%ow# 'ou might use lin"

conditions instead o a *ecision tas"!

owever# the *ecision tas" simplifes the wor"%ow!

I 'ou do not speci' a condition in the *ecision ta

the Integration Service evaluates the *ecision tas

to True!

?sing the *ecision Tas"

?se the *ecision tas" instead o multiple lin"

conditions in a wor"%ow!

Instead o speci'ing multiple lin" conditions# use

the predefned condition variable in a *ecision ta

to simpli' lin" conditions!

Example

=or example# 'ou have a ,ommand tas" that

depends on the status o the three sessions in th

wor"%ow! &ou want the Integration Service to run

the ,ommand tas" when an' o the three session

ails! To accomplish this# use a *ecision tas" with

the ollowing decision condition;

Q9>session!status =ILE* -5 QB>session!stat

=ILE* -5 QD>session!status =ILE*

&ou can then use the predefned condition variabl

in the input lin" condition o the ,ommand tas"!

,onfgure the input lin" with the ollowing lin"

condition;

*ecision!condition True

&ou can confgure the same logic in the wor"%ow

without the *ecision tas"!

.ithout the Decision tas4, you need to use

three lin4 conditions and treat the input lin

to the Command tas4 as *R lin4s!

Event Tas" &ou can defne events in the wor"%ow to speci' t

sequence o tas" execution!

The event is triggered based on the completion o

the sequence o tas"s!

?se the ollowing tas"s to help 'ou use events in t

wor"%ow;

Event-Raise tas# 0 Event05aise tas" represents a

user0defned event! $hen the Integration Service

runs the Event05aise tas"# the Event05aise tas"

DK


38/161

triggers the event! ?se the Event05aise tas" with the

Event0$ait tas" to defne events!

Event-"ait tas#0 The Event0$ait tas" waits or an

event to occur! -nce the event triggers# the

Integration Service continues executing the rest o

the wor"%ow!

To coordinate the execution o the wor"%ow# 'ou

ma' speci' the ollowing t'pes o events or the

Event0$ait andEvent05aise tas"s;

,rede0ned event 0 predefned event is a fle0

watch event! =or predefned events# use an Event0

$ait tas" to instruct the Integration Service to wait

or the specifed indicator fle to appear beore

continuing with the rest o the wor"%ow! $hen the

Integration Service locates the indicator fle# it starts

the next tas" in the wor"%ow!

User-de0ned event0 user0defned event is a

sequence o tas"s in the wor"%ow! ?se an Event0

5aise tas" to speci' the location o the user0defnedevent in the wor"%ow! user0defned event is

sequence o tas"s in the branch rom the Start tas"

leading to the Event05aise tas"!

$hen all the tas"s in the branch rom the Start tas"

to the Event05aise tas" complete# the Event05aise

tas" triggers the event! The Event0$ait tas" waits

or the Event05aise tas" to trigger the event beore

continuing with the rest o the tas"s in its branch!

Timer Tas"

&ou can speci' the period o time to wait beore the

Integration Service runs the next tas" in the

wor"%ow with the Timer tas"!&ou can choose to start the next tas" in the wor"%ow

at a specifed time and date!

&ou can also choose to wait a period o time ater

the start time o another tas"# wor"%ow# or wor"let

beore starting the next tas"!

The Timer tas" has the ollowing t'pes o settings;

A(sol$te ti%e0 &ou speci' the time that the

Integration Service starts running the next tas" in

the wor"%ow! &ou ma' speci' the date and time# or

'ou can choose a user0defned wor"%ow variable t

speci' the time!

Relative ti%e 0 &ou instruct the Integration Servi

t

infa notes .doc

Documents