1 informatica productivity pack save time and money while increasing the quality of your powercenter...

1

Informatica Productivity Pack

Save Time and Money while Increasing the Quality of Your PowerCenter Deployment

Louis Hausle

2

Are these issues you have run into?

• Best Practices Violations even though you have them documented!

• Abuse and misuse of SQL overrides, log verbosity or commit/cache/dtm buffer size?

• Zombie Workflows?

3

Devel-opment

38%

Data Validation and Testing

29%

Monitoring of P0werCenter

21%

Code Review and Best Practices Enforcement

10%

Where is Time Spent in a PowerCenter Environment?(source Informatica World 2012 Poll)

• Development accounts for less than 50% of time

• More Time and Money Spent on Testing and Monitoring

• You can shrink the TCO of PowerCenter by automated Testing and Monitoring

4

What Exactly is the Productivity Pack?

( it is a Sales Bundle of Three Products)

PowerCenter Productivity Pack

Proactive Monitoring for PowerCenter

- Governance

Proactive Monitoring for PowerCenter

- Operations

Data Validation

Option

DEV TEST PROD

5

Agenda

• Development – Proactive Monitoring for PowerCenter – Governance

• Testing – The Informatica Data Validation Option – DVO

• Production – Proactive Monitoring for PowerCenter - Operations

6

Every PowerCenter Customer Has Best Practices, Nearly Every One Struggles to Enforce Them!

7

Challenges in Data Integration & Governance

• Best Practice Enforcement• Most companies have no way to

enforce best practices

• Production Impact• Unchecked Development Practices

and Choices can lead to Fatal Mistakes in Production

• Measure and Quantify Development Work• How can you track many developers in

different locations?

8

1. Enforcement of Best Practices

Customer Challenges

• How do you monitor many developers in many locations, all the time?

• How do make your Best Practices Actionable and enforceable as they happen?

Mappings are created and saved without comments

Next Developer has no idea of

what logic is doing

Time is wasted and Mistakes can

be made

9

2. Production Impact of Development Options

Customer Challenges

• There are hundreds of development flags and options on Mappings, Workflows, Sessions and Transformations

• Customers can have dozens, hundreds and thousands of Workflows

A developer uses the ‘truncate target table’ option for testing in

development

He forgets to change the flag, and the

workflow is migrated into production

Data is lost, and recovery steps can take time and introduce new

data problems

11

Agenda

• Development – Proactive Monitoring for PowerCenter – Governance

• Testing – The Informatica Data Validation Option – DVO

• Production – Proactive Monitoring for PowerCenter - Operations

12

The Data Validation and Testing Dilemma

Informatica has over 4,000 PowerCenter Customers who believe using a metadata driven, graphical tool to move data is better than writing SQL and hand coding.

15 years later, 99% of those 4,000 Customers still use SQL and Excel to test the data they have moved in data integration products.

There has to be a better way…

13

Current Approach: Like a Photo Hunt

14

Source A

Source B

Target (DW or Operational

System

Sales Orders

Sales Amount

23 $150,000

Sales Orders

Sales Amount

30 $150,000

Sales Orders

Sales Amount

52 $290,000

DVO - tells you 1 Order and $10,000 is MissingIn a few clicks it does set, aggregation and value tests.

Data Integration

Or Movement

Data Validation Simple Illustration

15

DVO: Test CreationLeverage your PowerCenter Metadata!GUI with Pre-Built Test Operators with NO SQL

16

DVO: Test ResultsSummary of Test Results

17

Agenda

• Development – Proactive Monitoring for PowerCenter – Governance

• Testing – The Informatica Data Validation Option – DVO

• Production – Proactive Monitoring for PowerCenter - Operations

18

Informatica built these solutions to address three common customer concerns…

1. Customers asked for a “Complete, PowerCenter Centric” Monitoring Solution

2. Customers want to become “Proactive” in seeing issues before they turn into problems. Send email alerts instead of logging in to a monitor.

3. Scalable way to identify anomalies such as Workflows running longer/shorter than normal or returning more/less rows than normal.

19

What is Proactive PowerCenter Monitoring?

Prebuilt Rules to Monitor Informatica PowerCenter

Delivers Alerts to people, dashboards and systems

Complete Operational and Best Practices

Extensible – build unlimited number of new rules

20

Solution OverviewProactive PowerCenter Monitoring

Alert EnrichmentDBs, Web Services and more…

Alert Editing, Creation and Management

Custom:Environmental(CPU, Memory..)

DB (SQL)JMS

Web Services

• Operational• Design Time

• Start/Stop Workflows• Create Help Desk Ticket• Update Control Tables

Operational Monitoring viaContinuous Correlation(Sample Rules)

1 Workflow/session was successful but zero (0) records loaded

2 Rejected Records in Session

3 Disproportionate elapsed time between Workflow and Session

4 Session/Workflow Failures

5 Correlate session changes to workflow failures

6 SLA Violations: Workflow/Folder/Repository Level

7 Significant increase in Workflow/Session Elapsed Times

8 Significant decrease in applied row counts

9 Workflows missing schedules

10 Too many concurrent workflows

11 Automatic Restart of Zombie Workflows

12 Ping Domain/Integration Service/Repository Service

SQL Sources to Expose Repository Features and Allow Easy Rule and Report Building

Environmental Monitoring

Runtime Statistics Receiver via Web

Services Hub

Ping PowerCenter Services

Notification Framework for Alerts

Prebuilt EMAIL Responses

Prebuilt Dashboard Alerts

PowerCenter Workflow Response to Execute Commands

21

PowerCenter

Domain

Repository Service

Integration Service

Administrator

Dat

aba

seS

erve

r

App

lica

tion

Ser

ver

Proactive Monitoring for PowerCenterIllustration of Components that are Monitored

1. The PowerCenter Domain, Integration Services and Repository Services are

Monitored

2. The PowerCenter Repository Database is Monitored for Up/Down and

Tablespace Usage

3. The PowerCenter Server OS is Monitored for Memory and CPU Usage

4. Running Workflows and Sessions are Monitored

5. Over 500 Attributes can be used in rules

22

DEMONSTRATION of PROACTIVE MONITORING FOR

GOVERNANCE

• Check for Commit size violations• Check On Rollback Transactions on Errors• Naming Conventions and Check for

Comments