1 informatica productivity pack save time and money while increasing the quality of your powercenter...
TRANSCRIPT
1
Informatica Productivity Pack
Save Time and Money while Increasing the Quality of Your PowerCenter Deployment
Louis Hausle
2
Are these issues you have run into?
• Best Practices Violations even though you have them documented!
• Abuse and misuse of SQL overrides, log verbosity or commit/cache/dtm buffer size?
• Zombie Workflows?
3
Devel-opment
38%
Data Validation and Testing
29%
Monitoring of P0werCenter
21%
Code Review and Best Practices Enforcement
10%
Where is Time Spent in a PowerCenter Environment?(source Informatica World 2012 Poll)
• Development accounts for less than 50% of time
• More Time and Money Spent on Testing and Monitoring
• You can shrink the TCO of PowerCenter by automated Testing and Monitoring
4
What Exactly is the Productivity Pack?
( it is a Sales Bundle of Three Products)
PowerCenter Productivity Pack
Proactive Monitoring for PowerCenter
- Governance
Proactive Monitoring for PowerCenter
- Operations
Data Validation
Option
DEV TEST PROD
5
Agenda
• Development – Proactive Monitoring for PowerCenter – Governance
• Testing – The Informatica Data Validation Option – DVO
• Production – Proactive Monitoring for PowerCenter - Operations
6
Every PowerCenter Customer Has Best Practices, Nearly Every One Struggles to Enforce Them!
7
Challenges in Data Integration & Governance
• Best Practice Enforcement• Most companies have no way to
enforce best practices
• Production Impact• Unchecked Development Practices
and Choices can lead to Fatal Mistakes in Production
• Measure and Quantify Development Work• How can you track many developers in
different locations?
8
1. Enforcement of Best Practices
Customer Challenges
• How do you monitor many developers in many locations, all the time?
• How do make your Best Practices Actionable and enforceable as they happen?
Mappings are created and saved without comments
Next Developer has no idea of
what logic is doing
Time is wasted and Mistakes can
be made
9
2. Production Impact of Development Options
Customer Challenges
• There are hundreds of development flags and options on Mappings, Workflows, Sessions and Transformations
• Customers can have dozens, hundreds and thousands of Workflows
A developer uses the ‘truncate target table’ option for testing in
development
He forgets to change the flag, and the
workflow is migrated into production
Data is lost, and recovery steps can take time and introduce new
data problems
11
Agenda
• Development – Proactive Monitoring for PowerCenter – Governance
• Testing – The Informatica Data Validation Option – DVO
• Production – Proactive Monitoring for PowerCenter - Operations
12
The Data Validation and Testing Dilemma
Informatica has over 4,000 PowerCenter Customers who believe using a metadata driven, graphical tool to move data is better than writing SQL and hand coding.
15 years later, 99% of those 4,000 Customers still use SQL and Excel to test the data they have moved in data integration products.
There has to be a better way…
13
Current Approach: Like a Photo Hunt
14
Source A
Source B
Target (DW or Operational
System
Sales Orders
Sales Amount
23 $150,000
Sales Orders
Sales Amount
30 $150,000
Sales Orders
Sales Amount
52 $290,000
DVO - tells you 1 Order and $10,000 is MissingIn a few clicks it does set, aggregation and value tests.
Data Integration
Or Movement
Data Validation Simple Illustration
15
DVO: Test CreationLeverage your PowerCenter Metadata!GUI with Pre-Built Test Operators with NO SQL
16
DVO: Test ResultsSummary of Test Results
17
Agenda
• Development – Proactive Monitoring for PowerCenter – Governance
• Testing – The Informatica Data Validation Option – DVO
• Production – Proactive Monitoring for PowerCenter - Operations
18
Informatica built these solutions to address three common customer concerns…
1. Customers asked for a “Complete, PowerCenter Centric” Monitoring Solution
2. Customers want to become “Proactive” in seeing issues before they turn into problems. Send email alerts instead of logging in to a monitor.
3. Scalable way to identify anomalies such as Workflows running longer/shorter than normal or returning more/less rows than normal.
19
What is Proactive PowerCenter Monitoring?
Prebuilt Rules to Monitor Informatica PowerCenter
Delivers Alerts to people, dashboards and systems
Complete Operational and Best Practices
Extensible – build unlimited number of new rules
20
Solution OverviewProactive PowerCenter Monitoring
Alert EnrichmentDBs, Web Services and more…
Alert Editing, Creation and Management
Custom:Environmental(CPU, Memory..)
DB (SQL)JMS
Web Services
• Operational• Design Time
• Start/Stop Workflows• Create Help Desk Ticket• Update Control Tables
Operational Monitoring viaContinuous Correlation(Sample Rules)
1 Workflow/session was successful but zero (0) records loaded
2 Rejected Records in Session
3 Disproportionate elapsed time between Workflow and Session
4 Session/Workflow Failures
5 Correlate session changes to workflow failures
6 SLA Violations: Workflow/Folder/Repository Level
7 Significant increase in Workflow/Session Elapsed Times
8 Significant decrease in applied row counts
9 Workflows missing schedules
10 Too many concurrent workflows
11 Automatic Restart of Zombie Workflows
12 Ping Domain/Integration Service/Repository Service
SQL Sources to Expose Repository Features and Allow Easy Rule and Report Building
Environmental Monitoring
Runtime Statistics Receiver via Web
Services Hub
Ping PowerCenter Services
Notification Framework for Alerts
Prebuilt EMAIL Responses
Prebuilt Dashboard Alerts
PowerCenter Workflow Response to Execute Commands
21
PowerCenter
Domain
Repository Service
Integration Service
Administrator
Dat
aba
seS
erve
r
App
lica
tion
Ser
ver
Proactive Monitoring for PowerCenterIllustration of Components that are Monitored
1. The PowerCenter Domain, Integration Services and Repository Services are
Monitored
2. The PowerCenter Repository Database is Monitored for Up/Down and
Tablespace Usage
3. The PowerCenter Server OS is Monitored for Memory and CPU Usage
4. Running Workflows and Sessions are Monitored
5. Over 500 Attributes can be used in rules
22
DEMONSTRATION of PROACTIVE MONITORING FOR
GOVERNANCE
• Check for Commit size violations• Check On Rollback Transactions on Errors• Naming Conventions and Check for
Comments