powercenter real-time development - informatica … powercenter real-time development brian bunn,...

11

PowerCenter Real-Time Development

Brian Bunn, Project ManagerSerco

Jay Moles, Sr. Informatica DesignerSerco

Tom Bennett, Sr. ConsultantInformatica

2

Agenda

• Overview of PowerCenter Web Services

• Error Handling

• Performance Tuning

• Web Services Security

• Implementation – Case Study• SERCO

3

Overview of Informatica PowerCenterWeb Services

4

PowerCenter Web Services Hub

• Application Service• Client application gateway to PowerCenter Domain

• Batch Web Services• Integration Service Access• Repository Metadata Access

• Real-Time Web Services• Exposes Workflows as Web Services• PowerCenter Web Services Provider

5

Informatica PowerCenterWeb Services Provider

• Architecture

6

Informatica PowerCenterWeb Service Consumer

• Allow Integration with Web Services• 3rd Party• Provider

• Integration Sources• URL to Web Service• WSDL File• UDDI

7

Error Handling

8

SOAP Fault Messages

• FaultCode

• FaultString• Describes the error

• FaultFactor (Optional)• URI of the host generating the error

• Detail (Optional)• Describes the cause of the error

9

Error Handling

• No SOAP Faults: Capture Source

10

Performance Tuning/Scalability

11

Performance Tuning• Parameters to tune at the Workflow

12

Web Services Performance Tuning

• Parameters to tune at the Session level

Idle Time

Message Count

Reader Time Limit

13

Web Services Scalability

• Associate multiple repositories with a Web Services Hub

• Associate a repository with multiple Web Services Hubs

• Run multiple instances of a web service workflow

• Run web service sessions or workflows on a grid

Load Balancer

14

Web Services Security

15

Security Use Cases

• Protect a web service from unauthorized usage (by requiring username and password)

• Manage web services as part of overall security policies and integrating with LDAP

• Prevent a web service from DOS attacks or authentication attacks

• Allowing a secured web service to be load balanced across multiple WSHs for performance and scalability

16

Web Service Hub Security

• Encryption• Encrypts repository information in the configuration file and

in responses to Web service clients for login requests

• Authentication• Authenticates Web service clients’ user name and

password

• Authorization• Checks Web service client has execute permission on a

folder to run the service

3

17

Informatica WS-Security

• Support for Standard• Oasis WS-Security UsernameToken Profile 1.1• Single pass authentication – user specifies the appropriate

security headers in the actual request to the service

• Supports multiple WSHs load balancing scenario• Username Tokens are associated with a repository within a

specific domain and should be applicable to all WSHs associated to that repository

• Advanced support for encoded passwords• Hashed Passwords• Digested Passwords

• Prevent repeated DOS/login attack• Support for NONCE values

18

WS-Security Feature Details (Part 1)

• UsernameToken Format:

19

WS-Security Feature Details (Part 2)• Password Format Supported

• Clear Text• Hashed – Same as clear text, but the password is encoded

into a base64 20 bit hash value. Two hash algorithms supported: SHA-1 or MD5

• Digested – A generated password that is of the following formula:

• BASE64( SHA1_HASH (USERNAME + NONCE + CREATED) )• NONCE – random generated value that’s valid only once for that

specific username token, it’s provided to allow someone to specify a one-time only authentication token

• CREATED – timestamp of when the UsernameToken was made in UTC timezone format

• Example: 2008/08/11T18:06:32.425Z(yyyy/MM/dd’T’HH:mm:ss.SSS’Z)

20

WS-Security Feature Details (Part 3)

• Custom Property for NONCE and CREATED• NonceExpirationTime – custom property that can be set at the WSH to

determine how long the hub should hold down to a specific NONCE value of a UsernameToken. By default this value is set to 300 seconds (5 minutes)

• If a request is made containing the same NONCE value outside of this timeframe, the request will fail.

• CreatedFreshnessPeriod – custom property that determines how long a user has between the time the first request is made using the CREATED value and the next. By default this value is set to 300 seconds (5 minutes)

• If a request is made before this timestamp or after the expiration time following the timestamp, the request will fail.

• These two customer properties can be used independently of each other or together to ensure greater security.

21

Case Study : Implementation Architecture

22

Case Study: Architecture

23

Web Services Throughput (examples)

• Production Environment (Multiple Servers)• RowsPerSecond – Overall numbers range between 100 and 1000 rps

• If a request results in >10K rows, some customers have timed-out.• RequestsPerMinute – Overall numbers range between 10 and 100.

• If a request takes over 20 minutes, some customers have timed-out.

24

The ETL Behind-the-Scene

• Production Web-Service – ‘Broadcast’ing Data• ‘Filtered’ Data – SOAP request has ‘parameters’ for result set• ‘SQL’ Transformation – the data source can be ‘dynamic’ [access/name]• ‘Sequence Generator’ Transformation – XML output has multiple rows!

25

Web Services Latency (examples)

• Single Server Web-Service• Initialization – it takes the WSH from 3 to 11 seconds to prepare the ETL. • Processing – it then takes the WSH about 1 second to start processing

the ETL. At this point, the response time depends on the processing.

• Multiple Server Web-Service• Initialization – it takes the WSH from 3 to 11 seconds to prepare the ETL• Invocation – it takes the WSH about 3 seconds to prepare the SOAP

Request and invoke the internal web-service. • Processing – it then takes the internal WSH from 3 to 11 seconds to

prepare the ETL, then the response time will depend on the internal processing.

26

Timeout/Fault Handling (examples)

• Input – Web-Services Source• Validation – ETL tests/verifies content of the request

• Output – Web-Services Target• Response – ETL always sends same formatted output, either a ‘status

message’ or a pre-determined value

• Processing – depends on WSConsumer• Application Connection ‘Timeout’ – property set for internal WSH call

• Will wait for a connection to internal WSH indefinitely.• Will wait for a response from internal WSH indefinitely.

• These two properties prevent a forced timeout from the WSH.

27

Example – Our ‘Hello!’

• Simple Web-Service• No ‘input’ fields in SOAP Request, static text in SOAP response • Validates customer’s connection via SSL – ‘Hello!’

28

Example – Our ‘Addition’

• Two-Step Web-Service• Two ‘input’ fields in SOAP Request, ‘sum’ returned in SOAP response • Validates customer’s request will invoke internal web-service(s)

29

SSL Configuration

30

SSL Configuration – Setting Up SSL

• Keystore for Web Services Hub• Use the ‘keytool’ utility. You generate a keystore, generate a ‘signature

request’ from it, have it signed by an authority, then install the signed file into your keystore.

• Default keystore generated by the PowerCenter ‘installation’ process can be used for WSH, but be aware it is self-signed.

• Truststore for Web Services Hub• Add customers certificates to the ‘ca-bundle.crt’ files. • Use ‘openssl’ utilities to convert format if necessary. PEM format is

required to add certificate to the ‘ca-bundle.crt’ files.

31

SSL Configuration (examples)

• Creating the Keystore for Web Services Hub• Example of the ‘keytool’ utility. Prompts in black text, inputs in red.

32

SSL Configuration (examples)

• Creating the ‘Signature Request File’• Example of the ‘keytool’ utility using the created keystore.

• Have request ‘signed’ by your Authority• Independent firms: VeriSign.

• Install the ‘Signed Response File’• Example of the ‘keytool’ utility for the keystore and response.

33

SSL Configuration (examples)

• Installing the Customer Certificate• Customer certificates need to be in ‘PEM’ format.

• If certificate is not PEM, convert it using ‘openssl’ utility. If customer’s certificate file is in a binary format, DER for example, convert it.

• Add customer certificate text to the end of the ‘ca-bundle.crt’ files.• Add “PEM Data:” line before certificate text. Entry will look like this:

• File in PowerCenter Installation Directory, sub-folders ‘server/bin’ and ‘services/shared/bin’ – both!

34

SSL Configuration (examples)

• Using the Keystore in the Web Services Hub• Define the WSH service to use your signed-keystore. • URLScheme must be set to either ‘HTTPS’ or ‘HTTPandHTTPS’ for the

Web Services Hub to utilize SSL and access the keystore.• Enter the keystore name and password used to create your keystore.

35

Tips, Best-Practices, etc.

• SOAP Request/Response: WSDL or Manual?• Inputs – XML of the in-bound request: one or multiple rows? • Outputs – XML of the out-bound response: one or multiple rows?• Processing – Initially, set up a ‘pass-thru’ with an ‘exp’ transformation

and just the fields coming in and going out of the web-service. Basic ‘static’ test version. Then, add bits and pieces.

• TESTING your Web-Services• WSH and ‘TryIt!’ – ability to ‘run’ web-service, but limited.• soapUI– open-source tool (FREE!) that expands testing capabilities.

Basically a ‘SOAP Client’ GUI.

36

Informatica Velocity MethodologyVelocity is the blueprint for delivering efficient and successful Informatica solutions that solve business problems.

New Website• New search capability • Filtering/viewing content by

• project type• project phase • or other tags

• New accelerator tools• Hot links between the articles

Access at: mysupport.informatica.com

Visit the Informatica Pavilion at the Technology and Solutions Fair for more details. Check out

more than 100new articles!

3737

Questions?

38

Stay In Touch!

• LinkedIn• Informatica Power Center Real-Time

3939

Thank You