software performance engineering services

1

Collaborative Consulting: Software Performance Engineering Services

2

Contents and Overview

Value Proposition About Collaborative Services & approach

Performance Engineering’s high impact to Client brand and IT Risk

Mitigation

Collaborative has a rich history of success in

delivering client outcomes

Current and emerging client services

3




Mitigation




4

Why Collaborative? Our People, Value and Delivery

Our people

Our people are our strongest asset; we not only have the most senior individuals with deep domain expertise, but our IT technical support team is second to none in the industry.

Our strongest asset

At the heart of the organization is a team of committed individuals who are passionate about solving complex issues, and dedicated to providing the highest quality of service possible.

Heart of the organization

Our culture and work process is focused on pragmatism, problem-solving and results-oriented action.

Our culture and work process

We understand the “big picture”, as well as the macro business trends, but we really thrive on the specific mechanics and process of the business.

Our understanding

We bring a comprehensive methodology and approach that allow us to be confident in our work products, be rigorous in our approach, and speed “time-to-value” in all of our projects.

Our comprehensive methodology

We leverage our proven project accelerator tools and methodologies that reduce effort, improve quality and shorten timelines.

Our proven project accelerator tools

Our value Our delivery

5

Our solutions approach

Our flexible delivery model allows us to work with clients in the way that makes the most sense for the situation: as a trusted advisor, in a blended team scenario, or as a dedicated project team.

Flexibility

Enterprise Architecture

Business Technology

Technology strategy

Solution delivery

Business process optimization

Solution delivery

A keen understanding of enterprise

business issues and operations,

with practitioners

across multiple industries.

A discipline aimed at

building the right solution for

each client, based on their specific needs and business

objectives.

Aligning business and technology initiatives enables our clients to achieve results that deliver business value.

6

Services & Delivery Model

Life Sciences • Pharmaceutical • Medical Device

Understand and envision

Architect and design Develop and deploy Support and

maintain

Consulting Services Domestic Solution Center Services

Financial Services • Capital Markets • Insurance

Information Management

• Master Data Management

• Data Quality • Data Integration • Business Intelligence • Data Governance

Application Architecture and

Development • Architecture

Strategy • Application

Development • Systems Integration • Mobile Enablement • Application Support

Software Performance and Quality

• PE Advisory Services • Application Rescue • Performance Testing

& Tuning • Quality Assurance

Strategy • Software Quality

Testing

7




Mitigation




8

Performance Engineering Value

PE is the set of roles, skills, activities, practices, tools, and deliverables to be applied at each phase of the SDLC to ensure that a solution will be designed, implemented, and operationally supported

to meet the non-functional requirements defined for the solution.

What is PE?

• User experience: PE task and activities in the SDLC promote business and IT alignment through reliable, scalable, and performing systems. Performance is designed in early.

• Increased revenue: Ensure an organizations’ systems and applications are able to manage complexity, identify risk, and enable seamless and rapid business growth for increased revenue.

• Effective use and return on system resources. Capacity planning trending and forecasting can help manage IT system investments, eliminate budget surprises.

• Manage risk of technology adoption for business value. Performance evaluation and Proof-of-concept testing can help identify business risk.

Value delivered to the business

9

Benefits and Case for Action

We deliver high value, impactful services, resulting in business stability and client and customer satisfaction.

• Manage risk and complexity.

• Ensure service level agreement criteria is met and exceeded.

• Ensure application scales with business growth.

• Identify the system’s performance limits.

• Ensure the physical infrastructure

supports the business plan.

• There is a need for application service assurance and reliability.

• High availability and performance from enterprise applications are needed to achieve return on investment.

• There are multiple configuration options for distributed applications.

• Multiple integration points in a typical system installation can lead to performance and scalability issues.

Key Benefits Case for Action

10

The SPE Body of Knowledge – Key differentiator Kn

owle

dge

area

s

Enterprise Software Performance Engineering

Planning, coordination, information sharing, and control

SDLC & architecture

(SA)

Application performance management

(APM)

Problem detection

and resolution

(PDR)

Underlying competencies and roles for delivering

Techniques supporting the knowledge areas

Performance validation and testing

(PVT)

Capacity planning (CP)

Collaborative is the Performance Engineering Expert

11

PE: Methodology

PE encompasses the set of roles, skills, activities, practices, tools, and deliverables applied at every phase of the SDLC which ensures that a solution

will be designed, implemented, and operationally supported to meet the non-functional requirements defined for the solution.

Step-by-step framework for better software engineering

Collaborative’s PE Methodology is a step-by-step framework designed to ensure an organizations’ systems and applications are able to enhance competitive advantage, manage complexity, identify risk, and enable seamless and rapid business growth.

Focused on business objectives In short, our methodology helps companies determine if their technology can accommodate its business objectives.

Based on unique methodology

Our PE Methodology is unique in that it is applicable throughout a system’s entire life cycle. It also: • Describes the detailed approach to designing, building, executing and

interpreting performance benchmarks • Shows how to analyze performance of a system in production that falls

short of service level objectives

12




Mitigation




13




Mitigation




14

Performance Engineering Services

Software Performance Engineering

Collaborative’s performance

engineering services drive improvements

in information technology and

ensure scalability and stability for

existing and planned systems, including improving overall quality, reliability, capacity, security, and performance.

Advisory Services Is my current team delivering the right services? Organizational and operational planning, strategy and design services for transforming internal capabilities. Establishing PE Center of Excellence (COE)

Application and readiness

assessment

Is my application ready for the new volumes? or is the application response time getting worse over time. The goal is to technically assess the application to understand the risks to performance, scalability and the transaction response times.

Production Performance Rescue

A proven approach to production performance and stability issues, reviewing the end to end transaction. Minimizing disruption and quickly restoring functionality and capability.

Application & Service Performance Management

Are my key business transactions being monitored, do I know of slowdowns before my customers? Optimizing the performance of specific applications and services within a system. Define and implement an APM strategy. Identify key business transactions for monitoring. Measurement, and trending.

Technology optimization and

tuning

Is my technical architecture delivering the best end user response time and transaction throughput? Review technical components, includes hands on testing and measurement.

Benchmarks and Performance testing

Application performance and scalability test design and execution. Performance test lab definition and testing tool selection. We use local and remote team for test execution.

15

1. Advisory Services

• Review the current state of the team; review services provided, interaction model with development and operations teams, and work products

• Review the services provided to the organization • Determine the value provided • Understand what the organization needs from the PE team • Review the Application Selection Criteria for Performance Engineering and

performance testing • Engagement Model with the development teams, the operations teams, the

PMO process and the governance process.

Team and Organizational Assessment

• Services roadmap • Value proposition for the team • Organization structure, roles and responsibilities

Outcome

16

1. Advisory Services: Organizational Alignment Framework for PE

Communication & Education Programs




Competencies Provide PE processes, skills, and tools,

to excel in the new PE environment.

Resources Align resource allocation

changes with updates to role definition and responsibilities.

Assess the alignment of ongoing initiatives with the

business performance vision.

Business Vision Manage underlying policies and

conditions that influence behavior.

Culture Management

Ensure that the plan is successful by collaborating with key sponsors and leaders to

champion the initiative.

Sponsorship and Change Leadership

Assess the change to existing operations environment and business areas with regard to

workflow interactions, tools, resource allocation, responsibilities, and competencies.

Change Impact Assessment

17

1. Advisory Services Scope

Engineering Group Quality Assurance/ Performance Engineering Group

Operations

Platform/ Environment

Validation

Performance Benchmarking

Performance Regression

Performance Integration

Production Performance Monitoring

Configuration/ Customization

Design Implementation Application Testing Production

Support

Performance Engineering Advisory Scope

Define service level

objectives

Development Process

18

2. Application Readiness Assessment

Phase 1 Inspection of existing

application and technical architecture.

Significant risk

Phase 2 Risk evaluation. Code

inspection and performance testing.

Assessment report, recommendations, observations, and next steps.

Call to action: • Current application is slowing down, user response time is degrading. • Anticipating significant change in business volume. Will the current system scale? • Moving to a new technology platform or acquiring a new application. Will it support my business

volumes?

Yes

No

19

2. Application Readiness Assessment

Methodology is iterative and tactical

Review infrastructure & architecture

Define business activity profiles & service levels

• Identify risk areas • Review configuration

settings, topology, & sizing • Define points of

measurement

• Types & numbers of users • Business activities &

frequencies

Iterate testing & tuning

• Test data generation

• Create test scripts • User & transaction

profiles • Infrastructure

configuration

Design & build tests

Developing a clear picture of system capabilities helps companies form a strategy to enable seamless growth.

• Verify accurate inventories of applications and systems, and capabilities and limitations. • Determine whether certain applications need to be replaced, retired or upgraded. • Assess and improve performance engineering process and execution

20

3. Production Performance Rescue

Minimize disruption and quickly restore functionality and capability.

• Accelerated analysis and test design, including system architecture review

• Accelerated iteration of test execution and application and configuration changes

• Accelerated implementation of performance improvements.

• Mitigation/remediation of issue(s)

• Recommendation and planning of tactical short-term improvements and strategic longer-term enhancements

21

4. Application & Service Performance Management Align business and IT through proactive management of service levels

across IT and multiple lines of business.

Performance management data

repository

Performance repository administration:

• Create forecasts for new applications

• Modify existing forecasts

Performance data sources by: Tier, Application, Server

Platform Application Tier

Performance dashboard views: Canned Reports: J F M A M J J F M A M J J F M A M J

Legend: Performance Details Captured Performance Management Data Published

Combine monitoring and management tools with proven Performance Engineering experience and methodologies to link people, information and decision-making

processes throughout the organization.

22

4. APM: Watch Every Transaction for Every User

WEB

MQ/ESB

DCOM

CORBA

SERVER

APP SVR

PROXY

LDAP

Web Services

Datacenter

23

5. Technology Optimization and Tuning

• Identify business transactions and supporting system components

• Evaluate each tier in the technical architecture

• Leverage end to end transaction monitoring

• Use deep dive tool for code analysis and database analysis

Building a better user experience; and/or reduce resource consumption

• Improved and consistent user experience

• Improved scalability (process more transactions with the same system)

• Improved stability

• Transfer for knowledge to clients’ technical team

Outcome

24

6. Benchmarks and Performance Testing

What is changing? Where does it need to be validated? What (all) is being measured?

The project: Why are we testing and what are the success criteria?

Prerequisites:

Business goals and non-functional requirements are defined

Workload and scenarios defined

Environment defined (where are we testing)

Application in a stable state

Configuration management and release management ready

Core performance team identified

Supporting team committed

25

6. Benchmarks and Performance Testing

• Define a custom benchmark for clients, for a specific workload and technical architecture

• Application Baseline

• Stress and breakpoint

• Duration

• Peak load

• Component(s)

• Failover and Failback (baseline or with load)

• Antagonistic (degraded service times)

• Performance testing and code changes (iteration)

Performance Testing Categories

26

6. Performance test delivery – remote model

Distributed test execution team

Test execution options

• Remote team uses VPM to access client facility and execute tests

• Remote team use Cloud services to delivery load to client application

VPN

Cloud

Client facility Collaborative’s remote testing facility

Our performance engineer is onsite

Test execution team

27

• CASE • STUDY Healthcare Prescription Benefits Manager (PBM)

Approach

Objective •Major business impact due to production disruptions within customer’s PBM System. This resulted in major downtime of the system.

•No Performance Testing (PT) environment being used. Limited PT execution. Lack of disciple around PT and lack of coverage. No Performance Engineering occurring!

•Client looking to reduce risk and ensure that there will be no disruption in the future, and that all components of PBM System will scale and perform for all future releases.

Results

•In an effort to reduce risk and ensure a higher level of performance testing process, we were initially engaged to assess the PT and PE maturity across process, organization, roles and tools to identify and recommend areas where tactical changes can be applied to implement PE across the platform. •Following the assessment phase, presented roadmap including staffing and cost models to executive leadership. •Planned for partial FTEs and On Demand resources with a model to move towards more FTEs over time. •Focus has been on the entire Performance Engineering process across the SDLC, from design review, test execution, results review and analysis, and troubleshooting.

•Client has improved the ability to execute disciplined performance tests and to increase the number of meaningful tests. Client has been able to execute more tests for each release. •By improving the test preparation process and test review process, Client is able to reduce the number of retests due to improper configurations and settings. •Within first 6 months, Client has identified multiple significant issues, prior to Production deployment, resulting in immediate ROI, covering the cost of the group for the first two years. •Visibility into the performance of the application and infrastructure has been vastly increased by improving the monitoring process and increasing the number and type of metrics being collected. •Client is better positioned to manage risk and complexity across their PBM System.

28

• CASE • STUDY Financial Services Pension Provider

Approach

Objective

Client’s Online Group sought to proactively improve the performance of its systems and demonstrate previously achieved improvements. Client wants guarantees that the applications will perform without disruption, and given the frequency of enhancements and modifications being implemented for the platform, the organization wants to become more proactive and better understand the effect each enhancement has on the stability of the system before being deployed into production.

Results

•Phase 1: Assessment effort only. Began the strategic process of developing the PE process definition and PE roadmap, and PE process implementation. The assessment phase is a review of the key applications to assess the technical challenges of performance engineering each system. Within Phase 1 this included:

•Client secure account web application •Client Public site.

•Phase 2 began the tactical implementation of the roadmap, based on business priority.

We delivered an assessment including findings and recommendations, and developed a Roadmap including several key initiatives. Notably this included the following work streams:

•Baseline Client secure account web application and the Client Public site •PE test environment analysis •Client secure account web application and Public Site Production Performance Tuning •Organization Alignment •Non-production environment performance monitoring •Performance test data management •Application assessment – Administrative web application

29

• CASE • STUDY Retail

Approach

Objective • “Next generation” pharmacy system to replace legacy application is a highly complex system consisting of multiple

technologies and infrastructure components (2.6M lines of code; 25,000 function points). • Two PT environments leveraged to accelerate performance testing. Additional complexities and risks are also being identified

(e.g., system stability, level of code completeness) while executing performance testing. Concerns raised over the viability of aggressive PT timeframe.

• The company looking to reduce risk and ensure that the new solution will scale and perform for pilot and planned rollout to chain.

Results

•In an effort to reduce risk and ensure a higher level of performance testing process, Collaborative was initially engaged to assess the PT plan, process, organization, roles and tools to identify and recommend areas where tactical changes can be applied to improve the overall PT process.

•Following the assessment phase, Collaborative was engaged to manage and execute the large-scale performance testing. Current team includes a blend of resources from Collaborative, this company’s off-shore development partner based in India, and various on-shore support staff. Also working with key tech vendors (Oracle, IBM, EMC, etc).

•With aggressive program timelines, the team is running multiple shifts in a 5x24 mode.

•Focus has been on the entire testing process, from design review, test execution, results review and analysis, and troubleshooting.

•Collaborative technical architects are also working with the infrastructure team and the off-shore application development team to provide recommendations and resolve performance problems with the application and in the environment.

•Collaborative has improved the ability to execute performance tests and to increase the number of meaningful tests. By streamlining the testing process, Collaborative has been able to execute more tests for each shift during a 24 hour period.

•By improving the test preparation process and test review process, we were able to reduce the number of retests due to improper configurations and settings.

•Feedback provided to the test design process has enabled the test execution team to combine test cases into one test case. This also allowed the team to be more efficient with test execution.

•Visibility into the performance of the application and infrastructure has been vastly increased by improving the monitoring process and increasing the number and type of metrics being collected across all the tiers and components.

•The client is better positioned to manage their risk and complexity

30

Thank You