getting started with end-to-end application performance ... · what is apm? •apm = application...
TRANSCRIPT
![Page 1: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/1.jpg)
Getting Started with End-to-End Application Performance Management
Tom Quinn
CA Technologies
February 5, 2013
Session Number 12903
![Page 2: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/2.jpg)
Agenda
• Overview of APM
• Why you want to adopt Cross Platform APM
• The APM Cycle
• Recommendations
2
![Page 3: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/3.jpg)
What is APM?
• APM = Application
Performance Management is
the process and technologies
of evaluating, measuring and
analyzing performance data,
updating, validating and
repeating the cycle for
continual improvement on
application performance.
• Translating System Metrics
into Business Value
3
Evaluate
Measure
Analyze Update
Validate
![Page 4: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/4.jpg)
Why do I want to adopt Cross Platform APM?
• Formalize your monitoring
• To mediate an immediate issue
• Improve or create cross-team
collaboration
• End to End Application Visibility
• Faster Problem Resolution
• Quicker Time to Value
• Prioritize Resources to key
applications
• Move from a system only monitoring perspective to an
application perspective
4
![Page 5: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/5.jpg)
Where do I start?
• Before you begin the cycle, you must develop an overall
Application Performance Management strategy. Simply
ask yourself
5
Improved
Customer
Experience
Improve
MTTR and
Root Cause
Fix
Performance
Issues before
Production
Improve
Cross
Platform
Collaboration
Better
Resource
Utilization
What am I trying to accomplish?
SLA
Monitoring
and
Reporting
![Page 6: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/6.jpg)
How to Gain Leadership Buy-in
• Map your top 3 most business critical applications
• Identify the monitoring type\level at each hop along the path
• Determine cost of down-time
• Avg cost per customer x peak transaction volume
• Determine the cost of tool, hardware, staff
• Build your case (or not)
• Cost of an outage vs. Cost of APM
6
![Page 7: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/7.jpg)
Ok, all systems GO…now what?
• One you have decided that APM is right for you, it’s time to
define it for your organization.
• Assign a person or group who will own APM as a process
within your organization.
This helps with:
• Efficiency
• Consistency
• Focused Effort
7
![Page 8: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/8.jpg)
Who needs to be involved?
• Many people need to be involved to make APM effective
as there is no one “APM Expert” that can know and do all.
• APM Team
• Application Architects
• Developers
• Business Analysts
• Platform experts
• Monitoring Software Administrators
• Monitoring Teams
• Performance Analysts
8
APM Team
App Architects
Developers
Business Analysts
Platform Experts
Software Admins
Monitoring Teams
Performance Analysts
![Page 9: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/9.jpg)
Conversation Terminology
• business volume metrics (bvms) – the number of business
transactions executed during a given time interval with peaks identified
• system volume metrics (svms) – the number of system transactions
executed on behalf of a business transaction
• application performance budget – the resource constraints
(response time, cpu, etc) for a given application and its components
• critical path – the application path that is most used and provides the
most business value
• peak hour – identifying when system utilization is likely to spike based
on higher business/transaction volume
9
![Page 10: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/10.jpg)
Platform Terminology
Distributed
• Server
• VM
• JVM
• Reboot
• Chicken Farmer
Mainframe
• Mainframe\Enterprise Server
• LPAR
• Started Task
• IPL
• Crusty old mainframer
10
• It’s important to understand the different platforms use
different terminology, but there are a lot of similarities.
• Some technologies run on multiple platforms, for example,
DB2…be aware of that and don’t assume.
![Page 11: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/11.jpg)
11
Evaluate
Measure
Analyze Update
Validate
![Page 12: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/12.jpg)
Evaluate
• You must first evaluate your Company goals for
your applications
• Take inventory of your applications and assign
each to an application classification tier
• Begin APM with your Tier 1 applications
12
This is an important step to ensure you’re
consistently spending resources in the most
effective manor to meet business needs.
![Page 13: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/13.jpg)
Application Classification Tiers
13
•Business Critical
•Directly tied to revenue with immediate impact
•High Volume
•Customer Facing Tier 1
•Directly tied to revenue with impact within 5-7 hours
•High to medium volume
•Customer Facing Tier 2 •Directly tied to revenue with impact in 7-10 days or more
•Medium to low volume
•Customer facing or internal Tier 3 •Not directly tied to revenue; no real impact
•Low Volume
•Internal Only Tier 4 These are samples, you must evaluate your business and application make-up and build out appropriate tiers.
![Page 14: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/14.jpg)
Application Tiers
• How do I best determine what tier to classify app
• Ask questions
• Engage business analysts
• Engage Platform teams
• Create reports of historical data for transactions
• Define Business Transactions
• Map system transactions
• 1 Business transaction can equal 30 system transactions
14
![Page 15: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/15.jpg)
15
Evaluate
Measure
Analyze Update
Validate
![Page 16: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/16.jpg)
Measure
• Determine monitoring gaps and develop a strategy
to fill those gaps or determine they are acceptable
• Deploy more monitoring agents
• Purchase new software
• Utilize system metrics
• Gather relevant performance metrics
• Eliminate metrics you do not need
• Determine Monitoring Classes
• Current and Desired State
• Build Performance Budgets
16
Measure
![Page 17: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/17.jpg)
Application Monitoring Tiers
17
• 24x7 Real-time Monitoring of all components to a single dashboard
• Thresholds and Alerts on all monitored components
• End-user Experience Monitoring Tier 1 • All components monitored to a single dashboard
• Thresholds and Alerts on some monitored components
• End-user Experience Monitoring Tier 2 • Some components monitored to a single dashboard
• No thresholds or alerts in place
• Synthetic monitoring Tier 3 • Few to none components monitored
• System level monitoring only, not application level
• No thresholds or alerting in place Tier 4
![Page 18: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/18.jpg)
Performance Budgets
• Application Performance Budgeting is an approach to ensure
applications meet their response time objectives and proactively
mitigate performance concerns at any phase – development,
performance testing, production.
• Always updating and validating
• May assist in identifying performance bottlenecks and inefficiencies.
• A Performance Budget breaks down a transaction to allow for the
developer to code to requirements and enable operations to maintain
the application and continue meeting design specifications.
18
![Page 19: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/19.jpg)
Performance Budgets
• identify the required pieces of the
application before a single line of
code is written
• data sources and locations
• security/authentication
requirements
• hardware hops
• measure the infrastructure
response times.
before a single line of code is
written, you know the fastest
this application will ever run!
19
![Page 20: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/20.jpg)
Build the Application Performance Budget
- start with what you know. use existing metrics, industry
standards, vendor metrics
- start by allocating time across the tiers for each transaction
class to add up to the maximum response time.
20
Client Tier Network Tier Middle Tier Database Tier
Transaction Class 1 0 – 3 second objectives
.3 seconds 1.9 seconds .5 seconds .25 seconds
Transaction Class 2 0 – 5 second objectives
.45 seconds 2.9 seconds .9 seconds .7 seconds
Transaction Class 3 0 – 10 second objectives
.75 seconds 4.5 seconds 1.75 seconds 2.75 seconds
![Page 21: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/21.jpg)
Populate the Application Performance Budget
• - application performance budgets can vary in information,
look and feel, but the goal is the same – help develop well
performing transaction that meet response time goals
21
Class component name Tier # of execs cpu response cumulative cpu cumulative response
Balance Inquiry 1 abcd client 2 0.002 0.09 0.004 0.18
defg middle 1 0.001 0.15 0.001 0.15
wxyz db 3 0.025 0.83 0.075 2.49
mnop middle 1 0.1 0.43 0.1 0.43
0 0
0 0
0 0
0 0
0 0
total 7 0.128 1.5 0.18 3.25
![Page 22: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/22.jpg)
22
Evaluate
Measure
Analyze Update
Validate
![Page 23: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/23.jpg)
Analyze
• Analyze the Application Tiers and Monitoring Tiers
you have assigned
• Analyze the data you are collecting
• Are you collecting at adequate intervals?
• Are you collecting enough or too much data?
• At what granularity are you keeping that data at?
• Are the metrics within your guidelines or SLAs?
• Analyze the application flow and data collection for
any gaps
• Analyze the dashboards for meaningful metrics
23
Analyze
![Page 24: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/24.jpg)
24
Evaluate
Measure
Analyze Update
Validate
![Page 25: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/25.jpg)
Update
• Update the Application Tiers
• Have you added any new applications?
• Have you integrated functionality of applications?
• Have you created any new application
dependencies?
• Update the Monitoring Tiers for each application to
see if any have gone up or down a tier
• Have you filled monitoring gaps?
• Have you added thresholding and alerting?
• Update the Application dashboards
• Update the Performance Budgets
25
Update
![Page 26: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/26.jpg)
26
Evaluate
Measure
Analyze Update
Validate
![Page 27: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/27.jpg)
Validate
• Validate your updates
• Are you collecting the metrics you’re expecting?
• Are you missing any metrics?
• Compare your monitoring to your application
architecture diagram
• Validate your Performance Budget with metrics
from production
• Are you seeing the volume and response times you
expected?
• Validate your Application dashboards are
adequately representing the data
27
Validate
![Page 28: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/28.jpg)
Repeat the cycle
28
Evaluate
Measure
Analyze Update
Validate
Repeat
RE
PE
AT
Rep
eat
![Page 29: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/29.jpg)
Recommendations
• Develop a Performance and
Availability Team
• Include members from all
platforms
• Standardize on tooling on each
platform
• Standardize on metrics
provided\measured where
possible
• Standardize on a single pane of
glass application view
• Standardize on dashboard design
29
![Page 30: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/30.jpg)
Recommendations
• Get Performance engaged early
• Build performance reviews into project plan
• Include capacity planners
• Schedule design reviews with platform
performance teams
• Require application specific dashboards built in
development or testing phase before moving
into production
• Deploy monitoring with the application
30
![Page 31: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/31.jpg)
Recommendations
• Change your mindset from a system perspective to an
application perspective
• Use both real user monitoring and synthetic monitoring
• What is the performance within our firewall?
• What is the customer experience from external sites around
the world?
• Produce meaningful application level reports from this data
• This is a great tool for your capacity planners and Business
Analysts
31
![Page 32: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/32.jpg)
The SDLC and APM
• APM must be part of the Software Development Lifecycle
32
Get Engaged Early
APM
System Analysis
Design
Development
Implementation
Validation
Evaluation
Performance Gateway
GO/NO GO
![Page 33: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/33.jpg)
Recommendations
• Create a performance gateway where the
performance team reviews the
performance testing numbers and must
sign off before the application can be
implemented into your production
environment.
• This is the final opportunity to keep poor
performing applications from being
implemented into production
• This also gives you an opportunity to
acquire necessary capacity planning
numbers
33
![Page 34: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/34.jpg)
Reasons APM Fails
• Lack of commitment from Leadership
• Organizational Silos
• No communication across teams
• Too much data – Big Data
• Dashboard complexity
• Keep it simply, avoid flashy dashboards
• Build a dashboard that answers critical performance
questions in one click or less
• Focus on showing basic, easy to understand real-time
metrics
34
![Page 35: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/35.jpg)
Books on APM
• APM Best Practices: Realizing Application Performance
Management
Michael J. Sydor
• Application Performance Management: A Practical
Introduction
Dr Sampath I Prakash, John J Sikora
Both books available on Amazon.com in print and Kindle format as of this writing.
35
![Page 36: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/36.jpg)
Questions? Comments? Experiences?
36
![Page 37: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/37.jpg)
SESSION NUMBER 12903
Please provide feedback
37
Share.org\SFEval
Thank you!
![Page 38: Getting Started with End-to-End Application Performance ... · What is APM? •APM = Application Performance Management is the process and technologies of evaluating, measuring and](https://reader031.vdocuments.net/reader031/viewer/2022022707/5be51acf09d3f2f4628db170/html5/thumbnails/38.jpg)
• Tom Quinn is a Principal Consultant, Mainframe
Technical Sales at CA technologies responsible for
mainframe performance monitoring products. Tom
joined CA in 2009, bringing with him over 15 years of
technical and leadership experience in Information
Technology. Having worked for a large Mid-Western
insurance company in mainframe system and
application performance, Tom understands the technical
challenges a large organization can face and utilizes his
ITIL and ScrumMaster experience to discover and build
customer solutions. You can reach him at
[email protected] or [email protected].
38