a ranger4 guide to application performance management

A Ranger4 Guide to

Application Performance Management

www.ranger4.com © Ranger4 2014 1

http://www.google.com/url?q=http%3A%2F%2Fwww.ranger4.com&sa=D&sntz=1&usg=AFQjCNFVDEXbd4GQxVLiqSaw6TEFPEyAtQ

Contents

1.0 What is Application Performance Management?1.1 APM and DevOps

2.0 Why should you do it?3.0 What you should look for in a vendor

3.1 End-user experience monitoring3.2 Application topology discovery and visualization3.3 User-defined transaction profiling3.4 Application component deep dive3.5 IT operations analytics3.6 Questions to ask

4.0 Considerations at implementation time4.1 Cultural Change4.2 Process Change

4.2.1 Educate and Evangelise4.3 SAAS or On-Premise4.4 Verification4.5 Discovery and Configuration4.6 Alerts

5.0 Ranger4 Recommendations



1.0 What is Application Performance Management?

Application Performance Management (APM), is the monitoring and managing ofperformance and availability of software applications. APM strives to detect and diagnoseapplication performance problems to maintain an expected level of service. APM is thetranslation of IT metrics into business meaning (i.e. value).

Five functional dimensions are required for successful APM (as defined by Gartner in thelatest 2013 Magic Quadrant for APM):

1. End-user experience monitoring2. Application topology discovery and visualization3. User-defined transaction profiling4. Application component deep dive5. IT operations analytics

1.1 APM and DevOps

The DevOps movement has been born out of the increasing conflict between IT developmentand operations teams as a result of businesses’ need to deliver more innovation to theircustomers via applications and infrastructures of ever-increasing complexity. Wheredevelopment are all about change, operations are driven to control the stability of theenvironments they manage and deliver the highest possible uptime to the business - andchange puts all of that at risk.

APM helps development and operations work better together, collaborate more effectively,as it enables rapid problem analysis allowing the teams to quickly understand where theissue is with the software released and resolve it. Many organisations have typically set up‘war-rooms’ when there is a system outage where the team responsible for the application(developers, system administrators, DBAs, architects, security, testing etc) have beencaught in a cycle of denial and blame whilst they manically try and identify what broke. Theonly truly important thing is to have the system running properly again in the shortestpossible timeframe, but without effective diagnostic tooling this can be very hard to do.APM accelerates the time to recovery so developers can carry on innovating and operationscan achieve system stability again - without wasting time arguing and blaming each otherfor the problem.


http://www.google.com/url?q=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FSoftware&sa=D&sntz=1&usg=AFQjCNHla_7ciibkP7RbaHOAAEj0qRExlQ

http://www.google.com/url?q=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FPerformance_metric&sa=D&sntz=1&usg=AFQjCNERmHt_c7-Cya6yBTOsgS1uCuMV4A


2.0 Why should you do it?

High level benefits from using an APM solution include:

- Reduced MTTR*- Increased uptime- Reduction in lost transactions/business profits- Improved ability to provide proactive support- Less time spent troubleshooting- More time spent innovating and developing- Fix problems before they impact business- Reduced operational running costs (hardware, support etc)- Improved user experience

** The MTTR is the Mean Time to Repair, Resolve or Resolution - each of the definitionsmeans the same and can be used interchangeably

Our customers tell us of scenarios where applications have failed and they first they haveknown of the outage or performance degradation has been when they have been told by acustomer. Not only are they losing money while customers cannot transact, they are losingcredibility, reputation and probably losing their customers to their competition.

All of our customers are under pressure to continually innovate in order to compete andmost businesses we work with are conducting more and more of their business online.



3.0 What you should look for in a vendor

At a high level, APM tools must be selected with the five following strategic end goals inmind:

1. Simple to install and use2. Able to go wide3. Able to go deep4. Able to understand business transactions5. Built for cloud environments

Your APM tool must be able to cope with the change your business constantly experiences.

Remembering from section 1.0, Gartner’s five functional dimensions of APM, let’s exploreeach a little further.

3.1 End-user experience monitoring

This is the capture and presentation of data about how end-to-end application availability,browser, network and server latency, execution correctness and quality appear to the enduser. IT operations should be able to compare end user experience across geographies andbrowser types and advise development accordingly.

We believe business transactions are the only true measure of application performance andyou should look for a tool that provides real-time business transaction monitoring with thecapability to measure and track all transactions:

- Response Time- Load- Error Rate- Slow Rate- Stalls

3.2 Application topology discovery and visualization

You should look for a tool that handles the discovery of the software and hardwareinfrastructure components executed by the application, and the possible ways in which thesecomponents communicate to deliver the application and makes it possible to visualize andmanage entire applications. Look for tools that make it possible to auto-discover and mapapplication tiers and services, and see the relationships between them, thus eliminating



blind spots.

You tool should enable you to follow the flow of your transactions across all tiers andservices (even in a highly distributed environment) so you can identify bottlenecks superfast.

3.3 User-defined transaction profiling

The tracing of user-grouped events, which comprise a transaction as they occur within theapplication as they interact with components discovered in the second dimension(application topology discovery and visualization); this is generated in response to a user'srequest to the application.

3.4 Application component deep dive

Your APM tool of choice should provide fine-grained monitoring of the resources consumedby, and events occurring within, the components discovered in the application topologydiscovery and visualization dimension. These may include both server and client sidecomponents, devices and interfaces.

You should want to be able to identify root cause with complete code diagnostics and havefull visibility into code execution in production. Your APM tool should add very little overheadand require little or no manual configuration.

3.5 IT operations analytics

The burden of overseeing and managing applications has increasingly moved away fromapplication architects and developers. Now, it is IT operations and infrastructureprofessionals who are often shouldering the responsibility for ensuring application health andperformance.

This is particularly true with respect to distributed applications. A fundamental change inthe way that applications are built, deployed, and scaled has caused web applications toshift from being monolithic in nature to many components distributed over multiple tiersand nodes, all connected through service-oriented architectures. This trend has contributedto IT operations and infrastructure professionals being the ones whose are alerted whenapplications fail to meet business SLAs.

You chosen tool should use a number of techniques including event processing, statisticalpattern discovery and recognition, unstructured text indexing, search and inference,



topological analysis, and multidimensional database search and analysis to discovermeaningful and actionable patterns in the typically large datasets generated by the first fourdimensions of APM. It should enable you to detect business impact and performance spikesand your tool should learn "normal" performance behavior and "normal" code execution pathsfor all Business Transactions and Application Services, at all hours, days and weeks of theyear - so anomalies can be detected automatically.

Your tool of choice should ship with pre-built reports for your line of business people,dashboards for the operations teams and have a release comparison feature so that thedevelopment teams can see and understand the impact of their releases.

3.6 Questions to ask

Here are some questions to ask when talking to APM vendors:

1. What is the installation process up front?2. What kind of post-install work is required?3. How does the tool map the application?4. How does it display that mapping?5. Does it keep up with the needs of agile development?6. How much manual instrumentation is needed for complete visibility?7. Does your tool have insight into the entire distributed environment?8. Are there any blind spots in your current architecture?9. If it does map the environment, how does that data come back to you? Is it simple

to view?10. Can your APM tool go deep as well as wide?11. How does it isolate problem areas at the code level? • How much overhead does it

add? Less than 2%?12. How do you define “business transactions”?13. How does your tool represent them?14. How does your tool speak the “business language” of the application?15. Does the tool learn and react from performance baselines dynamically? If not, how

are they set and maintained?16. Is your tool built for a cloud environment “ground up,” or has it been retrofitted?17. Can the tool manage application performance with respect to virtual and clouds

environment? If so, how? • How many nodes can the tool monitor in the cloud?18. How does it adapt to the dynamic nature of the cloud?19. Can it dynamically provision resources across the cloud?20. What cloud-ready environments has it monitored in the past? What are the customer

names?



4.0 Considerations at implementation time

Implementing APM and DevOps requires change - and change can be painful and is oftenresisted. Here are some pointers on how to make your APM implementation project gosmoothly.

4.1 Cultural Change

You’ll probably find that the stakeholders in IT and business are delighted to hear that theywill have reports available on system performance and you already have a number ofevangelists, however, the key to perceived success is often in expectation, so werecommend you have a clear, documented set of requirements upfront from the teamdescribing what they want to see in the reports and how they want them delivered.

It’s a good idea at this point to have a view on your maturity level when it comes to APM -how are you finding out there’s a problem with your apps? Are your customers telling you orare you getting alerts from other tooling? What is the other tooling (make an inventory andclassify the existing tools (monitoring for database, network etc) and how many alerts areyou getting? How easy is it to interpret them? Who is currently using them and how? Whatdo you like / not like about them? Perform a gap analysis based on the capabilities describedin section 3.

4.2 Process Change

It’s unlikely your project’s going to get off the ground without a solid business case so findand identify one or two (probably business critical) applications that are being significantlyimpacted by outages / performance issues - if you can calculate the revenue/profits lostduring downtime all’s the better.

Additionally, you will, we hope, have baselined a number of quantifiable key metrics (MTTR,volume/severity of outages, number of people/time spent troubleshooting etc) and as partof your business case and project plan have some stated goals to achieve. Some of thesemay directly impact your SLAs and require a change in process - certain alerts for examplemay mean that your developers receive the pager call at 2am to tell them a severity oneoutage has occurred that requires their immediate attention. Everyone needs to be ready torespond to what your APM system can tell your organization.

Here are some questions to ask yourself as you prepare and deploy your APM solution:

● Have you defined the alerts’ routing to the people defined in your support and recovery process?



● Has everyone who will be receiving or has visibility of the new alerts been made aware of what they are and how to interpret them?

● If someone hears about your new APM solution and wants to receive alerts from it, how do they request this? What is your process for servicing this request?

● Is there a process to swiftly on-board a new application?● Is the APM tool integrated with other corporate systems?

4.2.1 Educate and Evangelise

Your vendor may provide training materials, train-the-trainer or onsite/online training courses on your new APM tool. You’ll need training material for:

- Basic usage- Advanced concepts (memory leaks, policies, dashboard creation, etc)- Operations (alerts/events) training- Reports interpretation for line-of-business and development

You need to make sure the people who will touch the product or consume the data have the information they need to be successful. Their success drives your success - you are all in this together.

Evangelise your new tool - broadcast your success and, wherever possible, quote quantified metrics (a percentage improvement in uptime, the new average MTTR etc). Your vendor should be able to and want to work with you to monitor, record and socialise the success of the tool. They may even ask you to be a reference - supporting the tools you love helps ensure their longevity and is great for your career development.

For every problem you solve with your new APM tool take a few minutes to document the success and make it available to the team and stakeholders. Collect the following information:

● Problem Description: Which application, what happened, what was the impact?● Resolution: What was the root cause and how was it resolved? (Include screenshots)● Business Impact: What was the time to resolution and how did that compare to the

baselined MTTR? What was the quantified business impact and how did that compare to life before APM?

4.3 SAAS or On-Premise

One of the first decisions you’ll need to make at implementation (and you’ll more than likelyhave already performed a POT/POC so may already have discussed this at some length with



your chosen vendor) is whether to host your own controller or use the vendor controlledSAAS. It’s easier (less work for you) to go the SAAS route since the vendor provides andmanages the platform but you may have considerations and requirements (around securityin particular) that mean you would rather run the infrastructure yourself. Check your serverprocurement and set-up processes early if this is the case to avoid your project slowingdown. Once the APM solution is up and running it’s time to install the agents.

4.4 Verification

After you’ve deployed your agents (whether straight into production or advancing throughyour route-to-live) and you have started used the monitored application, you’ll want to lookat the user interface to see if the information contained within looks correct.

● Look at your application flow map and try to identify any missing applicationcomponents

● Check the business transactions - are all the expected transactions there andreporting metrics?

● Are your end user experience metrics displaying?● Do you have transaction snapshots showing your custom code executing in the run

time?● Send out test alerts to see if they make it to their destination

If things don’t look right you’ll need to work out why. Maybe your application is differentthan you had initially conceived, or perhaps there a problem with the monitoring. Resolveany issues you see before declaring deployment and configuration victory.

NOTE: Production Load Cannot Be Simulated Exactly

To realize the most value from your APM purchase you must run it in production. No matter how good your Quality Engineering team is they cannot possibly code all of the weird and wonderful things your users will try to do in production. It can also be very difficult to duplicate your application environment in production. Example, you have 5000 JVMs spread across multiple cloud provider data centers. Replicating that environment would be time consuming and really expensive.

4.5 Discovery and Configuration

If you’ve picked the right tool, this bit should be easy.

Your APM tool should not require more than a series of simple steps in order to be installedand should be able to perform auto-discovery. It should be up, running, and instrumenting



the distributed application within hours or even minutes. The process looks something likethis: the end user installs the agents on all managed servers and virtual machines, installsthe controller, and re-starts the application. At that point, the tool itself should be able tohandle the challenge of mapping all the databases, tiers, and nodes within the distributedapplication, as well as displaying those relationships in an intuitive and visual way.

4.6 Alerts

Do you like alerts? Not many people do. Why? Because it’s difficult to set meaningful onesand ones that preempt real system issues. When you’re setting your alerts, set your mind tothinking about business impact - not IT infrastructure resource utilisation.

The first question to ask is:

“How can alerting be done the right way without spending more time and money than itcosts to develop and run my applications?”

Your most important and actionable alerts should be based off metrics that are directly associated with business impact. Here are some examples:

● End user response time (good indicator of regional issues)● Business transaction response time (good indicator of systemic issues)● Business transaction throughput rate (are we seeing the same amount of traffic as

usual?)● Number of widgets sold (is there a problem preventing users from buying?)

And remember, you might not always get all of this right first time and what’s right today might not be right tomorrow. But your APM tool, used properly, will enable you to deliver higher quality innovation, faster.



5.0 Ranger4 Recommendations

At Ranger4 we recommend AppDynamics for APM. In Gartner’s 2013 Magic Quadrant(released December 2013) AppDynamics was recognised as a leading vendor in this marketwith the following strengths highlighted:

- AppDynamics delivers analytics-driven root cause analysis- Corrective actions can be applied through run book automation assuring high availability- AppDynamics has capability across all 5 dimensions and DB monitoring too- A powerful visual user interface

We also like Splunk for additional log-file analysis and for use with Big Data to makevalue-based business transformation decisions.

We can make recommendations for you based on what we learn about your profile and workthrough a proof of technology or concept with you. We are also well-practised in writingbusiness cases with our customers wherever they start on the maturity scale and measuringprogress through and beyond the project.

We often find that APM is one solution of several that will help an organisation benefit froma full-blown DevOps implementation and have developed our DevOps Maturity Assessmentwhich baselines and scores an organization’s current capabilities on our DevOps MaturityIndex and provides a fit-assessed roadmap to a desired future state.

To find out more, please visit www.ranger4.com.



a ranger4 guide to application performance management

Technology