Issue 2

Achieving autonomous IT operations with HPE Operations Bridge

2

Achieving autonomous IT operations with HPE Operations Bridge

2Achieving autonomous IT operations with HPE Operations Bridge

7 Research from Gartner: Align Your Next-Generation Monitoring Tools With Future Infrastructure and Application Architectures

10 About Hewlett Packard Enterprise

Salvador Dali and IT

Is anyone driving an autonomous car yet? Probably not, and we’ll doubtless refrain for quite a while. But think back, remember how lots of us wouldn’t adopt automatic gearboxes, now so prolific?

Are they just a luxury or more practical than we think?

Like it or not we need to change and adapt to the business environment, much of which is modified by the force of time at least.

Add to that technology refreshes in infrastructure with software defined datacenters and hyper-converged resources, you’d be forgiven for seeing a Dali type picture of the IT Landscape.

“People don’t like change. But make the change fast enough and you go from one type of normal to another”Sir Terry Pratchett

Achieving autonomous IT operations with HPE Operations Bridge is published by HPE Software. Editorial supplied by HPE Software is independent of Gartner analysis. All Gartner research is © 2016 by Gartner, Inc. All rights reserved. All Gartner materials are used with Gartner’s permission. The use or publication of Gartner research does not indicate Gartner’s endorsement of HPE Software’s products and/or strategies. Reproduction or distribution of this publication in any form without prior written permission is forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Gartner shall have no liability for errors, omissions or inadequacies in the information contained herein or for interpretations thereof. The opinions expressed herein are subject to change without notice. Although Gartner research may include a discussion of related legal issues, Gartner does not provide legal advice or services and its research should not be construed or used as such. Gartner is a public company, and its shareholders may include firms and funds that have financial interests in entities covered in Gartner research. Gartner’s Board of Directors may include senior managers of these firms or funds. Gartner research is produced independently by its research organization without input or influence from these firms, funds or their managers. For further information on the independence and integrity of Gartner research, see “Guiding Principles on Independence and Objectivity” on its website.

Image courtesy of http://my-photogalore.blogspot.fr/2008/12/salvador-dali-paintings_13.html

3

As Gartner indicates in this newsletter, IT must change too. As digital transformation means modern applications, those pesky new technologies that developers use defines a diversity of modern business services designs and new manageability challenges.

Next generation…what?

As if the challenges of managing IT weren’t already enough in such a highly competitive environment, change is rife. One such change comes from the need to be a more strategic partner to executives, serving up the agility that businesses need to stay competitive as well as serving users who are now used to “always on” everything, IT operations are faced with multiple dilemmas, spanning Big Data, new technologies, doing more with less, all add to the need to transition to a service mature organization and exploit cloud securely and cost effectively.

Challenges indeed, but some of these represent strong opportunities too.

• As business applications adopt containerization to isolate and secure application logic and customer data, so too monitoring tools must adopt such architectures for scalability and flexibility in deployment

• As business applications adopt Big data, so too should monitoring tools

• As the Internet of Things connects us all to each other and devices, so too monitoring tools must exploit that connectivity and the data we can thus access

• As statistics and the analytics that generate them become key to measuring progress in business, monitoring tools must embed analytics

• As IT is now the business, monitoring capabilities must expose what’s happening as it happens to all stakeholders. Yesterday’s news is too late, stakeholders need to be empowered to see and make decisions faster

Sounds almost like a specification, but also common sense. These define next generation monitoring and manageability.

Whilst those changes probably define the evolving classical monitoring methods and tools, it’s also true that you don’t have more people with more skills waiting in the wings to simply evolve.

Doing more with the same or fewer people is going to require powerful techniques and tools that offer automation, self and machine learning leading to autonomous operations which free IT from repetitive tasks and exploits analytics at the speed of business.

HPE Operations Bridge monitoring tools from HPE Software are introduced now with all of the above in their roots to sense, analyze and adapt and address IT challenges and change.

Let’s have a look how this is offered.

Sensing the state of IT

Sensing isn’t just monitoring.

It’s about continuous discovery of objects to be managed and the dependencies between them, and using that data to perform analysis of events to determine root cause.

HPE Operations Bridge performs these aspects of sensing across all of Hybrid IT i.e. traditional, virtual and cloud.

It is designed to manage HPE system collectors including end user experience monitoring, agents and agentless monitoring tools, as well as integrating IT data from 3rd party management tools, which many people already have such as Nagios, Microsoft SCOM, and many others using connectors developed and supported by HPE Software as well from our partners. Please see our integrations ebook for more details.

Big Data – headache or opportunity

No-one knows ahead of issues exactly which information might be the most pertinent data needed to solve issues.

Whatever your current toolset, it’s clear the sheer quantity of information fed by a proliferation of sensors in devices, smartphones, industrial systems, our cars, literally everywhere. All these are sources of information that can provide vital insight.

The Internet of things defines this proliferation of connected sources of information, and its users have higher than ever expectations of business services.

Thus the data generated creates both a headache but also an opportunity for IT to have the information needed at hand and use it to at least handle issues in some case even before any impact is felt.

Autonomous Operations through a business lens

Source: HPE Software

4

The HPE Operations Bridge solution exploits HPE Vertica, an industry leading Big Data information management and analytics platform, coupled with our own patented topology based on event correlation and predictive analytics. This matches industry expert advice in AIOps and ITOA recommendations.

Monitoring Cloud service delivery

When cloud services are included in modern architectures, one might think that this negates the need to monitor. But imagine that you are the provider of the service, and try to derive differentiation, revenue and satisfaction from it. In that case any issue which might impact those goals should really be detected and resolved. But if that is due to somebody else’s datacenter, then how will you know?

Similarly, if you are consuming somebody else’s service, can you rely on them to inform when things change, when performance degrades? How will you determine if that’s due to your datacenter, your provider or connectivity issues?

The moment anything like that occurs, service delivery will be impacted.

Flying blind, not knowing means accepting the risk, unknown risk probably.

Applying appropriate monitoring assisted by automation can solve this issue, and provide service assurance.

Executing this automatically is what the HPE Operations Bridge does. Automated discovery and monitoring of your services in public and private cloud then allows algorithmic intelligence to be applied to that data, determine proactively when anomalies are occurring, alerting operations to them and potentially executing remediation that respects standard IT processes, goes a long way to providing that service assurance.

Docker based architecture designs are covered as well, with monitoring integration with the Docker hosts, OpsBridge discovers the application in the container and extracts vital information e.g. when containers repetitively fall over, are starved of resources or connectivity.

Automated Intelligence focuses IT operations on the opportunity

As we have just discussed, information storms can swamp tools and operators with mind boggling volumes of events and performance information. It’s easy then to expect that compute power be used to good effect, applying intelligent algorithms to that data at speeds faster than any human mind ever could, and learn over time to adapt and predict. Algorithms driving analytics can be simple such as intelligent and adaptive filtering to handle information storms,

Source: Gartner, G00296333, Will Cappelli, August 2016

“Utilize big data and machine learning technologies to achieve a data-centric approach to availability and performance monitoring”

HPE Operations Bridge manages AWS, Docker and Azure

Source: HPE Software

5

determine which information pertains to true root causes versus symptomatic issues. Other algorithms facilitate machine learning to accelerate finding the needle in the haystack, guiding people through the maze of data orders of magnitude faster than ever before and with far superior visibility along the way.

Analysis coupled with business context, such as topological analysis, is the right focus. Even when a root cause is identified, what will you do if several issues are occurring? Service impact analysis and visibility is proven to define prioritization. IT Operations need the means to focus on the most business impactful incidents that analysis isolates and then allocate for action accordingly.

Also as DevOps teams choose from an ever growing catalogue of technologies, and often mobile applications, to support business processes, and as IT choices steer towards Hybrid IT architectures to support them, then private and public cloud, containers and integration techniques abound to render the implementation landscape more complex. IT management solutions must cover all these different architectural choices, and exploit monitoring data generated by them all.

Autonomous IT Operations

Gartner indicates that modern solutions must apply automation to discover and monitor services (applications and the infrastructure that supports them) deployed across Hybrid IT.

Built into the HPE Operations Bridge is an IT service tree built from monitored data from HPE agent based and agentless native collectors as well as data imported from integrated 3rd party products. This process is continuous, meaning it maintains a dynamically updated true picture of the IT landscape.

Associated with the intelligent filtering to isolate the most pertinent events, automated topological analytics is then used to identify which constitute root causes and show operations which are most likely to perturb vital services.

Having the means to identify is one thing, but what happens when issues occur whilst IT is occupied or unavailable?

Best practice remediation can be automated, and the OpsBridge is equipped with the industry leading Operations orchestration with over 8000+ predefined workflows to choose from and customization at hand. Thus closed loop execution of corrective action and execution of well-defined and compliant processes are feasible. Examples include changes to allocate extra capacity, re-route transactions, extend a database whilst processes would typically include opening of ITSM tickets in service management tools, and generation of audit reports.

These artefacts contribute to relieving IT Operations of repetitive tasks, ensure consistency and eliminate human errors. The consistency also facilitates compliance. Artefacts such as monitoring automation, automated discovery, automated analytics and remediation, have provided high levels of autonomous Operations at Swiss Mobiliar and many other customers using the HPE Operations Bridge.

Putting the “O” in DevOps

As for Swiss Mobiliar and others, DevOps is here to perturb well known processes and create ways for those new technologies to provide good use for agile competitive differentiation. But how does

Operations Bridge help with DevOps teams?

Monitoring tools hold a great deal of information that can be put to great use. Monitored data characterizes the behavior of applications and infrastructure and resulting analytics of it can be like gold to help developers and testers to refine and release better versions.

But how do they get access to information?

HPE Operations Bridge provides several ways to assist them;

1. “CNN” for IT

Firstly a revolutionary easy way to access this data on popular tablets and other devices.

A famous Chinese proverb says ““Tell me and I’ll forget; show me and I may remember; involve me and I’ll understand.”

Show me! Something the business has been asking IT for a very long time. And the truth is, when they can see what’s happening, when IT shares this as it happens, the chances are a great deal of collaboration can result.

Our Business Value Dashboard is meant to facilitate exactly that. It shows what’s happening as it happens, in quick and easy formats they can access securely, digest quickly and IT can provide in a jiffy.

“Autonomous operations are important to save money and to be faster and we can concentrate more on the real tough issues“

Thomas Baumann, Swiss Mobiliar - Watch the video

6

A famous Chinese proverb says “Tell me and I’ll forget; show me and I may remember; involve me and I’ll understand.”Source: HPE Software

2. ChatOps

Secondly, DevOps teams can gain access to that valuable information without needing any account on the HPE Operations Bridge platform using ChatOps.

Robotized functions provide interactive access to information using popular messaging.

In cases where people collaborate to resolve issues, the dialogue and action taken is thus captured and new remediation applied.

See how you can get autonomous operations through a business lens

Source: HPE Software

ChatOps in action (slack was used in this example but is not imposed)

Source: HPE Software

See a demo of ChatOps

Source: HPE Software

Next generation… now

HPE Operations Bridge has been providing very strong value for years to IT in companies across financial services, telecommunication, healthcare, logistics and life sciences.

But just like their business, we don’t stand still. “HPE Operations Bridge goes modular” could be the headline as we apply the very technologies we have discussed, the technologies also recognized by Gartner in this newsletter.

We know the challenges you face with modern business transformation. We scale whether you are a large multi-national enterprise or smaller. We adapt to change the face of your business, such as the internet of things.

Some of our customers are applying the distributed monitoring capabilities coupled with analytics and real time dashboards, to satisfy railway regulations. They use the real time monitoring capabilities to monitor and govern locomotive movement, providing a safer transportation experience.

Others monitor Wind farms, where sensors galore generate information by the boat load, all useful in determining reliability and facilitating transmission of ecological power.

Others consider how transmission towers and transmission equipment can impact the user experience, and apply analytics to proactively manage anomalies before any user is impacted, any service degrade.

How can we help you?

To read more details see our new ebook

See our webinar that explains “5 New Ways to Focus Operations through a Business Lens with Bridge Suite”

www.hpe.com/software/opsbridge

7

Align Your Next-Generation Monitoring Tools With Future Infrastructure and Application Architectures

Research from Gartner

The architectures governing business applications are being increasingly structured around microservices, NoSQL databases and streaming analytics. Infrastructure and operations leaders need to implement monitoring systems that effectively manage new-generation infrastructures and application stacks.

Key Challenges

• The move to “monitoring as a service” will result in increased and nondeterministic demand for monitoring data among IT stakeholders.

• The volume and complexity of the datasets required to understand and modify the behavior of systems built according to modern architectural principles require a set of skills beyond those possessed by typical IT operations practitioners.

• Many IT monitoring tools in production were not designed for the dynamic nature associated with cloud-native architectures.

• Migrating to new monitoring architectures without causing subsequent monitoring service disruptions is rare; hence, it is rarely performed.

Recommendations

Infrastructure and operations leaders should:

• Ensure the scalability and resiliency of their monitoring architectures by buying or building systems out of modular components.

• Require automated service discovery to be built into their monitoring architectures, or require the ability to interface with a directory service.

• Use a four-layer process to access monitoring data: flexible querying and visualization for data search and aggregation; statistical analysis of metrics embedded in the data; machine-assisted discovery of correlations among data items; and, finally, machine-assisted extraction of causal paths from a network of correlations.

• Weight heavily the ability to incrementally deploy the monitoring technology as part of their tool selection criteria.

Introduction

If infrastructure and operations (I&O) leaders are to build an infrastructure that is increasingly scalable and dynamic to meet the needs of a digital business, then they must ensure that the IT organization’s monitoring tools are able to keep pace. In essence, there is a new “stack” of technologies being adopted “under the covers” of many of today’s modern monitoring tools (see Figure 1).

Figure 1 is a sampling of some of the technologies that we see being adopted in segments of the monitoring arena. Traditional enterprises may not need the capabilities of a Facebook or a Google, which can monitor millions of instances and collect gigabytes of data per second; however, the increasing availability and performance needs of future digital businesses could mandate entirely new monitoring architectures to remain competitive.

Provided here are some of the key capabilities needed for the modern monitoring needs of today’s digital businesses. Almost every major cloud or web-scale service provider has made multiple iterations on its monitoring toolset, so enterprises should not expect to move to this type of architecture overnight. Adding NoSQL data management and more-sophisticated analytical capabilities is usually a good place to start,

Source: Gartner (September 2016)

FIGURE 1Technologies That Are Part of Modern Monitoring Architectures

8

because the technology required can often be deployed with only minor adjustments to monitoring functionality and products already in place.

Avoid solutions that promise to do everything via a monolithic platform (even if that platform is built from modular components). Proceed incrementally and nondisruptively, and continuously assess where monitoring improvements can be made. Select only tools that enable future migrations in your monitoring architecture.

Analysis

Ensure the Scalability and Resiliency of Your Monitoring Architecture by Buying/Building Systems From Modular Components

Many traditional and non-SaaS monitoring systems are monolithic — for example, the data collection (checks), aggregation and even computational functions (e.g., alerting and data storage) are executed within the same core component. This means a significant percentage of the available CPU cycles are consumed by context switching. Although upgrading hardware (vertical scaling) may temporarily alleviate the potential performance issue, this remains a short-term fix, due to:

• Unpredictable data consumption patterns by an increasing number of end users. More consumers of monitoring data results in more metrics being collected. Making matters worse, the data collection process is likely to become increasingly unpredictable, as these new metric consumers “experiment” with their data granularity and frequency needs.

• The ever-increasing numbers of code releases that result from agile-oriented and continuous-delivery processes could create metric traffic storms that might overwhelm less-resilient areas of the monitoring architecture.

Recommendations:

• Build or deploy monitoring architectures that support the decomposition of major functions (see Figure 2). Modules need to be simple and connected via well-defined interfaces. This enables optimization only where needed, without requiring a complete implementation forklift upgrade and disruptions to your monitoring activities.

• Replicate components at a geographic (or some other) level, where needed, to reduce overall data volumes and better distribute processing using aggregated roll-ups of localized data. This autonomous or semiautonomous approach will improve overall monitoring system availability.

Ensure That Automated Service Discovery Is Within Your Monitoring Architecture or You Can Interface With a Directory ServiceService discovery remains a continuing challenge, even for many of the major web-scale service providers. This is because of the related issues of time and space:

• The time aspect refers to the increasing ability to rapidly provision and deprovision resources whether it is in the cloud, or even on-premises, using technologies such as containers and

Source: Gartner (September 2016)

FIGURE 1An Example of a Modular Monitoring Architecture

9

microservices. Microservices are often part of so-called immutable infrastructures, which can exacerbate the discovery problem, because the virtual machines (VMs) or the containers in which they are enveloped are created and destroyed more often than normal. This is because the focus with immutability is on achieving the desired state via replacement, rather than repair.

• The space-related issue is the targeted domain of managed objects. Most enterprises don’t need to manage the millions of objects that might reside in a web-scale cloud company. However, the number will certainly continue to arise with the advent of microservices and architectural styles made possible by the availability of web-scale cloud services. During the next few years, Gartner expects to regularly see enterprise monitoring systems that need to manage hundreds of thousands of objects across a landscape of private and public cloud implementations.

Recommendations:

• Require that your monitoring solution interface with key infrastructure orchestration APIs. Be mindful of the growing significance of open-source stacks in this domain; so, in addition to supporting platforms that you already have in place, support interfaces for technologies such as Kubernetes and Mesos (Marathon).

• Use a monitoring solution that supports external systems used for configuration monitoring and discovery. Be ready to support open-source technologies, such as Apache ZooKeeper and even Netflix OSS Eureka (which is supported by Pivotal Spring Cloud), as well as systems supported by your enterprise.

Use a Four-Layer Process to Access Monitoring DataTraditional monitoring technologies have attempted to focus on incorporating more visually attractive and intuitive interfaces to support a wide range of monitoring team needs and skills. Although this aligns well with the increasing “consumerization” of IT, it creates a new set of challenges:

• Commercial graphical user interfaces (GUIs) can inhibit more-sophisticated use case scenarios in which there’s demand for flexible, command-line and query-driven interfaces that enable sophisticated data processing and manipulation (such as creating quantiles or buckets of metrics for rapid data inspection).

• Metric naming in many common enterprise toolsets involves long labels or names that impede flexible filtering and searching.

Recommendations:

Build or buy a monitoring system that supports four levels of data access and analysis:

• A rich querying language enhanced by nonsymbolic visualization capabilities to pinpoint specific data items in complex datasets, while suggesting large-scale patterns

• Statistical analytics for the metrics embedded in data items to surface times series and other numerically based structures

• Machine-assisted discovery correlations among data items — numerical, textual, semantic and graphical — to support the surfacing of correlations that are too deeply hidden or complex for human practitioners

• Machine-assisted extraction of causal paths from among the correlations that have been previously established — only with this last level can genuinely action-oriented relationships among data items be determined

Your monitoring system must apply these four layers of access and analysis to streaming data as it is being generated by the applications and infrastructures being managed, as well as to large accumulations of historical data. This will almost always require a NoSQL database management platform.

Weight Heavily the Ability to Incrementally Deploy the Monitoring Technology as Part of the Tool Selection CriteriaMany enterprise-monitoring architectures remain static, due to the normally huge costs incurred when a migration to a new platform is undertaken. There is also the potentially additive issue of losing telemetry in the cutover process. These problems arise because, in traditional monitoring architectures:

• Client libraries (agents) running on target systems can report only to their own upstream aggregators and/or controllers.

• Integrations with other (competitive) monitoring systems are often not a priority in many commercial monitoring technology providers, because the goal of these firms is to be the single “source of truth” within the context of any other monitoring and telemetry tools in an enterprise.

Recommendations:

• Buy and deploy only those tools that have client libraries (agents running on a server) that are able to write to any aggregator or console for output to reduce rigid ties between agents and managers.

• Initially, add the new capabilities as increments to existing systems and plan to displace those systems gradually.

Source: Gartner Research Note G00303196, Cameron Haight, Will Cappelli,

28 September 2016

10

Hewlett Packard Enterprise is an industry leading technology company that enables customers to go further, faster. With the industry’s most comprehensive portfolio, spanning the cloud to the data center to workplace applications, our technology and services help customers around the world make IT more efficient, more productive and more secure.

For HPE Operations Bridge see hpe.com/software/opsbridge.

For HPE Business Value Dashboard see hpe.com/software/bvd.

For HPE Operations Analytics see hpe.com/software/opsanalytics.

For our ITOM management solutions please see hpe.com/software/itom.

See also our Blog channel here.

About Hewlett Packard Enterprise