hybrid cloud usage poses new challenges ......it experts noted there are numerous challenges to...

March2018

HYBRID CLOUD USAGE POSES NEW CHALLENGES FOR MONITORING SOLUTIONSA Global Survey of Cloud Professionals

Sponsored by

© 2018 Dimensional Research.All Rights Reserved. www.dimensionalresearch.com

Dimensional Research | March 2018

IntroductionA total of 502 cloud professionals completed a global survey on managing and monitoring hybrid IT environments to gather hard data on current adoption rates, and operational and monitoring challenges. The research focused on application data sharing trends, hybrid monitoring tool capabilities and gaps. The research also examined whether automation driven by machine learning was an applicable approach to managing the dynamic hybrid IT environment.

Executive SummaryThis research finds that more than 9 out of 10 companies already use multiple public cloud vendors. IT cloud professionals surveyed indicated that over the next 12 months the use of all cloud solutions types - IaaS, PaaS, and SaaS - will increase dramatically. More than 8 out of 10 companies share data between public cloud solutions and back to on-premises applications. This complexity led a majority of those surveyed to state that resolving cloud-based application issues is more difficult and slower than remedying on-premise app issues, which impacts the business.

IT experts noted there are numerous challenges to monitoring a hybrid IT environment such as security, real-time topology, and access to cloud infrastructure. Participants indicated a comprehensive monitoring solution is required to properly operate a hybrid IT environment. Nearly all participants want a sophisticated monitoring solution enabled with machine learning that automates responses and repairs to complex hybrid environments.

Key findings• Cloud Adoption Trends

- 92% use multiple cloud vendors - 88% of cloud-based apps share data & services with on-premise apps - 64% indicate cloud-based apps share data and services with other cloud-based apps

• Cloud Drives New Challenges - 96% indicate public cloud app issues are resolved too slowly, impacting the business - 64% state resolving performance issues and failures is more difficult for cloud-based apps

• New Cloud Solutions Needed - 62% of companies expect improved performance with hybrid IT monitoring - 98% would utilize automatic actions determined by machine learning

Sponsored by



www.dimensionalresearch.com© 2018 Dimensional Research.

All Rights Reserved. DAM9494


Detailed FindingsHybrid IT is the Standard IT Environment This research validated that 92% of respondents will be increasing their use of the cloud over the next 12 months, but more importantly this survey also discovered that the same percentage (92%) were already using multiple cloud vendors today. In fact, nearly 3 out 10 companies are using public cloud infrastructure from 6 or more providers. This finding indicates that hybrid IT environments utilizing multiple public and private clouds along with traditional infrastructure is already the new ‘normal’ for IT.

This diagram illustrates the possible data flows between applications running on-premises, in public clouds and with SaaS based solution. Companies may share no data externally while others could have data sharing with all infrastructure options.

8%

63%

17%12%

0%

10%20%

30%

40%

50%60%

70%

1 2 - 5 6 - 10 More than 10

Approximately, how many different public cloud vendors does your company use?

Public Cloud(s)PaaS, IaaS

On-PremisePrivate Cloud(s)

Traditional Infrastructure

SaaS Offerings

Data Flow

Data Flow Data Flow





Data is Flowing Between Cloud Apps and On-premise Services Just a few years ago security and data control were key obstacles slowing companies from adopting the cloud as the primary infrastructure for business applications. Today 88% of participants report not only that they have cloud-based applications, but that they are actively sharing data back to applications and services running on premise. This practice of data sharing places additional burdens on end to end visibility to ensure performance and data security as well as meet compliance requirements.

Majority of Public Cloud Applications Shared Data with SaaS-Based Solutions This research finds there is a growing data-sharing trend indicating a comfort and confidence in controlling and securing data, as 64% of those surveyed are sharing data with 3rd party SaaS-based applications and services. This has often been a concern as companies do not control the security or infrastructure for SaaS solutions. This continues to put a premium on application and infrastructure visibility to monitor that data is flowing to maintain application performance and reliability SLAs.

Yes 88%

No 12%

Do most of your company's applications running in IaaS/PaaS environments share data or services

with on-premesis applications and services?

Yes 64%

No 36%

Do a majority of your company's applications running in IaaS/PaaS environments share data or services with

other 3rd party cloud-based apps or services?

Micro Focus Operations Bridge Survey Research Report

Micro Focus Operations Bridge is the industry’s first containerized IT operations monitoring product. It monitors your hybrid IT infrastructure and automatically reduces operations data noise.

By proactively detecting problems before they have a major impact, automatically executing remediation, and improving collaboration across tool silos it maximizes uptime.

Operations Bridge supports the top public clouds i.e. AWS, Google and Azure along with private clouds based on VMware, OpenStack, Docker containers and Azure Stack, and traditional on-premise systems.

Customers report 70x faster root cause detection, 50% improved MTTR and a reduction in overall cost of IT operations.

For more information, visit - https://www.microfocus.com/opsbridge





Cloud-Based Applications are More Difficult to ResolveThis rapid adoption of a hybrid IT environment accepts the reality that much of the infrastructure is located elsewhere with limited access and control, and in the case of SaaS offerings, no control at all. This can create serious challenges in having all the information and access necessary to resolve application problems and performance issues. This research finds that 64% of IT professionals stated it is more difficult to resolved issues with applications located in the public cloud. These increased resolution challenges typically create delays to remediation and ultimately impact the business.

Cloud-Based Application Issues Often Cost Companies MoreNearly all participants indicated that public cloud issues directly impact the business. But the specific responses cloud professionals selected illustrate that resolving public-cloud-based application issues involves extra work such as additional time to schedule and coordinate remediation with employees from the cloud vendor. Additionally, participants state there are follow-on tasks after issue resolution, such as determining whether the issue breached SLAs or contractual agreements, followed by negotiating compensation. In some cases, cloud-based application issues required companies to invest additional resources in order to mitigate business loss. Thus, when hybrid IT issues do occur with cloud-based infrastructure the cost of resolution, and consumption of internal resources is higher than traditional applications.

Yes 64%

No 36%

Are resolving public cloud application failures or performance issues more difficult than resolving

on-premise application issues?

4%

41%

57%

61%

65%

0% 10% 20% 30% 40% 50% 60% 70%

There are no business impacts when cloud application issues occur

Utilizing expensive backup (or failover) environments

Prolonged downtime

Extra time required to resolve SLAs and contractual agreements

Resources tied up collaborating with cloud vendor

Which of the following business impacts occur when public cloud applications have failures or performance issues?





Cloud Professionals Have Numerous Challenges Monitoring Hybrid ITGiven the rapid progression to Hybrid IT and additional obstacles in resolving cloud-based application issues, this survey finds IT professional are further challenged with inadequate tools. When asked what their top monitoring challenges are for monitoring a hybrid environment nearly four answers tied at the top with the 5th answer just a few percentage points behind. This trend indicates a systemic problem and lack of necessary capabilities with monitoring solutions used today. The key challenge is the inability to get a comprehensive view (topology) of the entire system supporting an application. This is further inhibited by lack of access to cloud infrastructure as well as traditional infrastructure monitoring failing to supply needed information in a Hybrid IT environment. Lastly, security issues took two of the top 5 issues indicating that security remains a priority along with performance and availability.

2%

30%

31%

33%

38%

41%

41%

42%

42%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

There are no challenges for monitoring Hybrid IT applications and infrastructure

Inadequate public cloud cost metrics

Inadequate metrics from cloud providers

Lack of real-time metrics

Monitoring introduces additional security risks

Traditional monitoring agents don’t work in public-cloud

Inadequate security monitoring capabilities

Limited access to cloud infrastructure

Diff iculty of discovering and creating topology for hybrid IT (services that cross cloudand on-premises)

Which of the following are the top challenges for monitoring hybrid IT applications and infrastructure?





Hybrid IT Requires Comprehensive MonitoringThe research drilled down to ask participants what types of monitoring were truly needed for a hybrid environment. Supporting the foregoing report findings, security monitoring (76%) took the top spot, but basic infrastructure monitoring (68%) placed next, showing its key requirement to managing hybrid IT. Cloud professionals selected compliance monitoring (61%) in the third spot followed by end user monitoring (56%). These findings indicate Hybrid IT needs to be secure and compliant with visibility into the infrastructure fabric, but the end user requires a positive experience for the applications to meet business objectives.

Hybrid IT Is Complex and Requires Enterprise Class MonitoringThe scale and diversity of Hybrid IT can be astounding and given the numerous areas that must be monitored, this research sought to determine the attributes a monitoring solution must possess to be capable of handling a hybrid environment. Those managing hybrid IT systems stated high availability was the top requirement, but in second place was the need for automation, providing insight that IT operators need assistance to keep the complex environment functioning. The public cloud portion of a hybrid IT environment often prevents clients’ specific monitoring and thus the requirement to ingest data from other sources and systems (45%) was third on the list The vast number of devices in the infrastructure and amount of data that can be generated on a real time basis drove

1%

41%

44%

50%

56%

61%

68%

76%

0% 10% 20% 30% 40% 50% 60% 70% 80%

None of these are needed for monitoring a hybrid IT environment

End to end topology

Business metric monitoring

Cloud costs

End-user experience monitoring

Compliance monitoring

Infrastructure monitoring

Security monitoring

Which of the following are key types of monitoring that are needed for a hybrid IT environment?





the need for elastic scalability (44%), but the potential for huge volumes of data resulted in the need for a big data platform not only to manage and store the data but to process it as well for real-time information. Tied at 41%, cloud professionals want the solution to work with existing applications, hardware and cloud vendors, minimizing the need for customization and extra work, as well as having the ability to integrate and support a DevOps process.

Hybrid IT Monitoring Solutions Should Improve Performance and Reduce Costs Participants were then asked what they thought were the top benefits of a monitoring solution designed for a hybrid IT environment. Perhaps, as expected, improved availability and performance led all answers at 62%, followed by improved efficiencies (59%) and then reduced expenses (53%). However, when examined, the essence of all the answers and their effect on IT and business that cloud professionals shared was that a dedicated hybrid IT monitoring solution will deliver improved performance with reduced costs.

16%

25%

29%

35%

38%

41%

41%

41%

44%

45%

48%

59%

0% 10% 20% 30% 40% 50% 60% 70%

Built-in support for ChatOps

Machine learning

Business value dashboards

Modern user interface

Rapid time to value

Built in support of continuous operations (DevOps processes)

Built-in support for variety of applications, infrastructures and public cloud providers

Big data platform

Elastic scalability

Ability to analyze other types of data

Automation

High availability

What are the most important features for hybrid IT monitoring solutions?

1%

42%

46%

53%

59%

62%

0% 10% 20% 30% 40% 50% 60% 70%

There are no benefits from Hybrid IT monitoring

Reduced mean time to repair

Improve cloud cost management

Reduce operations expenses

Improve efficiencies

Improved availability and performance

What are the most important benefits expected from a hybrid IT monitoring solution?





Automation Driven by Machine Learning WantedThis research further investigated where cloud professionals would see value in automation and specifically where automation derived from machine learning fit. Nearly every participant (98%) indicated there was value from machine learning based automation. From a research perspective, the small percentage range from the top answer, automate load balancing (61%), to the bottom answer, system restart (48%), had a difference of only thirteen percent indicating strong value for machine-learning-driven automated actions.

2%

48%

51%

52%

54%

56%

57%

61%

0% 10% 20% 30% 40% 50% 60% 70%

We would not allow any of these actions to be performed automatically

Automatic system restart for hung systems

Automatically restart stopped application processes

Automatically detect new systems and apply monitoring

Automatically add storage capacity when disk is full

Automatically add compute capacity for overloaded system

Automatically remove access to suspected security threat

Automatically adjust load balancing

Which of the following actions determined dynamically by machine learning would your company be willing to use?





Lack of Knowledge Inhibiting Machine-Learning-Based AutomationGiven the strong desire for machine-learning-driven automation, this research inquired what the top barriers were to adopting this technology in their companies. Lack of expertise (44%) led all answers, closely followed by the perception that no capable solution exists (43%) and then a lack of maturity of monitoring and analytics tools necessary to generate and process the data for machine learning (40%). This finds that the market for machine-learning-driven automation, though still in early days, is not limited by concept, control, budget or acceptance concerns, but by the belief that solutions are still developing and those managing hybrid IT environments need to become more familiar with the topic of machine-learning-driven automation.

ConclusionThis research concludes that hybrid IT is no longer a future approach for IT as nearly all companies are using it today and hybrid IT use will continue to grow. In fact, a majority of companies are already sharing data and services between applications and across cloud environments; on-premise applications, and 3rd-party SaaS providers. But Hybrid IT is complex, with much of the infrastructure located elsewhere and not directly accessible, making it more difficult to resolve application issues when they arrive, and when issues do occur, they often take longer to resolve and consume more resources to fully remediate both the technical and business ramifications.

This report clearly finds that cloud professionals are challenged today to properly monitor and maintain a hybrid environment and are in need of a solution that can manage the size and complexity of and the dynamics of changing hybrid IT environments. Traditional IT infrastructure monitoring is failing to provide needed visibility. Those operating a hybrid IT environment need a solution designed to work in that environment and able to provide visibility to in-house and 3rd-party infrastructure and the applications that run across them, which requires the ability to accept data and metric feeds from external vendors.

Monitoring is not just needed for application performance, availability and problem resolution but to supply the visibility to manage and secure data to ensure compliance and minimize risk to the company. The need for a comprehensive monitoring solution for hybrid environments continues the trend documented in a 2016 research

8%

22%

34%

38%

40%

43%

44%

0% 10% 20% 30% 40% 50%

There are no barriers preventing adoption of automated actions

Don’t trust technology (want human involvement)

Regulations limit automated actions

Diff icult to configure and maintain

Immature monitoring and analytics

Lack of solution that combines monitoring, machine learning and automation

Lack of expertise

What's preventing your company from utilizing more automated actions across IT operations?





report1 where 94% stated an integrated hybrid cloud management solution was required. This shows a disturbing trend that while hybrid IT continues to become more complex, IT teams’ lack essential tools to manage it. Those surveyed believe that a monitoring solution designed for a hybrid IT environment will lead to improved performance with reduced costs.

Hybrid IT will continue to grow, driving increasing size and complexity, and utilizing dynamic scaling and provisioning which can quickly make the task of actively managing it daunting. This progression is driving those surveyed to be very accepting of automation driven by machine learning to ensure that Hybrid IT delivers increasing value to the business. It is now that IT professionals need to look for specific hybrid IT monitoring solutions that can take them forward and make their job easier before they become buried by the increasing tasks and fire drills of running a hybrid IT environment.

Survey Methodology A total of 502 cloud professionals from companies worldwide completed the survey; participants had direct management responsibility for clouds solution including both public and private (IaaS, PaaS, SaaS) clouds. All participants were from enterprise companies with more than 5,000 employees. Participants represented each of the five continents and the survey was administered electronically. Participants were offered a token compensation for their participation.

1Research, D. (2016). IT is Flying Blind in Hybrid Cloud Management.

Working with cloud environments is a

substantial part of my job 78%

Working with cloud environments is my

entire job 22%

Cloud Rate

50%

64%

83%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

PaaS

IaaS

SaaS

Clouds Used

5,000 to 10,000 35%

More than 10,000

65%

Company Size

1%

1%

2%

2%

18%

78%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Middle East or Africa

Australia or New Zealand

Asia

Mexico, Central America, or South America

Europe

United States or Canada

Location





About Dimensional ResearchDimensional Research® provides practical marketing research to help technology companies make their customers more successful. Our researchers are experts in the people, processes, and technology of corporate IT and understand how IT organizations operate. We partner with our clients to deliver actionable information that reduces risks, increases customer satisfaction, and grows the business. For more information, visit www.dimensionalresearch.com.

About Micro Focus Micro Focus and HPE Software have joined to become one of the largest pure-play software companies in the world. Bringing together two leaders in the software industry, Micro Focus is uniquely positioned to help customers maximize existing software investments and embrace innovation in a world of hybrid IT—from mainframe to mobile to cloud.

For more information, visit - https://www.microfocus.com

hybrid cloud usage poses new challenges ......it experts noted there are numerous challenges to...

Documents