data-driven it automation: a vision for the modern cio€¦ · it data management researc indstry...

IT & DATA MANAGEMENT RESEARCH,INDUSTRY ANALYSIS & CONSULTING

Data-Driven IT Automation: A Vision for the Modern CIO

By Dennis Nils Drogseth and Dan Twing An ENTERPRISE MANAGEMENT ASSOCIATES® (EMA™) Research Report SummaryApril 2020

Sponsored by:

Table of Contents

©2020 Enterprise Management Associates, Inc. All Rights Reserved. | www.enterprisemanagement.com


Executive Summary ............................................................................................... 1Report Highlights ............................................................................................ 1

Demographics Overview ....................................................................................... 2Qualifiers and Initial Strategic Highlights .............................................................. 4

The Critical Role of IT Analytics ..................................................................... 5The Four Use Cases.......................................................................................... 7

Problem, Incident, and Availability Management Automation Highlights ............ 9Technology Adoptions ................................................................................... 10Effectiveness ................................................................................................... 11

Configuration, Change, and Capacity Management Automation Highlights ...... 12Technology Adoptions ................................................................................... 13Effectiveness ................................................................................................... 14

DevOps Automation Highlights .......................................................................... 15Technology Adoptions ................................................................................... 16Effectiveness ................................................................................................... 17

Application Performance and Workload Optimization Automation Highlights (AppWLO) .......................................................................................................... 18

Technology Adoptions ................................................................................... 19Business Context ........................................................................................... 20Effectiveness ................................................................................................... 21

Cross-Use Case Correlations ................................................................................ 22Unifying Automation More Strategically ............................................................ 24Maturity-Related Insights .................................................................................... 27

Process Considerations and Recommendations Going Forward .................... 28Conclusion: Summing Up .................................................................................. 31

http://www.enterprisemanagement.com



Executive SummaryBased on more than a decade of research, EMA has seen that IT’s adoption of automation in the past was largely ad hoc, an attempt to address specific issues and tasks one at a time. However, more recent research is indicating that this is beginning to change—that it’s finally time to engage IT organizations in a discussion about how to make their automation investments more unified, more integrated, and more strategic.

Seeing these changes in attitude and need became the spark behind doing “Data-Driven IT Automation.” EMA reached out globally in Q1 of 2020 to understand the objectives, issues, technologies, and organizational dimensions surrounding automation adoption in the present and the future. To do this, EMA targeted four key use cases:

1. Problem, incident, and availability management

2. Configuration, change, and capacity management

3. DevOps

4. Application performance and workload optimization

Each use case was examined in depth individually and also compared to the others for commonalities and differences. As a part of this process, for quite possibly the first time in the industry, EMA bridged what had been very separate worlds and very separate technologies across a broad array of IT teams (operations, ITSM, development, etc.) and a long list of technologies that are quite often examined under a separate magnifying glass—for instance, looking at workflow and runbook on the one hand, and workload automation on the other hand, across four common threads.

EMA also examined the degree to which IT organizations were preparing to unify their automation strategies not only within each use case, but

across use cases more broadly. This included its own set of drivers, obstacles, benefits, and technology preferences, which were somewhat different than more use case-specific priorities.

One of the most striking areas of investigation was the handshake between AIOps or advanced IT analytics (AIA) and automation—hence the use of data-driven automation in the title. As this report will show, that linkage is becoming more and more significant and plays strongly to supporting more progressive levels of automation, more unified approaches to automation, and more effective use case outcomes.

Finally, it should be pointed out that EMA has also developed a four-stage automation maturity model from this research. While that is intended as a separate offering, this report will conclude by providing insights into what evolutionary factors most strongly align with automation maturity and effectiveness, with an eye to assist in planning how to implement and optimize a staged approach to automation adoption.

Report Highlights• On average, most respondents had more than 50% of their manual

processes documented.

• IT executive oversight was the more progressive and effective group. For instance, executive oversight led for those who saw unifying automation across use cases as extremely important and well underway.

• Product costs and complexity were major obstacles across all four use cases examined.

• 85% indicated that AIOps or AIA deployments were either a major initiative or well underway.




• More active automation integrations with AIOps/AIA correlated strongly with success, progressive levels of automation adoption, more focus on unified strategies, and other critical maturity factors, such as having an automation center of excellence.

• Slightly more than half of the respondents selected two use cases as leading for them, while most showed involvement at some level in more than one use case, and some in as many as three or four.

• IT operations was overall the most dominant driving group across all four use cases, but the IT executive suite more often led in purchasing decision-making.

• Most use cases indicated support for about five different stakeholder roles, while broader role support correlated consistently with success.

• When asked about automation technologies in play, most use cases also indicated a total of between 4 and 6, with increased numbers of active automation technologies also correlating with success. Workflow within and across IT was most often in a leading position.

• EMA saw very significant correlations between a focus on automation unification across use cases and other parameters for maturity and effectiveness, such as success rates, having an automation center of excellence, a more progressed capability to let analytics drive automation, and more progressive levels of automation across IT.

• AIOps/AIA led as the platform of choice to integrate with automation for enabling more strategic, cross-use case automation effectiveness.

• 50% claimed to have an automation center of excellence.

• Agile/Scrum and regulatory compliance led as the most relevant best practices for automation initiatives.

• 75% indicated that digital transformation was driving their automation initiatives or that the two were tightly coupled.

Demographics OverviewThe research was conducted in March of 2020, with 405 respondents: 200 from North America, 107 from EMEA (England, France, and Germany), and 98 from Asia-Pacific (China, Australia, Japan).

Company size was evenly divided between small, medium, and large, while eliminating companies with fewer than 500 employees:

• 36% between 500 and 2,499 employees

• 35% between 2,500 and 9,999 employees

• 30% with 10,000 or more employees

The five leading industry verticals were:

1. High-technology software provider

2. High-technology service provider

3. Manufacturing

4. Finance, banking, insurance

5. Retail/wholesale distribution




In order to better understand role priorities, EMA condensed the data from Figure 1 into four separate groups:

1. IT executive oversight

2. Managerial and other oversight

3. Automation planning, strategy, and assessment

4. Other (including hands-on administrative support, active stakeholders, and architecture and planning)

In general, IT executive oversight was the more progressive and effective group. For instance, executive oversight led for those who saw unifying automation across use cases as extremely important and well underway:

• 32% of IT executive oversight prioritized unifying automation across use cases

• 25% of automation strategy and planning prioritized unifying automation

• 22% of managerial + other oversight prioritized unifying automation

• Only 11% of other groups prioritized unifying automation

As indicated in Figure 1, EMA also sought to capture role-specific perspectives.

Figure 1: Given the strategic focus of the research, EMA looked for strong executive presence, with 34% in the role of executive oversight. Also critical was the 17% in automation planning, strategy, and assessment.

Which of the following best describes the active role you play in your company's automation initiatives?

34%

27%

2%

17%

11%

8%

2%

0% 5% 10% 15% 20% 25% 30% 35% 40%

IT executive oversight (Director and above)

Manager-level oversight

Other oversight

Automation planning/strategy/assessment

Hands-on administration and support

Architecture and planning

Active ongoing stakeholder/user




Differences were also of interest among these groups in other areas. For instance, when asked about the lead driver for unifying automation across use cases:

• Executive and managerial oversight both cited improving IT-to-business alignment

• Automation strategy and planning indicated cutting costs, along with having superior automation integrations

• The other groups indicated improved overall efficiencies in automation performance

Qualifiers and Initial Strategic HighlightsSince this research was targeted at strategic automation adoption, EMA leveraged its first qualifier by excluding those respondents with fewer than 21% of their processes automated, or 90 respondents out of 495. The data in Figure 2 shows a substantial bell curve peaking at between 50% and 60% of processes automated.

Figure 2: The average respondent before disqualification estimated that automation was applied to slightly more than 40% of IT tasks.

To what degree do you estimate you have implemented IT automation in your organization?

7%

11% 12%

14%12%

15%14%

8%

4%3%

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

Less than10%

10-20% 21-30% 31-40% 41-50% 51-60% 61-70% 71-80% 81-90% 91-100%

Percentage of Tasks Automated

Those less than 21% automated were

terminated from the remainder of the

survey.




When those who were disqualified were asked about obstacles, the top five in ranked order were:

1. Product complexity

2. Lack of human/financial resources to support integration

3. Too much customization required

4. Resistance to change

5. Lack of integration between analytics and automation (tied with) lack of executive support

As will be shown later, product costs and complexity were also major obstacles across all four use cases examined.

The Critical Role of IT AnalyticsThroughout the research, EMA examined the impact of advanced IT analytics, and AIOps in particular, on automation strategies and effectiveness. When respondents were asked about their AIOps or other AIA adoptions, the answers were striking:

• 33% said that AIOps/AIA was currently a major initiative

• 42% indicated that AIOps/AIA was actively underway

• 15% indicated that they were currently evaluating AIOps/AIA

• 7% said that AIOps/AIA was planned for the future

• Only 4% had no plans in this area

In general, more active deployments of AIOps/AIA correlated strongly with success, progressive levels of automation adoption, more focus on unified strategies, and other critical factors, such as having an automation center of excellence. Figure 3 highlights how AIOps/AIA adoptions correlated with the data in Figure 2—or more progressive levels of automation adoption.

Figure 3: Prioritizing AIOps or other forms of advanced IT analytics strongly correlated with more progressive levels of automation adoption.

To what degree is AIOps, IT operations analytics, or some form of advanced IT analytics (AIA) part of your environment?

by To what degree do you estimate you have implemented IT automation in your organization?

21%

46%

33%

32%

43%

24%

54%

32%

14%

0% 10% 20% 30% 40% 50% 60%

A major initiative

Actively in progress

All others

21-40% tasks automated 41-70% tasks automatedMore than 70% tasks automated




Also associated strongly with AIOps adoption was combining AI and analytics with automation. When asked about priorities for combining analytics and automation overall:

• 36% indicated it was an extremely high priority

• 54% indicated that it was a very high priority

• 9% said it was a second-tier priority

• Only 1% said it was not a priority

The correlations between combining analytics with automation and AIOps/AIA adoption are both logical and striking, as indicated in Figure 4.

Figure 4: Not surprisingly, AIOps/AIA deployments correlated strongly with priorities to combine analytics with automation.

To what degree is combining automation with AI/analytics a priority in your organization for 2020? by

To what degree is AIOps, IT operations analytics, or some form of advanced IT analytics (AIA) part of your environment?

64%

35%

1%

26%

69%

5%

18%

52%

30%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Extremely

Very high

All others

A major initiative Actively in progress All others




The AIOps/AIA and automation handshake has been an ongoing theme across a number of EMA research reports, and the data is very consistent and logical. Automation brings speed. Analytics bring insight. Combining the two enables intelligent, and potentially critical, repeatable actions.

The Four Use CasesEMA’s automation research was directed in part at four critical use cases. EMA began by asking which use case was the furthest along in delivering value, as shown in Figure 5.

Figure 5: Configuration, change, and capacity management got the highest rating in being furthest along in delivering value, but the differences across use cases were minimal.

Of the areas listed, which one do you feel is furthest along in delivering value?

27%

26%

23%

23%

0% 5% 10% 15% 20% 25% 30%

Configuration, change, and capacitymanagement automation

Application performance andworkload optimization

Problem, incident, and availabilitymanagement automation

DevOps automation




Figure 6: One of the discoveries in this research was the degree to which IT professionals spread their attention across multiple use cases. Each wheel here shows percentages of respondents who answered questions for other use cases.

In order to enable meaningful responses directed at the research’s four automation-relevant use cases, EMA asked each respondent to select the one in which they were most involved. However, if they felt they were equally involved in two, they could select two. Application performance and workload optimization got the highest number of respondents at 208, while

DevOps got the lowest number at 135. Many respondents, 203 out of 405, or slightly more than half, selected two. Figure 6 helps to underscore this diversity, showing how each use case was answered based on respondent makeup, as some respondents indicated some level of involvement in as many as three or four.

63

3035

60

Change Use Case188N

Incident Change DevOps AppWLO

11

35

29

60

DevOps Use Case135N


54

6060

34

AppWLO Use Case208N


29

63

11

54

Incident Use Case157N





Problem, incident, and availability management automation, referred to here as incident management automation for brevity, is one of the most pervasive and critical sets of automation requirements across all of IT. It topped the list, for instance, when respondents were asked where they were also actively involved, beyond their top two.

When EMA did research1 on digital war room trends and requirements, the company singled out five processes against which automation technologies should ideally come into play. These were:

1. Automation to support initial awareness, driven from events and other metrics, ideally through proactive analytic insights.

2. Response team engagement workflows, sometimes combined with social IT, to ensure that just the right group of IT professionals is brought together for resolution.

3. Automation associated with triage and diagnostics to drill down further into problem areas for complete root cause analysis.

4. Remediation-driven automation, often requiring configuration management changes, or engaging alternate infrastructure components, or in some cases code-related issues with applications. Here IT process automation and workload automation can play a coordinating, or governing role.

5. Validation and auditing also requires automation for efficiency and effectiveness.

Across these five processes, integrated security also played a key role, as was indicated by the data in this research.

1 EMA Research: “Unifying IT Decision Making for Digital War Rooms,” April 2018.

Needless to say, not all IT organizations have defined processes—even manually—for all of these five areas, the most neglected being auditing and validation. However, there was a clear correlation with success when more processes were defined and automated.

In this research, in order to get a feel for how incident management automation is underway, EMA asked about stakeholder roles driving these automation initiatives, as well as leading in purchasing.

The top five groups for driving incident management automation were:

1. IT operations – 34%

2. IT executive suite – 18%

3. Defined incident management team – 8%

4. IT service management team – 7%

5. Security, fraud risk management – 6%

The striking lead role for IT operations was expected. It also turned out to be largely consistent with other use cases.

Purchasing, on the other hand, was driven first and foremost by the IT executive suite, then by IT operations, both of which strongly overshadowed all other groups.

On average, respondents indicated that their incident management automation initiative was serving more than five different stakeholder roles, with operations clearly in the lead, followed by applications management and ITSM.

Problem, Incident, and Availability Management Automation Highlights




Technology Adoptions

Figure 7: Workflow within and across IT, security-related automation, and workload automation were viewed as both the most critical and the most prevalent technologies in support of incident management automation.

To better understand how many different automation technologies might be in play, EMA asked respondents to choose from a list of relevant options. The average response indicated that about five and a half different automation technologies were actively in play for incident management automation.

When asked which was the most critical automation technology, workflow within and across IT was strongly in the lead, as seen in Figure 7, correlating strongly with the prior question on prevalence.

What are the top three automation technologies in terms of relevance and usage?

27%22%

20%18%18%

17%16%

15%13%13%

12%11%11%

9%9%9%

6%5%

0%

0% 5% 10% 15% 20% 25% 30%

Workflow within and across ITSecurity-related process automation (and/or playbooks)

Workload automationAutomated root cause analysis

Automation in support of incident team communicationAutomation to capture user input to drive IT workflows

Integrated trouble ticketingAlert-driven notification

Automated event remediationAutomation in support of ADDM

Runbook or IT process automationWorkflow combined with social IT

Automation in support of infrastructure inventory and discoveryRobotic process automation

Automation in support of data assimilation and reconciliationAutomated tracking and documentation of remediation actions

AI botsVirtual agents

Other




Figure 8: More progressive levels of automation adoption strongly correlated with reduced mean time to repair.

In terms of integrations, the top five were:

1. Security-related tools

2. Monitoring tools for application performance

3. Incident management records/audits

4. Collaboration tools

5. Knowledge management

For analytic priorities, incident response led for both prevalence and criticality.

EffectivenessIn order to gauge use case-related efficiencies, EMA looked for a number of parameters. Among these were:

• Proactive incident remediation – the average response indicated detection and remediation of more than 40% of incidents before the impacted end user or business outcomes.

• Mean time to repair – the average time for MTTR once an alert is in was about four hours.

• Technical metrics – on average, respondents indicated about five technical metrics were in play, with more mapping to improved efficiencies and effectiveness.

• Business impact metrics – similarly, the average number of business impact metrics indicated was about five, while more business metrics mapped to improved outcomes in other areas.

When EMA evaluated efficiencies in these parameters across success rates, more progressive levels of automation adoption, more evolved strategies for unifying automation across use cases, and other factors, there were consistently strong connections. For instance, Figure 8 highlights the striking correlation between more progressive levels of automation and reduced mean time to repair.

How much of an improvement do you believe this was in MTTR prior to when your automation initiative was underway?


24%

41%

24%

11%15%

39%

23% 23%

0%

19% 19%

63%

0%

10%20%30%

40%50%60%70%

Up to 25% 26% to 50% 51% to 75% 76% or more

21-40% tasks automated 41-70% tasks automated More than 70% tasks automated




Configuration, Change, and Capacity Management Automation HighlightsConfiguration, change, and capacity management automation, referred to here as change automation for brevity, has been at the hub of IT transformational issues for years because it touches directly on process priorities, including an evolving shift in emphasis away from manual review toward automated change-making. This shift has become especially apparent in the next use case, DevOps, with accelerating requirements for far more frequent application changes and a growing number of production application releases annually. In fact, automating changes impacts the efficiencies of all four use cases examined here, in contexts such as incident remediation and job scheduling, among others.

One of the key areas to consider for change management automation and DevOps is best practices, which reflect a shift from the IT infrastructure library (ITIL) toward agile/scrum (see Figure 20). However, it should be mentioned here that “automation” is a pervasive part of ITIL4. EMA still sees a role for ITIL’s Change Advisory Board when critical infrastructure changes, or complex, traditional application releases, are underway.

Another factor to consider in change management automation is service modeling, including configuration management databases (CMDBs), configuration management systems (CMSs), and application discovery and dependency mapping (ADDM). In this EMA research, these integrations didn’t score high for change management automation (the combined

2 EMA Research: “IT Service Modeling (CMDB/CMS & DDM) in the Age of Cloud and Containers,” November, 2019.

integration score for CMDB/CMS and ADDM was 39%), but these integrations do play a major role in support of change-related analytics, including AIOps. In recent EMA research,2 the powerful bidirectional handshake between service modeling and IT analytics for context, currency, and impact-related insights was quite possibly the biggest single finding.

In this research, EMA found that the top five groups for driving change management automation were:


2. IT service management team – 16%


4. IT infrastructure team – 11%

5. Chief Digital Officer – 8%

Leadership in purchasing was a virtual three-way tie between the IT executive suite, IT operations, and ITSM.

On average, respondents indicated that their incident management automation initiative was serving slightly under five different stakeholder roles, with operations once again clearly in the lead, followed by the executive suite.




The average response indicated that about five automation technologies were actively in play for change management automation. When asked

which was the most critical automation technology, workflow within and across IT was once again strongly in the lead.

Figure 9: Workflow within and across IT, capacity management, analytics-driven automation, and automation to capture user input for driving IT workflows led as the most critical/relevant options.


Which automation technologies are the top three in terms of relevance and usage?

28%

23%

20%

18%

16%

14%

14%

14%

12%

12%

12%

12%

11%

10%

9%

9%

7%

6%

0%

0% 5% 10% 15% 20% 25% 30%

Workflows within and across IT

Capacity management/analytics-driven automation

Automation to capture user input to drive IT workflows

Workflows combined with social IT

Dynamic resource capacity and optimization

End-to-end automated provisioning and deployment

Automated inventory and discovery

Automated application discovery and dependency mapping

Patch management automation

Endpoint (PC or mobile) configuration automation

Workload automation

Asset management-driven automation

Automation in support of data assimilation and reconciliation

Runbook or IT process automation

AI bots

Robotic process automation

Core configuration tools (such as Chef, Puppet, Ansible)

Virtual agents

Other





1. IT asset management tools


3. Cloud performance-related management tools

4. Knowledge management

5. Monitoring tools for infrastructure performance

In terms of analytic priorities, incident response led for prevalence, but AIOps led for criticality.

EffectivenessIn order to gauge change management automation efficiencies, EMA looked at:

• Changes made without requiring rework – the average response indicated nearly 50% of changes were made without requiring rework.

• Automated changes without requiring approval – on average, more than 40% of automated changes were made without requiring approval or review.

• Technical metrics – an average of 4.7 technical metrics were in play.

• Business impact metrics – an average of 4.43 business impact metrics were in play.

Once again, the connections were consistent and striking when these were correlated with success-related factors. Figure 10 shows how more committed approaches to unifying automation strategies across use cases help minimize the number of changes requiring rework.

Figure 10: Those with an initiative to unify automation across use cases well underway were significantly more effective in minimizing the number of changes requiring rework.

How many changes are done without requirements for rework and without impacting service performance, security, or other areas of change-related impacts?

by To what degree are you seeking to unify automation investments across use cases?

15%

21%

64%

27%

44%

29%

48%

33%

19%

0% 10% 20% 30% 40% 50% 60% 70%

0-40%

41-60%

More than 60%

Extremely Significantly All others




Recent EMA research on DevOps underscored a key finding from this research, which EMA calls the “More Syndrome.” The same list of factors played a role on both datasets: more stakeholders, more metrics, more automation technologies, more analytic technologies, more best practices, etc. all contribute directly to more effectiveness and success.

Another factor in common was the growing role of containers in both private and public cloud. In the DevOps use case, 92% of respondents indicated a use of containers and microservices when private and public cloud resources were combined. The next two highest were change management automation and AppWLO both at 75%, with incident management automation at 61%.

Yet another factor that stood out in EMA’s DevOps research was the need for automation in support of the critical handshake between operations and development teams, as well as integrated ITSM teams. Automation wasn’t just key in supporting faster delivery of releases and release updates; it also helped to strongly solidify team cooperation and codify processes across what has become one of the most rapidly changing landscapes in terms of processes and politics within IT.

The top five groups for driving DevOps automation were:



3. ITSM team – 9%

4. Cloud – 7%

5. Application development teams affiliated with lines of business – 5%

The general message here is that automation in support of DevOps still resides largely on the operations side of the equation. However, it’s worthy of note that central application development teams were at 4%, bringing the total of the two together to 9%, tying for third place. Interestingly, only 2% indicated that site reliability engineers (SREs) were driving automation initiatives for DevOps. Nevertheless, SREs are taking an increasingly important role in overseeing the development/operations divide. Leadership in purchasing was a virtual tie between the IT executive suite and IT operations.

On average, respondents indicated that their DevOps automation initiative was serving slightly fewer than five different stakeholder roles, with operations and ITSM tied for prevalence, but ITSM leading for criticality.

DevOps Automation Highlights




The average response indicated that more than five automation technologies were actively in play for DevOps automation. When asked which automation technology was the most critical, workload automation took first place, with workflow within and across IT in second place. The strong

showing for workload automation underscored a growing trend that EMA has documented multiple times in which WLA is evolving far beyond job scheduling to become an overall source of coordination across multiple automation routines.

Figure 11: Workload automation and workflow across IT, automation for pre-deployment application configuration, and automation for cloud-specific configuration led for DevOps automation criticality and relevance.


Which automation technologies are the top three in terms of usage and relevance?

25%

21%

20%

20%

17%

15%

15%

14%

13%13%

13%

13%

12%

11%

10%

10%

9%

8%

0% 5% 10% 15% 20% 25% 30%

Workload automation

Workflow within and across IT

Automation for pre-deployment application configuration

Automation for cloud-specific application configurationAutomated application discovery and dependency mapping

Integrated automation in support of Q/A testing

Continuous attack testing and defense stack validation

Automated ADDM in pre-deployment

Release management automation

Automated discovery and inventory for preproduction

Virtual agents

Robotic process automation

AI bots


Autoscaling capacity optimization

Manifest-driven automation throughout the lifecycle

Workflow combined with socialRunbook or IT process automation




Figure 12: DevOps success correlated strongly with increased volumes of production releases due to automation.


1. Cloud performance-related management tools

2. CI/CD tools

3. Cloud cost-specific management tools



In looking at analytic priorities, AIOps led for both prevalence and criticality.

EffectivenessIn order to gauge DevOps automation efficiencies, EMA looked at:

• Releases into production without rework – on average, respondents indicated that about 50% of application releases were introduced into production without rework.

• Number of annual production releases – on average, respondents indicated that about 160 releases were introduced into production annually.



Critical indicators of success also correlated to superior DevOps effectiveness. Figure 12 shows how self-assigned success rates in DevOps automation correlated well with increases in annual production releases.

How much has the number of production releases delivered annually increased since putting higher levels of DevOps-related automation in place?

byHow successful have you been in your DevOps-driven initiatives to date?

42%

30%

18%

9%

0%

6%

68%

19%

4%

3%

0%

30%

39%

24%

6%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Increased dramatically

Increased significantly

Increased meaningfully

Increased somewhat

Not changed at all

Extremely successful Very successful All others




The first three use cases are well-defined categories. The application performance and workload optimization use case was created to capture application performance monitoring automation, workload automation, and other scheduling technologies. EMA defined this area for survey respondents as follows:

“Application performance” includes tools that detect and diagnose complex application performance problems to maintain expected service levels. “Workload optimization” includes tools that coordinate and run workloads in the best environment with properly balanced resource requirements. It also includes workload scheduling technologies that manage dependencies and business timing requirements, along with data and analytics needed to optimize the orderly completion of workloads.

The top five groups for driving AppWLO automation were:


2. ITSM – 14%


4. Application management – 9%

5. DevOps-driven team – 9%

The IT executive suite led for leadership in purchasing followed by ITSM.

On average, respondents indicated that their AppWLO automation initiative was serving slightly under four different stakeholder roles, with IT operations in the lead for relevance and criticality, followed closely by ITSM and the IT executive suite.

Application Performance and Workload Optimization Automation Highlights (AppWLO)




Figure 13: Workflow across IT followed by workload automation led for usage, relevance, and criticality in AppWLO initiatives.

Technology AdoptionsThe average response indicated that more than four automation technologies were actively in use, with workflow within and across IT mentioned most at 31% and workload automation in second place at 26%.

Which are the top three automation technologies in terms of usage and relevance?

31%

26%

18%

17%

14%

14%

13%

13%

13%

12%

12%

11%

11%

11%

10%

10%

3%

0% 10% 20% 30% 40%


Workload automation


Workflow combined with social IT

Comparative performance reporting for each build/release

Dynamic resource allocation and optimization

Automated testing and proactive health checks

Autoscaling capacity optimization

Automated application discovery and dependency mapping

Profiling and performance analysis for acceptance and load testing

Runbook or IT process automation

Managed file transfer

AI bots

Job scheduling

Event triggers and complex dependencies

Virtual agents

RPA orchestration





1. IT asset management tools


3. Cloud performance management tools

4. Incident management records/audits


For analytic priorities, big data lakes led for prevalence but AIOps led for criticality.

Business ContextWithin application performance management and workload automation, many IT organizations have been adding more dashboards and reports for business stakeholders. They have also been incorporating more linkages to business outcomes and business impact context for IT to more strongly align with business goals. Those providing business impact context as part of the automation implementation were far more successful.

Figure 14: Effective business context strongly aligned with AppWLO success.

To what extent do your application performance and workload optimization tools provide business impact context?

byHow successful have you been in your application performance and workload

optimization automation initiatives to date?

67%

33%

0%

0%

0%

41%

45%

11%

2%

1%

16%

51%

15%

7%

10%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Specific impacts and dependencies areclear and actionable

Business context and some actionableinformation is provided

Some business context but no actionableinformation is provided

Minimal business context is provided

No impacts or dependencies are identified

Extremely successful Very successful All others




Figure 15: More progressive levels of automation strongly helped to accelerate the number of jobs completed without the need to restart and without impacting business performance.

In order to gauge AppWLO automation efficiencies, EMA looked at:

• Percentage of jobs without restart – on average, respondents indicated that about 60% of job scheduling-related jobs were completed without the need for restart.

• Improvements due to automation – 68% believed that automation has significantly or dramatically improved job completion effectiveness.



Critical indicators of success also correlated to superior AppWLO effectiveness. Figure 14 shows how more progressive levels of automation can impact value, this time in terms of the percentage of jobs completed without the need to restart.

Effectiveness

How many jobs are completed without a restart and without impacting service performance or other dependencies?


22%

47%

31%15%

48%37%

0%13%

87%

0%

20%

40%

60%

80%

100%

0-40% 41-60% More than 60%

21-40% tasks automated 41-70% tasks automated More than 70% tasks automated




Cross-Use Case CorrelationsWhile each automation use case clearly had its own dimensions and priorities, there were quite a few similarities. In EMA’s analysis, the following stood out.

• Chief obstacles per use case:

◦ Problem, incident, and availability management – product costs and complexity

◦ Change, configuration, and capacity management – product costs and complexity

◦ DevOps – deployment complexity/heavy lifting

◦ Application performance and workload optimization – organizational conflicts within IT

• Top benefits achieved:

◦ Problem, incident, and availability management – IT cost savings

◦ Change, configuration, and capacity management – IT cost savings

◦ DevOps – IT cost savings

◦ Application performance and workload optimization – Increased IT personnel productivity

• Top purchasing priorities:

◦ Problem, incident, and availability management – scalability

◦ Change, configuration, and capacity management – usability

◦ DevOps – scalability

◦ Application performance and workload optimization – scalability




Figure 16: An analysis of the top 8 stakeholders per use case shows some striking similarities, most obviously the leading role of IT operations in all but DevOps, where ITSM took the lead.

Top three most important groups/stakeholders served by automationTop 8 Responses

41%27%

18%18%

17%16%16%

15%

0% 10% 20% 30% 40% 50%

IT operationsIT executive suite

Business executive suiteApplication management

ITSM teams,incident management

End-user managementNetwork operations

Incident

44%21%

18%18%

15%15%

14%14%

0% 10% 20% 30% 40% 50%


Application managementSecurity/fraud risk

DevOps-driven teamAsset management

Data centerITSM team

Change

44%38%

33%19%

13%13%13%

10%

0% 10% 20% 30% 40% 50%

ITSMIT operations

IT executive suiteApplication management

Engineering/developmentCloud

Security/fraud riskLOB App development

DevOps

37%29%29%

19%16%

15%13%

11%

0% 10% 20% 30% 40%


iTSMApplication management

DevOps-driven teamAsset management

IoTCloud

AppWLO

The net takeaway is that IT automation across use cases has far more in common than what market and traditional buying priorities might have assumed. This message strongly underscores the value of stepping back and

thinking more strategically about unifying automation investments across use cases.




Unifying Automation More Strategically

Figure 17: A focus on prescriptive analytics driving automation correlated significantly with a more progressive focus in unifying automation across use cases. Both factors played strongly to automation maturity levels and success.

While lack of integration across automation tools and lack of integration with analytic tools on a per-use case basis were persistently cited as obstacles by respondents, EMA wanted to explore a larger concern: strategic integrations across multiple use cases. When asked, EMA found:

• 23% of respondents said that unifying automation investments across use cases was well underway.

• 41% of respondents said that unifying automation investments has become a priority

• 28% indicated that strategic automation unification was under discussion

• 6% were just beginning to consider unification as a priority

• Only 1% said that their focus for automation was purely on a use case basis.

In examining how this mapped to other trends, EMA saw very significant correlations between a focus on automation unification across use cases and other parameters for maturity and effectiveness, such as success rates, having an automation center of excellence, a more progressed capability to let analytics drive automation, and more progressive levels of automation across IT. Figure 17 highlights the rather striking correspondence between a focus on unification and letting prescriptive analytics drive automation.

To what degree does your organization enable automated actions to be taken based on AI/analytics recommendations?

by To what degree are you seeking to unify automation investments across use cases?

67%

24%

7%

0%

1%

23%

57%

15%

4%

1%

5%

47%

32%

11%

5%

0% 10% 20% 30% 40% 50% 60% 70% 80%

Automated actions are driven directly from prescriptiveanalytics with the option of human override

Automated actions are enabled by human oversight,looking at prescriptive recommendations

Automated actions are enabled with human oversighton observable patterns

We don't allow AI to drive automated actions except forproven routines, such as batch processing

Automation is currently not coupled with AI/analyticstools or platforms

Extremely Significantly All others




When asked about drivers for unification, the following responses surfaced as the top ten:

1. Improving IT-to-business alignment

2. Improved overall efficiencies in automation performance

3. Cutting costs by having superior automation integrations

4. Unifying IT to operate more effectively as a whole

5. Creating a centralized automation center of excellence

6. Reducing administrative overhead for automation

7. Need to reduce complexity in the automation ecosystem

8. Cutting costs through superior OpEx efficiencies

9. IT executive pressure

10. The need for improved governance

In assessing these and related drivers, several overarching factors leapt out: cost cutting, improved efficiencies, organizational transformation, and governance. Given the strategic nature of these drivers, it’s not surprising that executive (or top-down) pressure is also a factor in promoting more effective, cross-use case automation.

When EMA asked about leading automation technologies for enabling cross-use case automation, Figure 18 shows that IT process automation, sometimes called “runbook,” led for both prevalence and criticality.

Figure 18: IT process automation (ITPA) led as the choice for unifying automation across use cases in terms of prevalence and relevance/criticality.

What automation technologies are leading the charge in support of your unified automation initiative?

74%

68%

36%

30%

23%

46%

27%

10%

11%

6%

0% 10% 20% 30% 40% 50% 60% 70% 80%

IT process automation or runbook


Workload automation (WLA)

AI bots

RPA orchestration

Leading Critical




This was not surprising. EMA has been tracking trends in ITPA for more than a decade as it continued to evolve from traditional runbooks into a more multidimensional way of bringing different automation technologies, such as workflow, configuration management, and incident management alerting together.

In parallel, as shown in Figure 18, AIOps/AIA led as the platform of choice to integrate with automation for enabling more strategic, cross-use case automation effectiveness. This reinforced the growing and powerful handshake between IT analytics (AIOps in particular) and more progressive levels of automation.

Figure 19: AIOps was clearly the dominant platform for unifying automation across use cases, followed by ITSM platforms.

What platform technologies are leading the charge in support of your unified automation initiative?

47%

35%

30%

28%

26%

19%

0% 10% 20% 30% 40% 50%

IT analytics/AIOps

ITSM (IT service management)

IT operations management suites

IT analytics (other)

A dedicated automation platform

DevOps platforms

When asked about benefits from cross-use case automation, the top five were:

1. Increased levels of innovation across IT

2. A more unified way of working across IT

3. Better alignment across IT silos

4. Improved IT-to-business alignment

5. Improved use case outcomes

Once again, these achieved benefits underscore the transformative nature of more strategic automation unification. The impact is not just on per-task efficiencies, but new ways of working across IT as a whole, with superior relevance to business outcomes and overall operational efficiencies.




In the segment of research devoted to maturity, EMA once again dove more deeply into analytics, this time very specifically in terms of IT’s readiness to let analytics actually drive automation directly. Figure 20 reflects a range

of options, with the clear leader in pervasiveness being automated actions are enabled by human oversight while looking at prescriptive recommendations.

Maturity-Related Insights

Figure 20: 27% of respondents had analytics drive automation with the option of human override. Only 2% had no AI coupling.

To what degree does your organization enable automated actions to be taken based on AI/analytics recommendations?

27%

46%

19%

6%

2%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Automated actions are driven directly from prescriptiveanalytics with the option of human override

Automated actions are enabled by human oversight,looking at prescriptive recommendations

Automated actions are enabled with human oversighton observable patterns

We don't allow AI to drive automated actions except forproven routines, such as batch processing

Automation is currently not coupled with AI/analyticstools or platforms

Clearly, the trend to evolve to fully automated solutions in which auditing takes the place of enforced review relies 100% on more advanced forms of prescriptive analytics to marry intelligence with action. Figure 20 provides

a map of options to trace this trend toward higher levels of automation, as informed more by prescriptive courses of action than observable patterns requiring human, and often expert, assessments.




EMA gathered data relevant to automation maturity from a number of perspectives. In addition, EMA looked at a very basic, yet fundamental parameter: the number of processes actually documented. The data showed that:

• 18% of respondents had documented less that 30% of their manual processes

• 25% had documented between 30% and 50% of their manual processes

• 23% had documented between 50% and 70% of their manual processes

• 25% had documented more than 70% of their manual processes

This data presented a bell curve that actually peaked at 30-40% of manual processes documented, gradually tapering down toward more progressive levels. Not surprisingly, having more processes documented strongly correlated with other success-related factors, such as more progressive levels of automation.

In alignment with process issues, EMA also wanted to examine best practices, both in terms of prevalence and priorities. In terms of prevalence, the top five best practices associated with automation initiatives were:

1. Regulatory compliance (SOX, HIPAA, etc.)

2. ISO security 27001/27002

3. Knowledge-centered support

4. Service integration and management

5. IT balanced scorecard

Figure 21 highlights the most relevant or critical best practice initiatives associated with automation, in which agile/scrum leapt to the top from sixth place in prevalence to tie with regulatory compliance. This trend underscores the need for IT to become more agile overall, as IT’s culture of restricting change has shifted toward more dynamic change accommodations in support of cloud migrations, DevOps, and more dynamic integrations between security and operations.

Process Considerations and Recommendations Going Forward




Figure 21: Agile/Scrum and regulatory compliance led as the most relevant best practices for automation initiatives.

Which best practices are the most relevant?

10%10%

9%9%

7%7%7%7%

6%5%5%

4%3%3%

2%2%

1%1%1%1%

0%0%

0% 2% 4% 6% 8% 10% 12%

Agile/ScrumRegulatory Compliance (SOX, FISMA, HIPAA, etc.)

ISO Security 27001/27002Service Integration and Management

Six SigmaCMMI - Capability Maturity Model Integration

Control Objectives for Information and Related TechnologyKnowledge-Centered Support (KCS)

Continuous OperationsITIL4

IT Balanced ScorecardITIL v2, v3

PRINCE2 or IPMA (for Project Management)TOGAF - The Open Group Architecture Framework

CSI Benchmarks for AWSNIST SP800 series

IT4ITISO 19770-1

Other ISO Security 27xxxFFIECOther

We are not supporting best practices

While the average respondent indicated a little greater than three best practices in support of automation initiatives, higher levels of maturity and effectiveness correlated consistently with more best practices in play. When

technology is transformative, attention to both process and communication are key.




As an extension of this, EMA asked respondents about digital transformation.

• 29% said that digital transformation was driving their automation initiative

• 46% said that digital transformation and automation were tightly coupled

• 18% said that digital transformation and automation were related, but not yet tightly coupled

• 8% had no plans to integrate the two

In reviewing other parameters, digital transformation was also significantly contributive to automation maturity and effectiveness—yet another player in aligning IT with business objectives, and IT performance with business performance.

Not surprisingly, having an automation center of excellence was another key parameter for automation maturity. When asked:

• 50% claimed to have an automation center of excellence

• 39% were planning to have an automation center of excellence

• 11% had no current plans

This data indicates a growing interest in a more strategic approach to automation. Figure 22 shows how having an automation center of excellence correlates well with more progressive approaches to unifying automation across use cases.

Figure 22: Not surprisingly, having an automation center of excellence went hand in hand with more progressive approaches to unifying automation across use cases and across IT.

To what degree are you seeking to unify automation investments across use cases? by

Do you currently have an automation center of excellence or an equivalent organization overseeing automation strategies across IT?

36%

43%

16%

3%

0%

9%

43%

41%

6%

1%

15%

28%

37%

13%

7%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Extremely: Unifying automation investments across use casesis well underway

Significantly: Unifying automation investments across usecases has become a priority

Meaningfully: Unifying automation investments across usecases is actively in discussion

Somewhat: Unifying automation investments across usecases is something we are beginning to consider

Not at all: We focus our automation investments exclusivelybased on use case

Yes No, but planned No, not planned




In terms of overall recommendations for going forward with more advanced levels of automation, the respondents flagged the following for their top ten:

1. Optimize processes before you automate them

2. Evaluate technologies with due diligence based on your unique needs

3. Make sure that you have adequate skillsets in place

4. Integrate with advanced deployments in AIOps and analytics

5. Start with easy processes for quick wins and build from there

6. Assess what’s needed and establish use case priorities first

7. Create new roles and functions within IT to accelerate adoption

8. Put metrics in place to measure and quantify your progress

9. Educate users ahead of time

10. Socialize your investments, directions, and KPIs with all relevant stakeholders

Conclusion: Summing Up Building on the data just shared, if the goal is to progress step-by-step and stage-by-stage into a more mature and effective automation capability, this research combined EMA’s history in IT consulting and suggests the following ten parameters:

1. Document and optimize your manual processes first.

2. Pay attention to best practices.

3. Prioritize based on use case and need.

4. Establish an automation center of excellence.

5. Establish both technical and business metrics to assess your progress, in terms of present and future objectives.

6. Invest in technologies based on a solid evaluation and what’s most relevant to you at each stage as you evolve.

7. Understand who your stakeholders are in your current environment and who they might need to be going forward.

8. Seek IT executive involvement, if it’s not already there, since IT executive leadership is critical in promoting more transformative ways of working.

9. Socialize and communicate as you evolve, each step of the way.

10. Integrate your automation investments with AIOps and other analytics as you progress from leveraging analytics as a reference point to actually having analytics drive your automated processes prescriptively.




In support of item number six, this research provides a high-level roadmap of which technologies are in play, how they’re being prioritized, and how they’re being used. It also indicates how technology adoption (in terms of automation and analytics) plays to many of these parameters. In terms of technology evaluations, EMA found the following top priorities for investing in automation across use cases:

• Usability

• Scalability

• Price

• Easy integration with monitoring and other tools

• Easy integration with AIOps and advanced analytics

• Leading-edge, innovative technologies

• Functional extensibility

• Easy integration with other automation tools

• Compliance

• Strong visualization capabilities—an integrated dashboard

In summing up parameters surrounding higher levels of progress and effectiveness that affect any roadmap for going forward with automation, this data once again affirms what EMA has called the “More Syndrome.” This would include:

• More automation technologies in play

• More stakeholders engaged

• More metrics in play

• More best practices supported

• More analytic deployments and integrations

• More processes automated per use case

• More integrated automation across use cases

• More teams defined specific to automation needs

However, it’s always important to remember the “more” so often ignored—more communication and planning. It is just as important to understand your stakeholders, your overall environment and needs, and your IT and business requirements as it is to be diligent in evaluating and adopting technologies, stage-by-stage, to address your evolving priorities.


About Enterprise Management Associates, Inc.Founded in 1996, Enterprise Management Associates (EMA) is a leading industry analyst firm that provides deep insight across the full spectrum of IT and data management technologies. EMA analysts leverage a unique combination of practical experience, insight into industry best practices, and in-depth knowledge of current and planned vendor solutions to help EMA’s clients achieve their goals. Learn more about EMA research, analysis, and consulting services for enterprise line of business users, IT professionals and IT vendors at www.enterprisemanagement.com or blogs.enterprisemanagement.com. You can also follow EMA on Twitter, Facebook or LinkedIn.

This report in whole or in part may not be duplicated, reproduced, stored in a retrieval system or retransmitted without prior written permission of Enterprise Management Associates, Inc. All opinions and estimates herein constitute our judgement as of this date and are subject to change without notice. Product names mentioned herein may be trademarks and/or registered trademarks of their respective companies. “EMA” and “Enterprise Management Associates” are trademarks of Enterprise Management Associates, Inc. in the United States and other countries.

©2020 Enterprise Management Associates, Inc. All Rights Reserved. EMA™, ENTERPRISE MANAGEMENT ASSOCIATES®, and the mobius symbol are registered trademarks or common-law trademarks of Enterprise Management Associates, Inc.

Corporate Headquarters: 1995 North 57th Court, Suite 120 Boulder, CO 80301 Phone: +1 303.543.9500 Fax: +1 303.543.7687 www.enterprisemanagement.com3969.042520


http://blogs.enterprisemanagement.com/

http://twitter.com/ema_research

https://www.facebook.com/enterprisemanagementassociates

http://www.linkedin.com/company/25620


data-driven it automation: a vision for the modern cio€¦ · it data management researc indstry...

Documents