speech analytics product and market report...

Speech Analytics Product and Market Report Reprint

Sponsored by:

- i - © DMG Consulting LLC

Table of Contents

1. Speech Analytics Vendor Satisfaction Analysis ............................................ 1 1.1 Summary of Survey Findings and Analysis ................................................... 3 1.2 Detailed Survey Findings and Analysis ......................................................... 5 About CallMiner .................................................................................................... 8 About DMG Consulting LLC .................................................................................. 8

- 1 - © DMG Consulting LLC

1. Speech Analytics Vendor Satisfaction Analysis

DMG Consulting conducts an independent customer satisfaction survey to measure end-user satisfaction with vendors and their products as well as their service, support, professional services, training and innovation. DMG conducts these satisfaction analyses because they provide great insights into general market trends and challenges, and end-user issues and concerns about vendors and products. DMG strongly recommends that prospects conduct their own reference checks by contacting at least 3 customers from their preferred vendor before making an investment. References are often very helpful in sharing their lessons learned. This satisfaction study evaluates the performance of the 7 leading and contending vendors analyzed at a detailed level in the 2014 - 2015 Speech Analytics Product and Market Report: Avaya, Calabrio, CallMiner, Genesys, NICE, Uptivity/inContact and Verint. DMG obtained a completed survey from 3 customers for each of these vendors. Since confidentiality is important to survey participants and respected throughout our process, the names of references and the companies that they represent are not directly attributed to any of the comments or ratings contained in this Report. The research for DMG Consulting’s customer satisfaction survey was conducted as follows:

1. The speech analytics vendors were asked to provide contact information for five customers who had agreed in advance to participate in the satisfaction survey.

2. DMG Consulting sent the participating customers an email with a link to a Web survey.

The survey consisted of 12 questions, which fell into the following categories:

1. Customer background – name of their speech analytics vendor, the top 3 - 5 primary investment drivers for purchasing a speech analytics solution, the top 3 - 5 ways that speech analytics was being used, the top 3 - 5 benefits realized, and which business units were using the information surfaced by speech analytics.

2. Product satisfaction – detailed questions about 21 aspects of product satisfaction including: ease of use/configuration/maintenance, ease of integration with third-party applications, ease of creating searches and queries, accuracy and business relevance of results, ability to fine-tune results, call analysis capabilities, emotion detection/sentiment analysis


capabilities, ability to conduct automated root cause analysis, ability to conduct discovery and surface previously unknown issues or new and breaking trends, ability to correlate seemingly unrelated issues, real-time capabilities, reporting and dashboards, ability to improve customer service, ability to improve the customer journey, ability to improve first contact resolution, ability to reduce operating costs, ability to improve adherence to regulatory requirements, ability to serve as an early warning system to identify issues before they escalate and impact a large number of customers, ability to improve customer retention, ability to increase sales and collections, and ability to realize the expected payback from the system.

3. Vendor performance – satisfaction levels with the vendor’s implementation, training/workshops, ongoing service and support, professional services, innovation, responsiveness to product enhancement requests, communication, product pricing, and the customer’s overall satisfaction level with the vendor/solution.

4. Open-ended questions – top 3 - 5 strengths and top 3 - 5 challenges of the speech analytics solution; biggest impact that speech analytics has had on the contact center; what the vendor should do to help end users more quickly realize the benefits and payback from the speech analytics solution; and an open invitation for participants to express general thoughts about their experience with the vendor and solution.

A total of 21 customer responses (3 from each vendor) were received and analyzed. The vendor satisfaction ratings reported in this analysis were calculated by averaging the scores provided by customers for each vendor and category, using the point scale illustrated in Figure 1. Figure 1: Customer Survey Rating Categories Satisfaction Rating Points Assigned Completely satisfied 5 Highly satisfied 4 Satisfied 3 Somewhat satisfied 2 Not satisfied 1

Source: DMG Consulting LLC, August 2014


1.1 Summary of Survey Findings and Analysis

The Figures and analyses that follow represent the results of DMG Consulting’s 2014 – 2015 speech analytics customer satisfaction survey. This section of the analysis examines customer satisfaction ratings across the 10 major survey categories, including: product, implementation, training/workshops, ongoing service and support, professional services, innovation, responsiveness to product enhancement requests, communication, product pricing, and the customer’s overall level of satisfaction with the vendor/solution. Vendor product satisfaction scores were derived by averaging the results of the 21 product satisfaction components found in Figure 3: Product Satisfaction Ratings, by Category. As seen in Figure 2, a majority (37.1%) of the satisfaction scores for all vendors in the 10 major categories surveyed fell into the highly satisfied range (4.0 to 4.66). 30% of the average ratings were in the satisfied range (3.0 to 3.95); 22.9% were in the completely satisfied range; and 10% were somewhat satisfied. Overall, the speech analytics references were highly satisfied (4.04 to 4.33) in 6 of the 10 categories they were asked to rate: training, ongoing service and maintenance, professional services, innovation, communication and overall vendor satisfaction. The remaining four categories – product, implementation, responsiveness to product enhancement requests and product pricing – achieved an average satisfaction score of satisfied (3.90 to 3.96). It is important to note that the ranges between the high and low score in 8 of the 10 categories were 2.0 or higher. This suggests significant disparities among the speech analytics solutions and vendors. The most notable results were shared by CallMiner and Verint. CallMiner achieved a perfect score, 5.0, in 6 categories: ongoing service and maintenance, professional services, innovation, communication, pricing and overall vendor satisfaction. CallMiner earned top score in 2 other categories: product (4.95) and implementation (4.96). Verint achieved a perfect score, 5.0, in 6 categories: training, ongoing service and maintenance, professional services, innovation, responsiveness to product enhancement requests and communication. Verint also achieved top score for implementation (4.66). Calabrio achieved a perfect score in 4 categories: ongoing service and maintenance, professional services, communication and overall vendor satisfaction.


Figure 2: Average Satisfaction Ratings, by Category

Avaya Calabrio CallMiner Genesys NICE Uptivity/ inContact Verint

Category Average Range

Product 1 2.82 4.39 4.95 3.42 4.29 3.18 4.71 3.96 2.13 Implementation 3.0 4.33 4.66 3.33 3.66 3.66 4.66 3.90 1.66 Training 3.0 4.66 4.66 3.33 4.66 4.0 5.0 4.18 2.0 Ongoing service and maintenance 3.33 5.0 5.0 3.66 4.33 4.0 5.0 4.33 1.67

Professional services 2.33 5.0 5.0 3.66 4.66 3.66 5.0 4.18 2.67 Innovation 2.0 4.66 5.0 3.33 4.66 3.66 5.0 4.04 3.0 Responsiveness to product enhancement requests 2.0 4.66 4.66 3.0 4.33 3.66 5.0 3.90 3.0

Communication 2.33 5.0 5.0 3.33 4.33 3.66 5.0 4.09 2.67 Pricing 3.0 4.66 5.0 2.66 4.33 3.66 4.66 3.99 2.34 Overall vendor satisfaction 2 2.33 5.0 5.0 3.66 4.33 4.0 4.66 4.14 2.67 Notes:

1. Product satisfaction scores were derived by averaging the results of the 21 product satisfaction components in Figure 3. 2. These scores do NOT represent the average of the other categories; this is a separate category where customers were asked to rate their overall satisfaction with

all aspects of their vendor relationship.



1.2 Detailed Survey Findings and Analysis

Product Satisfaction by Category

Survey participants were asked to rate their satisfaction with their vendor’s speech analytics solution for 21 key components, including: ease of use/configuration/maintenance, ease of integration with third-party applications, ease of creating searches and queries, accuracy and business relevance of results, ability to fine-tune results, call analysis capabilities, emotion detection/sentiment analysis capabilities, ability to conduct automated root cause analysis, ability to conduct discovery and surface previously unknown issues or new and breaking trends, ability to correlate seemingly unrelated issues, real-time capabilities, reporting and dashboards, ability to improve customer service, ability to improve the customer journey, ability to improve first contact resolution, ability to reduce operating costs, ability to improve adherence to regulatory requirements, ability to serve as an early warning system to identify issues before they escalate and impact a large number of customers, ability to improve customer retention, ability to increase sales and collections, and ability to realize the expected payback from the system. Figure 3 shows that most (39.5%) of the average satisfaction ratings for the 21 product categories surveyed fell into the highly satisfied (4.0 to 4.66) range, 24.5% fell into the satisfied range (3.0 to 3.66), 21.8% of the scores were completely satisfied (5.0), and 14.3% were only somewhat satisfied (2.33 to 2.66). Again it’s important to note that the ranges between the high and low score in 20 of the 21 categories were 2.0 or higher. This suggests significant disparities among the speech analytics solutions. CallMiner took the lead in the category of product satisfaction with an overall average score of 4.95. CallMiner achieved a perfect score, 5.0, in 18 of the 21 product component categories. Verint came in second place for product satisfaction with an average score of 4.71, and earned a perfect score in 9 of the categories. Calabrio was in third place with an average score of 4.39 and a perfect score in 5 of the categories.


Figure 3: Product Satisfaction Ratings by Category



Ease of use/configuration/ maintenance 3.44 4.33 5.0 3.0 4.0 3.66 4.66 4.01 2.0

Ease of integration with third-party applications 2.66 4.33 5.0 2.33 4.0 4.0 3.66 3.71 2.67

Ease of creating searches and queries 4.33 4.33 5.0 3.66 4.33 4.0 5.0 4.37 1.34

Accuracy and business relevance of results 3.66 4.0 5.0 4.0 4.0 3.0 5.0 4.09 2.0

Ability to fine-tune results 3.66 4.33 4.66 3.66 4.0 3.0 5.0 4.04 2.0

Call analysis capabilities 3.0 4.33 5.0 4.0 4.33 3.66 5.0 4.18 2.0

Emotion detection/sentiment analysis capabilities 2.0 3.33 4.66 2.0 3.66 2.33 4.0 3.14 2.66

Ability to conduct automated root cause analysis 2.0 3.66 5.0 2.66 4.0 3.0 5.0 3.61 3.0

Ability to conduct discovery and surface previously unknown issues or new and breaking trends

2.33 4.0 5.0 4.0 4.66 3.0 4.66 3.95 2.67

Ability to correlate seemingly unrelated issues 2.0 3.66 5.0 4.0 4.33 2.66 5.0 3.80 3.0

Real-time capabilities 2.66 4.33 5.0 2.0 4.33 4.0 5.0 3.90 3.0

Reporting and dashboards 2.66 4.66 4.66 3.0 4.33 3.66 4.66 3.94 2.0

Ability to improve customer service 3.0 5.0 5.0 4.0 4.66 4.0 4.66 4.33 2.0

Ability to improve the customer journey 3.0 4.66 5.0 3.66 4.66 3.33 4.33 4.09 2.0

Ability to improve first contact 2.66 4.33 5.0 3.66 4.66 3.0 4.66 3.99 2.34


Figure 3: Product Satisfaction Ratings by Category



resolution Ability to reduce operating costs 2.66 4.66 5.0 4.0 4.66 3.0 5.0 4.14 2.34

Ability to improve adherence to regulatory requirements 3.0 5.0 5.0 4.0 4.33 3.0 4.66 4.14 2.0

Ability to serve as an early warning system to identify issues before they escalate and impact a large number of customers

2.33 5.0 5.0 3.66 4.33 3.0 4.66 3.99 2.67

Ability to improve customer retention 2.66 5.0 5.0 3.66 4.33 2.33 4.66 3.94 2.67

Ability to increase sales and collections 3.0 4.33 5.0 3.66 4.33 2.33 4.66 3.90 2.67

Ability to realize the expected payback from the system 2.66 5.0 5.0 3.33 4.33 3.0 5.0 4.04 2.34

Vendor Average 2.82 4.39 4.95 3.42 4.29 3.18 4.71 3.96 2.13



About CallMiner

CallMiner helps businesses and organizations improve contact center performance and gather key

business intelligence by automating their ability to listen to every customer interaction. CallMiner’s

Eureka product portfolio automates the overwhelming process of monitoring information from 100%

of interactions – calls/audio, chat, email, surveys and social – to uncover consistent and reliable

information about agent performance. Real time business intelligence can be leveraged by

enterprises to dramatically improve customer service and sales, reduce the cost of service delivery,

mitigate risk, and identify areas for process and product improvements. For more information, please

contact CallMiner at (781) 547-5666, or visit www.callminer.com.

About DMG Consulting LLC

DMG Consulting LLC is a leading independent research, advisory and consulting firm specializing in

contact centers, back-office and real-time analytics. DMG provides insight and strategic guidance and

tactical advice to end users, vendors and the financial community. Each year, DMG devotes more than

10,000 hours to producing primary research on IT sectors, including workforce optimization (quality

management/liability recording), speech analytics, workforce management, performance

management, desktop analytics, surveying/voice of the customer, text analytics, cloud-based contact

center infrastructure, dialing, interactive voice response systems and proactive customer care. Our

actionable solutions are proven to deliver a lasting competitive advantage, and often pay for

themselves in as little as three months.

This reprint is excerpted from the 2014 – 2015 Speech Analytics Product and Market Report, which

was released in August 2014 with the permission of DMG Consulting LLC. More information about this

Report and DMG Consulting is available at www.dmgconsult.com.

http://www.dmgconsult.com/


© 2014 DMG Consulting LLC. All rights reserved. This Report is protected by United States copyright law. The reproduction, transmission or distribution of this Report in whole or in part in any form or medium without express written permission of DMG Consulting LLC is strictly prohibited. You may not alter or remove any copyright, trademark or other notice from this Report. This Report contains data, materials, information and analysis that is proprietary to and the confidential information of DMG Consulting LLC and is provided for solely to purchasers of this Report for their internal use. THIS REPORT AND ANY DATA, MATERIALS, INFORMATION AND ANALYSIS CONTAINED HEREIN MAY NOT BE DISCLOSED TO OR USED BY ANY OTHER PERSON OR ENTITY WITHOUT THE EXPRESS PRIOR WRITTEN CONSENT OF DMG CONSULTING LLC. Substantial effort went into verifying and validating the accuracy of the information contained within this Report, however, DMG Consulting LLC disclaims all warranties as to the accuracy or completeness of this information. DMG Consulting LLC shall not be liable for any errors or omissions in the information contained herein or for any losses or damages arising from use hereof.

speech analytics product and market report...

Documents