interaction mining: the new frontier of call center analytics

© 2011 interAnalytics 1

Interaction Mining: the new frontier of Call Center Analytics

Vincenzo Pallotta Rodolfo Delmonte Lammert Vrieling

David Walker


Outline

• Call Center Analytics• Automatic Argumentative Analysis for

Interaction Mining• Experiments with Call Center Data• Conclusions

3

CALL CENTER ANALYTICS

© 2011 interAnalytics


Call Center Analytics

• Call centers data represent a valuable asset for companies, but it is often underexploited for business purposes because:– it is highly dependent on quality of speech recognition

technology– it is mostly based on text-based content analysis.

• Interaction Mining as a viable alternative:– more robust– tailored for the conversational domain– slanted towards pragmatic and discourse analysis


Mainstream Call Center Analytics

Does not unveil real

insights about customer

satisfaction


Call Center Analytics: metrics and KPIs

• Agent Performance Statistics: – Average Speed of Answer, Average Hold Time, Call Abandonment Rate,

Attained Service Level, and Average Talk Time. – Quantitative measurements that can be obtained directly through ACD

(Automatic Call Distribution), Switch Output and Network Usage Data.• Peripheral Performance Data:

– Cost Per Call, First-Call Resolution Rate, Customer Satisfaction, Account Retention, Staff Turnover, Actual vs. Budgeted Costs, and Employee Loyalty.

– Quantitative, with the exception of Customer Satisfaction that is usually obtained through Customer Surveys.

• Performance Observation: – Call Quality, Accuracy and Efficiency, Adherence to Script,

Communication Etiquette, and Corporate Image Exemplification. – Qualitative metrics based on analysis of recorded calls and session monitoring

by a supervisor.


Four objectives

1. Identify Customer Oriented Behaviors, – which are highly correlated to positive customer ratings

(Rafaeli et al. 2007);

2. Identify Root Cause of Problems – by looking at controversial topics and how agents are able to

deal with them;

3. Identify customers who need particular attention – based on history of problematic interactions;

4. Learn best practices in dealing with customers – by identifying agents able to carry cooperative

conversations.


ARGUMENTATIVE ANALYSIS FOR INTERACTION MINING


Argumentative Structure of ConversationsDISCUSS(issue) <- PROPOSE(alternative)

1702.95 David: so - so my question is should we go ahead and get na- - nine identical head mounted crown mikes ? {qy} 61a

REJECT(alternative)1708.89 John: not before having one come here and have some people try it out . {s^arp^co} 61b.62a

PROVIDE(justification)1714.09 B: because there's no point in doing that if it's not going to be any better . {s} 61b+

ACCEPT(justification)

1712.69 David: okay . {s^bk} 62b

PROPOSE(alternative)

1716.85 John: so why don't we get one of these with the crown with a different headset ? {qw^cs} 63a

ACCEPT(alternative)1721.56 David: yeah . {s^bk} 63b1726.05 Lucy: yeah . {b} 1727.34 John: yeah . {b}

PROVIDE(justification)

1722.4 John: and - and see if that works . {s^cs} 63a+.64a 1723.53 Mark: and see if it's preferable and if it is then we'll get more . {s^cs^2} 64b1725.47 Mark: comfort . {s}

PROVIDE(justification)1714.09 John: because there's no point in doing that if it's not going to be any better . {s} 61b+

Why was David’s proposal on microphones rejected?


Automatic Argumentative Analysis

• Based on the GETARUNS system1.• Clauses in Turns are labelled with Primitive Discourse

Relations: – statement, narration, adverse, result, cause, motivation,

explanation, question, hypothesis, elaboration, permission, inception, circumstance, obligation, evaluation, agreement, contrast, evidence, hypoth, setting, prohibition.

• And then Turns are labelled with Argumentative labels:– ACCEPT, REJECT/DISAGREE, PROPOSE/SUGGEST,

EXPLAIN/JUSTIFY, REQUEST EXPLANATION/JUSTIFICATION.

1 Delmonte R., Bistrot A., Pallotta V.,Deep Linguistic Processing with GETARUNS for spoken dialogueUnderstanding. Proceedings LREC 2010 (P31 Dialogue Corpora).


Evaluation

Correct Incorrect Total Found Precision

Accept 662 16 678 98%

Reject 64 18 82 78%

Propose 321 74 395 81%

Request 180 1 181 99%

Explain 580 312 892 65%

Total 1826 421 2247 81.26%

Precision: 81.26% Recall: 97.53%

ICSI corpus of meetings (Janin et al., 2003)

Delmonte R., Bistrot A., Pallotta V.,Deep Linguistic Processing with GETARUNS for spoken dialogueUnderstanding. Proceedings LREC 2010 (P31 Dialogue Corpora).


EXPERIMENTS WITH CALL CENTER DATA


Rationale: implement the four objectives

1. Identify Customer Oriented Behaviors, 2. Identify Root Cause of Problems 3. Identify customers who need particular

attention 4. Learn best practices in dealing with

customers


The Data

• Corpus of 213 manually transcribed conversations of a help desk call center in the banking domain.

• Average of 66 turns per conversation.• Average of 1.6 calls per agent. • Collected for a study aimed at identifying

customer oriented behaviors that could favor satisfactory interaction with customers (Rafaeli et al. 2007).


Identify Customer Oriented Behaviors

• Based on the work of Rafaeli et al. 2006.• Customer Oriented Behaviors– anticipating customers requests 22,45%– educating the customer 16,91%– offering emotional support 21,57%– offering explanations / justifications 28,57%– personalization of information 10,50%


Significant correlation with argumentative labels


Identify Root Cause of Problems

• Cooperativeness score – a measure obtained by

averaging the score obtained by mapping argumentative labels of each turn in the conversation into a [-5 +5] scale.

• Sentiment Analysis module.

Argumentative Categories Cooperativeness

Accept explanation 5

Suggest 4

Propose 3

Provide opinion 2

Provide explanation/justification 1

Request explanation/justification 0

Question -1

Raise issue -2

Provide negative opinion -3

Disagree -4

Reject explanation or justification -5


Top 20 Controversial Topics with average cooperativeness scores and sentiment


Cooperativeness of speakers on top discussed topics


Identify problematic customers


Select a specific customer


Visualize a selected call


CONCLUSIONS


Conclusions

• New Generation Call Center Analytics requires Interaction Mining– Call Center Qualitative metrics and KPIs can be only

implemented with a full understanding of the customer interaction dynamics

• Argumentation is pervasive in conversations.– In order to recognize argumentative acts, advanced Natural

Language Understanding is necessary.• Future work:– Scalability: need to process millions of call per day!– Multi-language: call centers all over the world.


The Team

www.interanalytics.ch…find us at

interaction mining: the new frontier of call center analytics

Technology