modeling and analyzing ivr systems, as a special case of self
TRANSCRIPT
Modeling and Analyzing IVR Systems, as aSpecial Case of Self-services
Research Thesis
Submitted in Partial Fulfillment of theRequirements for the Degree of
Master of Science in Operations Research and System Analysis
Nitzan Carmeli
Submitted to the Senate of theTechnion - Israel Institute of Technology
Sivan, 5775 Haifa June 10, 2015
This Research Thesis Was Done Under the Supervision ofProfessor Haya Kaspi and Professor Avishai Mandelbaum in the Faculty of
Industrial Engineering and Management.
I would like to gratefully thank Professor Haya Kaspi and ProfessorAvishai Mandelbaum for their endless guidance and support, for giving methe wonderful opportunity to work with them and learn from them. It has
been a great privilege, and a pleasure.
I would also like to thank Dr. Galit Yom-Tov for her valuable guidance,Arik Senderovich, Yuval Michael and Anat Bernshtein for their
contribution to this work, and greatly thank the SEELab team: EllaNadjharov, Igor Gavako and Dr. Valery Trofimov for all their support and
assistance.
The Generous Financial Help of the Technion
is Gratefully Acknowledged.
Contents
1 Introduction 2
2 Literature Review 52.1 Methodologies for evaluating IVRs . . . . . . . . . . . . . . . . . 52.2 Designing and optimizing IVRs . . . . . . . . . . . . . . . . . . 62.3 Modeling a Call Center with an IVR . . . . . . . . . . . . . . . . 72.4 Stochastic Search in a Forest . . . . . . . . . . . . . . . . . . . . 82.5 Predicting Search Success in Self-Service Systems . . . . . . . . 9
3 Modeling Customer Flow as a Stochastic Search on a Tree 113.1 The Search Model . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.1.1 Building Blocks . . . . . . . . . . . . . . . . . . . . . . 123.1.2 State Space and Search Protocol . . . . . . . . . . . . . . 14
3.2 Properties of Candidates . . . . . . . . . . . . . . . . . . . . . . 173.3 Modeling the Search Protocol as a Rooted Graph . . . . . . . . . 24
4 Admissible Tree Creation 304.1 Algorithm Description . . . . . . . . . . . . . . . . . . . . . . . 314.2 Algorithm Formulation . . . . . . . . . . . . . . . . . . . . . . . 344.3 The Equivalence between Admissible Tree and the Set of Admis-
sible Candidates . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.4 Proofs of Lemmas 5–7 . . . . . . . . . . . . . . . . . . . . . . . 414.5 Proofs of Lemmas 8–10 . . . . . . . . . . . . . . . . . . . . . . . 44
5 Index Calculations Over the Admissible Tree 505.1 Index Calculations . . . . . . . . . . . . . . . . . . . . . . . . . 505.2 Finding Optimal Search Policy Algorithm . . . . . . . . . . . . . 52
5.2.1 Optimal Policy Algorithm . . . . . . . . . . . . . . . . . 54
6 Exploratory Data Analysis 566.1 Data Description . . . . . . . . . . . . . . . . . . . . . . . . . . 566.2 Demand for IVR Services . . . . . . . . . . . . . . . . . . . . . . 59
ii
6.3 Customer Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . 616.4 Success Rate of IVR Services . . . . . . . . . . . . . . . . . . . . 656.5 Further Support to an Abandonment Hypothesis . . . . . . . . . . 72
6.5.1 Customer Experience Effect on Time Spent in IVR Services 73
7 Model Implications 797.1 Comparing Different IVR Designs . . . . . . . . . . . . . . . . . 81
7.1.1 Estimating the Model Parameters . . . . . . . . . . . . . 817.1.2 Numerical Example . . . . . . . . . . . . . . . . . . . . 86
7.2 Further Implications . . . . . . . . . . . . . . . . . . . . . . . . . 96
8 Summary and Discussion 988.1 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
A IVR Data Table 101
B IVR Last Services Distribution 103
C Measurements - Reading Time of Menu Options 106
iii
List of Figures
1.1 Call center with an IVR . . . . . . . . . . . . . . . . . . . . . . . 3
3.1 An example of a rooted tree, representing an IVR system . . . . . 153.2 An example of a proper candidate represented by a path in GF .
Here G is taken from Figure 3.1. . . . . . . . . . . . . . . . . . . 283.3 A partial example of a Proper Graph created from an IVR tree G. 29
4.1 Example of a vertex in F 4exc . . . . . . . . . . . . . . . . . . . . . 33
4.2 Example of a partial Proper Graph and excluded paths . . . . . . 354.3 Example of a partial Admissible Tree . . . . . . . . . . . . . . . 374.4 Examples of Proposition 3, case IV . . . . . . . . . . . . . . . . . 394.5 Example of paths representing unwanted sequences, resulting from
adding vertices in F 2exc . . . . . . . . . . . . . . . . . . . . . . . 43
4.6 An example of Lemma 8 . . . . . . . . . . . . . . . . . . . . . . 464.7 An example of Lemma 8 . . . . . . . . . . . . . . . . . . . . . . 47
6.1 ILBank IVR menu . . . . . . . . . . . . . . . . . . . . . . . . . 586.2 ILBank Hybrid animation (link) . . . . . . . . . . . . . . . . . . 626.3 ILBank Network animation (link) . . . . . . . . . . . . . . . . . 636.4 Recent Account Activity duration, N = 1, 983, 161 . . . . . . . . 666.5 Account Summary duration, N = 1, 081, 992 . . . . . . . . . . . 666.6 Account Activity Today duration, N = 353, 328 . . . . . . . . . . 676.7 Credit Card Vouchers duration, N = 608, 529 . . . . . . . . . . . 676.8 IVR services duration . . . . . . . . . . . . . . . . . . . . . . . . 686.9 Fitting mixture of distributions, Recent Account Activity . . . . . 706.10 Fitting mixture of distributions, Account Summary . . . . . . . . 716.11 Distribution of the time in the ID phase, as a function of the number
of prior calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 736.12 Zoom in - Distribution of the time in the ID phase, as a function of
the number of prior calls . . . . . . . . . . . . . . . . . . . . . . 746.13 Distribution of the time in ‘Recent Account Activity’, as a function
of the number of prior calls . . . . . . . . . . . . . . . . . . . . . 756.14 Distribution of the time in ‘Recent Account Activity’,as a function
of the number of prior calls, zoom in from 0 to 20 seconds . . . . 76
iv
6.15 Distribution of the time in ‘Recent Account Activity’, as a functionof the number of prior calls, zoom in from 45 to 75 seconds . . . . 76
6.16 Distribution of the time in ‘Recent Account Activity’, as a functionof the number of prior visits to the service, across calls . . . . . . 77
6.17 Distribution of the time in ‘Recent Account Activity’, as a functionof the number of prior visits to the service, within one call . . . . 78
7.1 Calculating organizational profit - Simple example. . . . . . . . . 807.2 Original IVR design. Bold represents services with positive reward 897.3 Shallow IVR design. Bold represents services with positive reward 907.4 Deep IVR design. Bold represents services with positive reward . 91
A.1 IVR data table . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
B.1 Recent Account Activity duration . . . . . . . . . . . . . . . . . 103B.3 Account Summary duration . . . . . . . . . . . . . . . . . . . . . 104B.5 Account Activity Today duration . . . . . . . . . . . . . . . . . . 104B.7 Credit Card Vouchers duration . . . . . . . . . . . . . . . . . . . 105
C.1 Time measurement of ILBank IVR menu options . . . . . . . . . 108
v
List of Tables
6.1 Total number of IVR calls by their outcome, May 2008 to June 2009 576.2 Total number of IVR calls, by customer type, May 2008 to June
2009 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576.3 IVR services - relative demand frequency, May 2008 to June 2009 606.4 Frequent paths . . . . . . . . . . . . . . . . . . . . . . . . . . . 646.5 Fitting mixture of distributions, Recent Account Activity . . . . . 706.6 Fitting mixture of distributions, Account Summary . . . . . . . . 716.7 Statistics of time in the ID phase, as a function of the number of
prior calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
7.1 Average sojourn time within ILBank IVR, May 2008 to June 2009 857.2 Average patience, by customer type, based on Khudyakov et al.
[18] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 867.3 Rewards and costs, by Aksin et al. [2] . . . . . . . . . . . . . . . 877.4 Laplace transform for successful and unsuccessful service durations 887.5 Results, optimal paths and expected utility, High priority . . . . . 927.6 Results, optimal paths and expected utility, Medium priority . . . 937.7 Results, optimal paths and expected utility, Low priority . . . . . 947.8 Numerical example - average number of relevant calls by customer
type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 957.9 Organizational revenue . . . . . . . . . . . . . . . . . . . . . . . 95
vi
GlossaryAbbreviation Full FormIVR Interactive Voice ResponseEDA Exploratory Data Analysis
Notation1 ExplanationG = (V,E) A rooted tree.M The set of tree leaves.s The root vertex of G = (V,E).A(i) The set of all immediate successors of vertex i.pre(i) The immediate predecessor of vertex i.dep(i) Depth of the unique path leading from the root s to vertex i.Γ(i) The sub-tree spanning from vertex i.MΓ(i) The set of leaves in the sub-tree spanning from vertex i.c Cost per unit of time.τ ∼ exp(θ) Customer patience.Pi Success probability of vertex i.ri Reward earned from a successful visit at vertex i.tser(i) Time spent in vertex i given a successful visit.tF (i) Time spent in vertex i given an unsuccessful visit.Ti Time spent in vertex i.N(j) The number of times that option (vertex) j was previously ex-
plored.tN(j)i,j The exploration time of edge e = (i, j) ∈ E, given that option j
in menu i was previously explored N(j) times.Placei(j) The place of option j in menu i.li The leftmost son of vertex i.N A vector representing the number of repeated explorations of each
vertex.R∗(v) The expected utility of the tree spanning from vertex v.R∗u, v The index of edge (u, v).LX(s) The Laplace transform of X.
1Listed here are only notations which appear in more than one chapter. The rest are explained intheir relevant chapter.
vii
Abstract
Call centers play a prominent role in today’s economy. They serve as the maincustomer contact channel in various enterprises, which makes them highly labor-intensive operations. Thus, call centers look for means to reduce the number ofagents handling calls, and trying to do so without degrading service level. Inter-active Voice Response (IVR) systems are presently one of the main self-servicechannels employed by call centers. They are used as means to reduce operatingexpenses derived from agent employment costs.
The goal of our research is to improve and enhance IVR systems, aiming tocreate a body of knowledge that will generalize to other self-service systems. Wemodel customers flow within an IVR system as a stochastic search in a directedtree. The search goal is to find the optimal path on the IVR tree, which will resultin maximal expected discounted utility for customers. We show that a calculableindex can be assigned to each feasible option, and the optimal policy is to choosethe option with the highest index at each stage.
Our model building blocks were created through an Exploratory Data Analysis(EDA) of real IVR transactions, in a call center of a large Israeli bank. The EDArevealed interesting phenomena regarding customer abandonments and learningwithin the IVR.
Our work enables the comparison between alternative IVR designs, both fromthe customer and the enterprise point of view. This complements related researchin other fields, such as Human-Factor-Engineering and Telecommunication.
The model for IVR systems that we developed can easily be implemented toother self-service systems such as Internet websites which have become prevalent.
1
Chapter 1
Introduction
Call centers play a prominent role in today’s economy. Indeed, they serve asthe main customer contact channel in various enterprises [1], public or private,product-based or service-based. Call centers are also highly labor-intensive; theyemploy sometimes hundreds, or even thousands of Customer Service Representa-tives (CSRs or Agents) to handle incoming calls. Typically, 60%-70% of the over-all operating expenses of call centers are derived from agent employment costs[10]. Reducing the number of agents handling calls, without degrading servicelevel, is thus of interest and importance, and enabling customers to self-serve isone of the basic means for doing so. As customers self-serve, the agent workloadis being reduced, and less agents are required in order to maintain a certain servicelevel. Interactive Voice Response (IVR) systems, also known as Voice ResponseUnits (VRU), are one of the main self-service channels [18], along with Internetwebsites and designated smart-phone applications.
IVR systems, if properly designed, can increase customer satisfaction andloyalty, cut staffing costs and increase revenue by extending business hours andmarket reach [3]. Poorly designed IVR systems, on the other hand, will cause theopposite effect and lead to dissatisfied customers, increased call volume and mighteven increase agent turnover, as agents would serve frustrated customers [7]. APurdue University study showed that more than 90% of US consumers are formingtheir image on a certain company based on their experience with its call center.Furthermore, more than 60% stopped using the products of a company in whichthey had a negative call center experience [7]. Since the IVR system is the frontgate of most call centers, having an effective, efficient, and customer-friendly IVRsystem is extremely important.
The goal of our research is to improve and enhance IVR systems, aiming togeneralize to other self-service systems. To do so, we model and analyze customerflow within an IVR system. The model building blocks were established and in-spired by an Exploratory Data Analysis (EDA) of real IVR transactions in a callcenter of a large Israeli bank, based on more than one year of data which includesmillions of calls. The theoretical basis for our model then relies on the work of
2
Denardo et al. [8] and Granot and Zuckerman [12], both using stochastic search ina forest to model R&D project management.
In our work, we represent the IVR system as a rooted tree and model customerflow within it as a stochastic search. We model the search of a group of customers,which is commonly characterized by its perceived rewards and costs, its successprobabilities, its service time distribution within each segment of the system, andits patience distribution. Similarly to [8] and [12], the goal of our search is to findthe optimal path in the IVR tree, which will result in maximal expected discountedutility for customers within each group. We also show that, at each stage, an indexcan be assigned to each feasible edge, and the optimal policy is to chose the edgewith the highest index at each stage.
Our EDA revealed interesting phenomena, suggesting that substantial modi-fications to [8] and [12] are needed in order to appropriately describe customerflow within an IVR system. For example, we realized that customers reaching oneof the offered services in an IVR system, may abandon this service shortly after-wards. Thus reaching one of the tree leaves in itself does not guarantee a successfulcompletion of the search. In addition, the search may visit multiple leaves within asingle IVR visit. This led us to expand our state space into three-dimensional statesin order to maintain the tree structure.
System Description
Figure 1.1 presents a scheme of a call center with an IVR system. A customer
IVR
Abandonm
ent
Success
Returns
Agents
Arrivals
Figure 1.1: Call center with an IVR
usually starts the service in the IVR and then, if necessary, continues to an agentservice. The call may be either connected immediately or queued. Some customersmay abandon the queue as their waiting time exceeds their patience. After receiv-ing a service from an agent, the customer might be transferred to a different agent,
3
back to the IVR or simply exit the system. One of the main observations derivedfrom the EDA was that some customers leave the IVR system without getting anyrelevant information. These customers may either leave the call center or join theagents’ queue (opt-out) to receive the desired service. In both cases, we say thatthese customers abandon the IVR service.
When customers are self-served, finding whether their service was successfulis not an easy task. The subject of identifying abandonments from self-servicesystems, such as IVR, is thus of interest and we shall address this issue in ourwork.
Optimizing IVR Systems
Modeling and analyzing customer flows within the IVR is important for optimizingIVR design, solve usability problems, shorten service durations, decrease abandon-ment and opt-out rates and improve routing to the right group of agents. The subjectof optimizing IVR design has been addressed in the literature from various aspects.In the Human-Factor-Engineering (HFE) field, for example, we surveyed many pa-pers dealing with optimal IVR architecture, mainly comparing broad and shallowIVR designs [6, 15, 19, 21]. In other research fields, algorithms for designing anIVR system which minimizes its service time were presented [20]. Our modelenables the comparison between alternative IVR designs, both from the customerpoint of view and from the enterprise point of view, thus complementing researchin other fields such as Human-Factor-Engineering and Telecommunication Engi-neering. An example for such a comparison is given in Chapter 7.
Although this research focuses on IVR systems, we believe that both the the-oretical model and some of the methods presented in our EDA can be easily im-plemented in other self-service systems, which becomes more and more relevantthese days, such as Internet websites. To support this view, we refer to the work ofHassan et al. [13] on predicting search success in Web search engines.
Thesis Structure
The rest of this Thesis is structured as follows: Chapter 2 is a literature review ofrelated research. In Chapter 3, we describe our search model, define the search can-didates and specify the modeling of the search protocol as a rooted tree. In Chapter4, we present an algorithm called ‘Admissible Tree Algorithm’: it constructs arooted tree representing all admissible, possibly optimal, search candidates. Weassign indices to the edges of the resulting rooted tree in Chapter 5 and show howto infer the optimal search policy via a simple algorithm. Our EDA is presented inChapter 6. In Chapter 7 we discuss a numerical example that compares three IVRdesigns, based on the search model and the exploratory data analysis described inthe former chapters. We conclude and discuss further implications of our work, aswell as future research directions in Chapter 8.
4
Chapter 2
Literature Review
We cover the following subjects that pertain to our research: methodologies forevaluating IVRs, designing and optimizing IVRs, modeling a call center with anIVR, stochastic search in a forest, and predicting search success in self-servicesystems.
2.1 Methodologies for evaluating IVRs
Suhm and Peterson [23][24] presented a comprehensive methodology for IVR us-ability evaluation and redesign. It is claimed there that the standard IVR usabilitytests and standard IVR reports are not sufficient for assessing the true performanceand usability of an IVR. Most of the reports being used by call center managers arebased on measures related to IVR utilization, for example, the fraction of customersthat left the IVR without seeking a live agent, where this fraction is interpreted asthe success rate of the IVR service. After analyzing thousands of end-to-end calls,which included the interaction with the IVR and the customer-agent dialogs, it wasdiscovered that although 30% of the customers completed their service in the IVR,only 1.6% of the customers actually got relevant service.
In [23] they also presented a new measure for quantifying IVR usability andcost-effectiveness. This measure is defined as the agent time being saved by han-dling the call, or part of the call, in the IVR, compared to handling the call onlyby a live agent. It was also suggested that, by using User-Path diagrams, one canidentify usability problems, such as nodes in the IVR that are rarely visited, nodeswith high volume of abandonment and nodes with high volume of customers seek-ing live agent assistance. By analyzing end-to-end calls, one can also comparethe categorical distribution of original call reasons, which is revealed through thecustomer-agent dialogs, to the call-type distribution that arose from the IVR logs[24]. Differences between those two categorical distributions indicate that cus-tomers are not navigating correctly within the IVR.
5
2.2 Designing and optimizing IVRs
The subject of improving and optimizing IVR design has been addressed from dif-ferent aspects. One of the main issues that appears in Human-Factor-Engineering(HFE) research is IVR architecture – mainly, comparing broad (shallow) designsand deep (narrow) designs.
Schumacher et al. [21] recommended that IVRs with command-like optionsare to be limited to four or less items in each menu. The authors claim that sinceall the output is auditory, broad designs, with more than four options per menu,place a heavy demand on the working memory and therefore should be avoided.Marics and Engelbeck [19] also recommend that menus are to be limited to fourcommands (not including global commands such as Help and Exit). Both papersare relying on Miller’s work from 1956 which basically argued that the number ofobjects an average human can hold in working memory is 7±2. In [19] it was alsostated that menu items should be ordered according to frequency of use, unless themenu choices have a distinct natural or functional order.
Commarford et al. [6], on the other hand, argued that broad designs do notoverload the working memory. They claim that the user does not need to rememberall the options in the menu but to hold the best option, compare it with the new oneand then save the better option. Therefore, the user only needs to hold up to twooptions in the working memory. Creating a deep, divided design in order to limitthe number of options in every menu can increase complexity and create confusionbecause it might not fit the user’s mental model.
Huguenard et al. [15], and [6], both conducted experiments showing that cus-tomers who used broad IVR design performed tasks faster, with greater satisfactionand with lower error rate than users who used deep IVR designs. These results chal-lenge the belief, which relies on Miller (1956), that menus should contain fewerthan 7± 2 items.
Other researches address the issue of optimizing IVR design by presenting al-gorithms to reduce the service times in the IVR. Salcedo-Sanz et al. [20] introducedan evolutionary algorithm to optimally design IVRs, based on Dandelion encoding.Specifically, the IVR is considered to be a service tree, where each announcement(menu) is a node and each service is a leaf. The tree edges represent the differentoptions in each announcement. The algorithm assumes that the time spent in eachannouncement is linearly related to the number of options in each announcement,and that a customer listens to all the options in the announcement before making aselection. The algorithm aims to reduce the average time to reach a desired service.If M is the number of services, ti is the time required to reach a certain service i,and pi is the probability that a customer will ask for service i, then the optimal
IVR design will minimize the function f(T ) =M∑i=1
tipi . The basic idea behind
the algorithm is to associate the most frequently requested service with the shortestpath. The problem is equivalent to assigning code words to a set of messages tobe transmitted, such that the mean code word length is minimized. The suggested
6
algorithm was tested in a real call center of an Italian mobile telecommunicationscompany and in synthetic experiments. The results showed that the algorithm isable to obtain results which are very close to a lower bound for the problem, al-though it was not proved that the algorithm will yield optimal results. The lowerbound of the problem was derived using the noiseless coding theorem: it states that“the average codeword length l, using an alphabet of k symbols, is always largerthan the uncertainty measure represented by the entropy of the system”, which,
under the assumptions of [20] IVR tree, means: l ≥ −M∑i=1
pi · logk(pi). Therefore,
the average time to reach a desired service, for a k-ary tree with M services whose
probabilities are p1 ≤ p2 ≤ . . . ≤ pM , is: T = lkdann ≥ −kdannM∑i=1
pi · logk(pi),
where dann is the common duration of each option in the announcement.
2.3 Modeling a Call Center with an IVR
Srinivasan et al. [22] used a Markovian model of a call center with an IVR to de-termine the number of trunk lines (N) and agents (S) required to meet a certainservice level. Specifically, this measure is defined by the probability of wait foran agent (after the IVR service) and the probability that an arriving call will beblocked (all lines being busy). If a call arrives to the call center and all trunk linesare occupied (busy signal), the call is lost. Otherwise, the call spends some timein the IVR and then can either request an agent service with probability p, or leavethe system with probability 1 − p. If there is no available agent, the call will jointhe queue. Khudyakov et al. [17] expanded the model presented by [22] and addedcustomer impatience to the model. The model parameters and assumptions are:Arrivals according to a Poisson process with constant rate λ; IVR processing timesare i.i.d exponential random variables with rate θ; agent service times are i.i.d ex-ponential random variables with rate µ; customer patience (while waiting in queuefor an agent service) is exponentially distributed with parameter δ. An asymptoticanalysis in the QED (Quality and Efficiency Driven) regime was performed in [17].The asymptotic analysis provided QED approximations of frequently-used perfor-mance measures (such as the waiting probability, the probability of a busy signaland the mean waiting time given waiting).
Behzad and Tezcan [4] proposed a model of a call center with a flexible IVRsystem. They assumed that each call can be routed into one of two IVR designs.The specific design of each system was not considered. To quantify the differencesbetween the two designs they focused on three performance measures: call reso-lution probability – proportion of calls completing the service in the IVR; Opt-outprobability – proportion of calls that were transferred to a live agent; Abandonmentprobability – proportion of calls abandoning from the IVR. The IVR performancemeasures are affecting the call center staffing. For example, if the call resolution ishigh, then fewer agents are needed and vice versa. Therefore, IVR design must be
7
synchronized with call center staffing. A two-stage stochastic program was formu-lated in order to determine the optimal staffing level and the proportion of calls thatshould be routed to each IVR system, in a way that will minimize total costs. Thetotal costs consist of agent cost and abandonment cost; thus the objective functionis:
minN,π
cN + aEΛ[AbΛ],
where c is the cost of each agent per unit of time, a is the cost per abandonment(either from the IVR system or from the queue while waiting for an agent service),and AbΛ is the rate customers abandon the system in steady state, given that thearrival rate is Λ. In [4], the staffing level N is determined first, and then, oncethe exact arrival rate is known (with N being fixed from the previous stage), onedetermines how to route customers to the two different IVR systems.
Feigin [9] presented a statistical analysis of customer patience in a call center ofa US bank. The system considered was the queue of customers waiting to receiveagent service, after completing the IVR and the post-IVR phases. The post-IVRphase may consist of announcements, made by the system, which are designed towarn the customers of a heavy load in the system. In times of heavy system load,the system announces an expected waiting time (< 1 min, 1 min, 2 min, etc.) andrecommends that the customer return to the IVR system. It was shown that lessthan 1.5% of the customers who heard the announcement actually returned to theIVR.
One of the factors that may affect customers’ patience is how much time theyhave already invested in the call, namely, how much time they spent in the IVRand the post-IVR phases. Feigin compared the patience of customers who spent ashort time in the IVR (less than 100 sec) with the patience of customers who spenta longer time in the IVR, by analyzing their survival probability. There was a clearseparation between the two survival curves. Customers who invested more timein the IVR were more patient while waiting for an agent service. This fact shouldinfluence operation management, especially when aiming to minimize abandon-ment. It suggests that the priority of customers entering the agent queue should beinversely related to the time they had already spent in the IVR and in the post-IVRphase. This means that customers who tend to be less patient in the IVR and post-IVR phases will be given a higher priority in the agents’ queue and will wait less,while customers who tend to be more patient could wait longer.
2.4 Stochastic Search in a Forest
Denardo et al. [8] and Granot and Zuckerman [12] presented models for a stochas-tic search in a forest, in the context of research and development projects. In [12]it was assumed that a project consists of a collection of R&D activities, for whichthe precedence relationship is well defined. At each stage, one activity is chosenfrom a set of feasible activities. The project state is defined by the set of suc-cessfully completed activities. The model was described as a directed tree, where
8
edges represent activities and vertices represent the project state. This means thatvertex i is compatible with a successful completion of the sequence of activitiesrepresented by the edges leading from the source vertex to vertex i. At every ver-tex i, one can choose either to; (a) end the project and receive a reward of ri, (b)continue to one of the activities represented by edges emanating from vertex i. Ifan edge is successfully attempted, it can be followed by either attempting one ofits immediate successors or by termination. An unsuccessful attempt of an edgecan be followed by attempting another edge emanating from the same vertex, or bytermination. They defined a policy as a program specifying the next activity to bechosen at every stage as well as a stopping rule for ending the project. The goal isto find a policy maximizing the expected discounted net utility of the project. Theyassigned a ratio to each edge, and showed that the optimal policy is to attempt theavailable edges at each stage by a non-increasing order of their ratio.
The search protocol in [8] extended the search protocol described by [12]. Theydefined a state S as a set of edges in which no edge is a predecessor of the other.The project is in state S if the predecessors of all the edges in S have been at-tempted successfully, and no edge in S has been attempted yet. While in state S,any edge in S can be chosen, as well as the option to terminate the search (project).Their goal was to maximize the expected utility, with both linear utility functionand exponential utility function. As in [12], they also assigned a ratio to each edge,which was considered to be the edge index. They showed that an optimal policyis to attempt, at each stage, a feasible edge whose index is the largest, with stop-ping when a leaf is successfully attempted or when all feasible edges have negativeindices.
2.5 Predicting Search Success in Self-Service Systems
Hassan et al. [13] addressed the issue of predicting success of user searches in Websearch engines. They defined a search goal as an “atomic information need, result-ing in one or more queries”. Given a search goal, their objective was to predictwhether that goal was successfully reached or not. User search is composed of aset of queries and several (possibly zero) clicks on search results for each query.It can also be represented by an ordered sequence of user actions along with thetime between them. The authors have extracted patterns describing user behaviorfor each search goal and used those patterns to construct two Markovian models.The first model characterizes user behavior in successful goals, and the other onecharacterizes user behavior in unsuccessful goals. Given a new user goal, they es-timated the log likelihood that this sequence of actions was generated from each ofthe models. The ratio between those two log likelihood functions was then used topredict the goal success. The authors also showed that the transition times in a se-quence of actions are very important in predicting the goal success. It was assumedthat there is a specific distribution that governs the duration of time users spend ateach transition, for successful and unsuccessful goals, where long durations are
9
associated with success. Given a new goal, they estimated the likelihood that thetransition times were generated from the success distribution and the likelihoodthat they were generated from the failure distribution. Again, the ratio betweenthose two likelihood functions was used as a predictor of goal success. The authorswere able to show that using models of user behavior, including the sequence of allqueries and clicks in a search goal, along with the time spent in each transition, isa better predictor for goal success than the common predictors which are based onquery-url relevance.
10
Chapter 3
Modeling Customer Flow as aStochastic Search on a Tree
An IVR system usually offers several services. Customers enter the IVR and thenfollow a series of menus in order to reach a desired service or services. In thischapter we present a model for customer flow within an IVR system. Recall thatwe perceive the path of an individual customer within the IVR as a representativeof a group of customers with the same characteristics. In Chapter 5, we show howthe model presented here can be used to identify the optimal policy of each indi-vidual customer. An optimal policy is the customer path within the IVR systemthat will yield maximum utility for the customer. That is, the optimal policy stateswhich IVR services will be visited by the customer, in what order, and what will bethe path from one service to the other. In Chapter 7, we discuss the implications ofthe model presented here on optimizing IVR design, not just at the customer levelbut also at the organizational level.
Our model relies on the following assumptions:
1. Each customer may be seeking several IVR services; however the number ofdesired services by each customer is relatively small (formally, up to 5).
2. Customers have finite patience. This means that, after some exogenously-given time, they abandon the IVR system, if their service at the IVR has notbeen completed by then.
3. A customer path within the IVR constitutes a series of menus, each withseveral options, ultimately leading to an IVR service. At each menu, thecustomer can choose one of the menu options. At each time (either whilein the menu or the IVR service), the customer may choose to go back to theformer menu or back to the main menu (the first menu in the IVR system).The customer may also choose to end the IVR service and leave the callcenter, or to end the IVR service and opt-out to an agent service.
11
4. As customers gain experience with the IVR system, the time they spendlistening to menu options can only be reduced. Nonetheless, the order ofthe menus and the order of the options in each menu is predetermined. Thismeans that the time it takes to reach a specific option within a menu alwaysexceeds the time it would take to reach any of the prior options in the samemenu.
We model the IVR system as a rooted tree, and model customer paths within theIVR as a stochastic search on that tree.
3.1 The Search Model
Let G = (V,E) be a rooted tree. This means that G is a directed acyclic graph,and every vertex in G, except for the root vertex, has an in-degree of one. Thegraph G represents an IVR system. Let M ⊂ V be the set of tree leaves. It willbe occasionally convenient to denote |M | = m. Each vertex i ∈ M represents aservice offered by the IVR system. All other vertices (non-leaves) in G representIVR menus. If there is a directed edge from i ∈ V to j ∈ V , then i is called animmediate predecessor of j, and j is called an immediate successor of i. Movingalong edge e = (i, j) ∈ E means choosing the option represented by j in menui. The root vertex, denoted by s, represents the IVR main menu. Note that eachvertex j (except for the root) plays a dual role: of a menu and a menu option (foreach immediate predecessor of j).
As in the model of Granot and Zuckerman [12] and the model of Denardo etal. [8], our search model takes the form of a sequential decision process. Cus-tomers start their search at the root vertex s. A vertex could be explored only if allof its predecessors have been successfully explored. Hence, at the first step, onlythe immediate successors of the root vertex can be explored. If a vertex is success-fully reached, then the next available vertices are its immediate successors (movingforward), its immediate predecessor and the root vertex (moving backward). Cus-tomers may be looking for several services. If the exploration of a desired leaf(IVR service) is successful then a reward is gained. However, after reaching one ofthe desired services, this service will no longer be sought after. Hence, no rewardwill be earned from a leaf after the first time it was explored.
3.1.1 Building Blocks
Our search model requires some terminology and notations that we shall now re-view:
• A(i) - The set of all immediate successors of vertex i.If i ∈M , then A(i) = ∅.
• pre(i) - The immediate predecessor of vertex i.Since G is a rooted tree, every vertex i ∈ V \ s has exactly one immediate
12
predecessor. The root vertex s has no predecessors.
• dep(i) - Depth of the unique path leading from s to i.
• Γ(i) - The sub-tree spanning from vertex i.
• MΓ(i) - The set of leaves in the sub-tree spanning from vertex i.
• c - Cost per unit of time.
• τ - Customer patience. We shall assume that τ is exponentially distributedwith parameter θ: E[τ ] = 1
θ . Formally: τ ∼ exp(θ).
Each leaf i ∈M has the following properties:
• Pi - Success probability of leaf i, 0 < Pi < 1.
• ri - Reward earned from a success at leaf i, ri ≥ 0.
• tser(i) - Time spent in leaf i given that we have been successful in visit-ing it, tser(i) > 0.
• tF (i) - Time spent in leaf i given that we have not been successful invisiting it, tF (i) > 0.
• Time spent in leaf i, Ti, is given by Ti =
{tser(i) w.p. Pi
tF (i) w.p. 1− Pi
For each vertex i ∈ V \M : Pi = 1 and ri, tser(i), tF (i) and Ti are all equalto zero.
The time a customer spends in an IVR menu, i, in order to choose one of itsoptions, is a random variable which depends on the number of options in menu i,the place (rank-order) of option j in menu i, and the customer familiarity with themenu and its options. We denote:
• N(j) - The number of times that option (vertex) j is explored.
• tN(j)i,j - The exploration time of edge e = (i, j) ∈ E, given that optionj in menu i was previously exploredN(j) times.
We shall assume that the options in each IVR menu are ordered from right toleft, such that the rightmost option is given first and the leftmost option is givenlast.
Definition 1. Option k will be called a left brother of j if both are options of thesame menu, and k is ordered after j (placed to its left). Equivalently, option j willbe called a right brother of k.
13
Denoting:
• Placei(j) - The place of option j in menu i.
• li - The leftmost son of vertex i
Let k be a left brother of j. Then, by the underlying model assumptions:
t0i,kd≥ t0i,j ,
N(j) ≥ N(k), at all times,
tni,jd≤ tmi,j , if n > m, and,
tni,jd≤ tmi,k, if n ≥ m.
(3.1)
Figure 3.1 is now used to illustrate the terminology and notations presentedabove. For example, if the rooted tree shown in Figure 3.1 is G = (V,E), then:V = {a, b, ...}, E = {(s, a), (s, b), (a, c), ...}, M = {b, d, e, f, g}, A(a) ={c, d, e}, pre(c) = a, dep(c) = 2, Γ(a) is the sub-tree of G colored in red andMΓ(a) = {d, e, f, g}.
3.1.2 State Space and Search Protocol
Our state space has 3 components: The current vertex i, the total time spent in thesearch T , and a (|V | − 1) × 1 vector representing the number of repeated explo-rations of each edge, which will be denoted as N (component (j) of this vector isN(j), which was described above). Hence, formally, each state will be denotedby:
S =(i, T,N
).
Recall that customers have a finite patience, τ ∼ exp(θ). An attempt of anoption represented by edge e = (i, j), j ∈ A(i), while in state S =
(i, T,N
),
may result in a success, if τ ≥ T + tN(j)i,j , or failure if τ < T + t
N(j)i,j . Note
that if a customer reached state S =(i, T,N
), then the patience of that customer
exceeds T ; formally, τ > T . Therefore, the success probability of attempting edgee = (i, j), j ∈ A(i), while in state S =
(i, T,N
), is:
P(τ > T + t
N(j)i,j |τ > T
)=P(τ > T + t
N(j)i,j
)P (τ > T )
= P(τ > t
N(j)i,j
)due to the memoryless property of τ .
14
s
ab
cde
fg
( ), ,e e eP r T
( ),
N ea et
τ
Patience
Main menu
menu
service
( ) 1aPlace c =
Figure 3.1: An example of a rooted tree, representing an IVR system
The initial state(s, 0, 0
)will be denoted as S0.
While at state S =(i, T,N
), the customer can select one of the following ac-
tions:
1. Terminating the search and leaving the system;
2. Terminating the search and opting out to an agent;
3. Moving forward to one of i′s immediate successors, j ∈ A(i).
4. Moving backward to i′s immediate predecessor, j = pre(i).
5. Moving backward to the root vertex s.
We now elaborate on each of these 5 actions.
Termination: Choosing the first option will terminate the search and add no ad-ditional cost or reward. Choosing the second option will terminate the search andadd additional reward (or cost) of ropt−out.
Moving forward: If a customer in state S =(i, T,N
)attempts edge e = (i, j),
j ∈ A(i), and the attempt is successful, a transition occurs from state(i, T,N
)to
state(j, T + t
N(j)i,j + Tj , N + δi,j
), where δi,j is a (|V | − 1)× 1 vector defined as
15
follows:
δi,j(x)4=
1 j ∈ A(i) and(x = j or Placei(x) ≤ Placei(j)
)0 else.
Recall that, in this case, the attempt of j is equivalent to hearing and choosingoption j in menu i. From the underlying model assumptions, in order to hearoption j one must listen to all the options prior to it. Hence, the attempt of optionj ∈ A(i), results in an increase in all of N components corresponding to both jand its right brothers.
The attempt of j ∈ A(i) incurs cost of c per each unit of time spent explor-ing the edge (which means the time spent listening to option j and all its rightbrothers). If j ∈ M , one incurs an additional cost of c per each unit of time spentexploring vertex j, and a reward of rj is earned with probability Pj . (As mentionedbefore, if j was previously explored then no reward will be earned). If the attemptfails, the search is over, but still incurs cost of c per each unit of time exploring theedge prior to failure.
Moving backward: It will later turn out useful to conceptualize a backward tran-sition from i to pre(i), or from i to s, as adding respectively additional edges,(i, pre(i)) and (i, s), to E. When a customer reaches a new menu, the optionsof going backward (to a former menu or to the main menu), are considered onlyafter hearing all the options in that menu. Hence, the edges (i, s), (i, pre(i)) areplaced after li, the leftmost son of vertex i. Consequentially, Placei(pre(i)) =Placei(s) = Placei(li)+1. We therefore assume that the times it takes to exploreboth edge (i, pre(i)) and edge (i, s) equals the time it takes to explore the edge(i, li) plus a random variable representing the time it takes to decide on a backwardtransition after hearing the last option in the menu. Let tback be that last randomvariable; tback is assumed independent of all other variables and parameters. Thus,if u = s or u = pre(i):
tN(li)i,u = t
N(li)i,li
+ tback.
If i ∈M , i has no successors (sons), then, ti,pre(i) and ti,s are both simply equal totback.
If a backward transition from i to u = pre(i) or to u = s succeeds, a transitionoccurs from state
(i, T,N
)to state
(u, T + t
N(li)i,u , N + δi,li
). In this case, the
attempt of u is equivalent to hearing all the options in menu i and then decidingto go backward. Again, based on the underlying model assumptions, the attemptof u results in an increase in all of N components corresponding to immediatesuccessors of vertex i. Recall that if i ∈ M it has no successors, then δi,li = 0.Thus, if a backward transition occurs from a vertex i ∈ M , there is actually nochange in the vector N .
16
If a backward transition fails then the search is over. One way or the other, thisattempt incurs a cost of c per each unit of time spent exploring the edge (beforesuccessfully reaching u or before failure).
3.2 Properties of Candidates
In Section 3.1.2, we have defined the search protocol over the IVR tree representedby G. Following the terminology in [8], a candidate will now be defined as asequenceU = (i1, ..., in) of vertices in V . Recall that our goal is to find the optimalsearch policy that will yield maximum utility for the customer. A policy σ specifiesthe decision made at each state during the search - either moving to another feasiblestate, terminating the search (denoted by ter) or opting out to the agents queue(denoted by opt-out). An optimal policy is thus equivalent to a feasible sequenceof vertices in G with maximal utility.
Definition 2. A candidate U is proper if it starts with s, and if every two adjacentvertices in U , ik, ik+1, satisfy one of the following conditions:
ik+1 ∈ A(ik), or
ik+1 = pre(ik), or
ik 6= s , ik+1 = s.
Let Λ denote the set of all proper candidates.
Accordingly, a proper candidate U ∈ Λ, is a sequence of vertices which com-plies with the search protocol represented in Section 3.1.2. Meaning, a propercandidate represents a sequence of feasible decisions which is derived from a cer-tain search policy.
Let νσ(S) be the expected marginal utility obtained under policy σ when thedecision process starts with state S. Let Φ(S) be the optimal utility of state S, thatis:
Φ(S) = max {νσ(S) : all σ}
Consequentially, the optimal search policy is defined as:
σ∗ = argmaxσ{νσ(S0) : all σ}
Let X(U) be the random variable representing the net utility of exploring acandidate U , and R(U) = E [X(U)].
As in [8], the optimal search policy (starting at S0) σ∗ is equivalent to a propercandidate U ∈ Λ, with maximal utility. Φ(S0) can now be defined by:
Φ (S0) = max {R(U) : U ∈ Λ} .
17
Namely, an optimal candidate will satisfy:
R(U) = Φ(S0)
In Chapter 4 we introduce an algorithm that constructs a rooted tree from allproper candidates which may be optimal. In Chapter 5 we present an algorithm,following the concept of the ’Algorithm for Finding an Optimal Policy’ presentedin [12], to find the optimal policy σ∗ over this rooted tree.
Definition 3. A direct sequence from vertex i to vertex j is a (possibly empty)sequence of vertices starting with i and ending with j, in which each vertex isan immediate successor of the previous one. A backward direct sequence fromvertex i to vertex j is a (possibly empty) sequence of vertices starting with i andending with j, in which each vertex is an immediate predecessor of the previousone.
For example, in Figure 3.1, (a, c, g) is a direct sequence from a to g, while(g, c, a) is a backward direct sequence from g to a.
Lemma 1. Removing any subsequence from a proper candidate, such that it re-mains proper, could not increase the expected total exploration time of the candi-date.
This Lemma seems obvious; however, it could not be inferred immediately.Since transitions times are getting shorter as customers gain experience within thesystem, reducing a subsequence may increase the time of later transitions. There-fore we need to prove that reducing a subsequence could not increase the totalexploration time of U . The proof will be presented at the end of the present sub-section.
As a direct result of Lemma 1, we can now state the following proposition:
Proposition 1. A proper candidate U could not be optimal if it has at least one ofthe following properties:
1. The candidate U contains a loop, i → i1 → ... → in, such that in = i andall the vertices i1 → ...→ in have reward 0.
2. The candidate U contains a sequence i1 → ... → in → s, such that all thevertices i1 → ...→ in have reward 0.
3. The candidate U ends with the sequence i1 → ... → in, such that all thevertices i1 → ...→ in have reward 0.
4. The candidate U contains the sequence i → s → direct sequence to j,where pre(i) is a mutual predecessor of i and j.
18
The idea of Proposition 1 is based on the following observations: a subse-quence of a proper candidate U in which all the vertices have zero reward, couldnot add any positive reward to U . However, it will still incur a cost per unit of timeexploring it. By Lemma 1, removing this subsequence from U could not increasethe exploration time of U . Thus, if removing a subsequence with zero reward willresult in yet another proper candidateW , it is clear thatR(W ) > R(U). The proofis straightforward and will be presented at the end of the present subsection.
Let D denote the set of all proper candidates with at least one of the 4 proper-ties in Proposition 1.
Definition 4. Let Λ′ be the set Λ \ D. A candidate U ∈ Λ′ will be called anadmissible candidate.
Definition 5. We say that a vertex is redundant if the tree spanning from it doesnot contain a vertex with a positive reward.
Definition 6. Let Λ′i be the set of all candidates in Λ′ that have exactly i leaveswith positive reward.
Four corollaries stating properties of admissible candidates can be immediatelyderived from Proposition 1:
Corollary 1. An admissible candidate U will not include a redundant vertex.
Remark 1. It follows directly from Proposition 1 that any candidate containingredundant vertex or vertices could not be optimal, since it will have one of theproperties of Proposition 1. Thus, from now on, we shall assume that all the leavesin G have a positive reward; hence there are no redundant vertices. If this is not thecase, we can simply consider M as the set of leaves of G with positive reward.
Corollary 2. An admissible candidate U will not include a leaf more than once(see the proof of Lemma 3). This gives rise to two immediate conclusions:
1. If all the leaves of G with positive reward were already explored, the searchis over, which means that there will be no additional vertices in U .
2. If all the leaves with positive reward in the tree spanning from a specificvertex i ∈ V were already explored, the vertex i will not appear in U again.
Corollary 3. In an admissible candidate U , backward transitions to the root vertexs are only allowed from leaves of G (by property 2 of Proposition 1).
Corollary 4. If i and j are two sequential leaves in an admissible candidate U ,and pre(i) is a predecessor of j, then the path between them is i → pre(i) →direct sequence to j (by property 4 of Proposition 1).
19
Following Proposition 1 and the above four corollaries, we now show that thenumber of all admissible candidates in Λ′ is finite, and that each of the candidatesin Λ′ is finite. We will later use this fact in order to find the optimal candidate, thatis, the optimal search policy.
Proposition 2. The set Λ′ is finite, and each candidate in this set has a finitenumber of vertices.
In order to prove Proposition 2, we first state three Lemmas.
Lemma 2. Every admissible candidate U ∈ Λ′ starts with a direct sequence froms to a leaf i ∈M .
Lemma 3. The number of leaves in every admissible candidate U ∈ Λ′ is at most|M |.
Definition 7. A segment [i, j] will be a sequence between i and j, such that i ∈M ,j ∈M , and every vertex in the sequence between them is not in M .
Remark 2. According to Lemma 2 every admissible candidate U ∈ Λ′, starts witha direct sequence from s to a leaf i ∈ M . This sequence will also be called asegment and will be denoted by [s, i].
Lemma 4. Every segment [i, j] in a candidate U ∈ Λ′ can have only one of thefollowing forms:
1. i→ backward direct sequence→mutual predecessor of i and j with greatestdepth→ direct sequence→ j
2. i→ s→ direct sequence→ j
Notice that Lemma 4 implies that each segment in a candidate U ∈ Λ′ has afinite number of vertices.
We now conclude the discussion with proofs:
Proof of Lemma 1. Let U be a proper candidate containing the sequence i1 →i2 → .. → in, such that removing this sequence will result in yet another propercandidate, W . We show that, by extracting the sequence i1 → ...→ in, we alwaysreduce the expected total exploration time of the candidate.First, recall that by the underlying model assumptions, backward transitions fromleaves never influence the time of later transitions. Hence, we only discuss transi-tions from non-leaves vertices.Assume that the sequence i1 → ... → in includes a transition from a non-leafvertex j ∈ V to another vertex k ∈ V . In addition, assume that the vertex j alsoappears somewhere after vertex in in the proper candidate U , and is followed thereby vertex h. Assume that at the point of the aforementioned transition from j tok, N(k) = n and at the later transition from j to h, N(h) = m. Therefore, the
20
exploration time of j → k is tnj,k and the exploration time of j → h is tmj,h. IfPlacej(k) < Placej(h) (which means that either h is a left brother of k, or his a predecessor of j while k is a successor of j), then removing the transitionfrom j to k will not influence the time of the transition from j to h. Assume thatPlacej(k) ≥ Placej(h) (which means that either k is a left brother of h, k = h,or k is a predecessors of j). By the underlying model assumptions, the transitionfrom j to k increases N(h) by one. In addition, according to our assumptions:N(h) ≥ N(k) at all times. Therefore, if N(h) was equal to N(k) before the tran-sition from j to k, after this transitionN(h) = N(k)+1. IfN(h) was greater thanN(k) before this transition, it will stay greater afterwards. It follows that, m > n.If we remove the transition from j to k, we will increase the time of the later tran-sition j → h, so it will now be tm−1
j,h instead of tmj,h. For simplicity, we can assumethat tmi,j = 0. Therefore, by removing the transition from j to k we subtract tnj,k and
add tm−1j,h . It is left to show that tnj,k
d≥ tm−1
j,h . According to our model assumptions,
tnj,kd≥ tmj,h if m ≥ n.
Since m > n, m− 1 ≥ n and therefore:
tnj,kd≥ tm−1
j,h .
Thus, we are extracting a random variable which is stochastically larger then therandom variable we add. Hence, by extracting the sequence i1 → ... → in we arealways reducing the expected total exploration time of the candidate.
Proof of Proposition 1. We show that for every proper candidate U , which has atleast one of the above properties, we can find another proper candidate W suchthat R (W ) > R (U).
1. Let U be a proper candidate containing the sequence i → i1 → .. → in,where in = i and all the vertices i1 → ... → in have no reward. Extractingthe sequence i1 → ... → in will result in a proper candidate W . The se-quence i1 → ...→ in adds no positive reward. By Lemma 1, removing thissequence could not increase the expected total exploration time; hence couldnot increase the exploration costs. Therefore, R(W ) > R(U).
2. Let U be a proper candidate containing the sequence i1 → .. → in → s,such that all the vertices i1 → ... → in have no reward. Let i be the lastvertex before this sequence with positive reward. W.l.o.g we can assumethat i is followed immediately by i1 → ... → in. Extracting the sequencei1 → ...→ in will result in a proper candidate W . This candidate will havethe sequence i → s instead of the sequence i → ... → i1 → ... → in → s.Since i is a leaf:
ti,s = tbackd≤ tN(lin )
in,s= t
N(lin )in,lin
+ tback.
21
By Lemma 1, extracting the sequence
i1 → ...→ in
could not increase the total exploration time of the candidate, and since thissequence adds no positive reward, R(W ) > R(U).If there is no vertex with positive reward before this sequence, we can as-sume, w.l.o.g, that U starts with i1 → .. → in → s, where i1 = s. By thesame arguments as before, extracting the sequence i1 → ...→ in will resultin a proper candidate W , such that R(W ) > R(U).
3. Let U be a proper candidate ending with the sequence i1 → .. → in, suchthat all the vertices i1 → ... → in have no reward. Since no vertex in thesequence i1 → ... → in has a positive reward, this sequence only addsnegative utility to U . Extracting this sequence from U will result in anotherproper candidate W for which R(W ) > R(U).
4. Let U be a proper candidate containing the sequence
i→ s→ direct sequence to→ j
where i is a leaf of G with positive reward and pre(i) is a predecessor ofj (if i is not a leaf with positive reward then, by case 2, U could not beoptimal). The direct sequence from s to j contains pre(i) as well. Thereis no leaf in the direct sequence from s to pre(i); hence rk = 0 for ev-
ery k in this direct sequence. Recall that ti,pre(i)d= ti,s
d= tback. By
Lemma 1, removing the sequence s → direct sequence to → j could notincrease the expected total exploration time. Therefore, replacing the se-quence i → s → direct sequence to → j with the sequence i → pre(i) →direct sequence to→ j will result in a proper candidateW for whichR (W ) >R (U).
If U does not contain any vertex with a positive reward, then R(U) < 0. In thiscase, the policy of not exploring anything at all is more profitable than U .
Proof of Lemma 2. Let us assume that there is an admissible candidate U ∈ Λ′
which does not start with a direct sequence from s to a leaf i ∈ M . This meansthat U has one of the following properties:
• There are no leaves with positive reward in U .In this case U ends with a vertex j /∈ M . Since j /∈ M , rj = 0. U fulfillsproperty 3 of Proposition 1⇒ U ∈ D ⇒ U /∈ Λ′, which is a contradiction.
• There are two sequential vertices i, j /∈M in the prefix sequence of U , froms to the first leaf in U , such that j /∈ A(i). W.l.o.g. assume that the sequencefrom s to i is a direct sequence. This means that j is either pre(i) or j = s.If j = pre(i), and the sequence from s to i was a direct sequence, then U
22
contains the sequence pre(i) → i → pre(i), which is a zero reward loop⇒ U fulfills property 1 of Proposition 1⇒ U ∈ D ⇒ U /∈ Λ′. If j = s,U fulfills property 2 of Proposition 1 ⇒ U ∈ D, ⇒ U /∈ Λ′, which is acontradiction.
Proof of Lemma 3. Let U be a proper candidate in Λ. Assume that there is a leafi ∈ M which appears in U more than once. At the n > 1 time i appears inU it has no reward. W.l.o.g assume that i appears in U twice. If U ends withthe second appearance of i then U fulfills property 3 of Proposition 1. HenceU ∈ D ⇒ U /∈ Λ′. If the second appearance of i in U is followed by s thenU fulfills property 2 of Proposition 1. Hence U ∈ D ⇒ U /∈ Λ′. If the secondappearance of i in U is followed by pre(i) then U fulfills property 1 of Proposition1. Hence U ∈ D ⇒ U /∈ Λ′. Therefore, an admissible candidate U ∈ Λ′ can haveat most |M | leaves, where each leaf i ∈M appears in U no more than once.
Proof of Lemma 4. Let us assume that there is an admissible candidate U ∈ Λ′,which has a segment [i, j], which does not have one of the above two forms. Thismeans that the segment [i, j] contains at least one of the following sequences:
1. k → s where k /∈M .In this case U will have property 2 of Proposition 1; hence U ∈ D ⇒ U /∈Λ′, which is a contradiction.
2. k → l → k, where l ∈ A(k). In this case U will have property 1 ofProposition 1; hence U ∈ D ⇒ U /∈ Λ′, which is a contradiction.
Proof of Proposition 2. Let U be an admissible candidate in Λ′. According toLemma 3, U has at most m = |M | leaves (with positive reward), which meansit has at most m segments. Recall that Λ′i is the set of all candidates in Λ′ whichhave exactly i leaves (with positive reward). According to Lemma 2, since eachcandidate in Λ′ starts with a direct sequence from s to a leaf in M , |Λ′1| = m. Thenumber of candidates in Λ′i is at most:(
m
i
)(i!)2i−1.
Explanation: there are(mi
)options to choose the i leaves out of the total m leaves;
there are i! options of ordering these i leaves. There are i segments in each candi-date U ∈ Λ′i. According to Lemma 2, after ordering the i leaves, the first segmentis determined. According to Lemma 4, each of the last i − 1 segments can haveone of two forms. Hence, after we choose the i leaves and order them, there are atmost 2i−1 options for the particular segment combinations. Since m is finite |Λ′i|
23
is finite. Λ′ =m⋃i=1
Λ′i, and therefore also finite as a finite union of finite sets.
It is left to prove that each candidate in Λ′ has a finite number of vertices. This isclear due to the fact that each candidate has at mostm segments, and each segment,according to Lemma 2 and Lemma 4, has a finite number of vertices.
Notice that every admissible candidate in Λ′m contains all possible leaves inM . It is clear that every candidate in Λ′i, for i < m, is a prefix of a candidate inΛ′m
3.3 Modeling the Search Protocol as a Rooted Graph
In Section 3.1.2, we have defined our state space and listed all the possible tran-sitions from state to state (or to termination). These possible transitions betweenstates form the search protocol. In Section 3.2, we mapped the search protocol intoa set of proper candidates, where each candidate is a sequence of vertices com-plying with the search protocol. In this section we explain how the set of propercandidates, Λ, can be translated into a rooted graph, ultimately a tree. This rootedgraph will be denoted asGF = (V F , EF ) and will be called from now on a ProperGraph.
Let U be a proper candidate in Λ. According to the definition of proper candi-date, a vertex i ∈ V can appear in a candidate U more than once. However, recallthat our state space has 3 dimensions, including the total time spent in the search.Therefore, every new visit to a specific vertex is translated into a different state:it happens at a different time. We will translate vertices in U to their equivalentstates. We will explain how each state can be represented as a vertex in V F , andhow each transition between two states can be represented as an edge in EF . SinceU is proper, it starts with the vertex s. Hence, s will be the only root vertex ofGF = (V F , EF ).
Every vertex in V F will be defined by a base and an index. For example:u = ij , base(u) = i, index(u) = j. Let u, v be two vertices in V F , such that(u, v) ∈ EF . These two vertices in V F represent two sequential vertices in aproper candidate U . The bases of u and v represent their matching vertices in U .For example, if u = ij , it means that u represents the vertex i in U . The indexof a vertex in V F is a vector which may be empty. If u is a vertex in V F , thenthe components of index(u) represent the vertices prior to base(u) in a propercandidate U , from which backward transitions occurred. The order of the verticesin index(u) matches the chronological order of the backward transitions. If u =ij ∈ V F , it means that i is a vertex in a proper candidate U , and j is the only vertexprior to i in U from which a backward transition occurred. That is, in the propercandidate U , j was prior to i and was followed by either s or pre(j).
24
We will now introduce an example in order to understand the role of the baseand index of a vertex in V F . As before, consider the IVR tree G = (V,E) inFigure 3.1. Let U be the sequence s → a → c → f → c → a → d → s → b.Then U is a proper candidate; U will be represented by a path U in GF , which ispresented in Figure 3.2 below.
The base of a vertex in U ∈ GF represents the corresponding vertex in U .For example, in Figure 3.2, base(6) = d, which means that the sixth vertex in Urepresents a visit to vertex d while exploring candidate U . The index of vertex 6 inU represents that during the exploration of candidate U , prior to the current visit ofd, there were two backward transitions - the first one from vertex f and the secondfrom vertex c.
In general, let u, v be two vertices in a path U ∈ GF representing two sequen-tial vertices in a proper candidate U , such that base(u) = i and base(v) = j. If j ∈A(i), index(u) = index(v). If j = pre(i) or j = s, index(v) = (index(u), i).
For every vertex u ∈ V F we define:
Pu = Pbase(u)
tser(u) = tser(base(u))
tF (u) = tF (base(u))
ru =
{rbase(u) if base(u) /∈ index(u)
0 0
and as before: Tu =
{tser(u) w.p. PutF (u) w.p. 1− Pu.
In order to define immediate predecessor and immediate successors in theProper Graph, GF we will use the notations preF and AF (·), that is, if (u, v)is an edge in EF , then u = preF (v) and v ∈ AF (u).
As described in Section 3.1 our search model takes the form of a sequentialdecision process. Each state in our state space has 3 components representingthe current vertex i ∈ V , the total time spent in the search and the number ofrepeated visits to each edge. Each vertex in the Proper Graph, GF , is associatedwith a state in our state space. The root vertex s is associated with the initial stateS0 = (s, 0, 0).
We update the vector N (representing the number of repeated visits to eachedge) as we go from vertex to vertex in GF . The value of the vector N at vertexv ∈ V F will be denoted by Nv. Denoting preF (v) by u
((u, v) ∈ EF
), Nv will
25
be updated according to the following recursive formula:
Nv =
Nu + δbase(u),base(v) if base(v) ∈ A(base(u))
Nu if base(u) ∈MNu + δbase(u),lbase(u) otherwise
where:N s = 0.
The total time spent in the search is expressed as the time required to reachvertex v ∈ V F . This time will be calculated by the following recursive formula(again assuming preF (v) = u):
t(v) = t(u) + Tu + tu,v
where:
tu,v =
tNu(base(v))base(u),base(v) if base(v) ∈ A(base(u))
tback if base(u) ∈M
tNu(lbase(u))base(u),base(v) otherwise.
Explanation:
If base(v) ∈ A(base(u)) the time spent on the edge (u, v) depends on thenumber of previous visits to base(v).
If base(u) is a non-leaf vertex and base(v) is either pre(base(u)) or s, thenthe time spent on the edge (u, v) depends on the number of previous visitsto the leftmost son of base(u), lbase(u).
We conclude that every vertex v ∈ V F is associated with the state:
Sv =(base(v), t(v), Nv
).
For example, in Figure 3.2, S5 = (a, t(5), N5), where:
t(5) = t0s,a + t0a,c + t0c,f + Tf + tf,c(= tback) + t0c,a(= t0c,g + tback)
and
N0 =( s a b c d e f g
0 0 0 0 0 0 0 0);
after the transition s→ a:
N1 =( s a b c d e f g
0 1 0 0 0 0 0 0);
26
after the transition a→ c:
N2 =( s a b c d e f g
0 1 0 1 0 0 0 0);
after the transition c→ f :
N3 =( s a b c d e f g
0 1 0 1 0 0 1 0);
after the transition f → c:
N4 =( s a b c d e f g
0 1 0 1 0 0 1 0);
after the transition c→ a:
N5 =( s a b c d e f g
0 1 0 1 0 0 2 1).
The success probability of edge e = (u, v) ∈ EF is
Pu,v =P (τ > t(u) + Tu + tu,v)
P (τ > t(u) + Tu)= P (τ > tu,v) .
Figure 3.3 is a partial example of a Proper Graph, GF , which is based on theIVR tree, G, shown in Figure 3.1. The vertices and edges in dashed lines are thenew vertices and edges of GF (meaning that they belong to GF \G). The verticesin yellow are the set M - the set of leaves of G (representing the IVR services).
27
' :U G
U s a c f c a d s b
0
s
ab
cde
fg
( , ) :G V E
1
2
3
4
5
6
7
s
8
a
c
f
fc
fca
fcd
fcds
fcdb
Figure 3.2: An example of a proper candidate represented by a path in GF . HereG is taken from Figure 3.1.
28
Figure 3.3: A partial example of a Proper Graph created from an IVR tree G.
29
Chapter 4
Admissible Tree Creation
In Section 3.3, we introduced the construction of a Proper Graph, based on anIVR tree. We defined a Proper Graph as the translation of Λ - the set of all propercandidates, into a rooted graph. We now present an algorithm which constructsa rooted tree from Λ′ - the set of all admissible candidates: candidates which donot have any of the properties in Proposition 1 (are not included in D). This tree,denoted by T = (V T , ET ), will be called an Admissible Tree.
Recall that, our goal is to find the optimal search policy, that is, to find a se-quence of transitions between vertices of G which yields maximal expected utility.We defined a candidate (either just proper or also admissible) as a sequence ofvertices in an IVR tree G, in compliance to the search protocol. However, thissequence may contain backward transitions and loops. In Proposition 1 we provedthat only candidates included in Λ′ could be optimal. The purpose of the Admis-sible Tree - T , is to span all the sequences in Λ′ as direct paths in a tree, with nobackward transitions. Every state in our state space is translated into a vertex in T ,and all possible transitions from a specific state are modeled as edges emanatingfrom the corresponding vertex to its direct successors. In Chapter 5 we show thatan index can be assigned to each edge in T . Starting at the root vertex of T , we willalways choose the emanating edge with the highest index and move to the vertexit leads to (or terminate the search). This process will result in a direct path fromthe root vertex of T to one of its vertices. This direct path represents the optimalpolicy. That is, this path represents a sequence of transitions between vertices ofG, or equivalently, a sequence of transitions from state to state in our state space.
We say that a path U in T represents a candidate U in Λ′, if there a one-to-onematch between the bases of all the vertices in U (by their order) to the vertices inU (by their order).
Notice that T is a sub-graph of GF ; thus, vertices and edges in T will have thesame properties as vertices and edges in GF . To define the immediate predecessorand immediate successors in T , we use the notations preT and AT (·); that is, if(u, v) ∈ ET , then u = preT (v) and v ∈ AT (u)
30
4.1 Algorithm Description
The algorithm is initialized with T = G, where G is an IVR tree.
Remark 3. According to Remark 1, we are assuming that all the leaves in G havea positive reward. This is without loss of generality in the sense that, otherwise, allactions in the present chapter will be restricted to leaves with positive rewards, orto vertices that lead to such leaves. Formally, M will be the set of all leaves withpositive reward. Furthermore, MΓ(i) will denote the set of all leaves in the treespanning from i with positive reward and A(i) the set of all direct successors of iwith positive reward.
At every iteration, the algorithm adds new edges, emanating from leaves of Tand leading to new vertices, which leads to the T of the next iteration. Let MT
itr
be the set of all new leaves of T in a specific iteration. If u is a vertex in MTitr, the
algorithm will add edges from u to new vertices representing:
1. Immediate successors of base(u) in G: every k ∈ A(base(u)).
2. Immediate predecessor of base(u) in G: pre(base(u)).
3. The root vertex of G, s.
The new edges emanating from u represent all possible transitions from base(u)in G, according to the search protocol and the definition of proper candidates.
Every iteration of the algorithm has two stages:
1. Adding edges and vertices representing forward transitions.
2. Adding edges and vertices representing backward transitions.
At the first stage, for every u ∈ MTitr and for every k ∈ A(base(u)), the algo-
rithm adds to ET the edge (u, kindex(u)). However, since every path U ∈ T shouldrepresent an admissible candidate U ∈ Λ′ = Λ \ D, we will not add the edge(u, kindex(u)) (and the vertex kindex(u)) if it results in a path U which represents acandidate in D (a candidate with one or more of the properties of Proposition 1).The set of all such excluded k′s will be denoted by Fexc(u).
At the second stage, for every u ∈MTitr, the algorithm adds the edges
(u, pre(base(u))(index(u),base(u))),
(u, s(index(u),base(u))).
As stated before, we will not add edges to u ∈ MTitr if it results in a path U
representing a candidate U ∈ D. We distinguish between two types of backwardtransitions; transitions to immediate predecessor and transitions to the root vertex.
31
We denote by Bexc the set of all u′s for which adding an edge representing back-ward transition to pre(base(u)) will result in such an undesirable path. Recall that,by Corollary 3, backward transitions to the root vertex in an admissible candidateare only allowed from leaves of G. If base(u) ∈ M we can not add to it an edgerepresenting a backward transition to s, only if all the leaves of G were alreadyexplored in the path from the root vertex to u. We denote the set of all such u′s byB1exc.
We now formalize the definitions of the sets Fexc(u) and Bexc. Recall thatthese are the sets of vertices excluded from forward transition from a vertex u, andvertices from which there will be no backward transitions, respectively.
Definition 8. For every vertex i ∈ V T , Last index(i) will be the last vertex inthe vector index(i).
Definition 9. Let I(u) be the set (unordered) of all the vertices in the vectorindex(u).
For example: If we take vertex agc shown in Figure 3.3, thenLast index(agc) =c, and I(agc) = {g, c}; I(saa) is the set {a}.
Definition 10. Let Fexc(u) be the following set:
Fexc(u) = F 1exc(u) ∪ F 2
exc(u) ∪ F 3exc(u) ∪ F 4
exc(u),
whereF 1exc(u) = {k ∈ A(base(u)) : k ∈M and k ∈ I(u)}
F 2exc(u) = {k ∈ A(base(u)) : k = Last index(u)}
F 3exc(u) =
{k ∈ A(base(u)) : MΓ(k) ⊆ I(u)
}F 4exc(u) = {k ∈ A(base(u)) : Last index(u) ∈ Γ(k) \ {k}and ∀j ∈MΓ(k) s.t j /∈ I(u) ; pre(Last index(u)) is a predecessor of j
}.
Explanation of F 4exc(u): Assume that i is the last vertex in index(u). If k ∈
F 4exc(u), then i is in the tree spanning from k (but not k itself). This actually means
that the last backward transition, in the exploration prior to reaching u, was fromi. It also states that if there are leaves in the tree spanning from k which were notexplored yet, then the immediate predecessor of i, pre(i) is also a predecessor ofall those leaves.
Remark 4. Notice that if i = Last index(u) ∈ A(k), then Last index(u) ∈Γ(k) \ k. In addition, since pre(i) = k, k is also, by definition, a predecessor ofevery j ∈ MΓ(k). Thus, if Last index(u) ∈ A(k) it automatically means thatk ∈ F 4
exc(u).
32
The base of the vertex adeg in Figure 4.1 (colored in red) is an example of a ver-tex in F 4
exc(sdeg). Notice that base(sdeg) = s, a ∈ A(s) and Last index(sdeg) =
g. The tree spanning from a (not including a itself) is Γ(a) \ a = {c, d, e, g, f}and MΓ(a) = {d, e, g, f}. Within the set MΓ(a), only f /∈ I(sd,e,g). However,pre(g) = c is also a predecessor of f .
�
��
���
���
�
�
�
�
��
��
U s a d a e a c g s a= → → → → → → → → →
Figure 4.1: Example of a vertex in F 4exc
Definition 11. LetBexc = B1
exc ∪B2exc ∪B3
exc,
whereB1exc =
{u ∈MT : M ⊆ I(u) ∪ {base(u)}
}B2exc =
{u ∈MT : base(u) /∈M and Last index(u) /∈ A(base(u))
}33
B3exc =
{u ∈MT : MΓ(FDP (base(u))) ⊆ I(u) ∪ {base(u)}
},
where FDP (base(u)) is the predecessor of base(u) with depth 1.
Lemma 5 (Forward transitions). Let u be a vertex in MTitr, and let v be a vertex
such thatbase(v) ∈ Fexc(u).
Adding an edge (u, v) to ET results in a path U ∈ T representing a candidateU ∈ D.
Lemma 6 (Backward transitions to s). For every vertex u ∈ B1exc such that
base(u) ∈ M , if we add an edge to a new vertex v ∈ V T , such that base(v) = s,this will result in a path U ∈ T representing a candidate U ∈ D.
Lemma 7 (Backward transitions). For every vertex u ∈ Bexc, if we add an edge toa new vertex v ∈ V T , such that base(v) = pre(base(u)), this will result in a pathU ∈ T representing a candidate U ∈ D.
The proofs of the lemmas appear in Section 4.4.
Figure 4.2 presents a partial example of a Proper Graph, GF , based on theIVR tree, G, presented in Figure 3.1. Paths colored in red are paths that will not beincluded in an Admissible Tree, created by the Admissible Tree Algorithm whichis based on the same IVR tree, G.
We are now ready to formally present the Admissible Tree Algorithm.
4.2 Algorithm Formulation
The Admissible Tree Algorithm gets an IVR tree G = (V,E) as an input andreturns a rooted tree representing all admissible candidates in Λ′. The latter treewill be denoted by T =
(V T , ET
)and will be called an Admissible Tree. The set
of leaves of T will be denoted by MT .
Initialization:Set V T ← V .Set ET ← E.Set MT
itr ←M .Set MT
temp = ∅.
Note: Properties 1 and 2 of Proposition 1 immediately suggest that addingedges emanating from i ∈ V \ M and representing transitions to s or topre(i) will result in paths representing candidates in D. This is why we caninitialize T with G and add edges emanating from G leaves (with positivereward) only, and not from other vertices in G.
34
Figure 4.2: Example of a partial Proper Graph and excluded paths
35
1. Spanning the tree with forward transitionsSetET ← ET∪
{(u, kindex(u)) ∀u ∈MT
itr : k ∈ {A(base(u)) \ Fexc(u)}}
.Set V T ← V T ∪
{kindex(u) ∀u ∈MT
itr : k ∈ {A(base(u)) \ Fexc(u)}}
.Set MT
temp ←{kindex(u) ∀u ∈MT
itr : k ∈ {A(base(u)) \ Fexc(u)}}
.
2. Spanning the tree with backward transitions
Set
ET ← ET ∪{(u, pre(base(u))(index(u),base(u))
)∀u ∈MT
itr \Bexc}
∪{(u, s(index(u),base(u))
)∀u ∈MT
itr \B1exc : base(u) ∈M
}.
Set
V T ← V T ∪{pre(base(u))(index(u),base(u)) ∀u ∈MT
itr \Bexc}
∪{s(index(u),base(u)) ∀u ∈MT
itr \B1exc : base(u) ∈M
}.
Set
MTtemp ←MT
temp ∪{pre(base(u))(index(u),base(u)) ∀u ∈MT
itr \Bexc}
∪{s(index(u),base(u)) ∀u ∈MT
itr \B1exc : base(u) ∈M
}.
If MTtemp = ∅ then finish and return T =
(V T , ET
); Else
Set MTitr ←MT
temp.Set MT
temp ← ∅.Return to Step 1.
Figure 4.3 presents the first three iterations of our Admissible Tree Algorithm,based on the IVR tree G presented in Figure 3.1. Edges and vertices in purple rep-resent forward transitions. Edges and vertices in green represent backward transi-tions.
It is left to show that the output of the Admissible Tree Algorithm is indeed arooted tree.
Proposition 3. The application of the Admissible Tree Algorithm on an IVR tree,G = (V,E), results in a rooted tree, T =
(V T , ET
).
Proof. The algorithm is initialized with T = G, which is rooted at vertex s. Atevery step, the algorithm adds new edges emanating from leaves of T and leading
36
�
��
���
��
� � � �� �� �� �� � �
Initia
liza
tion
Itera
tion
1
� � � ��� ���� ���� �� � �� � �� �
Itera
tion
2
� � � �� � � �� � � �� �� �� ������ ��
Itera
tion
3
Figure 4.3: Example of a partial Admissible Tree
to new vertices. Hence, at every step, T is still connected and rooted. It is left toshow that every two distinct vertices in T have exactly one simple path betweenthem (a simple path is a path with no repeated vertices), or equivalently, show thateach vertex has exactly one immediate predecessor. Since T is connected, everyvertex in T , except from the root vertex s, has at least one immediate predecessor.Let v ∈ V T be a vertex with more than one immediate predecessor. Let u1 and u2
be v′s immediate predecessors.There are four possible cases:Case I: base(v) = s⇒
index(v) = (index(u1), base(u1))
index(v) = (index(u2), base(u2))
⇒ index(u1) = index(u2). (Recall that index(·) is a vector, the coordinates of which are ordered)
37
base(u1) = base(u2)
⇒ u1 = u2.
Case II: base(v) = pre(base(u1)) and base(v) = pre(base(u2))⇒
index(v) = (index(u1), base(u1))
index(v) = (index(u2), base(u2))
⇒ index(u1) = index(u2)
base(u1) = base(u2)
⇒ u1 = u2.
Case III: base(v) ∈ A(base(u1)) and base(v) ∈ A(base(u2))⇒
index(v) = index(u1)
index(v) = index(u2)
⇒ index(u1) = index(u2)
pre(base(v)) = base(u1)
pre(base(v)) = base(u2)
and since G is a tree⇒ base(u1) = base(u2)
⇒ u1 = u2.
Case IV: base(v) ∈ A(base(u1)) and base(v) = pre(base(u2))⇒
index(v) = index(u1)
index(v) = (index(u2), base(u2))
⇒ index(u1) = (index(u2), base(u2))
⇒ Last index(u1) = base(u2) ∈ A(base(v))
⇒ base(v) ∈ F 4exc(u1) (See Remark 4).
According to the specification of the Admissible Tree Algorithm, this last in-clusion would have prevented us from emanating an edge from u1 to v.Figure 4.4 depicts an examples of such a case.
38
s
ab
cde
g fds
da
Figure 4.4: Examples of Proposition 3, case IV
4.3 The Equivalence between Admissible Tree and the Setof Admissible Candidates
We start with the following
Definition 12. A path U ∈ T , from the root vertex s to a leaf i ∈ MT , will becalled a complete path.
Recall that we have defined Λ′m as the set of all candidates in Λ′ = Λ \D withexactly m = |M | different leaves (with positive reward). In this section, we willprove that every complete pathU ∈ T represents an admissible candidateU ∈ Λ′m,and that every admissible candidate in Λ′ can be represented by a path in T .
Theorem 1.An admissible candidate U is in Λ′m if and only if it is represented by a completepath
U ∈ T =(V T , ET
).
.Since every candidate in Λ′ \ Λ′m is a prefix of a candidate in Λ′m, it is also
represented as prefix of a complete path in T . Thus, Theorem 1 actually states that
39
the output of the Admissible Tree Algorithm is indeed a rooted tree containing alladmissible candidates in Λ′, and nothing but them.
In order to prove Theorem 1 we first show that every complete path in T rep-resents an admissible candidate in Λ′m. We do so via Lemmas 8 to 10, the proof ofwhich appear in Section 4.5. We then prove, via Lemma 11, that every admissiblecandidate in Λ′ is represented by a path in T . By that we have proved both sides ofTheorem 1 and the proof is concluded.
Lemma 8. Every complete path U ∈ T ends with a vertex representing a leafi ∈M .
Lemma 9. Every complete path U ∈ T has, among all its vertices, exactly mvertices representing all the leaves with positive utility in the original IVR tree, G.That is, every complete path U ∈ T has exactly m vertices with bases correspond-ing to different leaves of G.
Lemma 10. Every complete path U ∈ T represents an admissible candidate U ∈Λ′. Equivalently, a candidate U ∈ D cannot be represented by any complete pathU ∈ T .
Lemma 9 uses Lemma 8 to prove that every complete path U ∈ T represents acandidate with exactly m leaves. Lemma 10 states that a candidate represented bya complete path U in T has to be a candidate in Λ′. Since this candidate also hasmdifferent leaves, it follows that every complete path in T represents an admissiblecandidate in Λ′m. Thus, we have shown that every complete path U ∈ T representsan admissible candidate U ∈ Λ′m.Notice that since every complete path in T represents an admissible candidate inΛ′m, Proposition 2 implies that T is a finite tree.
It is left to show that every admissible candidate, U ∈ Λ′, is represented bya path U in T . In Section 3.3 we defined a Proper Graph, GF , as the tree repre-senting all proper candidates in Λ. Notice that T is a sub-tree of GF . Proving thatevery admissible candidate, U ∈ Λ′, is represented by a complete path U in T , isequivalent to proving the following:
Lemma 11. We assume that U is a proper candidate, represented by a path U inGF . If U does not appear in T , then the corresponding candidate U is in D.
To prove Lemma 11 we survey all the cases in which a path U in G will notbe included in T , and prove that the corresponding candidate U could not be anadmissible candidate. This entails the application of the following previous results:
1. U does not start with a direct path from s to a leaf i ∈M (this follows fromthe fact that T is initialized withG). However, by Lemma 2, the correspond-ing candidate U is in D.
40
2. U has an edge (u, v) such that base(v) ∈ A(base(u)) and base(v) ∈ Fexc(u).By Lemma 5 the corresponding candidate U is in D.
3. U has an edge (u, v) such that base(v) = pre(base(u)) and u ∈ Bexc. ByLemma 7 the corresponding candidate U is in D.
4. U has an edge (u, v) such that base(u) /∈ M and base(v) = s. The corre-sponding candidate U has property 2 of Proposition 1⇒ U ∈ D.
5. U has an edge (u, v) such that base(u) ∈ M , base(v) = s and u ∈ B1exc.
By Lemma 6 the corresponding candidate U is in D.
4.4 Proofs of Lemmas 5–7
Proof of Lemma 5. Let u be a vertex in MTitr. Let v be a vertex such that:
base(v) ∈ Fexc(u) = F 1exc(u) ∪ F 2
exc(u) ∪ F 3exc(u) ∪ F 4
exc(u).
Adding an edge from u to v represents a transition from base(u) to base(v) in aproper candidate U ∈ Λ, such that base(v) ∈ A(base(u)).
1. If base(v) ∈ F 1exc(u) it means that base(v) ∈ M and we already explored
this leaf at least once during our exploration of candidate U . Hence, addingan edge (u, v) ∈ ET will represent an additional visit to base(v) in U . Bythe proof of Lemma 3⇒ U ∈ D.
2. If base(v) ∈ F 2exc(u) then Last index(u) = base(v) which means that the
last backward transition in U was from base(v). The backward transitionwas either from base(v) to base(u) = pre(base(v)) or from base(v) to s.We shall assume that base(v) /∈ M , otherwise base(v) will be in F 1
exc(u).If the last backward transition was from base(v) to base(u), adding an edgefrom u to v will result in a sequence base(v) → pre(base(v)) → base(v)and U will have property 1 of Proposition 1⇒ U ∈ D (see for example thesub-path colored in red in Figure 4.5. The vertex in F 2
exc(u) in this examplewill be cgfc). If the last backward transition was from base(v) to s, thenU will have property 2 of Proposition 1 ⇒ U ∈ D (see for example thesub-path colored in blue in Figure 4.5).
3. If base(v) ∈ F 3exc(u), then MΓ(base(v)) ⊆ I(u), which means that all the
leaves in the tree spanning from base(v) were already explored. Hence byCorollary 2, adding an edge from u to v will result in a path representing acandidate U ∈ D.
41
4. If base(v) ∈ F 4exc(u) it means that the last backward transition in U , be-
fore reaching base(u), was from one of the successors of base(v), say,k ∈ Γ(base(v)) \ {base(v)}. This backward transition had to be from kto s; otherwise there had to be a backward transition from base(v) as well,in contradiction to the assumption that the last backward transition was fromk.We will assume that there is at least one leaf in the tree spanning frombase(v) (in the original IVR tree, G) that was not explored yet (otherwise,base(v) will be in F 3
exc(u)). Let h be a vertex such that base(h) is a leafin MΓ(base(v)) that was not explored yet. W.l.o.g. we can assume that his the only vertex with this property. From Corollary 2, we deduce thatfrom v we will have a path to a vertex h. This path will represent a directsequence from base(v) to base(h) in U . Hence, we get the following se-quence: k → s→ direct sequence to→ base(h). Since base(v) ∈ F 4
exc(u)it means that pre(k) is also a predecessor of base(h); therefore, this se-quence has property 4 of Proposition 1 ⇒ U ∈ D (for example, see thesub-path colored in red in Figure 4.1 and the matching candidate U ).
Proof of Lemma 6. The proof of Lemma 6 follows immediately from Corollary 2.
Proof of Lemma 7. Let u be a vertex in MTitr such that u ∈ Bexc = B1
exc ∪B2exc ∪
B3exc.
Let v be a vertex such that base(v) = pre(base(u)). Adding an edge from u to vrepresents a transition from base(u) to pre(base(u)), in a proper candidate U ∈ Λ.
1. If u ∈ B1exc it means that all the leaves in G were already explored in the
sequence from s to base(u) in the proper candidate U . By Corollary 2,adding an edge to any new vertex v will have to result in a path representinga sequence that will cause U to be in D.
2. Assume that u ∈ B2exc. u has an immediate predecessor k ∈ V T . The edge
(k, u) ∈ ET represents a transition in the proper candidate U , from base(k)to base(u); base(u) can either be reached from its immediate predecessor,pre(base(u)), or from an immediate successor of it, j ∈ A(base(u)). How-ever, since Last index(u) /∈ A(base(u)) it means that base(u) was reachedfrom its immediate predecessor; therefore base(k) = pre(base(u)). Thus,adding an edge from u to v where base(v) = pre(base(u)) will result inU having the sequence pre(base(u)) → base(u) → pre(base(u)). Sincebase(u), pre(base(u)) /∈M , this is a loop with zero reward⇒ U has prop-erty 1 of Proposition 1⇒ U ∈ D
42
Figure 4.5: Example of paths representing unwanted sequences, resulting fromadding vertices in F 2
exc
3. Assume that u ∈ B3exc. This means that all the leaves in the tree spanning
from the predecessor of base(u) in depth 1, FDP (base(u)), were alreadyexplored in the sequence from s to base(u) in the proper candidate U . It alsomeans that dep(base(u)) > 1. Hence, if base(v) = pre(base(u)) 6= s, thenMΓbase(v) ⊆ {I(u) ∪ base(u)}. By Corollary 2, adding edge (u, v) to ET
will result in a path representing a sequence that will cause U to be in D.
43
4.5 Proofs of Lemmas 8–10
We use for convenience the following two definitions:
Definition 13. Let L(i, j) be the Lowest mutual predecessor of i and j. L(i, j),is a mutual predecessor of both i and j, and has the largest depth among all suchmutual predecessors of i and j.
For example, in Figure 3.1, L(d, g) = a.
Definition 14. Let FDP (i) be the First Depth Predecessor of i. FDP (i) is apredecessor of i and dep(FDP (i)) = 1.
For example, in Figure 3.1, FDP (g) = a.
Proof of Lemma 8. Let v be the last vertex on a complete path U ∈ T , whichmeans that v is a leaf of T , v ∈ MT . Assume that base(v) /∈ M . In order for v tobe in MT , v needs to be in Bexc, which means that we cannot emanate edges fromv representing a backward transition from base(v). Another requirement, in orderfor v to be in MT , is that every immediate successor of base(v), k ∈ A(base(v)),will be in Fexc(v). This means that we cannot emanate edges from v representinga forward transition from base(v). We will assume that v ∈ Bexc and will eithershow a contradiction to the assumption the v ∈ V T , or show that there is at leastone immediate successor k ∈ A(base(v)) such that k /∈ Fexc(v).
1. Assume that v ∈ B1exc ⇒M ⊆ {I(v) ∪ base(v)}.
Since base(v) /∈ M ⇒ M ⊆ I(v). This means that all the leaves of theoriginal IVR tree, G, were already explored. Let u be the last vertex in Usuch that base(u) ∈ M . This means that all the other leaves in G are inI(u); therefore M ⊆ {I(u) ∪ base(u)} ⇒ u ∈ B1
exc ⇒ u ∈ Bexc. Thus,we will not add edges to u representing backward transitions from base(u).Since base(u) ∈ M , A(base(u)) = ∅; thus we cannot add edges from urepresenting forward transitions as well ⇒ u ∈ MT and v could not bereached.
2. Assume that v ∈ B2exc; then Last index(v) /∈ A(base(v)). Let u be v′s
immediate predecessor in U , such that (u, v) ∈ ET ; base(u) could not bein A(base(v)), otherwise Last index(v) will be base(u) ∈ A(base(v)).Therefore, base(u) = pre(base(v)). Since base(v) /∈ M it has at least oneimmediate successor, k ∈ A(base(v)). We will show that k /∈ Fexc(v),meaning k /∈ F 1
exc(v), k /∈ F 2exc(v), k /∈ F 3
exc(v) , and k /∈ F 4exc(v).
• Since v ∈ B2exc, Last index(v) 6= k ⇒ k /∈ F 2
exc(v).
44
• We assume that MΓ(base(v)) * I(u), otherwise base(v) will be inF 3exc(u) and the edge (u, v) will not be in ET .
Since base(u) = pre(base(v)), index(v) = index(u)and MΓ(base(v)) * I(v).This means that there is at least one leaf in the tree spanning frombase(v) in G that was not explored in the candidate represented by U .
– If k ∈ M we can assume, w.l.o.g., that k is this leaf. Thereforek /∈ F 1
exc(v), k /∈ F 3exc(v) and k /∈ F 4
exc(v)(since Γ(k) \ {k} = ∅).See for example Figure 4.6, where v = af and k = e.
– If k /∈ M , k /∈ F 1exc(v). We can assume, w.l.o.g., that there is a
leaf j that was not explored in the tree spanning from base(v) andis also in the tree spanning from k. Therefore, k /∈ F 3
exc(v).Assume that k ∈ F 4
exc(v)
⇒ Last index(v) ∈ Γ(k) \ {k}
andL(j, Last index(v)) = pre(Last index(v)).
However, since base(u) = pre(base(v))⇒ index(v) = index(u),therefore:
Last index(u) ∈ Γ(base(v)) \ {base(v)}
andL(j, Last index(u)) = pre(Last index(u))
⇒ base(v) ∈ F 4exc(u).
Hence (u, v) /∈ ET is a contradiction.
See for example Figure 4.7, where again v = af , u = sf , k =base(cf ) = c and the unexplored leaf j is the vertex g.
3. Assume that v ∈ B3exc. Let u be v′s immediate predecessor in U , such that
(u, v) ∈ ET .
• Assume that base(u) = pre(base(v)). This means that:
index(v) = index(u).
Since v ∈ B3exc, MΓ(FDP (base(v))) ⊆ {I(v) ∪ base(v)}
⇒MΓ(base(v)) ⊆MΓ(FDP (base(v))) ⊆ {I(v) ∪ base(v)}.
However, since base(v) /∈M
⇒MΓ(FDP (base(v))) ⊆ I(v) = I(u)
45
s
ab
cde
g
fa
f
fs
fe
Figure 4.6: An example of Lemma 8
⇒MΓ(base(v)) ⊆ I(u)
⇒ base(v) ∈ F 3exc(u).
Hence, (u, v) /∈ ET is a contradiction
• Assume that base(u) ∈ A(base(v)). This means that:
index(v) = (index(u), base(u)).
As before
MΓ(base(v)) ⊆MΓ(FDP (base(v)))) ⊆ {I(v) ∪ base(v)}.
Again, since base(v) /∈M , then
MΓ(base(v)) ⊆ I(v) = {I(u) ∪ base(u)}.
Since FDP (base(u)) = FDP (base(v))
⇒MΓ(FDP (base(u))) ⊆ {I(u) ∪ base(u)}
46
s
ab
cde
g
fa
f
fs
fc
Figure 4.7: An example of Lemma 8
⇒ u ∈ B3exc.
Hence, (u, v) /∈ ET is a contradiction.
Proof of Lemma 9. In Lemma 8 we proved that every complete path U ∈ T endswith a vertex representing a leaf with positive utility in G. Let u be the last vertexin a complete path U ∈ T , such that base(u) ∈ M . Assume that there is a leafi ∈ M which was not represented by a vertex in U . This means that u /∈ B1
exc;hence, by step 2 of the Admissible Tree Algorithm (presented in Section 4.2),there will be an edge (u, s(index(u),base(u))) ∈ ET , in contradiction to the assump-tion that u ∈MT .Assume that there is at least one leaf i ∈ M which is represented by two ver-tices v1, v2 ∈ U . This means that base(v1) = i and base(v2) = i. Assumethat dep(v1) < dep(v2). A leaf can only be reached from its direct predecessor,which means that there is an edge (w, v2) ∈ ET such that, base(w) = pre(i)and base(v2) = i. However, since i was already represented by v1 ∈ U ithas to be followed by a backward transition (either to pre(i) or to s); hencei ∈ I(w)⇒ v2 ∈ F 1
exc(w), thus (w, v2) /∈ ET .
Proof of Lemma 10. Assume that U is represented by a complete path U ∈ T . We
47
will show that if U has one of the properties of Proposition 1 (and hence U ∈ D),it could not be represented by a complete path U ∈ T .
1. Assume that U consists of the sequence i→ i1 → ...→ in, such that in = iand all the vertices i1 → ... → in have no reward. There are three possibleoptions for such a sequence:
(a) At some point there was a backward transition from a vertex with noreward ij to s. In this case U will have property 2 of Proposition 1.The proof for this case is presented below.
(b) At some point there was a sequence pre(ij) → ij → pre(ij), whereij is a vertex with no reward (and since pre(ij) /∈M , it also has no re-ward). Let (u, v) ∈ ET and (v, w) ∈ ET represent this sequence. Thismeans that base(u) = pre(ij), base(v) = ij , and base(w) = pre(ij).It also means that index(v) = index(u), index(w) = (index(v), base(v)).If ij ∈ M and has no reward, it means that it was already previouslyexplored in U . By Lemma 9, every complete path U ∈ T has, amongall its vertices, exactly |M | vertices representing all the leaves withpositive reward in the original IVR tree, G. Therefore, if ij appearsin U more than once, by Lemma 9, U could not be represented by acomplete path in T . We now consider the case where ij /∈ M . IfLast index(u) ∈ A(base(v)) then v ∈ F 4
exc(u) (see Remark 4), thusthe edge (u, v) /∈ ET . If Last index(u) /∈ A(base(v)) then v ∈ B2
exc,thus (v, w) /∈ ET .
(c) At some point there was a sequence ij → pre(ij) → ij . Let (u, v) ∈ET and (v, w) ∈ ET represent this sequence. This means that base(u) =ij , base(v) = pre(ij), base(w) = ij , and index(v) = (index(u), base(u)).Hence,Last index(v) = base(u) = base(w) and base(w) ∈ A(base(v)).This means that base(w) ∈ F 4
exc(v) (see Remark 4)⇒ (v, w) /∈ ET .
2. Assume that U consists of the sequence i1 → ...→ in → s such that all thevertices i1 → ...→ in have no reward.Let (u, v) ∈ ET represent the transition in → s. Recall that, the Admis-sible Tree Algorithm only adds edges from u to v, where base(v) = s ifbase(u) ∈ M ; thus, if in /∈ M , (u, v) /∈ ET . If in ∈ M but has no reward,it means that the leaf in was already explored before in U . By Lemma 9, ifin appears in U more than once, U could not be represented by a completepath in T .
3. Assume that U ends with i1 → ...→ in such that all the vertices i1 → ...→in have no reward. It was proven in Lemma 8 that every complete path in Tends with a vertex representing a leaf with positive reward in G. Hence, ifin /∈M , U could not be represented by a complete path in T . If in ∈M buthas no reward, it means that the leaf in was already explored before in U .By Lemma 9, U could not be represented by a complete path in T .
48
4. Assume that U consists of the sequence i → s → direct sequence to → j,where pre(i) 6= s is also a predecessor of j. Let u ∈ V T represent i in thissequence, that is base(u) = i. We will assume that i ∈M , otherwise U willhave property 2 of Proposition 1; this case was proven above. Let v ∈ V T
represent pre(i). The only backward transition in the sequence from i to j isthe first backward transition from i to s. Hence, for every vertex k in the pathrepresenting this sequence, except for u, Last index(k) = base(u) = i.Letw ∈ V T represent pre(i)′s immediate predecessor in the direct sequencefrom s to j. Therefore, under the assumptions here (w, v) ∈ ET . However,
Last index(w) = base(u) ∈ A(base(v))⇒ base(v) ∈ F 4exc(w)
(see Remark 4)⇒ (w, v) /∈ ET .
49
Chapter 5
Index Calculations Over theAdmissible Tree
5.1 Index Calculations
In Section 3.3, we explained how the search protocol on an IVR tree, G, can betranslated into a rooted tree. In Section 4.1, we presented an algorithm which con-structs such a tree, containing only admissible candidates. We also proved that theresulting tree, denoted by T = (V T , ET ), is finite. In this Section we will showthat the optimal candidate (search policy) can be found by calculating indices onthe edges of the Admissible Tree, T .
Let ru,v be the random variable representing the discounted utility of edge(u, v) ∈ ET , given that we successfully reached vertex u after tu units of time.Recall that customers have finite patience which is assumed to be exponentiallydistributed with parameter θ, τ ∼ exp(θ). Thus, a vertex u is successfully reachedonly if customer patience, τ , is greater than the time it takes to reach u, tu.
Let r∗(v) be the utility of the tree spanning from vertex v ∈ V T , given thatwe successfully reached it. Let R∗(v) be E[r∗(v)]. (Note that, as opposed to theoriginal IVR tree G, the Admissible Tree T may have non-leaves vertices withpositive reward. These vertices bases are leaves of G.) If v ∈ MT then r∗(v) issimply the discounted utility of this vertex, which will be denoted by r(v):
r(v) =
rve−αtser(v) −
tser(v)∫0
ce−αsds w.p. Pv
−tF (v)∫
0
ce−αsds w.p. 1− Pv(5.1)
where c is the cost per unit of time and α is a discount factor.If base(v) /∈M , then tser(v), tF (v) and rv all equal zero, thus, r(v) = 0.
50
Note that the success probability of visiting vertex v does not depend on thetime it takes to get there nor the customer patience. This means that the successprobability of visiting a vertex is independent of the success probability of reach-ing it.
Assume that v ∈MT , (u, v) ∈ ET , then:
Ru,v = E[ru,v] = E
I{τ≥tu+tu,v}
e−α(tu+tu,v)R∗(v)−tu+tu,v∫tu
ce−αsds
+I{tu+tu,v>τ≥tu}
− τ∫tu
ce−αsds
∣∣∣∣∣∣ τ ≥ tu
=E[I{τ≥tu+tu,v}
(e−α(tu+tu,v)R∗(v)− c
αe−αtu
(1− e−αtu,v
))]P (τ ≥ tu)
(5.2)
−E[I{tu+tu,v>τ≥tu}
cα
(e−αtu − e−ατ
)]P (τ ≥ tu)
. (5.3)
The numerator of 5.2 equals:
∞∫s=0
∞∫t=tu+s
(e−αtu · e−αsR∗(v)− c
αe−αtu +
c
αe−αtu · e−αs
)fτ (t)ftu,v(s)dtds
= e−αtu∞∫
s=0
(e−αsR∗(v)− c
α+c
αe−αs
)e−θ(tu+s)ftu,v(s)ds
= e−αtu · e−θtu[R∗(v)Ltu,v(α+ θ)− c
αLtu,v(θ) +
c
αLtu,v(α+ θ)
].
Hence,
(5.2) = e−αtu[R∗(v)Ltu,v(α+ θ)− c
αLtu,v(θ) +
c
αLtu,v(α+ θ)
].
51
The numerator of 5.3 equals:
∞∫s=0
tu+s∫t=tu
c
α
(e−αtu − e−αt
)fτ (t)ftu,v(s)dtds
=
∞∫s=0
tu+s∫t=tu
c
α
(e−αtu · θe−θt − c
αe−αtθe−θt
)ftu,v(s)dtds
=
∞∫s=0
c
αe−αtu ·e−θtu
(1− e−θs
)− cα
θ
α+ θe−αtu ·e−θtu
(1− e−(α+θ)s
)ftu,v(s)ds
=c
αe−αtu ·e−θtu− c
αe−αtu ·e−θtuLtu,v(θ)− c
α
θ
α+ θe−αtu ·e−θtu+
c
α
θ
α+ θe−αtu ·e−θtuLtu,v(α+θ).
Hence,
(5.3) =c
αe−αtu − c
αe−αtuLtu,v(θ)− c
α
θ
α+ θe−αtu +
c
α
θ
α+ θe−αtuLtu,v(α+ θ).
Finally we get,
Ru,v = e−αtu[Ltu,v(α+ θ)
(R∗(v) +
c
α
(1− θ
α+ θ
))− c
α
(1− θ
α+ θ
)].
(5.4)
5.2 Finding Optimal Search Policy Algorithm
We now introduce an algorithm which receives, as input, an Admissible Tree,T = (V T , ET ), and uses the indices calculated above to find the optimal searchpolicy. This algorithm will be called Optimal Policy Algorithm (and is based onthe ‘Algorithm for Finding Optimal Policy’ of Granot and Zuckerman [12]).
Let σ∗(u), u ∈ V T , be the optimal policy starting at vertex u. Recall that Tis a rooted tree; therefore, we start the search at root vertex s, and follow a pathalong the edges of T . This path will then be translated into a search protocol onthe original IVR tree, G. Note that the optimal path may not be a complete path.Complete paths in T are paths from the root vertex s to a leaf in MT . Thesepaths actually represent a search that went over all possible leaves (IVR services)of the IVR tree, G. However, it might be optimal to end the search after exploringonly part of the possible IVR services and not all of them. Hence, at every vertexu ∈ V T , there is an option of ending the search, either by terminating it and leavingthe system (will be denoted by ter), or by opting out to an agent service (will bedenoted by opt-out).
52
If the search reached vertex u ∈ V T , we would like to compare the possibleutility of each feasible action. The feasible actions are; (1) moving to one of theimmediate successors of u in T , v ∈ AT (u), (2) terminating the search with noadditional cost or reward, or (3) opting-out to an agent which incurs a (possiblynegative) utility of ropt−out.
Note: The time of decisions is restricted to completions of vertex visits. Thatis, a decision about the above 3 actions can only be made when a vertex visit iscompleted. This means that no actions can be made during transitions from vertexto vertex, or while visiting a vertex (in case this vertex represents IVR service andhas positive visit time). This is without loss since, by our formulation, rewards arereceived only at completions of vertex visits.
We showed that the expected discounted utility of edge (u, v) ∈ ET , given thatwe reached vertex u is given by Ru,v. However, while in u, it is obvious that everypossible reward from now on, will be discounted by the time it took us to get to u,tu. Hence, it is enough to calculate:
R∗u,v = Ltu,v(α+ θ)
(R∗(v) +
c
α
(1− θ
α+ θ
))− c
α
(1− θ
α+ θ
)(5.5)
for every v ∈ A(u) and compare the results (to each other and to ropt−out).R∗u,v will be the index of edge (u, v).
For convenience we will assume that ropt−out > 0. This means that after end-ing the search it is preferable to opt-out, instead of leaving the system. However, ifropt−out ≤ 0, then we will only substitute ropt−out with 0, and whenever the actionopt-out is chosen, we will chose the action ter instead.
The Optimal Policy Algorithm, receives an Admissible Tree, T , and dividesit into q levels. The first level, denoted by L1 includes all the tree leaves, the lastlevel, denoted by Lq includes the root vertex s. For every k = 2, ..., q − 1:
Lk =
{v : all edges emanating from v lead to a vertex in
k−1⋃i=1
Li
}.
First, the algorithm calculates the expected utility of vertices in L1 (tree leaves),according to equation 5.1. The algorithm then calculates the indices of edges em-anating from vertices in L2 according to equation 5.5. Based on these indices,the algorithm chooses the optimal policy for every vertex in L2 and calculates theexpected utility which will be earned from this vertex by performing this policy.The algorithm repeats this process with vertices in Lk, k = 3, ..., q. The algorithmreturns an optimal policy which is the action to be made at each stage, as well asthe expected utility of the search. We will now formally state the algorithm; furtherexplanations will be given within the algorithm itself.
53
5.2.1 Optimal Policy Algorithm
0. Initialization:
Divide the Admissible Tree T = (V T , ET ) into q levels in the followingway:
L1 ={v : v ∈MT
}∀k ≥ 2 : Lk =
{v : all edges emanating from v lead to a vertex in
k−1⋃i=1
Li
}Lq = {s}.
1. Step 1:∀v ∈ L1 Calculate R∗(v) = E[r∗(v)], according to equation 5.1(E[r∗(v)] = Pv
(rvLtser(v)(α)− c
α+c
αLtser(v)(α)
)+(1− Pv)
c
α
(LtF (v)(α)− 1
))k = 2.
2. Step 2:For all u ∈ Lk, ∀v ∈ AT (u) (the set of immediate successors of u in T ),calculate R∗u,v according to equation 5.5.
3. Step 3:For all u ∈ Lk,
• If maxv∈AT (u)
R∗u,v ≤ ropt−out:
Set R∗(u) = E[r(u)] + ropt−out, according to equation 5.1.
Set σ∗(u) = {opt-out}.
Explanation: If the indices of all the edges emanating from u are lowerthan ropt−out it is better to stop the search at vertex u and opt out to anagent.
• If maxv∈AT (u)
R∗u,v > ropt−out:
Set r∗(u) =
(ru + max
v∈AT (u)R∗u,v
)e−αtser(u) −
tser(u)∫0
ce−αsds w.p. Pu(max
v∈AT (u)R∗u,v
)e−αtF (u) −
tF (u)∫0
ce−αsds w.p. 1− Pu.
54
Set R∗(u) = E[r∗(u)]E[r∗(u)] = Pu
[(ru + max
v∈AT (u)R∗u,v)Ltser(u)(α)− c
α+c
αLtser(u)(α)
]
+ (1− Pu)
[(max
v∈AT (u)R∗u,v
)LtF (u)(α)− c
α+c
αLtF (u)(α)
] .
Set σ∗(u) = arg maxv∈AT (u)
{R∗u,v}.
Explanation: If base(u) ∈ M , it has a reward ru, earned with prob-ability Pu. The time spent in u itself is tser(u) with probability Puand tF (u) with probability 1 − Pu. Thus, if it is optimal to con-tinue the search after u we will choose the optimal action accordingto arg max
v∈AT (u)
{R∗u,v}. However, the total utility of u depends on whether
u′s exploration was successful or not, since we need to discount theutility of u by the time we spent in u and pay a fee of c units per eachunit of time we spent in u. In addition, if the exploration of u wassuccessful we need to add the reward from u itself, ru, to the expectedutility of all following subsequent which is expressed by max
v∈AT (u)R∗u,v.
Notice that if base(u) /∈M we simply get R∗(u) = maxv∈A(u)
R∗u,v
k = k + 1.
If k = q finish, else, return to Step 2.
55
Chapter 6
Exploratory Data Analysis
6.1 Data Description
The data for this research was generously provided by the Center for Service En-terprise Engineering (SEE) at the Technion: http://ie.technion.ac.il/labs/serveng/
The statistical analysis was carried out using the SEEStat platform developedby SEE. Our analysis is based on data from an IVR system of an Israeli Bank,which will be called from now on ILBank. ILBank call center operates 24 hours aday, 7 days a week, with 400-500 agents working on weekdays, and 50-170 agentson weekends. During peak hours, there are about 200 agents occupying the callcenter. Each day the call center processes, on average, 65K calls that start theirservice at the IVR system. About 65% of the calls complete their service in theIVR system. In the other 35%, customers seek additional service from an agent.These calls will enter the agents queue; some will receive service, and others willabandon the queue before receiving service. A transition from the IVR into theagents’ queue will be called opt-out.
The call center offers several types of services, the most common being “Pri-vate” (about 80% of the calls), which stands for retail banking. The customers ofILBank call center are divided into 4 priority groups: High, Medium, Low andUnidentified customers (customers which are not recognized by the system as theycall).
Our database consists of ACCESS tables of daily IVR logs from May 1, 2008to June 30, 2009. Table 6.1 presents the total number of calls completed in the IVRagainst the total number of calls requesting an agent service after the IVR service(calls resulting in opt-out), in that period. Table 6.2 presents the total number ofcalls from each priority group.
Our database also includes data on completed calls and agents, from April 2007to June 2009, with additional data such as: service duration, waiting time in queue,departure type (whether the customer received service, abandoned the queue, orwas disconnected), and the received service type.
56
Total Number of Calls Average per Day % Out of TotalCompleted in IVR 17,616,095 42,144 64.6%Requesting Agent Service 9,638,970 23,060 35.4 %Total 27,255,065 65,204 100%
Table 6.1: Total number of IVR calls by their outcome, May 2008 to June 2009
Customer Type Number of Calls Avg. Number of Calls per Day % Out of TotalHigh Priority 3,899,011 9,328 14.3%Medium Priority 9,942,048 23,785 36.4 %Low Priority 6,385,826 15,277 23.4%Unidentified 7,028,180 16,814 25.8%
Table 6.2: Total number of IVR calls, by customer type, May 2008 to June 2009
Figure 6.1 presents the ILBank IVR menu as it was described in the bank web-site, during the period of our data. The leaves of the IVR tree presented in Figure6.1 are the services provided by the ILBank IVR system. A customer may be look-ing for one or more IVR services. Each service can be reached by following a seriesof menus (non-leave vertices), which customers listen to and choose to follow.
A single IVR transaction may consist of more than one segment, with eachsegment occupying a record in the data table. Each record in the IVR databaseincludes a field called “event” and a field called “sub-event”. See Appendix A foran example of an IVR table in our database.
We conceptualized each service provided by the ILBank IVR as a combina-tion of an event and a sub-event. Events refer to a group of related services, whilethe sub-event specifies the particular service within this group. For example, anevent may be: “Stock Exchange” and the sub-events included within this event are:“Stock Exchange Israel”, “Stock Exchange USA”, “Stock Exchange Europe”, etc.There are two additional events which do not specify an IVR service: “Identifica-tion”, which is the opening event in most IVR transactions, and “Agent” whichspecifies a transition to the agents’ queue.
Our database consists of about 90 sub-events while, according to Figure 6.1,there are only about 40 IVR services offered by the ILBank IVR system. However,we were able to match most of the offered services to the relevant sub-events in ourdatabase, where sub-events that were not matched to any service represent eithersystem actions or follow-up actions/announcements to another sub-event (whichwas chosen to represent the IVR service). We should mention that although ex-tensive work was done in order to match the right sub-events to the IVR services,there still might remain some inaccuracies. We are rather confident, however, thatthis does not affect any of the results presented in the rest of this work.
57
Identification
Pla
y
Accou
nt
Bala
nce
Checking Account
and General
issues
Account Activity for Requested Date
Account Activity Today
Recent Account Activities
Account Summary
Current Balance
Account Activities
Checking Account and
General Activities
Deposit Activities
Credit Activities
Trust Funds Activities
Cheque Book Order
Transfer between Client Account
Change Password
Select Code
Change
Password by
SystemDaily Interest Deposit
Other Deposits Activities
Credit Balance Information
Credit Payment
Credit Establishing
Credit Balance Explanation
Trust Funds Sell
Stock Exchange
Securities Order Status
Selected Securities
Stock Exchange – Israel and World
Trust Funds Buy
Stock Exchange – Europe
Stock Exchange – Israel
Stock Exchange – USA
Stock Exchange – Asia
Stock Exchange – Options & Futures
Stock Exchange – Securities Rates
Stock Exchange – Interbank Rates
Investments
Securities Balance
Deposits Balance
Saving Plans Balance
Foreign Currency
Foreign Currency Current Account
Provident Funds Balance – by Fund Type
Provident Funds Balance – Specific Fund
Securities Portfolio Rates
Securities Portfolio Details Today
Securities Portfolio Details for Previous Day
Provident Funds Balance
Foreign Currency Current Account - by Currency
Foreign Currency Deposits Balance
Foreign Currency Deposits Balance – by Currency
Foreign Currency Rates
Loans, Mortgages
and Credit Card
Information
Loans and Guarantees Details
Credit Card Vouchers
Mortgages
Fax
Credit Card Credit
Figure 6.1: ILBank IVR menu
58
To explain the limitations of our data, let us consider a specific case. An IVRservice usually starts with identification, which is the root vertex of the tree inFigure 6.1. Assume that a customer enters the IVR system, uses an ID numberand a password for identification, chooses the menu ‘Current Account and Gen-eral Issues’ and then selects the first option within this menu which is the service‘Account Summary’. In our data, this customer call will have two segments, oc-cupying two records in the data table: ‘Identification’ and ‘Account Summary’.We are assuming that the duration of the first segment (Identification) includes theidentification time, as well as the time spent in the menu ‘Current Account andGeneral Issues’. The duration of the second segment (Account Summary) will bethe time spent in the service itself, from the moment the customer reached thisservice until leaving the system. If the customer subsequently chooses another ser-vice, for example, ‘Recent Account Activity’, we will have a third segment with thetime spent in this service, where the time spent in menus between the service ‘Ac-count Summary’ and the service ‘Recent Account Activity’ is apparently added tothe time spent in the service ‘Account Summary’. Thus, time between two activitiesis always added to the chronologically first one. Notice that there is no indicationwhich exact menus were visited by the customer until reaching a specific IVR ser-vice, nor the time spent in each menu.
6.2 Demand for IVR Services
ILBank IVR systems offer about 40 services. Table 6.3 presents the relative de-mand frequency of each service. For example, if service ‘Account Summary’ has arelative demand frequency of 5.7%, it means that 5.7% of the calls to the ILBankIVR system reached this service (once or more).
One can see that only 15 services out of the 40 available are reached in morethan 1% of the calls; only 8 services are reached in more than 2% of the calls,and only 3 services are reached in more than 5% of the calls. The most frequentservice is ‘Play Account Balance’ which is actually given automatically after theidentification. The fact that most of the available IVR services are rarely visited(0.00% stands for less than 0.5%of the calls) raises some significant and interestingquestions, which we shall elaborate on in Chapter 7.
59
# IVR Service Relative Demand Frequency1 Play Account Balance 68.24%2 Recent Account Activity 11.36%3 Account Summary 5.38%4 Credit Card Vouchers 3.73%5 Stock Exchange Israel 2.81%6 Selected Security 2.43%7 Transfer between Client Accounts 2.23%8 Account Activity Today 2.15%9 Selected Password 1.78%10 Account Activity for Requested Dates 1.51%11 Stock Exchange - Interbank Rates 1.30%12 Stock Exchange Europe 1.25%13 Securities Portfolio Rates 1.15%14 Stock Exchange USA 1.08%15 Securities Order Status 1.06%16 Current Balance 0.91%17 Change Password by System 0.62%18 Securities Portfolio Details Today 0.55%19 Deposits 0.51%20 Stock Exchange - Options and Futures 0.40%21 Provident Funds Balance - by Fund Type 0.34%22 Stock Exchange Asia 0.27%23 Stock Exchange - Securities Rates 0.18%24 Credit Card Credit 0.11%25 Saving Plans 0.08%26 Cheque Book Order 0.08%27 Daily Interest Deposit 0.06%28 Credit Balance 0.03%29 Foreign Currency Current Account 0.02%30 Credit Establishing 0.02%31 Foreign Currency Deposits Balance 0.01%32 Trust Funds Sell 0.01%33 Credit Information 0.01%34 Trust Funds Buy 0.00%35 Foreign Currency Rates 0.00%36 Foreign Currency Current Account - by Currency 0.00%37 Foreign Currency Deposits Balance - by Currency 0.00%38 Provident Fund Balance - Specific Fund 0.00%39 Credit Payment 0.00%40 Mortgages 0.00%41 Fax 0.00%
Table 6.3: IVR services - relative demand frequency, May 2008 to June 200960
6.3 Customer Paths
In order to present and analyze customer flow within the ILBank IVR system, weused SEEGraph, a structure-mining tool developed at the Technion SEELab. Withthe SEEGraph tool, we created animations, based on ILBank data from September16, 2008, of real-time customer transitions within the system. These animationsare somewhat similar to the user-path diagrams presented in Suhm and Peterson[23] and the process maps presented in Kaplan and Porter [16]. However, theanimations present the system dynamics as it evolves through the day, as well ascomplete customer paths, instead of an average overview of the system.
In Figure 6.2 and Figure 6.3, the blue rectangles represent IVR services. Theupper green ellipse represents the identification phase. The lower orange ellipserepresents opt-out to the agents’ queue. The edges between the rectangles representtransition between the different services, identification and opt-out. The animationof Figure 6.2 is accessible in Hybrid Animation. The animation of Figure 6.3 isaccessible in Network Animation.
In the Hybrid Animation (Figure 6.2), each colored circle moving along theedges represents a customer in the system. The time it takes a circle to move alongan edge from X to Y represents the customer sojourn time in activity X. Hence,relatively slow and fast motion of circles corresponds to long and short sojourntime, respectively. (Relatively, since the speed depends also on the length of thearc.) Circles moving along edges emanating from a rectangle and ending with a dotrepresent customers ending their service at the IVR and leaving the system. Thedynamic bar above each IVR service or phase (Identification, Agent) representsthe number of customers within this service or phase at a given moment. In theNetwork Animation (Figure 6.3), the thickness of an edge from X to Y correspondsto the number of customers that are in X and are about to move to Y (and similarlythere exists for edges representing customer exits).
A few interesting phenomena arise from the animation: (For orientation, wedivided the animation into 8 segments. The segment number appears on the bottomleft corner of the screen.)
• Almost all the calls start their IVR service at the identification phase(see segment 1 in the animations).
• Some of the customers leave the system right after the identification phase.According to our data analysis, around 8%-9% of the calls entering the IVRsystem leave the call center immediately after the identification phase.(See segment 2 in the animations; notice the arrow emanating from the iden-tification ellipse ending with a dot.)We shall define these customers as abandoning customers since it is clearthat these customers left the system without receiving any relevant service.
• After the identification phase, most of the calls either turn to ‘Play Account
61
Figu
re6.
2:IL
Ban
kH
ybri
dan
imat
ion
(lin
k)
62
Figu
re6.
3:IL
Ban
kN
etw
ork
anim
atio
n(l
ink)
63
Path Relative FrequencyIdentification→ Play Account Balance 21.85%
Identification→ Play Account Balance→ Opt-out (Agent) 17.88%Identification→ Opt-out (Agent) 12.30%
Identification 8.11%Identification→ Play Account Balance→ Recent Account Activity 5.27%Identification→ Play Account Balance→ Account Summary 1.30%
Not a Client→ Opt-out (Agent) 1.04%
Table 6.4: Frequent paths
Balance’ (which is apparently given automatically) or opt-out to the agents’queue. Most of the calls that exit ’Play Account Balance’ either leave thesystem or opt-out to the agents queue. This means that most of the customersuse the IVR system only for checking their balance or as a gateway to theagents.
• Some of the calls return to the IVR system after an opt-out to the agents’queue. We can learn from segment 3 in the animations that these customersprobably receive a password from the agent and then return to the IVR iden-tification phase.
• There are transitions from one service to the other, for example: from ‘Re-cent Account Activity’ to ‘Account Summary’ and the other way around. Thismeans that customers are indeed reaching more than one service in a singlecall (see segment 3 in the animations).
• There are loops between two services as well as self-loops. Most of theseloops occur in the Stock-Exchange services where customers receive infor-mation about several stock markets or several securities (see segment 4 inthe animations).
We used a process mining software called “Disco” (available at: http://www.fluxicon.com/) to extract the relative frequency of different customerpaths. The most frequent paths and their relative frequency are presented in Table6.4. The data considered was taken from 8 days in 2009: May 31, which was aSunday, hence a relatively loaded day, plus one week from June 14th to June 20th.During that period there was a total of 514,679 calls to the ILBank IVR.
We see again that most of the ILBank customers have very limited interactionswith the IVR system. Only 3 out of the 40 available IVR services are actuallyexpressed in the frequent paths. Notice that the paths presented here are all thepaths with relative frequency higher than 1%. Their total relative frequency is67.7%. This means that there are many more other paths (around 2000) with verysmall frequencies. This is due to the fact that even if there are several paths which
64
go through the same services, the order of the services within each path may bedifferent.
6.4 Success Rate of IVR Services
Our research deals with IVR systems offering several automated IVR services. TheILBank IVR system, for example, offers about 40 services. The exploration of IL-Bank IVR service durations revealed some very interesting phenomenon: the timedistribution of some services depict a distinct peak around 2-5 seconds. Figures 6.4to 6.6 are examples of services with such a distribution. Note that we are focusingon the time customers spent in a specific IVR service, from starting it until eitherreaching another IVR service, leaving the system or opting out to an agent. Weassume that the time spent in the identification phase, as well as in all the menusand prior services are not included in this duration.
An experimental “proof” that very short service durations correspond to aban-donments: ILBank had an online simulation of its IVR, which we experimentedwith. (It is presently inaccessible.) We were able to cover, and hence measurethe durations of most frequent IVR services, and in particular those presented inFigures 6.4 - 6.6. Our experiment conclusively demonstrated that customers whospent a very short time on a service, could not have received any useful informa-tion from that service. Thus, we deduce that the peaks near the origin correspondto customers reaching these specific services by mistake. We are assuming thatthe customers creating such peaks near the origin in a specific service, presum-ably, reached this service, realized within a few seconds that they have reached thewrong service, and left. We believe that this is the most reasonable explanation forthe observed phenomenon. We acknowledge, however, that there could be otherexplanations, such as system recording errors or mismatches between data logsand actual IVR transactions.
The histogram of the four services examined depict additional peaks, around20 seconds and around 50 seconds, (see Figure 6.8). According to our measure-ments (not including ‘Credit Card Vouchers’ which we could not measure), therecould be some information given in the first 20 seconds of the service. Hence, wededuce that the two peaks, around 20 seconds and around 50 seconds, represent theactual service duration. For example, the service ‘Recent Account Activity’ is com-posed of several announcements: information about current balance, withdrawals,deposits, cash-back, credit-card charges, etc. It may be that some of the customerschoose to listen only to the first announcement and others choose to listen to all theinformation.
65
0.000
1.000
2.000
3.000
4.000
5.000
6.000
7.000
8.000
00:02 00:11 00:21 00:31 00:41 00:51 01:01 01:11 01:21
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Current Account - Recent Account ActivityTotal for November 2008 to June 2009,All days
Threshold Value
Figure 6.4: Recent Account Activity duration, N = 1, 983, 161
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
11.00
12.00
13.00
14.00
15.00
00:02 00:11 00:21 00:31 00:41 00:51 01:01 01:11 01:21 01:31 01:41
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Account SummaryTotal for November 2008 to June2009,All days
Figure 6.5: Account Summary duration, N = 1, 081, 992
66
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
11.00
12.00
13.00
14.00
15.00
00:02 00:11 00:21 00:31 00:41 00:51 01:01 01:11 01:21 01:31 01:41 01:51
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Account Activity TodayTotal for November 2008 to June 2009,All days
Figure 6.6: Account Activity Today duration, N = 353, 328
0.000
0.250
0.500
0.750
1.000
1.250
1.500
1.750
2.000
2.250
2.500
2.750
3.000
3.250
3.500
00:02 00:17 00:32 00:47 01:02 01:17 01:32 01:47 02:02
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Credit Card VouchersTotal for November 2008 to June 2009,All days
Figure 6.7: Credit Card Vouchers duration, N = 608, 529
67
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
11.00
12.00
13.00
14.00
15.00
00:02 00:17 00:32 00:47 01:02 01:17 01:32 01:47 02:02
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank IVR_subevent_durationTotal for November 2008 to June 2009, All days
Current Account - Recent Account Activity Account Summary Account Activity Today Credit Card Vouchers
3 seconds
17 seconds 56 seconds
Figure 6.8: IVR services duration
68
How does one estimate the success probability of an IVR service, or equiva-lently its abandonment probability? A naive procedure, based on the distributionof a service duration, combined with our own experiments with that service, is toassociate with a service a threshold value X: Customers staying in a service lessthan its threshold value are considered to have an unsuccessful completion of thisservice. For example, the vertical line in Figure 6.4 represents a threshold valueof the service ‘Recent Account Activity’. This threshold value equals 6 seconds:13.5% out of all the visits to the service ‘Recent Account Activity’ ended within 6seconds or less; thus we estimate the success rate of this service by 86.5%. Thereare scientific procedures to determine such thresholds: classifications in MachineLearning [14]. Our threshold was based on our experiments with the actual IVRsystem and the service duration distribution. This threshold is somewhat a lowerbound since we realized no relevant information could have been received duringthe first 6 seconds of the service. However, as seen in the histogram, even if we setthe threshold value to be 10 seconds, it will not reduce the success rate by morethan 2%.
A more precise method of estimating the success rate of each IVR service wascarried out using SEEStat’s mixture-fitting tool. Using this tool, one can fit a mix-ture of theoretical distributions to the empirical distribution. The first step is tochoose the theoretical distributions form (either homogeneous mixture or hetero-geneous mixture). Following extensive experiments, we choose to fit a mixture ofsix Gamma distributions. The mixture-fitting tool then automatically provides thebest parameters and mixture proportions that fit the data. The Gamma distribu-tion has a scale parameter θ, and a shape parameter k. It was chosen since it is arich family of continuous distributions. Moreover, when k is an integer, the distri-bution represents the sum of k i.i.d exponential random variables with parameter θ.
In Figures 6.9 and 6.10, a mixture of six Gamma distributions was fitted tothe services ‘Recent Account Activity’ and ‘Account Summary’ respectively. Themixing proportion for both services and the distribution parameters are presentedin Tables 6.5 and 6.6.
69
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
00:02 00:11 00:21 00:31 00:41 00:51 01:01 01:11 01:21
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank IVR_subevent_duration, Current Account - Recent Account ActivityTotal for November 2008 to June 2009, All days
Fitting Mixtures of Distributions
Empirical Total Gamma Gamma Gamma Gamma Gamma Gamma
Figure 6.9: Fitting mixture of distributions, Recent Account Activity
Fitting mixture of distributions - Recent Account ActivityComponents Mixing Proportions (%) Scale Shape Mean STD1. Gamma 4.86 0.39 10.13 3.99 1.252. Gamma 8.70 0.02 134.46 2.29 0.1983. Gamma 36.73 3.32 6.30 20.94 8.344. Gamma 27.91 2.44 21.60 52.77 11.355. Gamma 11.98 0.21 273.52 57.50 3.486. Gamma 9.82 10.08 9.25 93.22 30.66
Table 6.5: Fitting mixture of distributions, Recent Account Activity
We assume that the mixture component (or components) that captures the peaknear the origin, corresponds to unsuccessful completions of this service (theirmeans are highlighted in green in Table 6.5 and Table 6.6). Thus, this compo-nent proportion in the mixture is an estimator for the complement of this service’s
70
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
11.00
12.00
13.00
14.00
15.00
16.00
17.00
00:02 00:11 00:21 00:31 00:41 00:51 01:01 01:11 01:21 01:31 01:41
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank IVR_subevent_duration, Account SummaryTotal for November 2008 to June 2009, All days
Fitting Mixtures of Distributions
Empirical Total Gamma Gamma Gamma Gamma Gamma Gamma
Figure 6.10: Fitting mixture of distributions, Account Summary
Fitting mixture of distributions - Account SummaryComponents Mixing Proportions (%) Scale Shape Mean STD1. Gamma 13.47 1.03 5.50 5.68 2.422. Gamma 15.93 0.02 143.18 2.32 0.193. Gamma 9.30 0.64 81.79 52.64 5.824. Gamma 35.20 5.36 4.19 22.49 10.985. Gamma 25.72 13.65 5.76 78.72 32.786. Gamma 0.35 5.96 39.95 237.93 37.64
Table 6.6: Fitting mixture of distributions, Account Summary
success rate. For example, in Figure 6.9, the first two components of the mixturecapture the peak at the origin, and their total proportion in the mixture is 13.5%.The success rate of the service ‘Recent Account Activity’ will be therefore 86.5%,as estimated by the threshold value method.
71
6.5 Further Support to an Abandonment Hypothesis
In this section, we explore the effect of cumulative customer experience within theIVR system on service durations and success rate.
Figures 6.11 and 6.12 (Figure 6.12 is a zoom in on Figure 6.11) present thedistribution of the time spent in the identification phase (given that it resulted in arecognized numeric value), as a function of the number of calls made to the IVRby the same customer prior to the current call. The relevant statistics are givenin Table 6.7. We counted calls only for identified customers who did not appearin our data prior to November 1st 2008; in particular, this means that if they hadprevious calls to the IVR, they had to be at least six months earlier (or they hadcalls as unidentified customers).
The thick blue line presents the total distribution of all customer calls. Clearly,as customer experience within the IVR system grows, the distribution of the timespent in the identification phase is shifted left. One can also see that the total dis-tribution is a mixture of the distributions of each customer’s call number. Table 6.7reveals that the average time spent in the identification phase decreases dramati-cally as customers gain more experience within the system. Recall that the time inthe identification phase presumably includes the time in subsequent menus, if therewere any. The phenomenon observed is therefore not surprising, and is actually anindicator for customers learning: as customers gain experience within the IVR sys-tem, they do not need to listen to the opening announcement, nor the followingmenus. Instead, they automatically give their identification number and choose theneeded options to reach a desired service. This learning process is manifested byshorter average time and smaller variations.
StatisticsN Mean (sec) STD (sec)
Total 852,371 54 551 267,988 83 732 100,213 59 513 61,922 53 454 43,263 49 39[5-10) 112,323 43 36[10-15) 54,397 36 24[15-20) 35,188 33 21[20-30) 45,668 30 19[30-40) 28,669 28 18[40-50) 19,461 27 18[50-100) 43,779 24 15
Table 6.7: Statistics of time in the ID phase, as a function of the number of priorcalls
72
0.000
0.500
1.000
1.500
2.000
2.500
3.000
3.500
4.000
4.500
5.000
5.500
00:02 00:32 01:02 01:32 02:02 02:32 03:02 03:32
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, ID time by experienceTotal for November 2008 to June 2009,All days
Total 1 2 3 4 [5 - 10) [10 - 15) [15 - 20) [20 - 30) [30 - 40) [40 - 50) [50 - 100)
Figure 6.11: Distribution of the time in the ID phase, as a function of the numberof prior calls
6.5.1 Customer Experience Effect on Time Spent in IVR Services
In Section 6.4, we presented the time distribution of several ILBank IVR services,focusing on distributions with a distinct peak near the origin. We assumed thatthese peaks are caused by unsuccessful completions of these services and sug-gested two methods to estimate success rate of a service. We are now interested inexploring the effect of IVR experience on the time spent in specific IVR servicesand their success rate. We will use the service ‘Recent Account Activity’ as a testcase.
Figure 6.13 presents the distribution of ‘Recent Account Activity’ duration as afunction of the total number of previous calls the customer has made to the IVR.As before, we considered all identified customers having no calls before November1st 2008.
The thick blue line in Figure 6.13 expresses the total service duration distri-bution. The red line expresses the service duration distribution for new customers(customers with no prior calls). The green line expresses the service duration dis-tribution for customers who had at least 50 calls to the IVR system. It seems likethe more experienced customers have a lower success rate in this specific service,which is depicted by the higher peak near the origin (see Figure 6.14 for zoom in).
73
0.000
0.500
1.000
1.500
2.000
2.500
3.000
3.500
4.000
4.500
5.000
5.500
00:02 00:08 00:14 00:20 00:26 00:32 00:38 00:44 00:50 00:56 01:02
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, ID time by experienceTotal for November 2008 to June 2009, All days
Total 1 2 3 4 [5 - 10) [10 - 15) [15 - 20) [20 - 30) [30 - 40) [40 - 50) [50 - 100)
Figure 6.12: Zoom in - Distribution of the time in the ID phase, as a function ofthe number of prior calls
In addition, the service duration of more experienced customers displays a widerdispersion in the second peak, around 55 seconds (see Figure 6.15 for zoom in).
The service ‘Recent Account Activity’ is composed of several announcementsregarding recent activities in the customer account, such as: withdraw, deposit pay-ment, deposit interest, cash-back and credit-card payment. The announcements aregiven automatically, one after the other. However, a customer may be uninterestedin all the announcements and quit after getting the relevant information. The du-ration of each announcement clearly depends on the customer account. As statedin Section 6.4, we believe that the second peak in the distribution, around 55 sec-onds, presents the service duration in cases of successful service completion. It istherefore consistent with the assumption that experienced customers have highervariability in their actions due to higher variability in their financial channels (dif-ferent deposits, etc.)
The fact that experienced customers demonstrate a lower success rate requiresfurther explanation since this seems counter-intuitive. It could be that the veryexperienced customers may be using the system “blindly”, as they believe theyremember the right path to a specific service without listening to the menus on theway; however, this kind of behavior leads to mistakes and causes them to reachthe wrong service and leave it within a few seconds. Conversely, inexperiencedcustomers are listening carefully to each menu, hence reaching the right service
74
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
11.00
12.00
13.00
14.00
15.00
00:02 00:11 00:21 00:31 00:41 00:51 01:01 01:11 01:21 01:31 01:41
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Current Account - Recent Account ActivityTotal for November 2008 to June 2009,All days
Total 1 2 3 4 [5 - 10) [10 - 15) [15 - 20) [20 - 30) [30 - 40) [40 - 50) [50 - 100)
Figure 6.13: Distribution of the time in ‘Recent Account Activity’, as a function ofthe number of prior calls
with higher probability. Notice that customer experience here is regarded as theirnumber of previous calls to the system. Still, they may have limited experience, orno experience at all, with this specific service. Therefore, we checked the effect ofcustomer experience with a specific service (‘Recent Account Activity) on the timespent in this service.
We distinguish between two types of experience: across calls experience, andwithin call experience. The customer experience with the service across calls isthe number of times the customer has reached this specific service in different calls(i.e. if a customer reached this service several times during one call, only the firsttime will be considered). Figure 6.16 presents the service duration distribution of‘Recent Account Activity’ as a function of customer experience with this serviceacross calls. The phenomenon observed in Figure 6.13 holds here as well.
The customer experience with this specific service within call is the numberof times this service was reached in a single call. Figure 6.17 presents the serviceduration distribution of ‘Recent Account Activity’ as a function of the number ofvisits to this service within one call. The first visit to this service in each call de-picts the same distribution as seen in Figure 6.4. However, if a customer reachesthis service again, within the same call, the probability that it was unintentional in-creases. This is very reasonable considering this specific service; this is in contrastto services such as ‘Selected Securities’, where customer returns are intentional,
75
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
11.00
12.00
13.00
14.00
15.00
1.50 4.00 6.50 9.00 11.50 14.00 16.50 19.00
Rel
ativ
e fr
eque
ncie
s %
Time (resolution 1)
ILBank, Current Account - Recent Account ActivityTotal for November 2008 to June 2009,All days
Total 1 2 3 4 [5 - 10) [10 - 15) [15 - 20) [20 - 30) [30 - 40) [40 - 50) [50 - 100)
Figure 6.14: Distribution of the time in ‘Recent Account Activity’,as a function ofthe number of prior calls, zoom in from 0 to 20 seconds
0.000
0.500
1.000
1.500
2.000
2.500
3.000
3.500
4.000
4.500
5.000
00:45 00:47 00:49 00:51 00:53 00:55 00:57 00:59 01:01 01:03 01:05 01:07 01:09
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Current Account - Recent Account ActivityTotal for November 2008 to June 2009,All days
Total 1 2 3 4 [5 - 10) [10 - 15) [15 - 20) [20 - 30) [30 - 40) [40 - 50) [50 - 100)
Figure 6.15: Distribution of the time in ‘Recent Account Activity’, as a function ofthe number of prior calls, zoom in from 45 to 75 seconds
76
for example, to learn about the value of several securities. Notice the additionalpeak at 11 seconds, which is yet to be explained.
0.00
2.50
5.00
7.50
10.00
12.50
15.00
17.50
20.00
00:02 00:11 00:21 00:31 00:41 00:51 01:01 01:11 01:21 01:31
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Current Account - Recent Account ActivityTotal for November 2008 to June 2009,All days
Total 1 2 3
Figure 6.16: Distribution of the time in ‘Recent Account Activity’, as a function ofthe number of prior visits to the service, across calls
To summarize, our exploratory data analysis revealed that experienced cus-tomers spend less time in IVR menus and services. Their familiarity with the IVRsystems enables them to reach IVR services without fully exploring all the menuson the way. However, it seems that this behavior leads to a higher mistakes rate,which means that they are prone to reach unwanted services by mistake and aban-don them.
77
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
11.00
12.00
00:02 00:17 00:32 00:47 01:02 01:17 01:32 01:47 02:02 02:17
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Current Account - Recent Account ActivityTotal for November 2008 to June 2009, All days
Total 1 2 3 4 5
Figure 6.17: Distribution of the time in ‘Recent Account Activity’, as a function ofthe number of prior visits to the service, within one call
78
Chapter 7
Model Implications
In Chapter 3, we modeled an IVR system as a rooted tree. In Chapter 5 we used thismodel to derive the policy of the individual customer. Notice that the individualcustomer in fact represents a group of customers with the same perceived rewardsand cost, the same success probabilities, service time distributions (within menusand IVR services) and patience distribution. Nonetheless, the power to change theIVR design is in the organization’s hand. This should be done in a way that willbe beneficial for the organization or, in other words and taking one view, increaseits profit. In the simplest case, where each customer is looking for a single IVRservice, the optimal path to reach this service is predetermined as the direct pathfrom the root vertex (main menu) to the desired leaf (service). Thus, in this case,the optimal policy inferred from our model will actually determine whether thecustomer’s patience will be long enough to reach the desired service or not. If thecustomer’s patience will not enable reaching the service, then the preferred policyis to not enter the IVR system at all (i.e., opt-out directly to agent service). Inthis simple case, we can fairly easily alternate our model in order to calculate theorganizational revenue from a specific IVR design, as we now briefly explain:
When each customer is looking for a single IVR service, there is no need toexpand the original IVR tree, as there will be no returns. The reward, and costparameters, r and c, will be the organization reward from each service and theorganizational cost per each unit of time in the system. We can think of all thecustomers entering the IVR system as a single customer, adhering to the above pa-rameters. For simplicity, we will call it a “Super customer”. Since the organizationis interested in total revenues by the end of each month/week/day, we can considerthe discount factor α to be zero, as the time spent in the IVR (minutes) is negligi-ble in terms of discounting. The demand frequency of each IVR service (as seen,for example, in Table 6.3) will be used as an additional parameter of the edges –the probability that the “Super customer” will choose that edge. Figure 7.1 is anexample of that; the numbers under each leaf are the relative demand frequencies;the red numbers on the edges are the corresponding probabilities of each edge.
Denote by Pu,v the probability that the “Super customer” will choose edge
79
s
ab
cde
fg
0.050.35
0.20.3
0.1
0.05
0.4
0.35
0.4
0.4
0.90.2
0.9
0.3
0.9
0.1 0.9
Figure 7.1: Calculating organizational profit - Simple example.
(u, v) while in vertex u. Since Pu,v is independent of all the other parameters, foreach edge (u, v) ∈ V , we get:
R∗u,v = Pu,vLtu,v(θ)(R∗(v) +
c
θ
)− c
θ(instead of R∗u,v from Equation 5.5).
Following the Optimal Policy Algorithm (5.2.1), we only need to change step3, and set:
R∗(u) =∑
v∈A(u)
R∗u,v,
for every vertex u ∈ V , and every value of R∗u,v, including negative values.The organizational revenue from its IVR system per day, for example, can be thenconsidered as X × R∗(s), where X is the number of customers per day and s isthe root vertex of the IVR tree, i.e. the starting point of the IVR main menu.
In the general case, a customer may be looking for several IVR services. Inthis case, the optimal policy derived from our model, for each individual customer,states which of the desired services will in fact be reached, in what order (whichof the services will be reached first, which will be reached second and so forth)and what will be the path between one service and the other. In this case, ourmodel can satisfy the optimal policy for groups of customers with homogeneouscharacteristics. The calculation of the organizational revenue from a specific IVRdesign can then be inferred from the aggregation of all the optimal policies, for eachgroup of customers. A partial example for such a calculation is given in Section7.1.2.
80
7.1 Comparing Different IVR Designs
In this section we show how one can combine our theoretical model with the ex-ploratory data analysis, to evaluate alternative IVR designs and compare them.This will be done using a numerical example, which is based on the ILBankdatabase.
7.1.1 Estimating the Model Parameters
In Chapter 3, we described the IVR system as a rooted tree, G = (V,E), in whicheach leaf represents a service offered by the IVR system and all the other verticesrepresent IVR menus. Moving along edge e = (i, j) ∈ E means choosing theoption represented by j in the menu represented by i. The root vertex, denoted bys, represents the IVR main menu.
The model parameters were as follows:
• Pi - Success probability of leaf i, 0 < Pi < 1.For non-leaves vertices, Pi = 1.
• tser(i) - Time spent in leaf i, given that we have been successful in visitingit, tser(i) > 0. For non-leaves vertices, tser(i) = 0.
• tF (i) - Time spent in leaf i, given that we have not been successful in visitingit, tF (i) > 0. For non-leaves vertices, tF (i) = 0.
• Ti =
{tser(i) w.p. PitF (i) w.p. 1− Pi
• tN(j)i,j - The exploration time of edge e = (i, j) ∈ E, given that edge (i, j)
was considered (heard) N(j) times before.
• ri - Reward earned from successful completion of leaf i, ri ≥ 0.For non-leaves vertices, ri = 0.
• τ ∼ exp(θ) - Customer patience.
• c - Cost per each unit of time.
• α - Discount factor.
Using exploratory data analysis, as presented in Chapter 6, we now estimatethe following parameters: Pi, Ti, t0i,j . Since we did not address the estimation ofthe other parameters, we used former results based on the ILBank database, merelyfor the sake of our numeric example.
81
Pi - Success Probability of Leaf i
Recall that each leaf in the tree G = (V,E) represents a service provided by theIVR system. Some customers may reach a specific service, realize that they havereached the wrong service and leave it without getting any relevant informationfrom it. Thus, reaching a service (a leaf in the tree) does not guarantee a successfulcompletion of it. The estimation of the success probability of each IVR service wasactually described in Section 6.4. We shall now review it shortly. We presented twomethods for estimating the success probability of each IVR service; by a thresholdvalue, and by fitting a mixture of theoretical distributions to the empirical serviceduration distribution. We showed, for example, that the success probability of theservice ‘Recent Account Activity’, in both methods, can be estimated by 86.5%.Actually listening to the actual service in the ILBank IVR system, we realizedthat no relevant information could have been obtained during the first six seconds,hence set the threshold value to be six seconds. Using SEEStat’s mixture-fittingtool we fitted a mixture of six Gamma distributions to the empirical distributionof the service duration; see Figure 6.9. We assumed that the first two distributionscapture the peak at the origin, which conceivably describes the distribution of un-successful completions of this service. The proportion of these distributions in themixture is 13.5%. We therefore deduce that the success probability of the service‘Recent Account Activity’ (or the leaf representing it) is 86.5%.
tser(i), tF (i), Ti - Time Spent in Leaf i
The time spent in each service is captured by its distribution; see Figures 6.4 to 6.7,for example. Fitting a mixture of several theoretical distributions to the empiricalone, we were able to distinguish between the distributions that capture the time ofunsuccessful services and the distributions that capture the time to successful com-pletion of the service. As mentioned above, a mixture of six Gamma distributionswas fitted to the empirical distribution of ‘Recent Account Activity’, where the firsttwo capture the time of unsuccessful services and the other four capture the timeof successful completions. Thus, tser of the equivalent leaf can be characterizedby the mixture of the last four distributions and tF by the mixture of the first two.Ti is therefore the combination of all six distributions. Notice that our model, andthe index calculations derived from it (see Section 5.1), only require the Laplacetransform of tser and tF .
The Laplace transform of a mixture of n Gamma distributions with parameters(ki, θi), i = 1, ..., n, and proportions λ1, ..., λn (0 ≤ λi ≤ 1, λ1 + ... + λn = 1),is given by:
n∑i=1
λi (1 + θis)−ki .
Table 6.5 presents the mixing proportion and distribution parameters of theservice ‘Recent Account Activity’. We assume, as before, that the first two distribu-tions are compatible with unsuccessful services and the other four distributions are
82
compatible with successfully completed customer services. It then follows that, forthe ‘Recent Account Activity’ service:
LtF (α) =4.86
4.86 + 8.70(1 + 0.39α)−10.13 +
8.70
4.86 + 8.70(1 + 0.02α)−134.46
and
Ltser(α) = 36.7386.44 (1 + 3.32α)−6.3 + 27.91
86.44 (1 + 2.44α)−21.6
+11.98
86.44(1 + 0.21α)−273.52 +
9.82
86.44(1 + 10.08α)−9.25 .
Remark 5. When the Laplace transform does not have a closed-form expression,use the Taylor expansion for functions of random variables. Specifically, if X is arandom variable, let
LX(s) = E[e−sX
], f(X) = e−sX .
By the Taylor expansion:
E [f(X)] ≈ f(µX) +f ′′(µX)
2σ2X ,
where µX and σ2X are the mean and variance of X respectively. Thus,
LX(s) ≈ e−sµX +s2e−sµX
2σ2X .
tN(j)i,j - Time Spent in Edges
Let (i, j) be a directed edge from vertex i to vertex j in the tree G = (V,E). Thismeans that vertex j corresponds to an option in the menu represented by vertex i.The time assigned to edge (i, j) is the time it takes to listen to menu i and chooseoption j. We assume that if a customer chooses option j, and this is the first visitto menu i, that customer will listen to all the options prior to j in this menu andwill choose j right after it was presented. We listened to the actual ILBank IVRsystem and measured this time for every option within every menu (see AppendixC). The options given in every menu are recorded messages with constant length.Hence, under the above assumption, the time t0i,j is actually deterministic for everyedge (i, j). However, this assumption does not always hold, as some customerslisten to the whole menu before making a selection, and some listen to the menu,or part of it, more than once. In addition, some customers may listen to the firstfew words of the desired option and then make a selection, while others will stalla little before making a selection. In Section 6.5, we saw that as customers acquiremore experience within the system, the time they spend in menus is getting shorter,since they do not have to listen to the whole menu in order to form their selection.
83
Thus, although each option is of constant length, the actual time a customer spendslistening to each option depends on familiarity with the system, and with the spe-cific menu in particular. Some customers are so skilled that they can automaticallychoose their desired option, or series of options, without listening to any menu.
Our model and index calculation require the Laplace transform of the time ineach edge, (i, j) ∈ E, as a function of the customer’s familiarity with the relevantmenu. If there is no closed-form expression for the Laplace transform, we used theTaylor expansion, as in Remark 5. This yields an approximation for the Laplacetransform, using only the mean and variance of the time on the edge.
Since our data does not reveal the time customers spend in menus, we couldnot evaluate tN(j)
i,j from our data logs. For the purpose of our numerical example,
we considered tN(j)i,j to be exponential random variables, for every i, j, and N(j).
E[t0i,j ] was calculated as the time it takes to fully listen to option j and all theoptions prior to j in menu i, according to our actual time measurements. Then,E[t
N(j)i,j ] for every N(j) > 0 was calculated in the following manner:
E[tN(j)i,j ] = max
{E[t0i,j ]
N(j) + 1, 1
}.
The time it takes to perform the operation of going backward (to the former menuor to the main menu), tback, was considered to be 5 seconds. Thus, if vertex iis a menu, E[t0i,pre(i)] and E[t0i,s] were calculated as the time it takes to listen to
the whole menu, namely t0i,li plus 5 seconds. In the same manner, E[tN(li)i,pre(i)] and
E[tN(li)i,s ] are equal E[t
N(li)i,li
] + 5. If vertex i is a leaf (IVR service), E[ti,pre(i)] =E[ti,s] = 5 seconds.
τ - Customer Patience
Recall that our model assumption was that τ ∼ exp(θ). Hence, we only need toestimate θ, or equivalently, average customer patience. The subject of customerpatience while waiting in the agents queue in a call center environment has beendiscussed extensively in several papers; [5], [10], [11], are examples. However, wefound no former research analyzing customer patience during self-service, suchas in IVR systems. We believe that our database enables a systematic analysis ofcustomer patience to some extent. However, this is beyond the scope of our currentwork and will be left to future research. Instead, we opt for the simplest procedureto estimate the mean of a censored exponential distribution.
Think of customers entering a service system at rate λ, with customer aban-donment rate being θ (i.e., average patience is 1/θ). The following relation holds[26]:
P{Abandonment} = θ × E[Wait]. (7.1)
84
Average Sojourn Time (sec) % Out of TotalCompleted in IVR 100 64.6%Requesting Agent Service 73 35.4 %
Table 7.1: Average sojourn time within ILBank IVR, May 2008 to June 2009
Equation 7.1 can be easily verified via Little’s Law, by
Abandonment rate = λ× P{Abandonment} = E[queue length]× θ (7.2)
with λ being the arrival rate, and, by Little’s Law
E[queue length] = λ× E[Wait].
We use the relation in 7.1 in order to establish reasonable range for the averagepatience and the θ parameter of the exponential patience in our model.
Table 7.1 presents the average sojourn time within the IVR system, for cus-tomers leaving the system after the IVR service and for customers opting out toreceive agent service afterwards. The results presented are based on the ILBankdatabase, from May 2008 to June 2009.
Using these results to calculate the average sojourn time in the ILBank IVRsystem, we get:
E[IV R sojourn time] = 90 sec.
Our data analysis also revealed that about 8.5% of the calls entering the IVRsystem ended, and left the system, after the identification phase. These calls arecounted as abandonments from the IVR system since no relevant service was re-ceived during these calls. In addition, in Section 6.4, we saw that some customersreach an IVR service but leave it within a few seconds, where it is clear that nouseful information could have been given within this short time. When such anunsuccessful service is not followed by any other IVR service, one can say that thecustomer abandoned the IVR service. Fitting a mixture of Gamma distributions tothe empirical distribution of IVR services durations, we saw that some relativelyfrequently visited services, such as Recent Account Activity and Account Summary,have up to 10%− 30% such abandonments (see Section 6.4 and Appendix B).
Using the relation in equation 7.1, and considering abandonment rate of 10%,we get an average patience which is equal to 900 seconds. Taking into considera-tion a much higher abandonment rate of 50%, we get an average patience which isequal to 180 seconds.
For the purpose of our numerical example, we took as anchor the average pa-tience calculated for the ILBank database by Khudyakov et al. [18]. They con-sidered customers waiting in queue for “Private” (retail banking) service type. Theaverage patience was estimated via survival analysis, for each customer type, basedon data from May 2007. The results, which are given here in Table 7.2, are takenfrom Table 5 in Khudyakov et al. [18].
85
High priority Medium priority Low priority UnidentifiedAverage Patience (sec) 412.54 886.32 958.55 262.23
Table 7.2: Average patience, by customer type, based on Khudyakov et al. [18]
Considering the relation in 7.1, this average patience is compatible with 10%−20% abandonments. Since customers’ patience while waiting in queue may be verydifferent from their patience while performing self-service, we ran our model withseveral very different values of the θ parameter, performing, in practice, parametricanalysis to this unknown parameter.
Our numerical example included three different IVR designs for each customerpriority (high, medium and low), resulting in 9 different Admissible Trees onwhich the indices were calculated (the specified setting is given in Section 7.1.2).The results for the average patience of 90 and 180 seconds and for the average pa-tience specified in Table 7.2 are given in Section 7.1.2. Although we estimated thepatience parameter using theory and results which are relevant for queues in a callcenter environment and not self-services, our parametric analysis validated the useof these parameters in our numeric example.
ri, c - Rewards and Costs
In this work, we did not address the issue of estimating customer perceived re-wards from different IVR services, nor the perceived cost per each unit of time inthe system. There are methodologies for estimating these perceived rewards andcosts, such as imputed costs and structural estimation methods. Aksin et al. [2]used structural estimation methods on the ILBank database to evaluate the meanand standard deviation of customer perceived rewards and costs. In their model,customers receive a reward from agent service and incur delay cost while wait-ing in queue, which is linear in their waiting. Since this is not within the scopeof our work, in the following numerical example we simply used their results (aspresented in Table 4 in [2]), as a first estimation for ri and c. To do so, we hadto assume that the cost per each unit of time in the IVR system depends on thecustomer priority, and is equal to the mean delay cost per second the customer willincur while waiting in queue. We also assumed that if a specific IVR service maybe of interest to a customer, then its reward will be the same as the mean rewardreceived from agent service, and it again depends on the customer priority. Table7.3 presents these parameters.
7.1.2 Numerical Example
Settings
The goal of the following numerical example is to show how our theoretical modelcan be used to compare different IVR designs and evaluate their profitability to the
86
Customer Type Cost per second ($) Reward from successful service ($)High priority 0.0178 6.309
Medium priority 0.0084 6.175Low priority 9.083E-06 5.299Unidentified 0.002 4.211
Table 7.3: Rewards and costs, by Aksin et al. [2]
organization. The numerical example will be based on the ILBank IVR system,the exploratory data analysis conducted in Chapter 6, and the resulting estimatedparameters, as described in Section 7.1.1.
Recall that our model describes the search of individual customers. We assumethat each customer is looking for one or more IVR services, but no more than Kservices. Our data analysis revealed that almost 95% of the calls reached no morethan 5 services, in addition to identification, and one of these services may be atransition to an agent. The reward from every IVR service, for each individualcustomer, is given as an input of the model. Since we assume that the number ofleaves with positive reward is no more than K, which is a relatively small constant(around 5), then the computational complexity of our problem is O(n), where n isthe length of the longest path in the remaining IVR tree. By the proof of Proposition2, the number of complete paths in the Admissible Tree, T =
(V T , ET
), on which
we calculate our indices, is bounded by K!2K−1. By Lemma 3 and Lemma 4, thelength of each such complete path can not exceed 2nK. Hence the total number ofvertices in T is bounded by K!2K−12nK.
Instead of solving the problem for each individual customer, we can look atgroups of customers which may be interested in the same services. In our example,we consider the group of customers which are only interested in basic accountinformation. We assume that these customers may have a positive reward from theservices: ‘Recent Account Activity’, ‘Account Summary’, ‘Account Activity Today’,‘Credit Card Vouchers’. However, within this group of customers, some customersmay reach one (or more) of these four services by mistake, and will not receive itsassociated reward. Thus, the percentage of customers within this group, reachinga specific service intentionally and successfully completing it, is given by Pi.
As stated in Section 7.1.1, we assumed that customers with different prioritiescould have different rewards and different costs, as given in table 7.3. We alsoassumed that, for a specific priority, all the four services incur the same reward.In our example, we focused on High, Medium and Low priority customers andignored Unidentified customers. We considered the discount factor to be α = 0.05,which means that a reward (or cost) of r units received (paid) after t units of time,will be equal to re−αt at present.
Table 7.4 presents estimators for Ltser and LtF , for the four services, in eachpriority group, for α = 0.05.
We shall compare three different designs: the original IVR design, a com-
87
Low Priority Medium Priority High PriorityIVR Service LtF (α) Ltser(α) LtF (α) Ltser(α) LtF (α) Ltser(α)
Recent Account Activity 0.8724 0.1919 0.8656 0.2031 0.8783 0.1961Account Summary 0.8887 0.2770 0.8907 0.2994 0.8605 0.2611Credit Card Vouchers 0.8736 0.2045 0.8645 0.2261 0.7974 0.2363Account Activity Today 0.8652 0.2387 0.8637 0.2242 0.8396 0.2152
Table 7.4: Laplace transform for successful and unsuccessful service durations
pletely shallow (broad) design and a completely deep (narrow) design (the originalIVR design is considered to be an intermediate design). The three designs are givenin Figures 7.2 to 7.4. The services in the shallow and deep designs are ordered ac-cording to their demand frequency, as presented in Table 6.3. The time neededto reach and choose each option in each menu, given that this is the first visit inthe menu (represented by t0i,j), is given above each edge. This time changes with
customer’s repeated visits to the menu (represented by tN(j)i,j ), as was explained
in Section 7.1.1. The time it takes to return to the previous menu or to the mainmenu is also calculated, as was stated in Section 7.1.1. For each IVR design andeach customer priority, we ran our model with different values of average patience,ranging from 90 seconds to 950 seconds. In the following example, we modelcustomer search after successfully completing the identification phase and hearingtheir account balance, which is assumed to be given automatically.
88
Iden
tificatio
n
Main
Me
nu
Pla
y
Account
Bala
nce
Current
Account and
General
Issues
Account Activity for Requested Date18
Account Activity
Today
6
Recent Account
Activity
Account
Summary
Current Balance
Account Activities
Checking Account and
General Activities
Deposit Activities
Credit Activities
Trust Funds Activities
Cheque Book Order
Transfer between Client Account
Change Password
Select Code
Change
Password by
SystemDaily Interest Deposit
Other Deposits Activities
Credit Balance Information
Credit Payment
Credit Establishing
Credit Balance Explanation
Trust Funds Sell
Stock Exchange
20
Securities Order Status
Selected Securities
Stock Exchange – Israel and World
Trust Funds Buy
Stock Exchange – Europe
Stock Exchange – Israel
Stock Exchange – USA
Stock Exchange – Asia
Stock Exchange – Options & Futures
Stock Exchange – Securities Rates
Stock Exchange – Interbank Rates
Investments
Securities Balance
Deposits Balance
Saving Plans Balance
Foreign Currency
Foreign Currency Current Account
Provident Funds Balance – by Fund Type
Provident Funds Balance – Specific Fund
Securities Portfolio Rates
Securities Portfolio Details Today
Securities Portfolio Details for Previous Day
Provident Funds Balance
Foreign Currency Current Account - by Currency
Foreign Currency Deposits Balance
Foreign Currency Deposits Balance – by Currency
Foreign Currency Rates
Loans,
Mortgages
and Credit
Card
Information
Credit Card Vouchers
Mortgages
Fax
20
20
Credit Card Credit17
10
3
12
14
Figure 7.2: Original IVR design. Bold represents services with positive reward
89
Identification
Main Menu
Credit Card
Vouchers
10
Recent Account
ActivityAccount
Summary
Stock
Exchange
Israel
Account
Activity Today
33 416
Selected
Security
Transfer
between Client
Accounts
Selected
Password
Figure 7.3: Shallow IVR design. Bold represents services with positive reward
Results
Our model finds the optimal policy for each individual customer, or each group ofcustomers with the same characteristics. The optimal policy is the customer pathwithin the IVR system that will yield maximum utility. That is, the optimal policywill state which IVR services will be visited by the customer, in what order, andwhat will be the path between one service and the other. If an IVR service with apositive reward is not considered in the optimal policy it means that it was morebeneficial to abandon the IVR system before reaching this service, and either leavethe call center or opt-out to an agent.
Tables 7.5 to 7.7 present the optimal policy and the expected utility for eachcustomer type, in every IVR design, for the average patience of 90 and 180 sec-onds and the average patience given in Table 7.2. IVR services within the paths arecolored in red.
We clearly see that, for customers looking solely for basic account information(which is given in the four aforementioned IVR services), the shallow design in-curs the highest utility and the deep design incurs the lowest utility, regardless ofcustomer priority or average patience.
Under the shallow design, the optimal path of all customer priorities includesall four services. However, the optimal path is not identical for all customer prior-ities, as the order of the services changes.
Under the deep design, the optimal path of high priority customers does notinclude the service Account Activity Today, which is placed at the deepest levelamong these four services. The optimal path of medium and low priority customersdoes include this service. This result is very reasonable since the cost per unit oftime for high priority customers is much higher than the cost per unit of time forthe other customer priorities.
90
12
6
11
Identification
Main Menu
Credit Card
Vouchers
11
Recent Account
Activity
Account
Summary
Stock Exchange
Israel
Account Activity
Today
8
7
3
Selected Security
Transfer between
Client Accounts
Selected
Password
M1
M2
5
M3
5
M4
612
M5
3
M6
8
M7
4
M8
Figure 7.4: Deep IVR design. Bold represents services with positive reward
91
IVR Design Averagepatience(sec)
ExpectedRevenue ($)
Optimal Path
Original 90 0.5816 Main Menu→ Current Account and General Issues→ Account Summary→ Current Account and Gen-eral Issues → Recent Account Activity → CurrentAccount and General Issues→ Account Activity to-day
180 0.6117 As above412.54 0.6298 Main Menu→ Current Account and General Issues
→ Account Summary→ Current Account and Gen-eral Issues → Recent Account Activity → CurrentAccount and General Issues→ Account Activity to-day→Main Menu→ Loans, Mortgages and CreditCard Info. → Credit Card Vouchers
Shallow 90 0.7244 Main Menu → Recent Account Activity → MainMenu → Account Summary → Main Menu →Credit Card Vouchers → Main Menu → AccountActivity today
180 0.7433 As above412.54 0.7546 As above
Deep 90 0.5811 Main Menu → Recent Account Activity → MainMenu → M1 → Account Summary → M1 → M2→ Credit Card Vouchers
180 0.6024 As above412.54 0.6153 As above
Table 7.5: Results, optimal paths and expected utility, High priority
There are two setups in which a change in the average patience influences theoptimal path. One is high priority customers with the original IVR design; the otheris low priority customers with the deep IVR design. We see that, in this case, whenthe average patience is relatively small (90 to 180 seconds), the optimal path doesnot include the service Credit Card Vouchers, while, for higher average patience(412 seconds and up) this service is being reached by the optimal path.
Organizational Profit
After solving the customers’ problem and identifying their optimal paths under ev-ery design, the organizational profit from each design can be calculated. Of course,in order to do so, we have to assume that customers will follow their optimal paths.
We now give a small, very simple example for calculating such organizational
92
IVR Design Averagepatience(sec)
ExpectedRevenue ($)
Optimal Path
Original 90 0.9371 Main Menu→ Current Account and General Issues→ Account Summary→ Current Account and Gen-eral Issues → Recent Account Activity → CurrentAccount and General Issues→ Account Activity to-day→Main Menu→ Loans, Mortgages and CreditCard Info. → Credit Card Vouchers
180 0.9824 As above886.32 1.0213 As above
Shallow 90 0.9848 Main Menu→Account Summary→Main Menu→Recent Account Activity → Main Menu → CreditCard Vouchers→ Main Menu→ Account Activitytoday
180 1.0478 As above886.32 1.0849 As above
Deep 90 0.7563 Main Menu → M1 → Account Summary → MainMenu → Recent Account Activity → Main Menu→ M1 → M2 → Credit Card Vouchers → M2 →M3→M4→M5→M6→ Account Activity today
180 0.8038 As above886.32 0.8452 As above
Table 7.6: Results, optimal paths and expected utility, Medium priority
profit. There are several performance measures which are related to the organi-zational profit from its IVR system. In our example, we only consider two, verybasic, performance measures: The agent time being saved and the cost of commu-nication.
The measure of the agent time being saved by handling the call in the IVR waspresented by Suhm and Peterson [23]. In order to calculate it, we will assume thatthe time spent in an IVR service is equal to the time it will take an agent to satisfythis service (any similar assumptions can be accommodated as well).
The cost of communication is relevant in 1-800 calls, where the organizationis paying for the call, and not the customer. In this case, we can calculate this costas the average time customers spend in the IVR system multiplied by the cost persecond of communication and the total number of customers.
In Table 7.9, we present the average time customers, from each priority group,spend in the IVR system, according to the IVR design and an underlying averagecustomer patience of 180 seconds. It also presents the average saved agent time foreach IVR design, customer priority and average patience. The average number ofrelevant calls, for each customer type, was calculated in the following manner:
93
IVR Design Averagepatience(sec)
ExpectedRevenue ($)
Optimal Path
Original 90 0.9028 Main Menu → Current Account and General Is-sues→ Account Summary→ Current Account andGeneral Issues → Account Activity today → Cur-rent Account and General Issues→ Recent AccountActivity → Main Menu → Loans, Mortgages andCredit Card Info. → Credit Card Vouchers
180 0.9475 As above958.55 0.9867 As above
Shallow 90 0.9472 Main Menu→Account Summary→Main Menu→Recent Account Activity → Main Menu → CreditCard Vouchers→ Main Menu→ Account Activitytoday
180 0.9874 As above958.55 1.0227 As above
Deep 90 0.7499 Main Menu → Recent Account Activity → MainMenu → M1 → Account Summary → M1 → M2→ Credit Card Vouchers → M2 → M3 → M4 →M5→M6→ Account Activity today
180 0.7891 Main Menu → M1 → Account Summary → MainMenu → Recent Account Activity → Main Menu→ M1 → M2 → Credit Card Vouchers → M2 →M3→M4→M5→M6→ Account Activity today
886.32 0.8294 As above
Table 7.7: Results, optimal paths and expected utility, Low priority
According to Table 6.1, 65% of the calls are completed in the IVR system. Basedon our data analysis, we found that about 60% of the calls reach an additional ser-vice besides ‘Play Account Balance’, or opting out to an agent afterwards. Basedon the demand frequencies of IVR services, we assume that no more than 35% ofthese calls are reaching services which are different from the four basic servicespresented in our numerical example. To summarize, in order to calculate the aver-age daily number of customers which may be interested in the four services in ournumerical example, we used the following formula: X × 0.65× 0.6× (1− 0.35),where X is the average daily number of calls from each priority group, accordingto Table 6.2.The results are given in Table 7.8.
Denote by d the organizational communication cost per second and assumethat the organizational cost per second of agent work is about 10×d. (The averagecost per second of agent work in ILBank was given in [25]. The communication
94
Customer Type Avg Number of Calls per DayHigh 2365
Medium 6029Low 3873
Table 7.8: Numerical example - average number of relevant calls by customer type
cost was derived from Web forums concerning corporate 1-800 numbers.) Basedon the above assumptions, the average organizational profit from each IVR design,under each underlying average customer patience, is given in the last column ofTable 7.9.
IVR Design Customer Priority Avg. Timeon Line(sec)
Avg. SavedAgentTime (sec)
Organizational rev-enue ($)
Original High 253,546 183,917Medium 978,873 648,223
Low 651,581 452,852Total 1,884,000 1,284,993 10,965,930 ×d
Shallow High 378,992 252,688Medium 962,341 648,224
Low 654,231 452,852Total 1,995,563 1,353,764 11,542,077 ×d
Deep High 253,287 183,658Medium 970,607 648,224
Low 659,531 452,852Total 1,883,425 1,284,734 10,963,915 ×d
Table 7.9: Organizational revenue
In order to find the most profitable design for the organization, one must repeatthis process for customers looking for all other services as well.
Assuming that d ≈ 0.02$ per minute, we get an average organizational profit of$3655 per day (from the group of customers looking solely for the aforementionedfour IVR services) using the original IVR design, $3847 per day using the shallowIVR design, and $3655 per day using the deep IVR design.
The results given in this numerical example suggest that if a substantial pro-portion of customers are looking only for basic account information, then an IVRdesign in which the aforementioned four services are placed as the first options ofthe first level, may be beneficial for the organization. However, in some cases, adesign which is optimal for the customers may not be optimal for the organization.First, customers’ most profitable design may sometimes involve longer time in thesystem. Secondly, there are other important factors which may influence the orga-nization’s profit. For example, abandonment costs (which differ across customer
95
types), reputation cost (which may be associated with waiting costs and, again,may vary with customer type), IVR services which imply higher reward for theorganization and more.
7.2 Further Implications
In the former section, we used a numerical example to show how our theoreticalmodel and exploratory data analysis can be used to compare alternative IVR de-signs, both from the customers’ point of view and from the organization’s point ofview. We believe that there are additional important implications to our model andthe data analysis methods suggested here. For example:
• Adding advertisements. Some organizations add advertisements to their IVRsystem. This may not affect the IVR design but prolong menu lengths. Withour model, we can estimate how advertisement will affect customer actionswithin the IVR system, and what will be the effect on the resulting organiza-tion’s revenue.
• Services with low demand. Understanding the reasons for IVR services withlow demand may be very important for the organization. There could be twomain reasons for very low demand for a service: either customers are notinterested in this service, or perhaps, customers are interested in the servicebut are experiencing trouble reaching it. One way to address this issue usingour model, is to gradually raise the perceived reward of services with lowdemand. If the optimal customer policy does not change, and these servicesare still not included within this policy, it means that these services may bebeneficial for customers but are hard to reach (customers lose patience onthe way). This result should lead to a change in the IVR design which willmake the low demand services more accessible. If low demand services aresupposed to be included in the optimal policy as their perceived reward goesup, then we can assume that in fact, customers are not interested in them,which means that their actual perceived reward is low. This result suggeststhat these services could perhaps be eliminated from the IVR system.This leads us to another interesting question which can be answered in theexact same manner: what should be the perceived reward from a service inorder to add it (or remove it) to (from) the IVR system?
• Do customers navigate optimally? If not, why? Comparing the model resultswith actual customer paths may shed light on usability problems with theIVR system. Suhm and Peterson [23] analyzed end-to-end calls, includingrecordings of customer-agent dialogs, in order to discover inconsistencieswhich indicate wrong customer navigations. Finding specific places withinthe IVR menu which lead to wrong navigations, is thus important. Using our
96
model, and assuming that customer parameters are known (or at least canbe estimated reliably), one can find these problematic places directly fromthe IVR logs, without any need to listen to customer-agent dialogs, whichis extremely time consuming. This could be done by comparing the optimalcustomer policies to their actual actions, and identifying places of inconsis-tencies. We should mention that even the services duration analysis, whichwas done in Section 6.4, can be used to discover usability problems. IVR ser-vices with high abandonment rate suggest that there is something confusingin the menus leading to them and perhaps a change in the menus’ wordingor IVR design is required.
• Imputed costs. The actual customer paths, implemented in our model, can beused to derive the perceived customer rewards and costs, or at least, the ratiobetween them. In a way, this is the opposite procedure from the previouspoint. The underlying assumption is that customers navigate according totheir optimal path. We can then try and find the model parameters that willlead to such an optimal path.
• Anticipating long waiting in agent queue, how does it affect customer behav-ior? Call centers workload usually changes throughout the day, as well asthe average waiting time in queue. If customers are anticipating long wait-ing time in the agents’ queue, their perceived opt-out reward may be smallerthan usual. A change in the perceived opt-out reward may lead to a changein the customer optimal policy, according to our model. This hypothesis canactually be examined by analyzing experienced customers’ paths during dif-ferent hours of the day. Such information can be used by the organizationto influence customers’ actions, for example, by announcing the estimatedwaiting time. In high workload hours, this could lead to higher usage of theIVR system and hence reduction in agent workload.
97
Chapter 8
Summary and Discussion
The goal of our research was to improve and enhance IVR systems, as a specialcase of self-service systems. We modeled the IVR system as a rooted tree andmodeled the customer flow within the IVR as a stochastic search on this tree. TheIVR services were modeled as the tree leaves; IVR menus were modeled as thetree non-leaves vertices, and the tree edges represented the different options withineach menu.
Our model relied on the work of Denardo et al. [8] and Granot and Zuckerman[12]; both presented stochastic search models for R&D projects. As in their work,the goal of our search was to find the optimal policy which will yield the highestexpected discounted utility for the customer.
The theoretical model we presented was inspired by an exploratory data anal-ysis, carried out on data logs from an Israeli Bank IVR system. This analysisrevealed some fascinating phenomena, which were incorporated into our searchmodel, as substantial modifications to the models in [8] and [12]. For example,we realized that a significant proportion of customers leave the IVR system with-out getting any relevant information, either right after they identify themselves, orafter reaching an undesirable IVR service. The proportions of the latter were trans-lated into the IVR services success probabilities. We also discovered that there is alearning process, which means that as customers gain more experience within thesystem, their response time gets shorter. This fact was incorporated into our modelas an additional dimension to the state space, representing the number of formerrepetitions along the search. We showed that our search protocol can be translatedinto a finite extended rooted tree, representing all possible optimal search candi-dates. An index was then assigned to each edge, where choosing the edge with thehighest index at each stage was shown to be the optimal policy.
We concluded our work with some practical implications of the theoreticalsearch model. This included a numerical example, based on our data, in whichwe compared three different IVR designs, both from the customer point of viewas well as the organization point of view. Our example reveals that our modelcan in fact supplement other research fields, such as Human-Factor-Engineering,
98
while establishing the optimal IVR design. We believe that our model can provideimportant insights for better IVR designs, as well as other self-service systems.Such implications were discussed in Section 7.2.
8.1 Future Research
A key observation in our research was the fact that some customers abandon theIVR service. We used very simple methods to evaluate the proportion of abandon-ing customers. While abandonments from queues are easily detected, identifyingabandonments from self-services is not an easy task. Hassan et al. [13], for exam-ple, showed that customer paths and transition times can be used in order to predictwhether a Web search was successful or not. We believe that our work may shedlight on this matter as well, and contribute to better analysis and understanding ofcustomer patience and customer success within self-service systems, such as IVRsystems and Web searches. However, the subject of identifying abandonmentsfrom the IVR systems, as a special case of self-service, and analyzing customerpatience accordingly, was not addressed in the present work. There is still a needfor comprehensive research in this direction. We believe that our exploratory dataanalysis can provide the very first steps in statistical analysis of customer patiencewithin self-service systems.
In this research, we have not addressed the interrelation between customer-IVR interaction and customer-agent interaction. However, this subject is of in-terest when modeling and analyzing call centers with an IVR system. For exam-ple, if the IVR is properly designed and all the simple services are successfullygiven via the IVR system, then customers reaching the agent services will demandmore complicated services, which may require longer service times and agentswith higher skills and more experience. The influence of customer-IVR interac-tion on the customer-agent interaction is thus of interest and may be integrated intostaffing models. Khudyakov et al. [17], for example, performed asymptotic anal-ysis of a call enter with an IVR system in the QED regime. They considered theIVR system as a “black box” with exponential service time, and provided approx-imations for several performance measures, such as the waiting probability andthe mean waiting time. Behzad and Tezcan [4] proposed a model of a call cen-ter with two IVR systems, which differ in their call resolution probability, opt-outprobability and abandonment probability. Their goal was to determine the optimalstaffing level and the proportion of calls that should be routed to each of the twoIVR systems. However, in both researches, the IVR design was not considered,nor do other issues addressed in our work, such as customer paths within the IVR,success probability of specific IVR services and actual IVR service times. Thus,we believe that call center models that integrate the IVR system, in the resolutionof our model, may be beneficial.
The customer-agent interaction may also influence the customer-IVR interac-tion. As previously discussed, customers anticipating a long waiting time in the
99
agents’ queue may have longer interactions with the IVR system, trying to com-plete their service within the IVR system in order to avoid long waits. On the otherhand, customers who are accustomed to highly personal agent service (for exampleVIP customers) may try to skip the IVR service even if they could have receivedsatisfactory service there.
The time customers spend in the IVR and their actions there, presents a greatopportunity for improving the on-line routing and managing of call center work-load. In ILBank, for example, the average time customers spend in the IVR is ofthe same order as the average waiting time in the agents’ queue. This means thatthere is a substantial amount of time in which customers are already in the callcenter, but are still not part of the offered load. Moreover, when customers are inthe IVR system, they may provide additional data that can be used in the on-linecall center operation. The gathered data on customers within the IVR system, com-bined with call center state information, can be used to control customer behavior,e.g., offering to return later or to leave a message, offering additional IVR services,etc. Feigin [9], for example, analyzed customer patience in queue as a function oftheir IVR service time. We believe that additional research in this direction may beboth interesting and useful.
Our search model was tailored to describe customer flow within an IVR sys-tem. However, we believe that it can be modified to describe other systems andother problems which can be solved using stochastic search. One example for suchmodification is in cases where the tree is not predetermined, and perhaps someoptions may be available at each stage with a certain probability. In this case, forexample, a Partially Observed Markov Decision Process may be required (insteadof a simple MDP) in order to find the optimal policy.
100
Appendix A
IVR Data Table
Figure A.1 presents the main fields in the IVR table of the ILBank database:
1. call id - universal identifier associated with the entire call.
2. customer id - customer ID (0 - unidentified customer).
3. cust type - the customer priority; 1 - High, 2 - Medium, 3 - Low, 4 - Uniden-tified.
4. event - identifier of a general customer transaction in the IVR.
5. prev event – previous event
6. segment num - ordinal number of the segment in a specific call
7. subevent – sub-event of the IVR event, identifier of a specific customer trans-action in the IVR.
8. segment start - time in seconds at which the sub-event starts.
9. segment end – time in seconds at which the sub-event ends.
10. duration - amount of time in seconds a customer spends in each sub-event inthe IVR
11. seg id – the state of the call; 1 - customer call start, 2 - customer call startand end, 3 - customer call end, 4 - customer call middle segment, 9 - flagmarking the middle segment before the next visit to the IVR in a specificcall (if exists).
101
call_id
customer_id
cust_type
event
prev_event
segment_num
subevent
segment_start
segment_end
duration
seg_id
900106162
04
30
13
1221589852
1221589926
74
1
900106162
04
53
21
1221589926
1221589936
10
3
989000001
1805664254
33
01
11221523204
1221523239
35
1
989000001
1805664254
37
32
18
1221523239
1221523260
21
3
989000002
1686873386
23
01
31221523204
1221523231
27
1
989000002
1686873386
27
32
18
1221523231
1221523243
12
4
989000002
1686873386
25
73
11221523243
1221523247
43
989000003
1542189479
33
01
11221523216
1221523235
19
1
989000003
1542189479
37
32
18
1221523235
1221523241
64
989000003
1542189479
35
73
11221523241
1221523245
43
989000004
1762256591
23
01
31221523217
1221523230
13
1
989000004
1762256591
27
32
18
1221523230
1221523250
20
4
989000004
1762256591
27
73
20
1221523250
1221523313
63
4
989000004
1762256591
27
74
25
1221523313
1221523386
73
4
989000004
1762256591
27
75
36
1221523386
1221523448
62
3
989000005
1798826054
23
01
11221523219
1221523236
17
1
989000005
1798826054
27
32
18
1221523236
1221523246
10
4
989000005
1798826054
27
73
19
1221523246
1221523248
24
989000005
1798826054
210
74
29
1221523248
1221523336
88
3
989000006
1472556965
23
01
11221523229
1221523243
14
1
989000006
1472556965
27
32
18
1221523243
1221523258
15
4
989000006
1472556965
27
73
20
1221523258
1221523288
30
3
989000008
1704796286
33
01
11221523235
1221523266
31
1
989000008
1704796286
37
32
18
1221523266
1221523272
64
989000008
1704796286
35
73
11221523272
1221523275
33
989000009
1476663897
23
01
11221523235
1221523241
61
989000009
1476663897
27
32
18
1221523241
1221523246
54
989000009
1476663897
27
73
20
1221523246
1221523248
23
989000010
1707342167
23
01
11221523238
1221523279
41
1
989000010
1707342167
27
32
18
1221523279
1221523300
21
4
989000010
1707342167
25
73
11221523300
1221523395
95
3
989000011
1453327173
33
01
11221523244
1221523266
22
1
989000011
1453327173
37
32
18
1221523266
1221523285
19
3
Figu
reA
.1:I
VR
data
tabl
e
102
Appendix B
IVR Last Services Distribution
The following figures present the time distribution of four IVR services. The blueline represents the distribution across all appearances of the relevant IVR service.The red line is the service time distribution only when it is the last service in thecall (not followed by any other service).
0.000
1.000
2.000
3.000
4.000
5.000
6.000
7.000
8.000
00:02 00:11 00:21 00:31 00:41 00:51 01:01 01:11 01:21
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Current Account - Recent Account ActivityTotal for November 2008 to June 2009,All days
Total customer call end
Figure B.1: Recent Account Activity duration
Ntotal = 1, 983, 161 Ncall end = 1, 312, 345
103
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
11.00
12.00
13.00
14.00
15.00
00:02 00:11 00:21 00:31 00:41 00:51 01:01 01:11 01:21 01:31 01:41
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Account SummaryTotal for November 2008 to June 2009, All days
Total customer call end
Figure B.3: Account Summary duration
Ntotal = 1, 081, 992 Ncall end = 413, 573
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
11.00
12.00
13.00
14.00
15.00
00:02 00:17 00:32 00:47 01:02 01:17 01:32 01:47 02:02
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Account Activity TodayTotal for November 2008 to June 2009,All days
Total customer call end
Figure B.5: Account Activity Today duration
Ntotal = 353, 328 Ncall end = 175, 774
104
0.000
0.500
1.000
1.500
2.000
2.500
3.000
3.500
4.000
4.500
5.000
00:02 00:17 00:32 00:47 01:02 01:17 01:32 01:47 02:02
Rel
ativ
e fr
eque
ncie
s %
Time(mm:ss) (1 sec. resolution)
ILBank, Credit Card VouchersTotal for November 2008 to June 2009, All days
Total customer call end
Figure B.7: Credit Card Vouchers duration
Ntotal = 608, 529 Ncall end = 91, 677
105
Appendix C
Measurements - Reading Time ofMenu Options
We measured the time required in order to fully hear every option in every menu,within the ILBank IVR system. The cumulative time in the menu is calculated asthe time it takes to hear a certain option plus the time it takes to hear all the optionsprior to it in the relevant menu. Figure C.1 presents our results.
Notice that at the time of our measurements, the ILBank IVR design was a littlebit different from the design presented in Figure 6.1. However, we could still usethe measured times presented here to derive the appropriate times for the designpresented in Figure 6.1.
106
Vertex
NumberDepth Vertex
Option
reading
time
Cumulative
time in
menu
1 1 Current Account and General Issues 3 3
1.1 2 Account Summary 6 6
1.2 2 Recent Account Activity 4 10
1.3 2 Account Activity Today 4 14
1.4 2 Account Activity for Requested Date 4 18
1.5 2 Current Balance 2 20
2 1 Account Activities 3 6
2.1 2 Checking Account and General Activities 8 8
2.1.1 3 Cheque Book Order 3 3
2.1.2 3 Change Password 3 6
2.2 2 Deposit Activities 3 11
2.2.1 3 Daily Interest Deposit 2 2
2.2.2 3 Other Deposit Activities 3 5
2.3 2 Credit activities 3 14
2.3.1 3 Credit Usage 7 7
2.3.2 3 Credit Payment 3 10
2.3.3 3 Credit Balance Information 5 15
2.3.4 3 Credit Balance Explanation 4 19
2.4 2 Trust Fund Activities 3 17
2.4.1 3 Trust Fund Buy 3 3
2.4.2 3 Trust Fund Sell 3 6
2.5 2 Credit Card 3 20
2.5.1 3 Restore code 4 4
2.5.2 3 Change Charge Date 3 7
3 1 Stock Exchange and Foreign Currencies 6 12
3.1 2 Stock Exchange Israel 3* 3
3.1.1 3 Securities Order Status 4 4
3.1.1.1 4 Order Status by Security Number 3 3
3.1.1.2 4 Order Status by Security Name 4 7
3.1.1.3 4 Orders Completed or Partially Completed 5 12
3.1.1.4 4 Orders to be Completed 3 15
3.1.1.5 4 General Explanation Regarding Order Status and Securities 4 19
3.1.2 3 Selected Securities 3 3
3.1.3 3 Stock Exchange Israel 3 6
3.1.4 3 Securities Rates 4 10
3.2 2 World Stock Exchange and foreign currencies 3 6
3.2.1 3 World Stock Exchange 3** 3
3.2.1.1 4 Stock Exchange Europe 3 3
3.2.1.2 4 Stock Exchange USA 3 6
3.2.1.3 4 Stock Exchange Asia 2 8
3.2.2 3 USA Stock Exchange - Option and Futures 4 7
3.2.3 3 Stock Exchange - Interbank Rates 6 13
107
Vertex
NumberDepth Vertex
Option
reading
time
Cumulative
time in
menu
4 1 Investment Information 3 15
4.1 2 Securities Balance 10 10
4.1.1 3 Securities Portfolio Rates 3*** 18
4.1.2 3 Securities Portfolio Details Today 4 22
4.1.3 3 Securities Portfolio Details for Previous Day 4 26
4.2 2 Trust Fund Balance 3 13
4.2.1 3 Trust Fund Balance - by Fund Type 3 3
4.2.2 3 Trust Fund Balance - Specific Fund 3 6
5 1 Loans, Mortgages and Credit Card Information 5 20
5.1 2 Credit Card Vouchers 12 12
5.2 2 Credit Card Charges 5 17
5.3 2 Mortgage 3 20
Figure C.1: Time measurement of ILBank IVR menu options
* There were additional 50 seconds in which a general status of Israeli stockexchange markets was given.
** There were additional 100 seconds in which a general status of stock ex-change markets around the world was given.
*** There were additional 15 seconds in which general securities informa-tion was given.
108
Bibliography
[1] Aksin, Z., Armony, M., and Mehrotra, V. (2007). The modern call center: Amulti-disciplinary perspective on operations management research. Productionand Operations Management, 16(6):665–688.
[2] Aksin, Z., Ata, B., Emadi, S. M., and Su, C.-L. (2013). Structural estimationof callers’ delay sensitivity in call centers. Management Science, 59(12):2727–2746.
[3] Aspect Communications Corporation (2003). Why your customers hate yourIVR systems... and what you can do about it. Technical report.
[4] Behzad, B. and Tezcan, T. (2011). Robust design and control of call centerswith flexible IVR systems. Available at SSRN 1895440.
[5] Brown, L., Gans, N., Mandelbaum, A., Sakov, A., Shen, H., Zeltyn, S.,and Zhao, L. (2005). Statistical analysis of a telephone call center: Aqueueing-science perspective. Journal of the American Statistical Association,100(469):36–50.
[6] Commarford, P. M., Lewis, J. R., Smither, J. A.-A., and Gentzler, M. D. (2008).A comparison of broad versus deep auditory menu structures. Human Factors:The Journal of the Human Factors and Ergonomics Society, 50(1):77–89.
[7] Delorey, E. (2003). Correlating IVR performance and customersatisfaction. Technical report. http://www.easyivr.com/tech-ivr-applications_108.htm.
[8] Denardo, E. V., Rothblum, U. G., and Van der Heyden, L. (2004). Index poli-cies for stochastic search in a forest with an application to R&D project man-agement. Mathematics of Operations Research, 29(1):162–181.
[9] Feigin, P. D. (2006). Analysis of customer patience in a bank call center. Work-ing Paper, Technion, Israel Institute of Technology. http://ie.technion.ac.il/serveng/References/CCA-Patience.pdf.
[10] Gans, N., Koole, G., and Mandelbaum, A. (2003). Telephone call centers:Tutorial, review, and research prospects. Manufacturing & Service OperationsManagement, 5(2):79–141.
109
[11] Garnett, O., Mandelbaum, A., and Reiman, M. (2002). Designing a call cen-ter with impatient customers. Manufacturing & Service Operations Manage-ment, 4(3):208–227.
[12] Granot, D. and Zuckerman, D. (1991). Optimal sequencing and resource allo-cation in research and development projects. Management Science, 37(2):140–156.
[13] Hassan, A., Jones, R., and Klinkner, K. L. (2010). Beyond DCG: user behav-ior as a predictor of a successful search. In Proceedings of the Third ACM Inter-national Conference on Web Search and Data Mining, pages 221–230. ACM.
[14] Hastie, T., Tibshirani, R., Friedman, J., Hastie, T., Friedman, J., and Tibshi-rani, R. (2009). The elements of statistical learning, volume 2. Springer.
[15] Huguenard, B. R., Lerch, F. J., Junker, B. W., Patz, R. J., and Kass, R. E.(1997). Working-memory failure in phone-based interaction. ACM Transactionson Computer-Human Interaction (TOCHI), 4(2):67–102.
[16] Kaplan, R. S. and Porter, M. E. (2011). How to solve the cost crisis in healthcare. Harv Bus Rev, 89(9):46–52.
[17] Khudyakov, P., Feigin, P. D., and Mandelbaum, A. (2010). Designing acall center with an IVR (Interactive Voice Response). Queueing Systems,66(3):215–237.
[18] Khudyakov, P., Mandelbaum, A., Trofimov, V., Maman, S., Nadjhrov, E.,Gavako, I., Liberman, P., and Kutsy, K. (2009). Empirical analysis of a callcenter in an Israeli commercial company. Technical Report, Technion, Israel In-stitute of Technology. http://ie.technion.ac.il/labs/serveng/files/sec3/The_Call_Center_of_IL_Bank.pdf.
[19] Marics, M. A. and Engelbeck, G. (1997). Designing voice menu applicationsfor telephones. Handbook of human-computer interaction, 2.
[20] Salcedo-Sanz, S., Naldi, M., Perez-Bellido, A. M., Portilla-Figueras, J. A.,and Ortı́z-Garcı́a, E. G. (2010). Evolutionary optimization of service times inInteractive Voice Response systems. IEEE Transactions on Evolutionary Com-putation, 14(4):602–617.
[21] Schumacher, R. M., Hardzinski, M. L., and Schwartz, A. L. (1995). Increas-ing the usability of Interactive Voice Response systems: Research and guide-lines for phone-based interfaces. Human Factors: The Journal of the HumanFactors and Ergonomics Society, 37(2):251–264.
[22] Srinivasan, R., Talim, J., and Wang, J. (2004). Performance analysis of a callcenter with Interactive Voice Response units. Top, 12(1):91–110.
110
[23] Suhm, B. and Peterson, P. (2002). A data-driven methodology for evaluatingand optimizing call center IVRs. International Journal of Speech Technology,5(1):23–37.
[24] Suhm, B. and Peterson, P. (2009). Call browser: a system to improve thecaller experience by analyzing live calls end-to-end. In Proceedings of theSIGCHI Conference on Human Factors in Computing Systems, pages 1313–1322. ACM.
[25] Technion, Israel Institute of Technology (2009). ILBank Call Center. Unpub-lished Manuscript.
[26] Zohar, E., Mandelbaum, A., and Shimkin, N. (2002). Adaptive behavior ofimpatient customers in tele-queues: Theory and empirical support. ManagementScience, 48(4):566–583.
111