recommender systems and consumer product … product recommendation...1.1 literature review ... the...
TRANSCRIPT
1
Recommender Systems and Consumer Product Search
(full paper, word count: 7773)
Vidyanand Choudhary Zhe (James) Zhang
University of California, Irvine University of Texas, Dallas
[email protected] [email protected]
Abstract
Recommender systems are popular tools used by online retail websites to suggest
products to consumers, such as those featuring music, movies, and other online content.
An online retailer can use this technology to entice consumers to buy the recommended
products, and consumers can avoid searching for alternative products if they accept the
recommended products. In this paper, we develop an analytical framework to examine
the optimal recommender system strategy of an online multi-product retailer and the
impact of the recommender system on society. We show that the retailer’s optimal
recommender system strategy is driven by consumers’ search cost and misfit cost, as well
as the probability that a consumer continues product search if she rejects the
recommendation. We find that when the recommender system is relatively less accurate,
the firm cannot mislead all consumers with the recommended products. Although these
recommended products are a perfect match for some consumers, other consumers may
reject the recommendation. When the firm improves the accuracy of the recommender
system, the net of aggregate consumer mismatch cost and search cost decrease. This
means that the firm’s use of recommender system generates externality that benefits
consumers. Moreover, the firm’s revenue increases, and thus, social welfare increases.
We also find that when the recommender system is relatively accurate, the firm
deliberately misleads all consumers by not giving them their ideal products. Nonetheless,
all consumers take the recommended products, and the retailer’s revenue increases. This
leads to a mismatch cost that is borne by the consumers, and aggregate consumer surplus
reduces; but interestingly, this does not lower social welfare.
2
1. Introduction
Product recommender systems are personalized sale assistance tools to make product
search easier for consumers before making their purchase decisions. In general, online
retailers or intermediaries use recommender systems to provide personalized product or
content recommendations that individual consumers may be interested in. These systems
are used in cases where there are a large number of differentiated products and become
extremely popular in recent years in a variety of applications, like books, movies, and
music. Figure 1 illustrates the personalized product recommendation provided by
Amazon’s recommender system. A consumer sees this set of recommended products at
the home page immediately after her logging in, and she can end product search if she
finds what she wants among these products. From consumers’ perspective, recommender
systems make their product search easier and save product search time. The question is
that how does a recommender system affect consumers’ product search process; and how
does an online retailer strategically manage the recommender system to maximize
revenue.
Figure 1: Amazon product recommendation
The value proposition of recommender systems is that they provide consumption
experience that is personalized to consumers’ tastes (Hosanagar, Fleder, Lee and Buja,
2014). Giving individual consumers what they want reduces their product search cost in a
context where consumers face many choices. In Figure 1, Amazon identifies this
consumer as someone who is looking for books about business intelligence or data
mining, and thus, recommends to her three books, Principles of Data Mining, Python in a
Nutshell, and Introductory Statistics with R. She would buy one of the recommended
3
books if it was the book she was looking for and finish the product search; otherwise, she
might start searching for other books using the in-store search engine.
As a result, a consumer has a better shopping experience with Amazon if the
recommended products are the ones she is interested in. Thus, from online retailers’
perspective, these tools are used to build loyalty and turn browsers into buyers. Google
News reports that use of recommendation increases articles viewed by 38% (Das et al.
2007); Amazon reports that 35% of sales originate from recommendations (Lamere and
Green 2008); and Netflix reports that 60% of their movie rentals originate from
recommendations (Thompson 2008). Thus recommender systems are highly relevant and
becoming increasingly powerful tool for retailers.
On the other hand, online retailers can manipulate the product search process of
consumers by providing information about only a selection of products or contents.
Online advertising-supported intermediaries, like online news sites and search engines,
expose advertisements to viewers in more prominent places than the content that is of
interest to consumers. This is referred to as search diversion by Hagiu and Jullien (2014).
One recent account is that Google allegedly manipulated search results—promoting its
own services and suppressing competitors’1 2. In addition, Amazon deliberately removed
pages promoting books by Hachette in midst of a dispute over e-book pricing3. Similarly,
an online retailer can deliberately direct consumers’ attention to specific products in the
form of personalized recommendation,
In this paper, we examine a context where consumers first get recommendation
from the recommender system, and then they may use a product search engine to find
alternative products before making their final purchase decision. The research questions
are: 1) What is the effect of product recommendation on consumers’ product search and
purchasing decision? 2) How does an online retailer strategically deploy a recommender
system? 3) What is the effect of product and consumer characteristics on the online
retailer’s optimal product recommendation strategy? 4) What is the impact of a
recommender system on consumer surplus and social welfare?
1 http://www.telegraph.co.uk/technology/google/9281639/Google-warned-by-EUs-antitrust-inquiry-of-search-manipulation-concerns.html 2 http://www.bidnessetc.com/22603-yelp-doesnt-like-googles-likely-antitrust-settlement-with-the-eu/ 3 http://bits.blogs.nytimes.com/2014/05/23/amazon-escalates-its-battle-against-hachette/?_php=true&_type=blogs&_r=0
4
Understanding consumer information search and acquisition has been an
important theme in economics and marketing, and information systems. In Bakos (1997),
the electronic market system provides product search function that lowers buyer search
costs and improves market efficiency. In recent years, online retailers have not only
improved product search functions, but also provided recommender systems to
consumers to improve their product search experience. Due to its popularity,
recommender systems have become a major theme of research (Murthi and Sarkar 2003,
Dellarocas 2009, Hosanagar et al. 2014), and its impact on consumer product search and
market structure is being studied (Kim, Albuquerque, and Bronnenberg, 2010).
Prior research has typically treated recommender systems as a tool or an add-on
service that provide accurate match between buyers and products (Häubl & Trifts 2000,
Tam & Ho 2006, Fong 2012). In this study, we develop a framework in a context where
products are differentiated and an online multi-product monopolist can skew
recommendations in favor of products with higher profit margin. Consumers accept or
reject the product recommendation depending on their preferences. If consumers reject
the recommendation, then they may continue to search for alternative products, thus
incurring search costs. We find that the firm can benefit from strategically recommending
products with higher profit margin. We also show that the firm’s decision about product
recommendation depends on product profit margin, consumer search cost and misfit cost.
In our analysis of consumer surplus and social welfare, we find that when the
recommender system is relatively less accurate, the firm cannot mislead all consumers
with recommended products. When the firm improves the accuracy of the recommender
system, both the consumer surplus and the firm’s revenue increase, thus, social welfare
increases. On the other hand, when the recommender system is relatively accurate, the
retailer deliberately misleads all consumers by not giving them their ideal product. This
leads to a mismatch cost that is borne by the consumers. Thus, when the firm improves
the accuracy of the recommender system, consumer surplus falls. Surprisingly, this does
not lead to any decrease in social welfare.
5
1.1 Literature Review
A simplified taxonomy of recommender systems divides them into two types: content-
based and collaborative filtering-based systems. Content-based systems use product
information (e.g., genre, mood, author) as criteria to recommend items similar to those a
user rated highly. On the other hand, collaborative filtering-based systems recommend
products that other consumers with similar tastes and preferences bought in the past.
There is a large body of work on designing recommender systems, and Adomavicius and
Tuzhilin (2005) provide an extensive review in the information systems literature.
The economic implications of recommender systems and how they affect society
are not well understood. There is a growing stream of research focusing on the economic
implications of electronic commerce in online market. Studies have examined the impact
of use of recommender systems on sales (Ansari et al. 2000, Das et al. 2007, Bodapati
2008) and future sales (Johar et al. 2014), the influence of recommender systems on
consumer search behavior (Kim et al. 2010), “long tail” effect of information goods
(Fleder and Hosanagar 2009, Oestreicher-Singer and Sundararajan 2012), and
personalization service (Murthi and Sarkar 2003, Hosanagar et al. 2014).
Another relevant stream of literature examines the impact of the reduction of
consumer search cost on firms’ profit and social welfare. Bakos (1997) studies the
implication of consumer search cost on online sellers’ pricing and product strategies, and
the resulting reduction on market inefficiency. Branco, Sun and Villas-Boas (2012)
develop a framework to study the role between graduate learning, search cost, and
consumers’ purchase decision. They discuss the impact of consumer search on profits and
social welfare, and how the seller chooses its price to strategically influence the extent of
consumers’ search.
We consider recommender systems as an integral part of the consumer product
search process, and consumers use recommender systems in combination with other
product search features, such as search engines, before making a purchase decision. We
consider the case where recommender systems can be used strategically to favor higher
margin products. The firm needs to balance the tradeoff between profit margin of the
6
recommended product and the likelihood of consumers’ acceptance of the
recommendation. In our framework, product recommendation is a substitute for the use
of search engines as consumers can learn about the product recommendation without
incurring search cost. Therefore, this paper also contributes to the literature on consumer
search.
We describe the model setup in §2 and the results in §3. We conclude with a
discussion and directions for future research in §4.
2. Model Setup
2.1 Consumers
In a world with differentiated product offerings, we consider a market of consumers with
heterogeneous preferences and one monopolist firm selling products that are spatially
differentiated along a single dimension. In our horizontal differentiation model, consumer
preference or location is denoted by, ix and it is distributed on the interval [0,1] . The
monopolist’s products are differentiated along the same dimension so that product
locations are evenly distributed on [0,1] .
A consumer incurs a “misfit” cost t per unit distance between her ideal location
and product location. This cost represents a consumer’s loss due to obtaining a product at
a location that is different from her ideal location per unit distance. All consumers have
identical unit demand, subject to a reservation utility V , and they are risk neutral. Utility
of a consumer at ix from buying a product at x is:
( , ) | |i iU x x V x x t (1)
All products seem ex ante identical to consumers. This implies that consumers
have no prior knowledge of the location of any product. Each consumer can learn a
product’s location by searching and evaluating the product. The time and effort to search
and analyze the product is the search cost, represented by parameter c .
7
A consumer sequentially searches and examines one product at a time. She
decides whether to purchase the examined products or to continue the search. She will
stop searching and accept a product if the expected gain in utility from an additional
search is less or equal to the cost of another search. It follows that a consumer will
purchase a product when it is within a certain distance to her ideal location. We refer to
this distance as the acceptance range, and we formally define this range in the next
section.
Figure 2: Consumer decision tree
There are two types of consumers: a proportion of consumers, , who are willing
to search and the remaining consumers, 1 , who do not search. Consumers of the first
type will search if they reject the recommendation. They will continue searching till they
find a product sx in their acceptance range. Consumers of the second type will never
search for products and will not buy any product if they reject the recommendation. This
may occur for several reasons for example some consumers may have an outside option
in the form of an existing or substitute product. We assume that the acceptance range is
identical for both types of consumer. Figure 2 illustrates the flow of consumer decisions.
2.2 Firm and product profit margin
The monopolist firm sells a continuum of horizontally differentiated products at
the same price. There are several examples that fit this setting in the online movie or
music subscription models (Netflix and Spotify), where online users pay monthly
subscription fee and do not pay for individual movie or music titles. This setting is also
applicable to the revenue models where online intermediaries such as search engines,
newspapers that charge little or no fee from users, and instead obtain revenue from
8
advertisers. The firm gains heterogeneous profit margin from sales of different products.
This can happen due to revenue and cost factors. For example, in the case of Netflix and
Spotify, the quality of digital media (non-HD and HD) impacts the price that retailer can
charge as well as the retailer’s marginal costs. In the case of the Google search engine,
the revenue from keyword advertising varies across different keywords. In general, the
profit margin can vary across products because they are provided by different suppliers
with different contract terms. To simplify our model, we assume that products have
heterogeneous profit margins and all products have the same exogenously determined
price. We assume a mapping from product location to profit margin for each product as
( )v x mx , where x is product location and m is the highest profit margin.
The firm knows the distribution of consumer locations, but not the exact location
of a consumer. The firm can invest time and effort to gather information about consumers
and such analysis can better inform the firm about the consumer’s location. In our model,
the firm is able to narrow a consumer’s possible location to a window. We refer to this
window of possible consumer locations as a bucket. The firm knows that the consumer’s
true location is within the bucket so that the probability of the bucket containing the
consumer’s location is 1. The range of a bucket is denoted by r . In this way, instead of a
consumer’s exact location, the firm identifies a bucket that the consumer belongs to,
which spans from ja to ja r . Figure 1 illustrates that the firm observes the bucket
[ , ]j ja a r associated with a consumer without knowing her ideal location, ix . We assume
that the consumer’s location is equally likely to be at any point within the bucket and this
is known to the firm. This implies that the firm has better information about the location
of consumers when the range of the bucket ( r ) is small. At the extreme values of r , when
1r the firm has no information about consumers, and when 0r , the firm has precise
information about consumers’ locations.
9
Figure 3: Consumer bucket [ , ]j ja a r for a consumer at ix
When a consumer arrives at the retailer, the firm does not observe the true
location of the consumer. Instead, he observes a bucket. In aggregate, the firm observes
buckets whose left border ( ja ) are uniformly distributed on the interval [0,1 ]r . When
the firm recommends a product at rx to a consumer associated with a bucket [ , ]j ja a r ,
there are three possible outcomes: 1) the consumer buys the recommended product and
the firm gets the profit margin rx m ; 2) the consumer rejects the recommendation, and
searches on her own, to purchase a product at sx , resulting in expected profit of [ ]sE x m to
the firm; and 3) the consumer rejects the recommendation, and does not buy any product,
resulting in the firm getting zero profit margin. The probabilities associated with these
three outcomes depend on the location of the product recommendation ( rx ), the left
border of the bucket ( ja ), and the range of bucket ( r ). The firm’s expected revenue from
a bucket whose left border is at ja is [ ( | )]ja rE R x r . Thus, the firm’s total expected revenue
consists of the expected revenue from all buckets and it is:
1
0[ ] [ ( | )]
j
r
a rE R E R x r da
(2)
In this paper, we make two assumptions. 1) We assume the price of all products is
zero so that consumers’ purchasing decisions depend on their baseline utility, misfit and
search costs. 2) We assume that for consumers, whose locations are close to the ends of
the line (0 or 1), they can always find the same number of products located on the left and
on the right of their individual locations.
3. Analysis
A consumer sequentially searches for product information on the retailer firm’s website.
In a sequential search, a consumer decides to stop or continue a search each time after
10
having searched a product. The theory of optimal sequential search states that a consumer
continues a search only if the marginal benefit of doing so outweighs the marginal cost.
This leads to a stopping criterion that we refer to as the Acceptance Range of a consumer.
3.1 Acceptance range
A consumer incurs cost c per search when she looks for a product and learns its
location. Moreover, a consumer incurs misfit cost t per unit distance between her ideal
location and the location of purchased product. A consumer will stop searching and
purchase the currently examined product if the gain in the expected utility of an
additional search is less than or equal to the search cost c . We define that a consumer is
indifferent between buying a product located at x distance away from her ideal location
ix and continuing searching. This critical distance ix is obtained from the following
equation:
2 ( )i
i
x x
ix
x x tdx c
(3)
The distribution of firm’s products is common knowledge, thus, a consumer
knows there is a product with probability dx along any differentiated part of the unit line.
On the left-hand side of the equation (3) is the expected gain of utility due to finding a
product that has a distance less than x to the consumer’ location, and on the right-hand
side of the equation is the search cost. Solving the equation (3), we obtain the acceptance
range cx t . This implies that a consumer will stop searching for products and buy the
currently examined product if it is closer than or equal to cx t distance from her ideal
location.
PROPOSITION 1: When search costs increase 0dx
dc
or misfit costs decrease 0
dx
dt
then the expected distance between the consumer’s ideal location and the location of the
purchased product will increase.
Proof is in the Appendix 3.A.
11
Proposition 1 establishes the stopping rule of an individual consumer who
strategically balances the trade-offs between her misfit cost and search cost. When her
search cost is high, having examined a product, the cost of finding a fit product (closer to
her ideal location) is likely to be high. Thus, she will stop searching when she finds a
product even if the product is distant from her ideal location when she has high search
cost. Figure 4 illustrates the concept of this stopping rule for a consumer, whose location
is at ix . She will buy a product if it is within the acceptance range ,[ ]i ix xc t c t ,
otherwise, she may continue to search for alternative products.
Figure 4: Consumer's acceptance range
3.2 In the absence of recommender system
Some consumers will search for products when the firm does not have a
recommender system. If a consumer at ix searches, she incurs cost c for each search, and
she stops searching till she finds a product that is within her acceptance range
,[ ]i ix xc t c t . Hence, the probability of finding products outside the acceptance
range on the first 1k searches and finding a product within the acceptance range on the
thk search is 1( ) (1 )kP k p p , where p is the probability of a random draw of a product
that falls in the acceptance range. Therefore, the number of searches till a product within
the acceptance range is discovered by a consumer follows a geometric distribution, and
the expected number of searches is 1/ p . For a consumer at ix , the probability of a
random draw of products being within the acceptance range is 2p c t as she would
buy any product within ,[ ]i ix xc t c t . Hence, the expected number of searches is
[ ] 1/ / 2E k p t c , and the expected cost of searches is the product of the expected
number of searches and the cost per search: [ ] / 2E kc ct .
12
A consumer will continue to search for products till she discovers a product sx
that is within the acceptance range ,[ ]i ix xc t c t . Since the products are evenly
distributed, the expected value of product sx is [ ]2
i i
s i
c t c tx xE x x
. Moreover,
proportion of consumers will search and 1 proportion of consumers will not search.
This implies that the firm’s expected profit from a consumer at ix in the absence of a
recommender system is:
[ ]s iE x m x m (4)
3.3 Product recommendation
The firm can recommend a product to each individual consumer when she arrives
at the website. Depending on the bucket [ , ]j ja a r associated with the consumer ix , the
firm recommends a product at ( )r jx a . This consumer will accept the recommended
product rx if it is within her acceptance range , [ ]i ix xc t c t as we have shown in
§3.1. If so, then the firm has profit margin rx m . The consumer at ix will reject the
recommended product rx if it is outside the acceptance range , [ ]i ix xc t c t . The
consumer will continue to search with a probability till she finds an acceptable product,
and thus the firm’s expected profit margin is ix m as shown in (4).
The firm prefers to recommend products with larger x , as the profit margin is
( )v x mx , which implies larger profit margin for larger x . Figure 5 illustrates the process
of the firm’s recommending product. The 1st step is that the firm understands the true
location of the consumer can be at any location within the range of the bucket [ , ]j ja a r
with equal probability. The second step is that the firm decides to recommend a product
at ( )r jx a , and expects three outcomes which depend on the relative position of the left
border of the bucket ( ja ), the range of a bucket ( r ), and the product recommendation
( ( )r jx a ). When the product recommendation is within the range of
( )j r j ja c t a c tx a r and ( )r j jx a a c tr , the consumer will accept the
13
recommendation rx if ( ) /r j i jx a c t x a r . The likelihood of a consumer accepting
the recommendation rx is: ( / )
( )j r
accept r
a r x c tL x
r
.
If ( ) /j i r ja x x a c t , then she does not accept the recommendation. If she
rejects the recommendation, then with a probability , she will search, and with a
probability 1 , she will leave the retailer’s website. Though range of the buckets
seems to be greater than the acceptance range ( /r c t ) in Figure 5, the concept is
applicable to the cases where /r c t .
Figure 5: The firm’s product recommendation and consumer’s decision when
( )j r j ja c t a c tx a r and ( )r j jx a a c tr
When ( )j r j ja c t a c tx a r and ( )r j jx a a c tr , the firm’s expected
revenue of recommending product at rx is:
/
/[ ( )] / /
j r
jr j
a r x c t
a r r i i ix c t a
E R x x m rdx x m rdx
(5)
The first term is the expected profit margin if the recommendation was within the
consumer’s acceptance range ( [ / , ]i r jx x c t a r ). The second term is the expected
profit margin if the recommendation was outside her acceptance range ( [ , / )i j rx a x c t ).
When r j c tx a r , the firm gets zero from recommendation. When /r jx a c t , the
second term is zero; while when /r jx a r c t , the consumer may reject the
recommendation and give up product search if the consumer location is greater than
/r i jx c t x a r . Thus, the firm’s maximization problem is:
14
max [ ( )]j
r
a rx
E R x
subject to r jx ca t
r j c tx a r
r j c tx a r
We solve the maximization problem and obtain
( (1 ) ( (2 ) ))0
rjm c t a r x
r t
as FOC. The value of recommendation
1
1/
2 2
ja rc tx
satisfies the FOC. Since 0 1 , the optimal product
recommendation 1 /ja cx r t , the constraints on the optimal product recommendation
are only r j c tx a r and jrx a c t . When /r jx a c t , the firm can increase the
value of rx to jrx a c t , which leads to higher expected profit margin from
recommendation (greater value of the first term in (5) and the second term remains as 0),
and thus, the greater expected revenue. Note that when 2r c t , /j jcr cta a t ,
thus, /r jx a c t is the only boundary solution.
When 2r c t , /j jcr cta a t , it is possible that
( )j r j ja c t a c tx a r which means that a consumer, whose location is either
extremely large or small within the bucket, may reject the recommended product rx . In
this case, the firm’s expected revenue of recommending a product at rx is:
/ /
/ /[ ( )] / / /
r r j
jr j r
x c t x c t a r
a r r i i i i ix c t a x c t
E R x x m rdx x m rdx x m rdx
(6)
The first derivative of the expected revenue in (6) is [ ( )] 2(1 )
/ 0ja r
r
dE R x mc t
dx r
.
This implies that the optimal solution is the highest possible value /r jx a r c t . Thus,
when 2 /r c t , the constraint is r j c tx a r , and /r jx a r c t is a boundary
solution.
15
We further assume that the search cost is low compared to the misfit cost, that is
9 4
t tc . The optimal product recommendation strategy is summarized in Lemma 1.
LEMMA 1: 1) When 0 /r c t , the optimal product recommendation is /r jx a c t
for buckets [0,1 / ]ja c t , and 1rx for buckets (1 / ,1 ]ja c t r ; 2) When 1/c t r r ,
the optimal product recommendation is 1rx x for buckets 1[0, ]ja a , and /r jx a c t
for buckets 1( ,1 ]ja a r ; 3) When 1 1r r , the optimal product recommendation is
1rx x for all buckets [0,1 ]ja r ; where 1
1/
2 2
ja rc tx
, 1
1 /
2
c tr
and
1
/
1
r ca
t
.
It is obvious to see that it is more like that a consumer will accept the
recommended product if the range of a bucket ( r ) decreases. Moreover, when every
consumer, who rejects recommended product, continues to search for alternative products
( 1 ), the firm recommends products that do not match with consumers’ preferences.
The firm recommends products outside the bucket if the range of buckets is relatively
small, or recommends the product at the most right border of the bucket when the range
is large. When consumers may give up product search all together with a probability, the
firm recommends products that are inside the bucket so that the recommendation may be
a perfect match for the consumer.
PROPOSITION 2: The location of the product recommended by the recommender
system rx is independent of the profit margin parameter m .
The firm’s strategic recommendation strategy is driven by heterogeneity among
products’ profit margin xm and the profit margin increase as either x or m increases.
Therefore, one may think that the firm’s product recommendation strategy will shift to
the right when profit margin m is higher. However, counter-intuitively, Proposition 2
says that the product recommendation does not depend on m .
3.4 Firm’s total expected revenue
16
Lemma 1 says that the firm’s recommendation strategy for each consumer bucket
depends on the range of a bucket ( r ), the consumer’s search cost ( c ), misfit cost ( t ), and
the probability that consumer will continue product search after rejecting the
recommendation ( ). Thus, the firm’s expected revenue of a bucket over [ , ]j ja a r also
depends on these factors, *[ ( | , , , , )]j ja aE R x r c t . Without considering the cost of
recommender system, we show that the firm’s expected total revenue is:
1
0[ ( )] [ ( | )]
j
r
a r jE R r E R x r da
, and this decreases as the bucket range increases (Figure 6).
Figure 6: Impact of accuracy of recommender system on the firm's expected revenue where 1m ,
0.2c , 1t , and 0.6
PROPOSITION 3: The firm’s expected total revenue increases as the range of
consumer buckets decreases (lower r ), [ ( )]
0dE R r
dr .
Decrease in the range of buckets means the firm can more accurately identify
each consumer, and thus it is more likely for a consumer accepting a recommended
product that has a higher profit margin. Thus, the firm’s total revenue increases as r
decreases.
PROPOSITION 4: When the range of buckets is relatively small, /r c t , the firm’s
optimal product recommendation is outside the bucket * [ , ]r j jx a a r .
17
When the optimal range of bucket is * /r c t (Lemma 2), therefore, the firm’s
optimal product recommendation is * /r jx a c t . Proposition 4 states that the firm will
never provide recommendation that perfectly matches the consumer’s preference if the
recommender system is sufficiently accurate ( /r c t ). This implies that the consumers
will accept the recommendation, but they never get a perfect match to their ideal
locations. This is intuitive as the firm will try to sell products that are profitable and
acceptable to consumers. When the cost of improving accuracy of the recommender
system is relatively low, the firm will invest to build a more accurate recommender
system. This allows the firm to sell the product that is acceptable to consumers but he
chooses to recommend products that are outside the consumers’ buckets. Such a
recommendation strategy yields higher profit margin to the retailer. Proposition 5
explores the firm’s product recommendation strategy when the cost of improving the
accuracy of the recommender system is relatively high.
PROPOSITION 5: When the range of buckets is relatively large, * /r c t , (1) the
firm’s optimal product recommendation is inside the bucket * [ , ]r j jx a a r ; and (2) a
consumer is more likely to accept the recommendation if is smaller ( )
0accept rdL x
d .
When 1/c t r r , the firm’s optimal product recommendation is 1rx x for
buckets start from 1[0, ]ja a , or /r jx a c t for buckets start from 1( ,1 ]ja a r ; or when
1 1r r , the firm’s optimal product recommendation is 1rx x for all buckets (Lemma 1).
In either case, the product recommendation is inside the range of buckets * [ , ]r j jx a a r ,
which implies that a consumer may get a perfect match with the product recommendation.
However, in this case a consumer may reject the recommendation if her location is
[ , / )i j rx a x c t , since the optimal range of bucket is relatively large, * /r c t .
Proposition 5 states that when the cost of improving accuracy of the recommender system
is relatively high, the firm will invest less and thus the recommender system is less
accurate, * /r c t . Therefore, a consumer may reject the recommendation, but she may
get a perfect match with the product recommendation.
18
Moreover, it is more likely that she accepts the recommendation when it is less
likely for consumers to continue to search if they reject the product recommendation
(smaller ), or the cost of is lower, or the acceptance range is greater. The intuition is
that when there is a probability that a consumer will give up product search and do not
buy any product if she rejects recommendation ( 0 1 ), the firm needs to sacrifice the
expected profit margin from recommending product (by recommending a lower rx ) as he
may lose a consumer if she rejects the recommendation. Therefore, the chance that a
consumer will accept the product recommendation is higher, if there is a higher chance if
the firm loses a consumer. \
3.5 Realized consumer surplus
The expected misfit cost for a consumer at ix who search products till she finds
an acceptable product is the expected loss from products that are within the range
[ / , / ]i ix c t x c t :
/
2 ( ) / 2i
i
x c t
ix
x x tdx ct
The expected number of attempts for a consumer who searches products till she
finds an acceptable product is / 2t c , and the expected search cost is / 2ct from §3.2.
Therefore, consumer surplus of a consumer who rejects project recommendation,
and continues to search for alternative product till an acceptable product is the baseline
utility minus the expected search cost and misfit costs:
/ 2 / 2s
iCS V cs cs V cs
The consumer surplus of a consumer at ix who accepts the product
recommendation rx is:
| |r
i r iCS V x x t
In order to examine the impact of the firm’s recommender system strategy on
consumer surplus, we need to assume a mapping from consumer location to consumer
19
bucket, / (1 )j ia x r . In this subsection, we assume consumer locations ix are distributed
uniformly on the interval [0,1] , and the left borders of buckets ja are distributed
uniformly on the interval [0,1 ]r .
Lemma 1 states that when the range of buckets is relatively small, /r c t , the
firm’s product recommendation is * /r jx a c t for all buckets. Since all consumers
accept the recommendation and none searches for products, consumers only incur misfit
cost (Proposition 4). For consumers whose locations are close to 1, (1 / ) / (1 )i c tx r ,
the product recommendation is * 1rx as it is the most profitable product; while for
consumers whose location are (1 / ) / (1 )i c tx r , the product recommendation is
* /r jx a c t . The aggregate consumer surplus is:
(1 / )/(1 )
(1 / )/(10 )
1
( / ) ( 1)r r
i r j i
c t r
cr i
t riCS CS x a c t dx CS x dx
(7)
When the range of buckets is medium, 1/c t r r , the firm’s optimal product
recommendation is *
1
1/
2 2
j
r
a rc tx x
for buckets 1[0, ]ja a , or * /r jx a c t for
buckets 1( ,1 ]ja a r . Some consumers ( 1 /ix x c t ) reject the product recommendation
because it is outside their acceptance ranges, and they continue to search and buy
products that are acceptable. Their expected misfit cost is / 2ct and search cost is
/ 2ct . Other consumers ( 1 /ix x c t ) accept the product recommendation, and they do
not search for other products. Their search cost is zero. Moreover, for consumers
1 1[ / , / (1 )]ix x c t a r , whose corresponding buckets are 1[0, ]ja a , the product
recommendation is *
1rx x ; while for consumers 1[ / (1 ),1]ix a r , the product
recommendation is * /r jx a c t . Thus, the aggregate consumer surplus is:
1 1
1 1
/ /(1 ) 1
10 / /(1 )
( ) ( / )x c t a r
s r r
i i i r i i r j ix c t a r
CS CS dx CS x x dx CS x a c t dx
(8)
Furthermore, when the range of buckets is relatively large, 1 1r r , the optimal
product recommendation is 1rx x for all consumers. Similar to the case where
20
1/c t r r , some consumers ( 1 /ix x c t ) reject the product recommendation, while
other consumers ( 1 /ix x c t ) accept the product recommendation. For consumers who
reject the product recommendation, they may continue to search for products, and they
incur search and misfit costs; while for consumers who accept the product
recommendation, they only incur misfit cost. The aggregate consumer surplus is:
1
1
/ 1
10 /
( )x c t
s r
i i i r ix c t
CS CS dx CS x x dx
(9)
We further assume that the unit misfit cost is 1t , and thus, 1 1
9 4c , following
the assumption in §3.2.
PROPOSITION 6: (1) The aggregate consumer surplus increases as the range of
buckets increases, 0dCS
dr , when the range of bucket is relatively small /r c t ; (2)
aggregate consumer surplus decreases as the range of buckets decreases, 0dCS
dr , when
the range of bucket is relatively large /r c t .
Proposition 6 states that when the accuracy of recommender system is not
accurate ( /r c t ), consumer surplus increases if the range of buckets decreases. This
implies that consumers benefit as the recommender system becomes more accurate in this
range. On the other hand, when the recommender system is already accurate ( /r c t ),
consumer surplus decreases if the range of buckets decreases. This means that consumers
become worse off if recommender system becomes more accurate (Figure 7).
21
Figure 7: Impact of bucket range on the realized consumer surplus
Proposition 6 argues that improvement of recommender system (shorter r ) by the
firm has heterogeneous impacts consumer surplus. The range of consumer buckets also
has heterogeneous impacts on social welfare. Figure 8 illustrates that social welfare
increases as the accuracy of the recommender system improves (shorter r ) when the
range of consumer buckets is relatively large ( / ,1)r c t , but is constant the range of
consumer buckets is relatively small (0, / ]r c t . Combined with what we have shown
about aggregate consumer surplus and the firm’s revenue, we show that when the
recommender system is relatively not accurate, improving its accuracy is beneficial to
both the firm and its customers. On the other hand, decrease in the range of consumer
buckets does not entail any increase in social welfare when the range is already relatively
small. The loss in the aggregate consumer surplus is the firm’s gain in revenue when the
range of consumer buckets decreases if the range is already relatively low ( /r c t ).
from accurately identifying individual consumer’s preference and recommending
products with higher margin.
22
Figure 8: Impact of bucket range on the realized social welfare
PROPOSITION 7: When the range of buckets decreases, (1) social surplus does not
change as the range of buckets increases, if the range of bucket is relatively small
/r c t ; (2) social surplus decreases if the range of bucket is relatively large /r c t .
When the range of consumer buckets is relatively large ( /r c t ), since the
aggregate consumer surplus (Proposition 6) and the firm’s revenue increases, social
welfare increases. This implies that when the firm does not have adequate information
about individual consumers’ preferences, improving accuracy of the recommender
system creates externality to consumers. At the same time, the firm’s revenue increases,
since more consumers accept the recommended products with higher profit margin when
the recommender system becomes more accurate.
On the other hand, when the recommender system is relatively accurate, the
retailer deliberately misleads all consumers by not giving them their ideal products.
When the firm improves the accuracy of the recommender system, all consumers are
given the recommended products that are further away from their preferences, but are
within their acceptance range. This leads to higher mismatch cost that is borne by the
consumers. Interestingly, even when the recommender system is accurate and misleads
all consumers, social welfare does not decrease. This happens because the firm’s revenue
increases at the same rate when the recommender system becomes more accurate.
23
4. Conclusion
In this paper, we develop a framework to examine the optimal recommender system
strategy of an online monopolist multi-product retailer. In the context of online products,
we consider consumers consider the personalized recommended products of the
recommender system before they decide to search for alternative products. The use of
recommender system reduces consumers’ search cost as they avoid product search by
accepting the product recommendation.
We show that the firm’s expected revenue increases when the accuracy of the
recommender system improves. We also find that the firm’s optimal recommender
system strategy is driven by consumers’ search cost and misfit cost, and the probability
that a consumer does not buy any product if she rejects recommendation. We find that
when the recommender system gets more accurate, the aggregate consumer surplus
decreases if the recommender system is relatively accurate, and increases if the
recommender system is relatively less accurate. This is surprising, as one may think the
recommender system would have a monotonic impact on consumer surplus.
More interestingly, we find that when the recommender systems gets more
accurate, social welfare increases if the recommender system is relatively less accurate,
but does not change if the recommender system is relatively accurate. This implies that
both the firm and consumers benefit from improving accuracy of the recommender
system when it is less accurate. On the other hand, the firm’s gain in revenue is exactly
the loss in consumer surplus, and thus, the social welfare does not change, if the
recommender system is relatively accurate.
Our finding suggests that when the firm does not have enough information about
consumers’ preferences, product recommendation are not accurate and only some
customers accept the recommended products. We refer to this stage as the “explore” stage
of the recommender system. When the firm gets more information, the recommendations
become more accurate, and thus more customers accept the recommended products and
skip the search for alternative products. In other words, the firm’s effort of improving
accuracy of the recommender system creates an externality that benefits consumers. On
the other hand, when the firm gets enough information about consumers, the
24
recommendation is accurate. This is the “exploit” stage of the recommender system. The
firm can target consumers with more “accurate” product recommendations that have
higher profit margin but are further away from consumers’ references. The firm’s
revenue increases on the expense of consumers—their mismatch cost increases. However,
society as a whole never gets worse off when the recommender system gets more
accurate.
Our results provide several insights about the use of recommender systems in the
context of online retail business. First, an online retailer should adopt a recommender
system and use it to maximize revenue by skewing towards products with higher profit
margin. Second, the recommender system has non-monotonic effects on consumers,
depending on the different stages of the recommender system — understanding of
individual consumers’ preferences by the retailer. Third, it is never worse off from the
societal point of view when the recommender system gets more accurate. This provides
counter-argument to voices of draw-backs and perils of personalized or target marketing
in the electronic marketplace (Zhang 2011).
In this paper, we assume a uniform price across all products, and homogenous
mismatch cost per distance between individual preference and a product among
consumers. In future study, we want to generalize these assumptions to accommodate the
firm’s product pricing decision and consumer heterogeneity in mismatch cost and search
cost.
References
Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender
systems: A survey of the state-of-the-art and possible extensions. Knowledge and Data
Engineering, IEEE Transactions on, 17(6), 734-749.
Ansari, A., Essegaier, S., & Kohli, R. (2000). Internet recommendation systems. Journal
of Marketing research, 37(3), 363-375.
Bakos, J. Y. (1997). Reducing buyer search costs: Implications for electronic
marketplaces. Management science, 43(12), 1676-1692.
Bodapati, A. V. (2008). Recommendation systems with purchase data. Journal of
Marketing Research, 45(1), 77-93.
25
Branco, F., Sun, M., & Villas-Boas, J. M. (2012). Optimal search for product information.
Management Science, 58(11), 2037-2056.
Das, A. S., Datar, M., Garg, A., & Rajaram, S. (2007). Google news personalization:
scalable online collaborative filtering. In Proceedings of the 16th international
conference on World Wide Web (pp. 271-280). ACM.
Fleder, D., & Hosanagar, K. (2009). Blockbuster culture's next rise or fall: The impact of
recommender systems on sales diversity. Management science, 55(5), 697-712.
Fong, N. M. (2012). Targeted Marketing and Customer Search. working paper. Available
at SSRN 2097495.
Hosanagar, K., Fleder, D., Lee, D., & Buja, A. (2013). Will the Global Village Fracture
Into Tribes? Recommender Systems and Their Effects on Consumer Fragmentation.
Management Science, 60(4), 805-823.
Häubl, G., & Trifts, V. (2000). Consumer decision making in online shopping
environments: The effects of interactive decision aids. Marketing science, 19(1), 4-21.
Johar, M., Mookerjee, V., & Sarkar, S. (2014). Selling vs. Profiling: Optimizing the Offer
Set in Web-Based Personalization. Information Systems Research, 25(2), 285-306.
Kim, J. B., Albuquerque, P., & Bronnenberg, B. J. (2010). Online demand under limited
consumer search. Marketing Science, 29(6), 1001-1023.
Lamere, P., & Green, S. (2008). Project aura-recommendation for the rest of us. In
Presentation at Sun JavaOne Conference. Slides last accessed (Vol. 25).
Murthi, B. P. S., & Sarkar, S. (2003). The role of the management sciences in research on
personalization. Management Science, 49(10), 1344-1362.
Oestreicher-Singer, G., & Sundararajan, A. (2012). Recommendation networks and the
long tail of electronic commerce. MIS Quarterly, 36(1), 65-83.
Tam, K. Y., & Ho, S. Y. (2006). Understanding the impact of web personalization on
user information processing and decision outcomes. Mis Quarterly, 865-890.
Thompson, C. (2008). If you liked this, you’re sure to love that. The New York Times, 21.
http://www.nytimes.com/2008/11/23/magazine/23Netflix-t.html?pagewantedDall&
_rD0.
Zhang, J. (2011). The perils of behavior-based personalization. Marketing Science, 30(1),
170-186.
26
Appendix A: Proof
PROPOSITION 1: When search costs increase 0dx
dc
or misfit costs decrease 0
dx
dt
then the expected distance between the consumer’s ideal location and the location of the
purchased product will increase.
Proof of Proposition 1:
Since /x c t , 1
2 ct
xd
dc
, and 1
2 t
xd
dt c
. Since 0c and 0t , 0dx
dc
and 0
dx
dt
.
Thus, we prove Proposition 1. ▪
LEMMA 1: 1) When 0 /r c t , the optimal product recommendation is /r jx a c t
for buckets start from [0,1 / ]ja c t , or 1rx for buckets start from (1 / ,1 ]ja c t r ; 2)
When 1/c t r r , the optimal product recommendation is 1rx x for buckets start from
1[0, ]ja a , or /r jx a c t for buckets start from 1( ,1 ]ja a r ; 3) When 1 1r r , the
optimal product recommendation is 1rx x for all buckets [0,1 ]ja r ; where
1
1/
2 2
ja rc tx
, 1
1 /
2
c tr
and 1
/
1
r ca
t
.
Proof of Lemma 1:
When 0 /r c t , then 1
2 / / ( / )1/
2 2 2
j ja r a c t c tcx
r c tt
is lower than
(2 ) 2 / //
2
j
j
a c t c ta c t
, thus, the optimal product recommendation is the
boundary solution /jrx a c t .
27
When /c t r , then solve the value of 1a where
1 1 1/ ( )a c t x a a , and we obtain
1
/
1
r ca
t
. Since the maximum value of
ja is 1 r , then solving 11 1
/1
1
r c ta r
, we
get 1
1 /
2
c tr
.
This implies that when 1/c t r r , if 1ja a , then 1 1 /x a c t , thus the optimal product
recommendation is 1rx x ; and if 1ja a , then 1 1 /x a c t , thus the optimal product
recommendation is /jrx a c t .
When 1 1r r , 1 1 /x a c t , thus, the optimal product recommendation is 1rx x .
Thus, we prove Lemma 1. ▪
PROPOSITION 2: The firm’s product recommendation rx is independent of profit
margin m .
Proof of Proposition 2:
Given the accuracy of the recommender system, it is obvious to see that the product
recommendation, /r jx a c t or 1
1/
2 2
ja rc tx
, is independent of m .
Thus, we prove Proposition 2. ▪
PROPOSITION 3: The firm’s expected total revenue increases as the range of
consumer buckets decreases (lower r ), [ ( )]
0dE R r
dr .
Proof of Proposition 3:
28
From Lemma 1, we show the firm’s product recommendation strategy depends on the
range of buckets. And there are three regions for the range of buckets: 1) 0 /r c t , 2)
1/c t r r , and 3) 1 1r r .
1) When 0 /r c t , the product recommendation is /r jx a c t for buckets
[0,1 / ]ja c t , or 1rx for buckets (1 / ,1 ]ja c t r . Thus, the optimal expected total
revenue is
1 1
0 1
/
/[ ( )] [ ( )] [ ( 1)]
/
( )( / )
2
j j
c t
jc
r
a r j at
r jE R r E R x da E Ra x dac t
m t cm c t r
t
.
We take the first order derivative of the optimal expected total revenue with respect to the
range of buckets, and we obtain: [ ( )]
0dE R r
drm .
2) When 1/c t r r , the optimal product recommendation is 1rx x for buckets start
from 1[0, ]ja a , or /r jx a c t for buckets start from 1( ,1 ]ja a r . Thus, the optimal
expected total revenue is 1
11
1
0[ ( )] [ ( )] [ ( / )]
j j
a r
a r j a r j ja
E R r E R x da E R x a c t ax d
.
We take the first order derivative of the optimal expected total revenue with respect to the
range of buckets, and we obtain:
3/2 2 2 2 2 3/2
2 3/2
( 3 (5 6 2 ) 2 (9 (1[ ( ) 3 (1 ) 7 6) )
6(2 )(
)]
1 )
m c c r t r r rdE R r
rd
r t
r t
.
We let 3/2 2 2 2 2 3/23 (5 6 2 ) 2 (9 (1 ) 3 (1 ) 7 6)A c c r t r r r r t , so the numerator is
mA . We then take the derivative of A with respect to , and we find that
26(3 2 ) ( (1 ) )r r tA
c td
d , and
2
2
212 ( (1 ) ) 0d
dr c t
Ar t
. This implies that A is at
the maximum when 2 / 3 . We substitute 2 / 3 into A , and we have
3/2 2 2 3/21(3 17 2 (4 7 )3
3( 2 / ) )c cr t r r tA . Since
29
1/ (3 1
/ ( / )4 3
2 / 3) cc t r r t r c t , and 2 / 1 3 /c t c t , we show that
( 2 / 3) 0A . This implies that 0A for all (0,1) .
Since 2 3/26(2 )(
[ ( )
)
]
1
dE R r
d
mA
tr r , and 2 3/26(2 )(1 ) 0r t , we show that
[ ( )]0
dE R r
dr for
all (0,1) .
3) When 1 1r r , the optimal product recommendation is 1rx x for all buckets
[0,1 ]ja r . Thus, the optimal expected total revenue is 1
10
[ ( )] [ ( )]j
r
a r jE R r E R x dax
.
We take the first order derivative of the optimal expected total revenue with respect to the
range of buckets, and we obtain:
2 2 2 2 3
2
(3 3 (1 )(1 ) ((1 ) 3(2 ) 2(1 ) ) )
6( )
[
2
( )] m c c r t r r t
r t
dE R r
dr
.
Since 0 1 , the numerator is positive, but the denominator is negative, and we have
[ ( )]0
dE R r
dr .
Thus, we show that [ ( )]
0dE R r
dr . ▪
PROPOSITION 4: When the range of buckets is relatively small, /r c t , the firm’s
optimal product recommendation is outside the bucket * [ , ]r j jx a a r .
PROPOSITION 5: When the range of buckets is relatively large, * /r c t , (1) the
firm’s optimal product recommendation is inside the bucket * [ , ]r j jx a a r ; and (2) a
consumer is more likely to accept the recommendation if is smaller ( )
0accept rdL x
d .
Proof of Propositions 4 and 5: When the range of buckets is relatively small, /r c t ,
/r j jx a c t a r . When the range of buckets is relatively large, /r c t , 1r jx x a r ,
and the likelihood of a consume may accept the recommendation is:
30
1( / ) / (1 ) / ( )( )
(2 )
j j j
accept r
a r x c t a r c t c t a rL x
r r r
, which decreases in , so
( )0
accept rdL x
d .
Thus, we prove Proposition 4 and 5. ▪
PROPOSITION 6: (1) The aggregate consumer surplus increases as the range of
buckets increases, 0dCS
dr , when the range of bucket is relatively small r c ; (2)
aggregate consumer surplus decreases as the range of buckets decreases, 0dCS
dr , when
the range of bucket is relatively large 1
2r
c
.
Proof of Proposition 6:
When /r c t , the product recommendation is /r jx a c t for buckets [0,1 / ]ja c t ,
or 1rx for buckets (1 / ,1 ]ja c t r . The aggregate consumer surplus from equation (7)
is (1 / )/(1 ) 1
0 (1 / )/(1 )( ( / ) ) ( (1 ) )j i i i
c t r
c t riCS V a c t x t dx V x t dx
. We get
2
2(1 )
c ct tS
r
rC V
. Take the derivative with respect to r , and we obtain
2
2
( )0
2(1 )
c tdCS
dr r
.
When 1/c t r r , the optimal product recommendation is 1rx x for buckets start from
1[0, ]ja a , or /r jx a c t for buckets start from 1( ,1 ]ja a r . We first solve for 1sa that
satisfies 1 1/ (1 ) /s sa r a c t . If 1/4 1/4 2( 4(1 ) )
2
c c c t
tr
, then 1 1sa a . This
implies when 1/4 1/4 2( 4(1 )
/)
2
c c c tc r
tt
, consumers 1 1/ (1 ) / (1 )i sa r x a r ,
will be given the product recommendation that is smaller than the true location
/r j ic xx a t ; and consumers 1 / (1 )i sx a r , will be given the product
31
recommendation larger than the true location /r j ic xx a t . Thus, the aggregate
consumer surplus from equation (8) is:
1 1 1
1 1 1
/ /(1 ) /(1 ) 1
10 / /(1 ) /(1 )
( ) ( ( ) ) ( ( / ) ) ( ( / ) )s
s
x c t a r a r
i i i j i i i j ix c t a r a r
CS V dx V x x t dx V a c t x t dx V x a cct t t dx
If 1/4 1/4 2( 4(1 ) )
2
c c c tr
t
, then 1 1sa a . We first solve for 2sa that satisfies
2 1 2/ (1 ) ( )s j sa r x a a . When 1/4 4 2
1
1/( 4(1 ) )
2
c c c t
tr r
, consumers
2 1/ (1 ) / (1 )s ia r x a r , will be given the product recommendation that is greater than
the true location 1r ix x x ; and consumers 1 / (1 ) ia r x , will be given the product
recommendation that is greater than the true location /r j ic xx a t . Thus, the
aggregate consumer surplus from equation (8) is:
1 2 1
1 2 1
/ /(1 ) /(1 ) 1
1 10 / /(1 ) /(1 )
( ) ( ( ) ) ( ( ) ) ( ( / ) )s
s
x c t a r a r
i i i i i i j ix c t a r a r
CS V dx V x x t dx V x x t dx V x a c t tct dx
When 1 1r r , the optimal product recommendation is 1rx x for all buckets [0,1 ]ja r .
Thus, the aggregate consumer surplus from equation (9) is
1 2
1 2
/ /(1 ) 1
1 10 / /(1 )
( ) ( ( ) ) ( ( ) )s
s
x c t a r
i i i i ix c t a r
CS V dx V x x t dx V x x t dxct
.
Since we assume 1t , we take the derivative with respect to r , and we obtain
2 2 2
2
(5 10 4 ) 2 (1 )(1 (1 ) 2 ) (1 ) (1 2(2 ) )
2(2 )(1 )
c c V V Vd
r r
CS
d
. Also, since
1/ 9 1/ 4c , 0 1 and V c , the numerator of the derivative is negative and the
denominator is positive, thus, 0.dCS
dr
Thus, we prove Proposition 6. ▪