Recommender Systems

Cristina AlaimoC.Alaimo@surrey.ac.uk

Department of Digital EconomySurrey Business School, UK

Jannis KallinikosJ.Kallinikos@lse.ac.uk

Department of ManagementLondon School of Economics, UK

Personalization increasingly mediates the experience of users on the Web. Online platforms

and organizations use personalization services to retain users, achieve longer user or

customer engagement and, ultimately, higher profits. Cast in this light, personalization is a

ubiquitous modality by means of which organizations seek to structure interaction with their

users. Amazon, for instance, mediates the buying experience of its customers through

computational systems that advance recommendations concerning relevant products to buy

upon nearly every transaction. Similarly, Spotify uses the listening habits of its users to

recommend tunes which they may find relevant to listen. In a rather different context,

Facebook modulates its news feed to the interests of individual users by mapping each user’s

ongoing interaction with his/her network of other users, and Google famously personalizes its

search engine results, gathering, in turn, relevant information on the search habits of users.

Collaborative filtering recommender systems are one amongst a complex and differentiated

landscape of technologies of personalization. Such systems are used alone or, increasingly, in

combination with other systems to develop and implement so-called hybrid recommenders

(for a useful taxonomy see Burke and Ramezani 2011). The form of user mediation the

various recommender systems offer may slightly vary in procedures and types of data

required but their logic and operations remain largely similar (Adomavicious and Tuzhilin

2005). All these systems work with a steady collection of carefully structured data produced

by myriads of individual transactions or interactions and the continuous feedback from users.

This engineering of experience through which individual actions are first produced, then

tracked, inferred, and transferred over to users presupposes a set of standardized and

automated procedures which —paradoxical as it may sound— become the backbone of

personalization. User interaction has to be shaped along actions standardized enough to leave

a computable data footprint underlain by carefully crafted user models that make individual

users comparable or better commensurable with one another.

Famously conveyed by Amazon’s: “Customers who bought this item also bought (…)”

collaborative filtering was first implemented in 1993 as a Usenet news recommender system

called GroupLens. The system recorded user ratings of news articles, stored them into a user

profile database and once it had enough data was able to relate ratings to recommend other

articles that users might find relevant (Konstan et al. 1997). A second experiment was done

with the site MovieLens. There, users were asked to rate movies on a scale of 5-stars and

paired into groups of similar users, that is, users who had rated movies similarly. Soon after

that, MIT built Ringo, a music recommender system (Shardanand and Maes 1995).

Riedl and Konstan are credited to have built one of the first collaborative filtering

recommender systems. Collaborative filtering, as the name suggests, uses information from

groups (hence, the collaborative dimension) to filter relevant items to individuals. Quite aptly

described as “any mechanism whereby members of a community collaborate to identify what

is good and what is bad” (Riedl and Konstan 1999: 330-331), collaborative filtering embeds

also elements of information retrieval and filtering, and a large list of automated and often

unsupervised computational operations. The “collaborative” element resonates with the so

called “wisdom of the crowd”, the expectation that information produced by masses of people

is somehow more precise or relevant than information produced by experts (Surowiecki

2005). Recommender systems take these insights along a distinctive route that reflects partly

the quest for personalization and partly the technological rendition of online experience we

outlined above.

Recommender systems are largely automated systems that derive from the application of AI

methods to information filtering and techniques of data representation and inference that have

their roots in the expert systems of the 1980s (Jannach 2010). Collaborative filtering gathers

information on user interactions, although users seldom interact with each other in these

settings, they only interact with the system to provide relevant information in the form of

buying transactions and ratings. Users, therefore, do not collaborate as the name collaborative

filtering might suggest. Nowhere is a community or group to be found. Those systems

compute affinities between users by gathering the independent preferences of atomized

individuals. Collaboration is euphemistically deployed to refer to these statistically-mediated

comparisons of user ratings. In fact, users are most of the times unaware of contributing to

the development and working of recommender systems and, when they happen to be, they are

seldom fully aware of the ways in which their contribution occurs. Advances in AI and

machine learning amplify such trends. As these systems grow in complexity and automation

the risk of overemphasizing implicit assumptions on the basis of which the system operates

also increases. Take the two-stage approach to personalization of the deep neural network

system for YouTube recommendation illustrated by Covington, Adams and Sargin (2016).

Here the system does not need to rely on user generated ranking anymore but has itself

learned how to rank. The system is made by two sub-systems: one for the generation of

videos to be recommended and another one for ranking those videos. The first network uses

collaborative filtering as it takes a user’s YouTube activity history and filters a smaller

sample of items to be recommended. The second system, called the ranking network,

automatically assigns a score to each video by gathering large volumes of data both on videos

and users (see fig 1). It is not by accident that YouTube has been at the centre of media

scrutiny recently as its modus operandi seeking prolonged user engagement led the system to

recommend mostly crude or offensive content1.

Figure 1: The deep neural network recommender system for YouTube. The figure illustrates the system architecture with its recommendation flow and relative data sources. From left to right: a. the video database b. the first sub-system of collaborative filtering c. the second sub-system of ranking and d. the recommendations to users. (Adapted from Covington, Adams and Sargin 2016).

The basic assumptions of collaborative filtering are that (i) the history of individual

preferences together with (ii) the history of the preferences of similar individuals are better

predictors than experts or the derivation of user preferences from market segments to which

1 See for instance: https://www.theguardian.com/technology/2018/feb/02/how-youtubes-algorithm-distorts-truth https://www.theguardian.com/technology/2018/feb/02/youtube-algorithm-election-clinton-trump-guillaume-chaslot

users are assumed to belong in standard marketing practices. In other words, the system

assumes that if users shared similar preferences in the past they will also share similar

preferences in the future. This broadly means that suggestions are tightly coupled with past

preferences and the preferences of similar others as mapped and recorded by the system.

As a general rule, in all recommender systems the kind of information selected and the ways

it is structured into databases and user profiles are directly connected with the core

technology of the system. In collaborative filtering, user profiles are modelled as a list

containing the history and quality of their ratings (Ricci et al. 2011); other systems may have

different user models and thus require different kind of information (e.g. data on items or a

mix between data on items and user behaviour). Information can be gathered by explicit

actions such as buying, rating, liking, watching, listening, etc. but they can also be collected

by using implicit behaviour such as browsing, searching, or time spent on webpages, etc. The

denominations of explicit and implicit refer to the indicator of consumer preferences which

are explicit when the indicators stem from actions that can be straightforwardly linked to

preferences; when the system does not have indicators of preferences, or data are sparse, it

infers them by interpreting as preferences any implicit actions. Other systems may use data

derived from items or from items and users and gather additional data from third-party

organizations. It is relevant to note that even when the system uses item data, it does not

simply record information but rather carefully designs specific data formats so as to fit the

core technology of the recommender system. A good example of the increased sophistication

in information design is given by the different ways music personalization techniques work2.

In collaborative filtering, once the system is up and running with a rating database and active

user profiles, it needs to cluster users or items into groups of similar users or items. To do so,

collaborative filtering recommender systems use two different approaches: user-based or

item-based algorithms. The first is the simplest and relies on the idea that given a rating

database the distance between users is determined by each individual user’s ratings of the

same item. The second instead computes the distance between items on the basis of how

closely users who have rated these items agree.

2 Pandora’s music genome project and The Echo Nest, the music intelligence platform empowering Spotify recommender engine, are two hybrid recommenders that approach the same problem (music personalization) in an entirely different way. See https://www.pandora.com/about/mgp and http://the.echonest.com/

Figure 2: Illustrative example of item-item personalization algorithm.

With a user-based algorithm, users who rate items similarly form neighbourhood of users.

With an item-based algorithm, items that are rated similarly form neighbourhood of items.

The grouping of users or items (called sometimes nearest neighbours or peer users) is how

the system establishes segments, that is, groups of similar users. Differently from traditional

marketing, in recommender systems this happens automatically and under computational

rules. There are no real similarities between user-user or item-item; users or items are

grouped together and deemed similar on the basis of the affinities emerging from rating

patterns provided by large amounts of data. The “segments” so created by the automated

recommender system become groups of predictors: for every item y that user x has not seen,

a prediction is computed on the basis of the rating of the item y made by nearest neighbours.

The majority of recommender systems use variants of a weighted, k-nearest-neighbour

prediction algorithm (roughly synthesized in “how much a target user u will like a target item

i by first selecting a neighbourhood of other users with tastes most similar to that of u”, (see

Konstan and Riedl 2012a; 2012b).

Figure 3: Illustrative example of the prediction based on nearest neighbours: “for every item y that user x has not seen, a prediction is computed on the basis of the rating of the item y made by nearest neighbours”.

To automatically select neighbours, however, the system needs first to compute a measure of

similarity between users or items. The computation of similarity is a fundamental problem for

the success of the prediction. That is, the measure by which two or more users or items are

deemed similar by the system and grouped together needs to be as accurate as possible. There

is no consensus yet to which of the different measures applied to the construction of the

similarity function works best (Alaimo 2013, 2014). A common approach remains the

Pearson’s correlation coefficient (another is the vector cosine similarity measure). “It

computes the missing rating of user u according to the average value of ratings made by its

neighbours weighted by each of their degree of similarity with the user u” (Ricci et al. 2011).

Figure 4: Pearson’s correlation coefficient. It is used to compute “the missing rating of user u according to the average value of ratings made by its neighbours weighted by each of their degree of similarity with the user u”.

The coefficient is successful in factoring ratings so as to make users commensurable. If a user

rates movies only with 4 and 5 stars and another user instead deploys the whole scale from 1

to 5, the ratings of the two users need to be adjusted in order to allow comparison. However,

the coefficient cannot solve sparsity problems, it cannot factor out the difference between the

ratings of two users if the former has rated 500 movies and latter only 5. Also, the coefficient

cannot make any difference between ratings that concern popular movies (most people like

Star Wars anyway) and ratings that concern less popular movies (which are arguably more

indicative of similarity of taste). Two other problems concern size and time. Size has always

been a problem for the successful computation of predictions. If the group of similar users or

items is too small, it becomes impossible to compute good predictions but if the size of the

group is too large this may be because the threshold of similarity is too low and predictions

will be inaccurate. Time instead refers to the complexity of re-computing the whole model

once a user adds a new rating. For this last reason, item-based collaborative filtering approach

is considered a better one. In this approach, the item-item algorithm calculates the distance

between items according to how much two users agree. It is important to bear in mind that

here similarity is computed between ratings, it is the pattern of user rating behaviour overtime

that is analyzed and computed to make two or more items nearest neighbours. Therefore, in

this case the distance between items is pre-computable as it is dependent on thousands of

available ratings thus remaining relatively stable overtime.

Customization and Personalization

The ideas presented above indicate that personalization and recommender systems are closely

associated with the online, data-intensive environments in which most organizations currently

operate. However, personalization is more than a technical response to the ubiquity of data

that characterize our age. It is above all an organizational practice that seeks to modulate a

space of interaction between organizations and users or customers in an economic, cultural

and social context that is increasingly marked by the fragmentation of consumer needs and

the individualization of consumption. Placing personalization within the larger historical

context of customization gives a better appreciation of the origin of this organizational

practice and the ways it has evolved with the use of digital technologies.

Customization has historically emerged as the organizational antidote to the typified

consumer experience characteristic of mass production and the long-driven standardization of

products or services which meticulous specialization and far-driven economies of scale have

brought about (Chandler 1977; Lampel and Mintzberg 1996). Despite significant variation

which the model of mass production has been subject to over the last few decades (Pine

1993), a great deal of products and services are still produced under conditions that recount

the exigencies of low unit cost, achieved through specialization, economies of scale and

standardization. Seen in this light, customization has been a response to this one-size-fits-all

consumer experience associated with standardized products and services. It is an

organizational practice that seeks to alleviate some of the negative implications of a long-

driven standardization and expand the possibilities of consumer choice.

In reality, customization operates by designing a space of interaction with consumers

whereby the latter are claimed to have ample freedom for exercising their choice. For

standard market segmentation techniques, individuals are just singular instances of wider

market segments associated with class, demographic, educational or income attributes. Under

these conditions an individual cannot but be one among a large group of similar others with

whom s/he shares a predictable set of needs. Customization assumes that such a rather

wholesale method for dissecting taste distribution among large populations is not any longer

well attuned to the faster production cycles of modern organizations nor to the context of

free-will individuals that seem to characterize hypermodern societies. As a response to these

changes, marketing segmentation techniques have been redefined to conceive individuals as

active consumers driven by desires mostly associated to lifestyle and other expressive

attributes of contemporary ways of living.

It is true that such a shift in marketing practices seldom moves far beyond the market

segmentation techniques with which standardized products and services have always been

closely associated. In fact, such consumer space of choice is more fictional than real. In a

great deal of cases, the exercise of personal choice assumed by customization is linked to

products and services that are mass-produced and then mass-customized. The adaptation to

individual customers is still mediated by segmentation techniques, certainly more finely

differentiated, that place individuals into smaller target groups (Zwick and Denegri Knott

2009; Zuboff and Maxmin 2003). Yet, targeting individual consumer desires instead of

product needs signalled a shift of capital importance for the development of the modern

individual consumer which we find closely relates to personalization. In this regard,

customization and personalization can be linked to a broader culture of individualism and

commodious consumerism characteristic of post- or late-modern societies which they

variously reinforce (Bauman 2000; Lipovetsky 2005). The current fragmentation of consumer

tastes is thus a broader societal phenomenon (Anderson 2006; Beck 1992) which has been

certainly escalated by the attempt of organizations to grapple with social changes (Zwick and

Cayla 2011). By redefining the space of consumption as a fictional space of limitless

consumer choice, organizations have reinforced the increasing fragmentation of taste

promoting a standardized notion of hyper-individuality.

From individuals to data

Personalization as an organizational practice and recommender systems as a primary

technology of personalization have transformed the ways data and data-based systems

mediate human experience of consumption and the space of consumer choice (Kallinikos

1992, 2007; Manovich 2001). This was noted as far back as 1999 when, in The New Yorker,

Malcom Gladwell (1999) wrote:

The really transformative potential of collaborative filtering, however, has to do with

the way taste products—books, plays, movies, and the rest—can be marketed.

Marketers now play an elaborate game of stereotyping. They create fixed sets of

groups—middle-class-suburban, young-urban-professional, inner-city-working-class,

rural-religious, and so on—and then find out enough about us to fit us into one of

those groups. The collaborative-filtering process, on the other hand, starts with who

we are, then derives our cultural ‘neighborhood’ from those facts. And these groups

aren’t permanent. They change as we change … (Italics added)

If we follow this account, personalization seems to go much further than customization.

Against the “lazy, prejudice philosophy” of demographic profiling, as Riedl and Konstan

(1999: 113) call traditional marketing segmentation, recommender systems claim to offer an

unbiased account of individual user needs and desires. Recommended systems aspire to

champion the resurgence of “who we are” out of the coarse taxonomies of traditional

marketing but also out of the late techniques of target groups and life styles characteristic of

customized marketing. Once again, personalization operates by re-engineering the space of

consumption. The space of limitless consumer choice is given an interesting tweak by being

transformed into a data field that is assigned the status of facts. Yet as we have seen,

recommender systems engineer the space of user interaction to fine-tune it to practices of data

gathering, user profiling and computation. Data are fashioned as facts by black-boxing the

numerous technological operations sustaining them which are far removed from user

interface and thus from user awareness. As distinct from past practices of customization,

interaction is now staged between an individual user and a digital system (not an organization

or other users) and shaped by iterative feedback loops between (i) programmed behaviour

and an initial data set, (ii) suggestions and (iii) user reactions and feedback by which the

system learns and readjusts its outputs.

Cast in the digital medium, the space of consumer choice articulated by the organizational

practice of personalization transforms traditional assumptions concerning individuals. On

most counts, personalization is the process of inference of what individuals and small taste

groups are or would like to be based on the clustering of data that are supposed to stand for

their actual preferences. There are few other cultural, social or personal references to which

the identity of individuals and groups is related. In the context of personalized

recommendations there are often no reference to data on actual buying behaviour but only

ratings and other clicking-related behaviours (Alaimo and Kallinikos 2017).

A critical look at these practices suggests that the practice of personalization is reductive in

ways that undermine any genuine concern for persons as unique cultural individuals.

Contrary to what Malcom Gladwell seems to suggest in the quote above, personalization -

mediated in recommender systems - is not concerned with persons but data. It compiles

profiles of individuals or taste groups out of digital marks (data) and what these last are

engineered to represent. Disturbing as it may seem, users online are seldom individuals in the

literal sense of the world. They are not real-world persons who exercise choice as the

outcome of their unique make-up of life experiences. They are rather data aggregates which

are put together on the basis of specific computational models (or profiles) of users imposed

by the core technology of the system (i.e. users are lists of such engineered operations such as

rating, listening, liking, etc.). This, in turn, conditions what kind of behaviour the system

needs to design so as to have the data required to perform adequately. As distinct from the

fictional lifestyles of late-consumerism and the rigid stereotypes of customization, individuals

online are just another digital item, constantly re-modulated, re-positioned and updated every

time a new click, like or rating is produced. Personalization systems help online consumers in

solving a problem that technology itself has caused. Overabundance of choices linked to

endless data-fields is a daunting scenario for users who therefore need to be aided by constant

personalized suggestions. In this respect personalization has many positive effects for

organizations as it is effectively correlated with higher user engagement and higher margins.

Personalization may also have some positive effects for users, as far as it helps users discover

new items, learn something new about themselves and overcome the anxiety of making

decisions (Anderson 2006). Yet, there is high price to pay for these functional gains. As

shown in this entry, whatever the implications of personalization, these need to be

appreciated within a larger time purview and the ways recommender systems encode and

engineer user experience.

Concluding remarks

In this entry we have reviewed the organizational practice of personalization by

deconstructing the ways collaborative filtering recommender systems work. Although

different recommender systems may be based on varying computational paradigms

(Adomavicius and Tuzhilin 2005; Burke and Ramezani 2011; Jannach, 2010; Ricci et al.,

2011) and may rely on several different data principles and algorithms, all of them work by

following a similar logic: (i) they need data on user and user behaviour (some gather data on

products as well); (ii) they need to construct and update a user model or profile whereby they

gather user preferences and past behaviour; (iii) they offer automated personalized

suggestions; (iv) they require continuous feedbacks from users to adapt and learn.

We have placed the emergence of personalization within the broader historical process of

customization and the quest of producing goods and services that are supposed to address the

distinctive needs of individuals and small groups. We have drawn attention to the mediating

properties of personalization systems and the current practice of data clustering and data-

based techniques that deeply impregnate personalization processes. Although the accuracy of

data-based techniques makes personalized services look as an empirically-grounded

mediation through which users can discover their own, allegedly true, needs and

predispositions, it is important to realize that there are no genuine individuals in these

systems, at least not in the sense we understand the term in real-life contexts (Alaimo and

Kallinikos 2016; 2017; Elmer 2004; Hildebrandt and Rouvroy 2011).

The datification of user experience which underlies personalization has a number of

consequences. Online involvement is heavily shaped by first translating individuals into user

profiles or computable models that render them steadily knowable entities. Any recommender

system actively fashions what an individual user or consumer is by creating models of users

which are highly dependent on the core technology-in-use. Such models constitute the

backbone for designing a set of clearly defined online interactions that enable the

computability of user preferences. In this respect, the individuality that personalization

constructs is no more than a changeable data profile. User models or profiles are assembled

out of strategies of attributing preferences to individuals through a complex journey of

technologizing experience whereby standardized expressions of individual behaviour —

clicking, liking or rating— are interpreted as expressions of taste and assessed by comparison

to a network of standardized behavioural expressions of others. Online individual consumers

are automatically fashioned by computational models as digital objects that are always

updatable and constantly in the making.

References

Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: A

survey of the state-of-the-art and possible extensions. IEEE transactions on knowledge and

data engineering, 17(6), 734-749.

Alaimo, C. (2013). Technology of consumption on social shopping platforms: deconstructing

Similarity. EGOS: Montreal, Canada.

Alaimo, C. (2014). Computational consumption: Social media and the construction of digital

consumers. Doctoral dissertation, London School of Economics, London, UK.

http://etheses.lse.ac.uk/975/

Alaimo, C., & Kallinikos, J. (2017). Computing the everyday: Social media as data platforms. The

Information Society, 33(4), 175-191.

Alaimo, C., & Kallinikos, J. (2016). Encoding the everyday: The infrastructural apparatus of social

data. In Big data is not a monolith: Policies, practices, and problems, ed. C. Sugimoto, H.

Ekbia, and M. Mattioli, 77–90. Cambridge MA: MIT Press.

Anderson, C. (2006). The long tail: Why the future of business is selling less of more. Hachette Books.

Beck, U. (1992). Risk society: Towards a new modernity. London: Sage

Bauman, Z. (2000). Liquid modernity. New York: Wiley.

Burke, R. (2002). Hybrid recommender systems: Survey and experiments. User Modeling and User-

Adapted Interaction, 12(4), 331-370.

Burke, R., & Ramezani, M. (2011). Matching recommendation technologies and domains.

In Recommender Systems Handbook (pp. 367-386). Springer US.

Chandler Jr, A. D. (1993). The visible hand: The managerial revolution in American business.

Cambridge, MA: Harvard University Press.

Covington, P., Adams, J., & Sargin, E. (2016). Deep neural networks for YouTube recommendations.

In Proceedings of the 10th ACM Conference on Recommender Systems (pp. 191-198). ACM.

Elmer, G. (2004). Profiling machines. Mapping the personal information economy. Cambridge, MA:

The MIT Press

Gladwell, M. (1999) The science of the sleeper, The New Yorker

https://www.newyorker.com/magazine/1999/10/04/the-science-of-the-sleeper

Hildebrandt, M., & Rouvroy, A. (Eds.). (2011). The philosophy of law meets the philosophy of

technology. London: Routledge.

Jannach, D. (2010). Recommender systems: An introduction. New York: Cambridge University Press

Kallinikos, J. (1992). The significations of machines, Scandinavian Journal of Management, 8/2: 113-

132.

Kallinikos, J. (2007). The consequences of information: Institutional implications of technological

change. Edward Elgar Publishing.

Konstan, J. A., Miller, B. N., Maltz, D., Herlocker, J. L., Gordon, L. R., & Riedl, J. (1997).

GroupLens: applying collaborative filtering to Usenet news. Communications of the

ACM, 40(3), 77-87.

Konstan, J. A., & Riedl, J. (2012a). Recommender systems: From algorithms to user experience. User

Modeling and User-Adapted Interaction, 22(1-2), 101-123

Konstan, J. A., & Riedl, J. (2012b). Recommended for you. IEEE Spectrum, 1-8

Lampel, J., & Mintzberg, H. (1996). Customizing customization. Sloan Management Review, 38(1),

21.

Lipovetsky, G. (2005). Hypermodern times. Cambridge: Polity.

Manovich, L. (2001). The language of new media. Cambridge, MA: The MIT Press

Mintzberg, H. (1979). The structuring of organizations. Englewood Cliffs: Prentice-Hall.

Pine, B. J. (1993). Mass customization: the new frontier in business competition. Cambridge, MA:

Harvard Business Press.

Ricci, F., Rokach, L., & Shapira, B. (2011). Introduction to recommender systems handbook.

In Recommender Systems Handbook (pp. 1-35). Springer US.

Riedl, J., & Konstan, J. (1999). Word of mouse: the marketing power of collaborative filtering.

Hachette UK.

Riedl, J., & Smyth, B. (2011). Introduction to special issue on recommender systems. ACM

Transactions on the Web, 5(1), 1-2

Shardanand, U., & Maes, P. (1995). Social information filtering: algorithms for automating “word of

mouth”. In Proceedings of the SIGCHI conference on Human factors in computing

systems (pp. 210-217). ACM Press/Addison-Wesley Publishing Co.

Surowiecki, J. (2005). The wisdom of crowds. New York: Anchor Books.

Wang, H., Wang, N., & Yeung, D. Y. (2015). Collaborative deep learning for recommender systems.

In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery

and Data Mining (pp. 1235-1244). ACM.

Zwick, D., & Cayla, J. (Eds.). (2011). Inside marketing: Practices, ideologies, devices. Oxford:

Oxford University Press.

Zwick, D., & Denegri Knott, J. (2009). Manufacturing customers: The database as new means of

production. Journal of Consumer Culture, 9(2), 221-247.

Zuboff, S., & Maxmin, J. (2003). The Support Economy: How corporations fail individuals and the

next episode of capitalism, London: Allen Lane.