recommendation technology: will it boost e-commerce?

4
May 2006 13 Published by the IEEE Computer Society INDUSTRY TRENDS M ost consumers who have used Amazon.com or other major e-commerce Web sites are accus- tomed to receiving rec- ommendations for a book, music CD, DVD, or article of clothing they might be interested in buying. Initially, recommendation tech- nology was relatively crude. It basi- cally just recommended products like other items the buyer had purchased. However, the technology has become considerably more sophisticated and is now an essential part of many on- line retailers’ economic models. The approach uses complex algo- rithms to analyze large volumes of data and determine what products that potential consumers might want to buy based on their stated prefer- ences, online shopping choices, and the purchases of people with similar tastes or demographics. It is creating new revenue opportunities and in- creasing both customer retention and the number of shoppers who be- come buyers. Major recommendation-technol- ogy vendors include AgentArts, ChoiceStream, ExpertMaker, Google, and Mavice. Big users include Amazon.com, Apple Computer, and the Netflix DVD-rental Web site. The technology is helping drive online sales, particularly in the booming music industry. “With such a huge market, it’s easy to see that personalizing makes sense,” said ChoiceStream CEO Michael Strickman. “One of our clients has seen users download almost four times more songs than before imple- mentation.” While recommendation technol- ogy continues to improve, it has gen- erated concerns. For example, some statistical approaches rely on large volumes of data that aren’t available to many smaller retailers. Also, the technology’s gathering and use of data from online retail ac- tivity and the creation of consumer profiles have sparked privacy- related concerns. WHAT DO YOU RECOMMEND? Recommendation technology be- gan to take shape in the early 1990s. The University of Minnesota’s GroupLens Research Project (www. cs.umn.edu/research/GroupLens/ index.html) was an early pioneer. Some of the first recommendation- technology users included Amazon. com and Net Perceptions, a vendor of Web site personalization software. The driving forces for companies to integrate recommendation en- gines into their e-commerce sites in- clude the desire to get consumers to not only buy more products but also to return to their sites in the future. Customers return to sites because recommendation engines make it faster and easier to find products they want, and they offer personal- ization that yields helpful purchas- ing suggestions, said Brian Stack, Mavice’s chief technology officer and cofounder. Vendors want to present cus- tomers with products they are inter- ested in but didn’t plan to buy. Currently, fewer than 25 percent of online shoppers make unplanned purchases, a far smaller percentage than customers at traditional stores, said analyst Patti Freeman Evans with JupiterResearch, a market research firm. Online vendors also use recom- mendation tools as a way to pro- mote and generate revenue from older or low-demand items, such as niche CDs and DVDs, said Gartner Research Director Mike McGuire. In addition, businesses use the tech- nology to collect customer data to improve marketing and sales, par- ticularly of available stock and pro- motional products. The business models that recom- mendation-technology vendors gen- erally employ involve either hosting the service for a company or licens- ing their engines to e-commerce businesses to run themselves. TYPES OF SYSTEMS Implicit engines provide recom- mendations based on analysis of multiple customers’ activities while browsing a company’s Web site. These engines basically tell users that customers who bought Product A subsequently purchased Product B. Explicit engines make recommen- dations based on customers entering words or phrases to describe the Recommendation Technology: Will It Boost E-Commerce? Neal Leavitt

Upload: n

Post on 23-Sep-2016

212 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Recommendation technology: will it boost e-commerce?

May 2006 13P u b l i s h e d b y t h e I E E E C o m p u t e r S o c i e t y

I N D U S T R Y T R E N D S

M ost consumers who haveused Amazon.com orother major e-commerceWeb sites are accus-tomed to receiving rec-

ommendations for a book, music CD,DVD, or article of clothing they mightbe interested in buying.

Initially, recommendation tech-nology was relatively crude. It basi-cally just recommended products likeother items the buyer had purchased.However, the technology has becomeconsiderably more sophisticated andis now an essential part of many on-line retailers’ economic models.

The approach uses complex algo-rithms to analyze large volumes ofdata and determine what productsthat potential consumers might wantto buy based on their stated prefer-ences, online shopping choices, andthe purchases of people with similartastes or demographics. It is creatingnew revenue opportunities and in-creasing both customer retentionand the number of shoppers who be-come buyers.

Major recommendation-technol-ogy vendors include AgentArts,ChoiceStream, ExpertMaker, Google,and Mavice. Big users includeAmazon.com, Apple Computer, andthe Netflix DVD-rental Web site.

The technology is helping driveonline sales, particularly in thebooming music industry. “With such

a huge market, it’s easy to see thatpersonalizing makes sense,” saidChoiceStream CEO MichaelStrickman. “One of our clients hasseen users download almost fourtimes more songs than before imple-mentation.”

While recommendation technol-ogy continues to improve, it has gen-erated concerns. For example, somestatistical approaches rely on largevolumes of data that aren’t availableto many smaller retailers.

Also, the technology’s gatheringand use of data from online retail ac-tivity and the creation of consumerprofiles have sparked privacy-related concerns.

WHAT DO YOU RECOMMEND?Recommendation technology be-

gan to take shape in the early 1990s.The University of Minnesota’sGroupLens Research Project (www.cs.umn.edu/research/GroupLens/index.html) was an early pioneer.Some of the first recommendation-

technology users included Amazon.com and Net Perceptions, a vendorof Web site personalization software.

The driving forces for companiesto integrate recommendation en-gines into their e-commerce sites in-clude the desire to get consumers tonot only buy more products but alsoto return to their sites in the future.

Customers return to sites becauserecommendation engines make itfaster and easier to find productsthey want, and they offer personal-ization that yields helpful purchas-ing suggestions, said Brian Stack,Mavice’s chief technology officerand cofounder.

Vendors want to present cus-tomers with products they are inter-ested in but didn’t plan to buy.Currently, fewer than 25 percent ofonline shoppers make unplannedpurchases, a far smaller percentagethan customers at traditional stores,said analyst Patti Freeman Evanswith JupiterResearch, a market research firm.

Online vendors also use recom-mendation tools as a way to pro-mote and generate revenue fromolder or low-demand items, such asniche CDs and DVDs, said GartnerResearch Director Mike McGuire.In addition, businesses use the tech-nology to collect customer data toimprove marketing and sales, par-ticularly of available stock and pro-motional products.

The business models that recom-mendation-technology vendors gen-erally employ involve either hostingthe service for a company or licens-ing their engines to e-commercebusinesses to run themselves.

TYPES OF SYSTEMSImplicit engines provide recom-

mendations based on analysis ofmultiple customers’ activities whilebrowsing a company’s Web site.These engines basically tell users thatcustomers who bought Product Asubsequently purchased Product B.

Explicit engines make recommen-dations based on customers enteringwords or phrases to describe the

RecommendationTechnology:Will It Boost E-Commerce?Neal Leavitt

Page 2: Recommendation technology: will it boost e-commerce?

14 Computer

I N D U S T R Y T R E N D S

Collaborative filteringMany recommendation engines,

such as those used by Amazon.comand Netflix, utilize collaborative fil-tering, which computes personalizedrecommendations by comparingmultiple customers’ information—collected via profiles, questionnaires,and purchasing histories—and find-ing those with similar tastes.

Collaborative filtering uses pat-tern-matching techniques to deter-mine correlations between productsthat customers have purchased andpotential future items of interest,and between the buying choices ofsimilar customers.

User-based filtering examines andthen leverages the history, prefer-ences, and similarities among a cur-rent online customer and previousconsumers, Russell explained.However, user-based filtering oftendoesn’t scale well because the analy-sis and comparison processes be-come more complex for Web siteswith many customers and products.

Item-based filtering identifies sim-ilarities among items, rather thanusers, explained Michael Strickman,ChoiceStream’s chief technology of-ficer. In other words, the technique,which companies such as Choice-Stream and Mavice utilize, deter-mines that users who liked Item Amight also like the similar Item B. Atits simplest, this approach could rec-ommend a sports-related product,such as a football book or golf clubs,to someone who purchased a sports-related computer game.

Item-based filtering, which Figure1 shows, is generally more scalablethan the user-based technique be-cause it’s easier to draw correlationsamong a limited number of prod-ucts, which are easy to categorize,than among millions of users, whoseactivities must be examined and an-alyzed, said Strickman.

QUALIFIEDRECOMMENDATION

Companies have several concernsabout recommendation technology,including system scalability. Accord-

type of products they are searchingfor.

Content-based systems recom-mend items similar to ones the cus-tomer has preferred in the past.These systems gather informationabout a customer’s preferences viaquestionnaires or past purchasinghistories stored in databases.

Collaborative-based engines pro-vide recommendations derived fromthe purchasing preferences of cus-tomers with similar interests or de-mographics, based on questionnaireresponses or profiles generated fromconsumers’ online activities.

“Hybrid models can use variousparts of content-based and collabo-rative models,” noted Mavice’sStack.

HOW THEY WORKRecommendation engines come up

with suggestions in different ways,including expert, rules-based ap-proaches, and Bayesian techniques,which make decisions via statisticallybased probability inferences.

For example, said ExpertMakerCEO Lars Hard, “To perform wellwith small amounts of information,some vendors add encoded knowl-edge into their system to reflect theexpertise of a skilled salesperson.”

By knowing the attributes of theproduct, whether it is clothing ormovies, vendors can better analyzeuser activity to provide relevant sug-gestions, added ChoiceStream’sStrickman.

Some recommendation-technol-ogy vendors, such as ExpertMaker,are starting to use one or more typesof artificial intelligence, such asneural networks, to learn to makemore accurate suggestions withgreater consistency.

Most major vendors provide sim-ple APIs so that their engines caneasily integrate with Web sites, in-stant messaging programs, TV-based or wireless applications, andother systems.

A recommendation engine can actas a Web service for one or more on-line stores. In these cases, the engine

and the e-commerce application acttogether over an IP backbone usingXML; the Simple Object AccessProtocol; the Web Services Descrip-tion Language; and universal de-scription, discovery, and integrationregistries.

Vendors such as ChoiceStreamoffer remote management, report-ing, and testing capabilities as partof a packaged recommendation ser-vice for e-commerce vendors.

In recommendation engines, rela-tional databases store user profiles,purchase history, and product cross-linking information. “Thus, a scal-able and robust database schema iscritical to success,” said MaviceCEO Greg Russell.

Data miningRecommendation engines gener-

ally use data mining technology,which looks for hidden patterns andrelationships in large amounts ofdata to predict future behavior.

Data mining algorithms analyzecustomers’ shopping and purchasinghistory, answers to user-preferencequestionnaires and surveys, and pro-files resulting from this information.They translate consumer activityinto product and preference rela-tionships that could indicate whatconsumers might be interested inbuying in the future.

Recommendation engines’ datamining systems typically update con-tent relationships nightly by lookingat changes in user profiles that haveoccurred during the day, notedAgentArts CEO Andrew Coates.

When deciding which products torecommend, some engines give thenewest purchases more considera-tion than older ones to better reflecta user’s current interests.

Recommendation engines are becominga critical part of many

e-commerce sites.

Page 3: Recommendation technology: will it boost e-commerce?

May 2006 15

costs, including those for hardwareand software. Another key reasonwhy the systems can be expensive isuser-interface development, saidCoates. “The way the storefront pre-sents recommendations is crucial ingenerating benefits,” he explained.“The UI layer is the critical differ-ence as to how visible and accessiblerecommendations really are and,therefore, how easy it is for cus-tomers to discover new contentwithout having to do any work.”

The Liveplasma.com beta musicand movie file-sharing site uses anunusual interface for its recommen-dation technology. The site graphi-cally maps users’ potential interests,showing clusters of circles of varioussizes. Each cluster indicates the songsor movies in which customers withsimilar preferences in one area mighthave interest. The bigger circles in-dicate greater potential interest.

Recommendation engines relyheavily on user profiling, which re-

ing to Netflix chief product officerNeil Hunt, his company’s customerbase grew rapidly from 2.6 millionin 2004 to 4.2 million in 2005 and isexpected to increase to an estimated5.9 million this year and 20 millionby 2012. “When you look at thatgrowth track, everything has to bescalable,” he said.

Many users are also concernedabout privacy and don’t like e-com-merce sites collecting personal in-formation based on their purchases.

AccuracyA key issue for recommendation

engines is making suggestions thataccurately reflect a buyer’s interests.According to AgentArts’ Coates,this could be a particular problemfor low-traffic e-commerce sites,which have relatively little customerdata from which to derive usefulcontent relationships. This couldalso be trouble for sites that offermany content items, which makesselecting recommendations morecomplex.

However, the problem can also af-flict large e-commerce operations.WalMart recently took down itsWeb site’s recommendation systemwhen customers who looked at aboxed set of DVDs that includedMartin Luther King: I Have aDream and Unforgivable Blackness:The Rise and Fall of Jack Johnsonwere told they might also enjoy anumber of other titles, includingsome that were potentially insulting,particularly to African-Americans.

The company has apologized forthe incident, saying it occurred whenits e-commerce site offered simple,direct links—rather than those de-termined by customer or productanalysis—among a large number ofunrelated DVDs. WalMart said thiswas done to promote video sales,without checking to determinewhether the links between somemovie pairs could be offensive.

CostMost recommendation systems

entail significant vendor start-up

quires storing, backing up, and en-crypting enormous amounts of data,which can increase operating costs.In addition, many of the techniquescan require considerable processingpower.

Meanwhile, return on investmentcan be difficult to measure. Thus,small and medium-size companiesmight not feel they can justify im-plementing recommendation tech-nology.

However, Coates added, large e-commerce providers might wantto do so because even a small in-crease in their sales and customer-retention rates could mean millionsof dollars in additional revenue.

R ecommendation technologycould take several new direc-tions in the near future. For ex-

ample, noted Gartner’s McGuire, thetechnology could add a location-based component. “I might have a

Item D

A B CD E F G

H I J

A. Choice setLimited to items selected,

rated, or purchasedby users. Includes only

Items B, C, and D.

B. Preference captureUser rates Item A.

C. Preference profilefor target user

• Item A

User ratings for Items A, B, C, and D

Item A Item B Item C Item D

Target user

User 1

User 2

User 3

Item D is most similar toItem A in terms of its ratings.

D. RecommenderFinds items rated similarlyto highly rated items in the

target user’s profile.

Recommended to target user.Source: ChoiceStream

Figure 1. ChoiceStream’s recommendation engine makes its suggestions to customers

based on multiple users’ product ratings. In this simplified example, a target user

gives Item A a good rating.Three other users had previously rated Items A through D.

Item D’s ratings were like Item A’s.Thus, the engine recommends Item D to the target

user, based on his earlier rating of Item A.

Page 4: Recommendation technology: will it boost e-commerce?

16 Computer

I N D U S T R Y T R E N D S

wireless communications device withGPS capability,” he said, “and as Iget near a store, I could get pingedwith a message saying there’s a salethere on a CD I might like.”

One burgeoning development ismatching consumer tastes acrossWeb businesses, such as usingknowledge of customers’ tastes inone area, like music, to sell themproducts in another area, like books.This will depend on future businessalliances and partnerships, ratherthan advances in the technology,said Mavice’s Russell.

Marketplace adoption of recom-mendation technology is still in itsearly stages. Recommendation sys-tems will have to increase personal-ization by collecting and processingadditional data without being intru-sive and time-consuming. In the fu-ture, consumers will not want toutilize the technology unless it ishighly functional and easy to use,said Rob Enderle, principal analystof the Enderle Group, a market re-search firm.

“Recommendation technologymust also be able to reach out tosmall and medium-size businessesand be robust and cost-effective,”said Mavice’s Stack. “Once this occurs, we’ll see recommendationtechnology truly become a ubiqui-tous piece of an online consumer’sdaily Internet experience.” ■

Neal Leavitt is president of LeavittCommunications (www.leavcom.com),a Fallbrook, California-based interna-tional marketing communications com-pany with affiliate offices in Brazil,France, Germany, Hong Kong, India,and the UK. He writes frequently ontechnology topics and can be reachedat [email protected].

Editor: Lee Garber, Computer,[email protected]

ISI Web of KnowledgeSM

Where ideas turn into knowledge.

The search for new knowledge is an

ongoing challenge … but the search for

the best research resources need not be.

ISI Web of Knowledge assures your

access to the critical information you

require. Whether the ideas you seek are

across the globe, have their roots in the

past, or can be found in today’s newest

discoveries, ISI Web of Knowledge

delivers all the research data and

tools you need to reach your goals.

Navigate freely.

Explore confidently.

Search successfully.

with ISI Web of Knowledge