Cricking: Customer-productInteraction in Retail usingPervasive Technologies

Rafael PousUbiCA Lab, DTICUniversitat Pompeu FabraTanger 122-14008018, Barcelona - Spainrafael.pous@upf.edu

Marc Morenza-CinosUbiCA Lab, DTICUniversitat Pompeu FabraTanger 122-14008018, Barcelona - Spainmarc.morenza@upf.edu

Joan Melia-SeguıUbiCA Lab, DTICUniversitat Pompeu FabraTanger 122-14008018, Barcelona - Spainjoan.melia@upf.edu

Zulqarnain RashidUbiCA Lab, DTICUniversitat Pompeu FabraTanger 122-14008018, Barcelona - Spainzulqarnain@upf.edu

Anna CarrerasUbiCA Lab, DTICUniversitat Pompeu FabraTanger 122-14008018, Barcelona - Spainanna.carrerasc@upf.edu

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bearthis notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee. Requestpermissions from permissions@acm.org.UbiComp’13 Adjunct, September 8–12, 2013, Zurich, Switzerland.Copyright c© 2013 ACM 978-1-4503-2215-7/13/09...$15.00.

http://dx.doi.org/10.1145/2494091.2496015

AbstractThe popularization of eCommerce has led to effectivecustomer shopping experiences. Pervasive computingcould bring the benefits of eCommerce to brick andmortar stores, merging both online and physical worldsinto a unique system. We define crick as the extension ofthe (c)lick and b(rick) concept, by means of pervasivetechnologies. In this paper, we summarize ourwork-in-progress research on using pervasive RadioFrequency Identification (RFID) to sense human-productinteraction. These cricks can be performed throughdiverse interfaces in the retail domain, and automaticallyreceive feedback in different manners. We believe thatintegrating RFID and other pervasive technologies in retailstores is the next step to obtain comprehensive customer’suser models and preferences. Retail managementimprovement, or personal and collaborativerecommendations, are envisioned to be successfulapplications of cricking.

Author KeywordsRFID, Human-product Interaction, Cricking, UserModeling, Location

ACM Classification KeywordsH.5.m [Information interfaces and presentation (e.g.,HCI)]: Miscellaneous.

Session: PeTRE: Workshop on Pervasive Technologies in Retail Environment

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

1023

IntroductionThe popularization of eCommerce has revolutionized thehow customers interact with retail stores. In online retailstores, customers access products with a simple click on awebsite, or through a smartphone application. A clickaction produces two outputs: on one hand, the customerreceives information about that product, and on the otherhand, the retailer knows that a click has been performedon that specific product.

The benefits of eCommerce are not limited to receivinginstant information about a product from the retailer.The customer can search online information about aspecific product from different sources, including searchengines, competitors, recommendation websites,specialized forums, etc. This gives the customer thepower to compare prices, brands, sellers or other productcharacteristics, and choose the best option. In this sense,eCommerce improves the customer shopping experience.

Figure 1: Customer interactingwith products in a store.

However, the eCommerce customer is not only benefitedby the instant access to product information. Theclick-stream of customers in a eCommerce website orsmartphone application has been successfully used tounderstand and model their interests and preferences.This is the case of Amazon [1] and Google [3] amongstothers. In these examples, eCommerces can effectivelypropose and recommend products tailored to the customerpreferences. Hence, eCommerces improved selling rates.

Up to present, brick and mortar retailers could not adoptneither benefit from the eCommerce improvements in itsrelation with customers [8]. In the remainder of this paperwe describe our ongoing research on the utilization ofpervasive technologies to bring the benefits of eCommerceto traditional brick and mortar retail, merging both theonline and the physical worlds.

Cricking to merge eCommerce with brick andmortar storesCurrent strategies from traditional brick and mortarretailers to adopt eCommerce benefits are known as bricksand clicks [9]. These strategies are far from actuallymerging both scenarios. Instead, it refers to thecoexistence of both models within a same retail agent.For instance, collecting a product at a store after havingpurchased it online. However, pervasive technologies havethe potential to move a step forward in this scenario,allowing a real integration of eCommerce into brick andmortar stores.

Let us imagine a scenario where a customer is shopping ata store, and is interested in a garment. At this point, thecustomer’s actual choices to obtain information from thatproduct are limited to reading the product’s label, askinga shopping assistant, or searching online through asmartphone. None of these options are a comparableexperience to online shopping with respect to obtainingautomatic feedback (i.e., information, recommendations,or alternatives), or comparison options.

Conversely, let us imagine a scenario (c.f. Figure 1) wherethe same customer takes the same garment, and receivesautomatic information on a screen about related itemsbased on the customers’ history of purchases and thatseason’s fashion. Moreover, the smartphone automaticallysuggests alternative stores in the vicinity with better offersand reviews [12] for the same garment. Finally, thecustomer’s smartphone easily shares the shopping activitywith her closest friends in an online social network, whichcan support the shopper telling that she looks great withthat garment.

Session: PeTRE: Workshop on Pervasive Technologies in Retail Environment

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

1024

Cricking is an extension of the (c)lick and b(rick) conceptby means of pervasive technologies. We can define cricksas human-product interactions at a brick and mortarstore, receiving automatic feedback based on theinteracted product, and thus, leading to a full eCommerceexperience. Cricking merges both the online and thephysical worlds, bringing the benefits of eCommerce tobrick and mortar stores.

Taxonomy of pervasive technologies to en-able cricks in the retail domainPervasive technologies play a basic role to make crickspossible. In this subsection we present a taxonomy withdifferent technologies we have surveyed to make crickingpossible in the retail domain.

Figure 2: RFID-enabled smartshelf front and back views.

When clicking on a retailer’s website, we interact withspecific products. Cricks on brick and mortar storesshould be equally specific. Hence, the system must beable to uniquely identify each product. Radio FrequencyIdentification (RFID) is a wireless technology whose maingoal is identify objects. Specifically, Electronic ProductCode Generation 2 (EPC Gen2) [11] is an RFIDtechnology intended for retail, consisting on low-costpassive tags (or electronic labels), powered by an RFIDreader plus one or more antennas. A tag should beattached to each product in a store, allowing a uniqueitem identification.

Besides unique identification, the position or location ofthe products must be known to let the system provideappropriate feedback to cricks. We define as smart shelf atraditional retail shelf enabled with RFID antennas (c.f.Figure 2). Hence, knowing the position of the antennasan accurate location of RFID-tagged products can beobtained.

Summing up, filling the smart shelf with RFID-taggedproducts allows customers to crick in the brick and mortarstore, by analyzing the RFID information and events.Each crick in a smart shelf (i.e., a human-productinteraction) can be performed through different interfacesas evaluated in our research:

• Augmented reality: A customer can observe thesmart shelf through a tablet or smartphone screen,and crick on a specific product. Thanks to theRFID provided identification and location, thecustomer could interact with each single product,improving computer vision capabilities. An examplecan be found in [6].

• Sensors on shelves: By placing specific sensors(besides RFID) on the smart shelf [7] the systemcan not only detect the product’s location but alsothe customer’s hand position. A crick would linkevents detection in both RFID antennas andsensors. An example can be found in [10].

• Gestures recognition: Sensors like Kinect [4]connected to the smart shelf, can be used asinteraction interfaces. By recognizing positions orgestures of our body, the system can interpret theseinputs as cricks on the retail products. An examplecan be found in [5].

• Deviceless human-product interaction: Taking aproduct out of the smart shelf is detected by theRFID system. Moreover, the antennas can recoverlow-level radio frequency communicationinformation such as the received signal strengthindicator (RSSI), to improve location resolution andduration of human-product interactions. Thisapproach relies only on the RFID information, and it

Session: PeTRE: Workshop on Pervasive Technologies in Retail Environment

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

1025

is envisioned as the long-term technique to crickdetection at brick and mortar stores.

Once the customer has performed the crick, it is expectedto obtain automatic feedback, similarly to the informationreceived after a click in eCommerce. Next, we summarizethe different interfaces to show crick’s output, beingsurveyed in our research. The interfaces are organizeddepending on whether the crick is performed through adevice, or deviceless:

• Device crick: The same devices’ screens used tofocus and crick on products, can be used tosuperpose information about these products.Similarly, Google Glasses [2] can also outputinformation.

• Deviceless crick: Cricks performed deviceless relyon other supports to obtain feedback. Embeddedscreens or image projections on the smart shelvescan provide useful feedback to cricks. Sound is alsopossible feedback interface, being specially relevantto visually impaired people. All of the above canprovide useful interaction and help customers intheir shopping experience.

Each crick can be registered by the retailer to obtaindeeper knowledge about customers’ preferences andfashion evolutions. Furthermore, if customers are properlyidentified in the system, a customer-preferences model canbe enabled.

Future work and discussionWe firmly believe that cricks are the next revolution incombining eCommerce and brick and mortar retail. Our

ongoing research in the laboratory and real stores isreporting results in this direction. Moreover, cricking isnot dependant of next-generation developments, but it isbeing tested in real scenarios, with current state-of-the-artproducts and technology.

Not only retail management improvement, but anyinventorying-dependant brick and mortar business couldbenefit from the cricking concept. For instance, customerscould obtain instant reviews, extended productinformation and tailored personal and collaborativerecommendations. The above are envisioned to besuccessful applications of cricking in retail.

AcknowledgementsThe authors would like to thank the anonymous reviewersfor their valuable comments and suggestions to improvethe quality of the paper. This work was partly funded bythe Spanish Government through project TIN2012-34965PIGALL, and Obra Social “la Caixa”-ACUP throughproject 2011ACUP00261.

References[1] Amazon Website. http://www.amazon.com/.[2] Google Glass.

http://www.google.com/glass/start/.[3] Google Website. http://www.google.com/.[4] Frati, V., and Prattichizzo, D. Using kinect for hand

tracking and rendering in wearable haptics. In WorldHaptics Conference (WHC), 2011 IEEE (2011),317–321.

[5] Gehring, S., Lochtefeld, M., Daiber, F., Bohmer, M.,and Kruger, A. Using intelligent natural userinterfaces to support sales conversations. InProceedings of International Conference onIntelligent User Interfaces. International Conference

Session: PeTRE: Workshop on Pervasive Technologies in Retail Environment

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

1026

on Intelligent User Interfaces (IUI-12), February14-17, Lisbon, Portugal, ACM (2012).

[6] Lochtefeld, M., Gehring, S., Schoning, J., andKruger, A. Shelftorchlight: Augmenting a shelf usinga camera projector unit. In In Adjunct Proceedingsof the Eighth International Conference on PervasiveComputing (2010).

[7] Matsushita, K., Iwai, D., and Sato, K. Interactivebookshelf surface for in situ book searching andstoring support. In Proceedings of the 2ndAugmented Human International Conference, AH ’11(2011).

[8] Motorola. What’s driving tomorrow’s retailexperience.http://downloads.visionid.ie/white_papers/

Motorola-Tomorrows-Retail-Experience.pdf, 2012.

[9] Roberti, M. Merging bricks and clicks (2011).http://www.rfidjournal.com/articles/view?8154.

[10] Spassova, L., Schoning, J., Kahl, G., and Kruger, A.Innovative retail laboratory. In Roots for the Futureof Ambient Intelligence. European Conference onAmbient Intelligence (AmI-09), 3rd, November18-21, Salzburg, Austria, o.A. (2009).

[11] Thiesse, F., and Michahelles, F. An overview of epctechnology. Sensor Review 26, 2 (2006), 101–105.

[12] von Reischach, F., Guinard, D., Michahelles, F., andFleisch, E. A mobile product recommendation systeminteracting with tagged products. In PervasiveComputing and Communications, 2009. PerCom2009. IEEE International Conference on (2009), 1–6.

Session: PeTRE: Workshop on Pervasive Technologies in Retail Environment

UbiComp’13, September 8–12, 2013, Zurich, Switzerland

1027