This article was downloaded by: [University of York]On: 02 December 2014, At: 06:20Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Applied Artificial Intelligence: AnInternational JournalPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/uaai20

OVERCOMING MOBILE DEVICELIMITATIONS THROUGH ADAPTIVEINFORMATION RETRIEVALDINA GOREN-BAR aa Information Systems Engineering Department , Beer-Sheva, IsraelPublished online: 16 Aug 2010.

To cite this article: DINA GOREN-BAR (2004) OVERCOMING MOBILE DEVICE LIMITATIONS THROUGHADAPTIVE INFORMATION RETRIEVAL, Applied Artificial Intelligence: An International Journal, 18:6,513-532, DOI: 10.1080/08839510490462876

To link to this article: http://dx.doi.org/10.1080/08839510490462876

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

u OVERCOMING MOBILEDEVICE LIMITATIONSTHROUGH ADAPTIVEINFORMATION RETRIEVAL

DINA GOREN-BARInformation Systems Engineering Department,

Ben-Gurion University of the Negev,

Beer-Sheva, Israel

The present study evaluates user preferences for personalized information access throughmobile devices. The research focuses on text search operations in three simulated deviceson a PC screen (cellular telephone, Personal Digital Assistant [PDA], and InteractiveTV [ITV] as control) through continuous search. The adaptation process implementedexplicit user feedback regarding the relevance of the retrieved information. The results showthat users preferred the adaptive information access system to the non-adaptive one for allthe devices (PDA, cellular phone, and ITV). Moreover, adaptation neutralizes the consist-ent preference for ITV (big screen, easy manipulation) upon other devices for textual infor-mation manipulation tasks.

Telecommunication enables an open market of information services throughmany available devices. This evolution fulfills the vision of ‘‘information anytime, at any place, in any form’’ (Magedanz et al. 1996).

Wireless devices are capable of receiving all the digital informationsources from the Internet without physical constraints. However, unlikethe traditional desktop Internet environment where users can browse throughthousands of pages of graphic-rich content, until recently the wireless Inter-net was mainly limited to text data. This was due to three main reasons: First,wireless networks were lacking the bandwidth to effectively transmit graphic-rich images. Second, the small screens of mobile phones and handheld deviceswere only sufficient to display text data and small icons, and, third, wirelessdevices had limited computing power. However, Internet accessibility is

This study was made possible through the contributions of Hadas Beeri, who assisted actively in the

research, and Sharon Weisblit, who ran the experiments and was involved in prior stages of the experi-

ments.

Address correspondence to Dina Goren-Bar, Information Systems Engineering Department,

Ben-Gurion University, P.O. Box 653, Beer-Sheva 84105, Israel. E-mail: dinag@bgu.ac.il

Applied Artificial Intelligence, 18:513�532, 2004

Copyright # Taylor & Francis Inc.

ISSN: 0883-9514 print/1087-6545 online

DOI: 10.1080=08839510490462876

513

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

changing rapidly. Currently, the wireless Internet market is being developedfrom several distinct technology types—the work-enabling wireless LAN; theinter-device wireless connection of Bluetooth and infra-red; and cellularnetwork-centered third-generation (3G), which is a cellular broadband tech-nology carrying Internet, enhanced SMS, e-mail and video-stream. DoCo-Mo’s i-Mode service in Tokyo was the first commercial service thatallowed broadband streaming of rich data into mobile phone handsets.i-Mode handsets have enhanced capacities such as video cameras that allowusers to make video calls and MP3 players which can stream music live fromthe Web; they are E-mailers, Web browsers, organizers, and, of course, tele-phones. In Europe, UMTS (3G) technology, which promises ‘‘wideband’’communication speeds of up to 2Mbps was commercially available in2003=2004.

As a result, the user will be able to access more information anywherethrough small-size display units. Therefore, the main usage of the wirelessInternet should be to allow the user to perform focused searches and thussatisfy specific and immediate information needs.

Personalization aims to get only relevant information for the user byfiltering out all irrelevant information, based on a ‘‘user profile’’ that repre-sents the user’s information needs (Belkin and Croft 1992).

Currently the personalization level that the Mobile Network Operators(MNO) and software companies are offering is relatively low and is per-formed by the user, not the system, thus enabling customization of services.Personalized services are based on SMS (Short Message Service), which isavailable in almost all mobile phones. The level of personalization is limitedto choosing from available content links, icon, and services (Bahattin et al.2001). For example, Nokia has developed the Artus Messaging platformfor MNOs, which acts as a gateway between information and applicationsresiding on the Internet or company intranets, and a mobile phone (www.nokia.com=networks=17=maxp.html). The messaging platform allows theMNOs to create value-added WAP and messaging applications for all mobileusers where users are able to select from the available content links and ser-vices the operator has provided. This allows each user to personalize andcontrol the information they see on their wireless devices. Other systemsavailable on the market today provide similar services.

Interactive TV (ITV) is another platform that enables users to accessinformation services through an Internet connection. It has several technicaladvantages over mobile devices: It has a large screen and multimedia (gra-phics, text, video, and audio) communication capabilities. Moreover, theusers’ attitude towards TV is highly positive: The device is well known, haswidespread access, and is commonly used by users from different ages andcultures. ITV exploits the advantages of the TV medium, transforming theusers from passive watchers to active users.

514 D. Goren-Bar

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

ITV and mobile devices fill similar user needs: (1) The user has a focusedinformation need; (2) The user has no patience and wants immediate answerto his request; (3) The user is mostly passive and is interested in minimalinteraction with the device; and (4) The interaction style with the device isvery limited through the mobile key-pad or the remote control unit throughup=down browsing (or zapping) and does not enable large text input(Robertson et al. 1996).

Both mobile devices and ITV differ substantially from computer Web usein terms of user expectations (perform extensive Web searches), user interac-tion with the device (enable a variety of interaction styles including extensivetext input), and the kind of information searched (professional, travel,general knowledge, etc).

Our work focuses on adaptive and personalized information servicesthrough different devices. We developed an intelligent information retrievalsystem and performed a user study to test whether adaptation is really neededand in which conditions. We intend to determine whether and how mobilecomputing (and with which device) may help us in performing our tasks.

ADAPTATION IN MOBILE DEVICES

Standynk and Kass (1992) describe information filtering as the need topresent each user with all information relevant to that user, blocking outirrelevant information. The basis for filtering irrelevant information is theknowledge stored in a user profile. This profile may contain general userdetails, fields of interests, information goals, etc.

Anderson et al. (2001) emphasize the need for reducing the number oflinks followed, thus preventing wireless users from spending considerabletime scrolling through pages on a small screen. The presented ‘‘Minpath’’algorithm combined with Naı̈ve Bayes model is relying on a user model tocompute the probability of possible page trails to be taken by a specific user.The model was learned offline, as the evaluation of the model ran in realtime. After a training session, the model was tested resulting in savings ofmore than 40% of links for wireless visitors.

Another adaptation approach relates to the display of pages normallybrowsed by large displays to the wireless limited device. Buyukkokten et al.(2001) demonstrated a browsing technique for PDAs and cellular phonesnamed ‘‘Accordion’’ since, in the presented technique, pages can be shrunkor expanded. This method uses three main techniques: page summarization,keyword-driven summarization, and automated-view transitions. Page sum-marization basically parses the page into several Semantic Textual Units(STUs), summarizing each STU into a single line and marking whetherSTU is longer than presented. The marker may expand and then shrink.Keyword-driven summarization completes the page summarization by the

Adaptive Information Retrieval 515

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

highlighting of keywords that are searched by the user, and automaticallyopening all STUs containing those words. Furthermore, in order to avoiduser disorientation upon content change of the Web page, automated viewtransitions are made for smooth scrolling.

The ‘‘end game’’ summarization model was tested on 15 individuals. Theresults were shrinkage of 82% in downloaded page size, and a significantreduction in information that a user must see while scrolling pages.

Aside from the reduction of links followed and adaptation of Web pagesto display, an urgent need for profile adaptation is presented (Billsus et al.2000). Their work presents a learning agent for wireless news access thatimplicitly learns about users’ interests in daily news stories by observing theirbrowsing behavior. This agent uses a content-based algorithm that was orig-inally designed for a Web-based client (Billsus and Pazzani 1999b). The DailyLearner, a learning online newspaper (available for public use at http:==dailylearner.ics.uci.edu) was adapted to the requirements and constraints ofwireless information access. The interface designed was aimed to minimizeboth interaction between user and device and the amount of data transmitted(due to cost, bandwidth, and size limitations). This is done by an implicit col-lection of the user’s news access patterns. By selecting each of the nine newscategories, a first set of personalized headlines is retrieved. Only four head-lines are displayed at once, rank-ordered by the user’s current interest profile.A ‘‘thumbs up’’ icon may be added to headlines that receive a rank greaterthan a threshold of 0.9. Users’ actions are converted into scores ranging from0 to 1. A fully downloaded story receives a rank of 1.0. In contrast, skippinga story receives a negative score of 0.2 from the system’s prediction. Theirsystem evaluation results show evidence of the utility of adaptive news access,compared with a non-adaptive system.

RESEARCH GOALS AND ASSUMPTIONS

Prior research compared the performance of content tasks in threedevices: cellular telephone, PDA, and ITV, (Goren-Bar 2002).

Results indicated that users clearly preferred interactive TV over mobiledevices for performing content tasks. TV has a big screen, it can broadcast inreal time, and can transmit multimedia content (graphics, text, video, andaudio). The user attitude towards TV is largely positive: The device is wellknown and widely accessible to a variety of users (adults, children, the eld-erly, the physically handicapped, etc.), and to people of different cultures.However, until recent years, broadcasting meant user passivity. ITV exploitsall TV advantages while giving the user an interactive experience.

Mobile devices (like PDA and the cellular telephone) are also interactiveand have mobility superiority over ITV. Nonetheless, these devices are small

516 D. Goren-Bar

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

and lack some of the advantages we have mentioned above. In order to helpusers to overcome the devices’ limitations, an adaptive model should be used.

To improve the information retrieval process in mobile devices, wedeveloped a personalized mechanism based on the IUIM model (Goren-Bar 2001) intended to show relevant news to the user according to her=hisuser profile and adapt them to its changing preferences based on a heuristiclearning algorithm.

To test the efficacy of the adaptive system, we developed a simulative in-formation search system which operates with three devices: cellulartelephone, PDA, and ITV.

This study aims to compare the influence of an adaptation method on theusers’ information search process in the three devices. The research tests thefollowing hypothesis:

1. Regarding the adaptation process: We assume that the adaptive systemwill be preferred to the non-adaptive one. Users will prefer the infor-mation items presented with the system applying the IUIM principlesand adaptation process in all three devices.

2. We assume that the adaptation process will not distinctively affect thepreference of a specific device over another because the same adaptationmodel is applied in all three devices.

3. Regarding device preference: Based on prior results, we assume that ITVwould be preferred to PDA, which would be preferred to the cellular tele-phone. We assume that larger screen display devices will be found moreadequate for performing content tasks, and therefore more preferablethan small screen display devices.

4. Regarding the influence of users’ prior experience with the device on users’interfaces judgment. We assume that when the device and interface aremore familiar to the user, he will find it more understandable and easierto operate.

SYSTEM ARCHITECTURE

The system architecture is client-server. The client is on the user’s deviceand the database is on the Web server (see Figure 1).

The following is a short description of the system components.

. User Device. The user performs the information search through a cellulartelephone, a PDA, or an ITV browser. The device gathers and saves theinformation of the current session and sends it to the server for analysis.Both, cellular and PDA devices connect to the server through a WAP gate-way. The user interface was developed in WML on the M3gate simulator,

Adaptive Information Retrieval 517

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

which resembles the device and its operation on a PC. The ITV connectsdirectly to the server through http, getting the query results in HTML.The PDA can also do the same. We simulated the ITV on InternetExplorer browser opened to full size with the screen display set to low res-olution. Following prior research (Goren-Bar 2002), we chose to continuewith ITV simulation (instead of using a standard Web browser) since ITVinvokes similar user cognitive processes as mobile devices, such as rapidbrowsing of information (known as zapping) with minimal investment ofcognitive effort. As stated earlier, both mobile devices and ITV differ sub-stantially from computer Web use on user expectations (perform extensiveWeb searches), user interaction with the device (enable a variety of inter-action styles including extensive text input), and the kind of informationsearched (professional, travel, general knowledge, etc.) (Robertson et al.1996).

. WAP Gateway. It checks the WML pages and converts them intoHTML=XML to be sent to the server. On the other side, it checks HTMLpages coming from the server and converts them into binary code suitablefor cellular communication.

. User KDB (Knowledge Data-base). This knowledge base saves the userprofile and the user topics of interests. It is located on the Web ser-ver. It is updated according to the analysis of the user actions on thedevice.

. DocDB. This database contains the information items for the experimentalsystem.

. Matching. This is a process performed on the server that matches userinterests and information items and sends the retrieved information tothe user.

FIGURE 1. Experimental system architecture.

518 D. Goren-Bar

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

. Learning. This process also performed on the server analyzes the userimplicit feedback and updates the user preferences.

EXPERIMENTAL ADAPTIVE USER MODEL

The experimental model adaptive system was based on the IUIM model(Goren-Bar 2001). The model defines three main components. User Model,Task Model, and Dialogue Model. These enable personalization and adap-tation based on machine learning algorithms. In the present work, weimplemented the following components:

. User Model. Defines the user profile, which includes demographic attri-butes and the user preferences of interesting information subjects.

. Task Model. Defines the information retrieval tasks to be performed in allthree devices.

. Dialogue Manager. Defines the learning mechanism and the adaptationalgorithm to show the user the most relevant news in its preferred subjects.

User Model

User Model DefinitionWahlster and Kobsa (1989) define a user model as a source of knowledge

containing explicit assumptions regarding all the aspects of the user, whichmay be relevant to adapting the discourse management to a particular user.These assumptions must be separated by the system from all the other knowl-edge residing in it. They even define a User Modeling Component (UMC). Itsrole is to: (1) Gradually build the user model; (2) Store, update, and deleteknowledge items; (3) Maintain model consistence; and (4) Furnish assump-tions regarding the user to other system components. The UMC wasaugmented by another role, which is drawing conclusions from initial=preliminary assumptions (Kobsa 1994).

The user model of the present study is comprised of two user profiles:demographic and content-based.

PDAs and ITV are not very popular in the entire population; therefore,we chose a user group that is acquainted with all three devices. TheDemographic User Profile includes the following characteristics: age(21�29); mother tongue (Hebrew); occupation (student); and education(undergraduate—third year in technological studies). The Content-BasedProfile includes the categories of news preferred, based on the overall choicesof the subjects of the group. The Content-Based Profile enables retrieving thelist of news categories to be displayed to the user when he=she asks for news.The Content-Based Profile is updated each time the user provides relevancefeedback regarding the retrieved news.

Adaptive Information Retrieval 519

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

User Profile AcquisitionThe most popular approach for acquiring knowledge about the user is

the explicit approach, which is based on user interrogation. Systems utilizingthis method usually require their users to fill out a form describing their areasof interest or other relevant parameters (Hanani et al. 2001). This method isused, for example, in WAIS (Brewester and Art 1991). Some systems providethe user with a set of terms that represent each domain from which he=shecan construct its personal profile (McCleary 1994). This method preventsthe semantic confusion observed in systems where users have the freedomto choose their own terms. An example is BackWeb (BRM 2000), which isconnected to predefined sources of data and has restricted predefineddomains.

The present study implements the explicit approach for acquiring the userknowledge. The reason for selecting this approach is that it resembles thestandard procedure with mobile devices nowadays. When a user buys a cellu-lar telephone, a PDA, or subscribes to ITV, it is necessary to fill out a formwith personal details. If the user is interested in information services, he=shehas to select the services from the available ones. Moreover, it is extremelycomplicated to fill a form in the mobile phone due to its screen size. In thepresent study, the user fills out a Web form when he enrolls the system.The information topics are selected from a predefined tree to prevent seman-tic confusion.

Information topics are arranged in a four-level depth tree. Each node hasfour leaves. The information topic labels for the first level are entertainmentand free time; sports; news; and vacation and touring. The total number ofinformation items at the lowest level is 256. Figure 2 presents a partial sampleof the topic tree.

The user’s topics of interest are updated by two mechanisms: by the userinitiative, through an update form in each one of the three devices, and by thesystem initiative as a result of the adaptation algorithm (see ‘‘DialogueModel’’).

Task Model

Task Models examine the way in which people perform their tasks,including the operations they perform, the objects they use, and the knowl-edge required of them to perform these operations from the user’s perspective(Dix et al. 1997).

Tasks are actions, which the user considers meaningful. In defining atask, one must address a dimension of intent, which does not exist in thedescription of a function. That is, the task is performed by the user for thepurpose of attaining a goal. Therefore, a task model should describe what

520 D. Goren-Bar

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

users do in the course of their work in formal terms and what the user shoulddo in order to attain a certain goal.

One of the prominent models in this area is the GOMS model developedby Card et al. (1983). The model’s name is the acronym of goals, operators,methods, and selection of rules. It has been widely used in the analysis ofword processing work. The model’s basic assumption is that user behaviorin the performance of a routine computer task may be described as a hier-archy of primary tasks and subtasks. On a more detailed level, that behaviormay be described as a search for a goal through physical (such as movingup=down arrows in a cellular phone) and mental (such as comparing thecontent of two information items) operators. A collection of operators exe-cuted in one routine is defined as a method (for instance, search). Sometimesmore than one method may be used to perform a specific task. In this case, aselection takes place, according to each user’s specific rules (in the GOMSmodel it is based on if-then statements).

Another popular method for task analysis is the Hierarchical TaskAnalysis (HTA), which performs a hierarchical decomposition or breakdownof tasks along with the sequence in which they are executed. The HTA resultis a hierarchy of task and subtasks, along with plans outlining the tasksequence and the conditions for their execution. Finding the most appropri-ate hierarchy is part of the HTA process (Dix et al. 1997).

FIGURE 2. A partial sample of the topic tree.

Adaptive Information Retrieval 521

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

The present work implemented HTA for the analysis of iterative search.This kind of search is required for enabling information filtering, learningfrom user actions, and implementing adaptation. Figure 3 shows the user’stask sequence when performing an iterative search task.

The same tasks were performed on all three devices in the same order.

Dialogue Model

The dialogue manager’s role is to manage the learning and adaptationmechanisms.

The learning component is needed in order to improve further filtering.Users may change their information needs; therefore, filtering systems mustinclude a learning process that detects shifts in users’ interests and updatesthe user-model by reinforcing, contradicting, or canceling existing knowledgeabout the users (Hanani et al. 2001).

There are two popular methods of learning: learning by observation andlearning by feedback.

Learning by observation compares new situations to known ones, todecide the course of action or to suggest an action. In a filtering situation,a system that observes user behavior for different data items can comparea new data item to known ones, in order to suggest whether to retrieve itor filter it out. This sort of mechanism is widely used in software agents.Billsus and Pazzani (1999a; b) implemented learning by observation in anagent that compiles a daily news program for individual users. They reportedan overall increased retrieval performance and higher precision and classi-fication accuracy. Grammex (‘‘Grammars by Example’’) is an agent trainedto recognize text by example (Lieberman et al. 2001). Editing grammar isdifficult and error-prone for end users. Grammex is a direct manipulation

FIGURE 3. Task sequence of iterative search.

522 D. Goren-Bar

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

interface designed to allow non-expert users to define grammar interactively.The user presents concrete examples of text that he or she would like theagent to recognize. Rules are constructed by an iterative process, whereGrammex heuristically parses the example, displays a set of hypotheses,and the user critiques the system’s suggestions. Actions to take upon recog-nition are also demonstrated by example.

In learning by feedback the system learns from explicit or implicit userfeedback. The user provides explicit feedback directly—by telling the systemhow to act in a similar situation or indirectly—by providing feedback infor-mation (such as a score of relevance of the data item). An information filter-ing system can learn about the relevance of a new data item from the userfeedback relating to similar data items. However, most of the users are un-willing to cooperate with explicit feedback. The implicit approach, implemen-ted by inference, does not require active user involvement. Instead, the user’sreaction to each incoming data item is recorded, in order to learn from itabout the actual relevancy of the data item to the user. GroupLens (Konstanet al. 1997), a collaborative filtering system, monitored reading times as anindicator for relevance. Other user behavior, like whether the user saves,discards, prints, or forwards the data items can also serve as an indicationof interest.

The present work implements learning by explicit feedback. The dialoguemodel enables to fine tune the user’s preferences by learning from explicitrelevance feedback provided by the user on each retrieved information item.The user grades each information item read on a scale from 1 to 6 (from notinteresting at all to very interesting). The learning is performed based on adecision tree that matches the tree of information topics presented in Figure2. We present the user two levels of the topic tree: the root, which displays thebasic topics entertainment, sports, news and travel, and the lowest level(leaves, level 4), which presents the information items. The user is not awareof levels 2 and 3 of the topic tree that represents the subtopics. This inner rep-resentation enables the learning algorithm to find the specific subjects thatreflect the user’s interests. The learning is performed based on the gradesthe user assigns to each information topic during the interaction with theleaves of the topic tree.

Learning and Adaptation Algorithm for the Experimental System

The following algorithm was developed for the purpose of the experi-mental system (see Figure 4).

1. The user starts the search by selecting an item from the root.2. The system retrieves randomly one information item from the lowest level

(level 4) from the same branch of the selected topic.

Adaptive Information Retrieval 523

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

3. The user grades the information item on a scale of 1�6 (from 1 [not inter-esting at all] to 6 [very interesting]).If the grade ¼ ‘‘1’’ or ‘‘2’’ (not interesting), the system goes to the‘‘father’’ at level 2 and retrieves randomly an information item (fromlevel 4) from one of the three other branches.If the grade ¼ ‘‘3’’ or ‘‘4’’ (interesting), the system goes to the ‘‘father’’ atlevel 3 and retrieves randomly an information item (from level 4) from oneof the three other branches (brothers).If the grade ¼ ‘‘5’’ or ‘‘6’’ (very interesting), the system goes to the‘‘father’’ at level 3 and retrieves randomly an information item (fromlevel 4) from the same branch (topic).

Step 3 is repeated until the user stops his search.The same algorithm is performed through all three devices. In any case,

no item will be retrieved twice for the same user while searching in any of thethree devices. This will impede the user from learning any specific infor-mation item and reinforce a specific positive or negative attitude towardsthe displayed item.

FIGURE 4. Adaptation algorithm for displaying information items based on user relevance feedback.

524 D. Goren-Bar

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

EXPERIMENT

Method

The purpose of this experiment was to compare the user’s satisfactionwith the retrieved results while comparing an adaptive system with anon-adaptive one. In order to enable adaptation, a continuous search taskwas performed using three devices (ITV, PDA, and cellular telephone).

SubjectsSixty-three male and female technology students participated in the

experiment. Most of the students were from the Department of IndustrialEngineering and Information System Engineering at Ben-Gurion Universityof the Negev. Their ages ranged between 21�29.

ToolsThe study used three instruments: registration forms (in HTML), simu-

lative operation of ongoing search in the three devices, and computerizedfeedback questionnaires. The cellular telephone and the PDA were simulatedand operated with the M3gate tool, developed by Numeric AlgorithmLaboratories, and the ITV was simulated and operated with the InternetExplorer browser opened to full size. The system architecture was client-server (see Figure 1). All the subjects’ data was saved on the Access 2000database that was on the Web server.

The system included an information items database built from afour-level tree (see Figure 2). The first level included four main topics: enter-tainment, sports, news, and travel. Each main topic was divided into foursubtopics (level 2), and each subtopic was divided into four sub-subtopics(level 3). Each topic in level 3 had four information items (level 4) for asum total of 256 information items. The information items were taken mainlyfrom Internet information sites (Walla, Yahoo, Ynet, LaMetayel.com, etc.).The information item length was fixed and ranged between 105 to 300 char-acters (including captions and spaces). In order to build the informationitems database according to the user’s stereotype, 320 information items werecollected at first, and, after a validation process (as explained next), 256information items were left.

The validation process included distribution of questionnaires (on paper)to 160 technology students (other than the 63 experimental subjects). Thenumber of students was determined so each one of the 16 subtopics fromlevel 2 would have 10 subjects to validate it. The questionnaire presentedall the information items from one subtopic in level 2 arranged randomly.Subjects were required to classify them into four separate categories andprovide a label for each category. Then frequencies for all the categories

Adaptive Information Retrieval 525

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

created on each topic were computed. If 20% or more of the subjects found aparticular information item unsuitable for that category, it was disqualified.

DesignThe experiment array was 3X2X3 factorial. The independent variables

were device type (within-subject, variable with three levels: ITV=PDA=Cell);adaptive=non-adaptive system (between-subject, variable with two levels);and search number (within-subject, variable with three levels). The dependentvariables were the user’s grade of the information item (explicit feedback)measured on a scale of 1 to 6; user’s previous knowledge level in each device(measured on a scale of 1 to 6), and device preference for task performance(categorical variable).

ProcedureSubjects were split randomly into two groups. The first group

(30 subjects) experienced the non-adaptive system, and the second group(33 subjects) experienced the adaptive system.

At the beginning, subjects filled out a computerized personal details ques-tionnaire. Then subjects experienced a three-step ongoing information searchin all three devices. Searching was done in one of the four main informationtopics, according to the user’s request. After each search, subjects graded theresults explicitly within the simulated system environment. The ongoingsearch was different for the two experimental groups: In the adaptationgroup, subjects provided explicit feedback after each information item pres-entation and the system performed the learning and adaptation processaccordingly (see Figure 4). In the non-adaptive group, the information wasselected randomly, regardless of the user’s feedback.

It is important to note that information items were not shown twice tothe same subject. Furthermore, the device sequence was selected randomly.On completion of the ongoing search tasks, subjects reported their priorknowledge on each device on a scale of 1 to 6. Finally, subjects chose theirmost preferred device for performing a search (device preference dependentvariable) in a computerized questionnaire.

Results

In order to examine the effects of adaptation on user’s preferences, wecompared the users’ grades on information items (explicit relevance feed-back) in the adaptive and non-adaptive system using the three devices.

We ran a two factorial ANOVA (3 device types X 2 levels of adaptation)with repeated measures on information grades (see Figure 5).

As expected, the main effect of adaptation was highly significant[F (1,61) ¼ 27.284, Mse ¼ 37.153 p < 0.0001] (as shown in Figure 5).

526 D. Goren-Bar

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

Furthermore, as was hypothesized, no significant interaction between thedevice type and adaptation was found.

We also ran a two factorial ANOVA on the computed variable: thedifference between information grades during consecutive searches. The usersperformed three searches on the same topic; therefore, there were two com-puted grades (between the first and the second search, and between thesecond and the third one). Each user performed three searches, one on eachdevice (PDA, cellular telephone, and ITV). This variable computes the differ-ences between the users’ grades on each search, regardless of the device (seeFigure 6).

FIGURE 5. Information fitness average for adaptive and non-adaptive system.

FIGURE 6. Effects of adaptation on information items’ grades during consecutive search.

Adaptive Information Retrieval 527

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

Figure 6 shows that, as expected, we found a significant interactionbetween the presence or absence of adaptation and the user grades on theinformation items presented during a consecutive search [F (1,61) ¼ 5.569,MSe ¼ 12.537, p < 0.05]. The group that performed the search with theadaptive system raised their grades between searches. On the other hand,the group that performed the search with the non-adaptive system loweredtheir grades between searches.

For the dependent variable device preference, we ran goodness of fit test(v2) on each content task (reading news and performing search). The resultsare shown in Table 1.

Table 1 shows that, as expected, ITV was found most suitable for contenttasks performance, the PDA was in second place, and the cellular telephonewas estimated as unsuitable.

We ran Pearson Rank Correlation between the device type and previousknowledge in the device. A connection was found only in the cellulartelephone. This correlation was statistically significant, but the correlationwas weak [R < 0.3, p < 0.05]. There was no connection between the devicetype and previous knowledge in PDA and ITV [R < 0.2, p ¼ NS].

DISCUSSION

The main experimental result relates to the effectiveness of the adaptationprocess. Users ranked information as more suitable in the adaptive systemthan in the non-adaptive one. This is compatible with other studies dealingwith adaptation. For example, Billsus et al. (2000) developed a learning agentdesigned to help users access interesting news stories through PDA. Whenthey tested their system in comparison with a non-adaptive system, theyfound evidence of the utility of adaptive news access. Another example isthe FAB system (Balbanovic and Shoham 1997), which is based on acontent-based profile and collaborative approach and recommends Webpages to users. In a comparison performed to three benchmarks, the FABsystem has been shown to improve its performance over time, while consist-ently producing pages from the other three systems. This was remarked uponin the users’ satisfaction report.

TABLE 1 Device Preference for Tasks Performance

Residual

Content task v2 (113,2) Cell PDA ITV

Read news 24.667 �15.0 �2.0 17

Information search 8.857 �11.0 4.0 7.0

<0.05

528 D. Goren-Bar

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

Our results also showed that the subjects in the adaptive system weremore satisfied with the information items retrieved and showed a positivetendency between searches. A surprising finding is the negative tendency inthe information items’ grades between searches that the subjects in thenon-adaptive group gave (see Figure 6). We would expect a flat line, whichmeans indifference between the search results. This negative tendency indi-cates disappointment from the subjects’ point of view. The subjects in thenon-adaptive group knew that the target of the experiment was to performan ongoing information search in three devices. In addition, the subjectscome from a technological background and they are familiar with infor-mation and Web searches and its problems. In light of these two aspects, theyprobably developed an expectation to get more compatible information fromsearch to search. But since there was no adaptation model, they weredisappointed.

Another interesting finding is that the subjects differentiate between theevaluation of the device (and its interface) and the evaluation of the retrievedinformation. The results were consistent: Subjects preferred PDA and ITV tothe cellular telephone, with no significant difference between PDA and ITV,as reported in our previous study (Goren-Bar 2002). As expected, the maineffect of the adaptive system was highly significant [F (1,61) ¼ 27.284,MSe ¼ 37.153, p < 0.0001], meaning that adaptation improved the resultsretrieved in all three devices, showing no significant interaction betweenthe device type and the user model.

Another consistent finding was that the PDA and ITV were preferred tothe cellular telephone for performing information search tasks on a signifi-cance level of P < 0.05 (see Table 1). Results showed that subjects preferredthe ITV in the first place and PDA in the second. However, the cellulartelephone was estimated as unsuitable.

A bias that could influence the users’ objective judgment of the cellularinterface is that they did not judge the interface in terms of the limitationsof the cellular telephone, but in terms of fitness to content tasks. Also, itshould be mentioned that the experiments were performed on a simulativesystem, which illustrates the devices on a PC platform. This parameter madethe device operation more difficult, particularly with the cellular telephone,which is difficult as it is. Unfortunately, due to the fact that simulated deviceshave been used in this study, we can’t assume that users interact in the samemanner with real devices. In order to understand the user preferences for thedifferent interfaces’ devices, interaction situations with real devices deserveempirical testing.

In spite of the experiments’ problems, the main factor that could result inthese findings is the physical limitations of the cellular telephone. There aretwo aspects that should be examined in this regard. The first is the cellulartelephone’s small display, which is significantly smaller than the screens of

Adaptive Information Retrieval 529

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

the other devices. The second aspect, which is influenced by the first, is theappropriate information length.

From the aspect of the small display effect, explanations and elucidationscan be found in other research that examine this effect. Small screen limita-tions are well explained in Kamba et al. (1996). The authors argued that theproblem with small devices is ‘‘the need to shrink the screen to a size that fitsinside a very small box,’’ and they referred to this as the ‘‘physical’’ limi-tation. They also referred to the ‘‘functional’’ limitations as ‘‘the need to keepthe screen sufficiently large to show enough information so that the device isactually useful.’’ This means that the system needs to have control objects aswell as text. Another finding showed that users seem to choose and preferdirect access strategies over less directed, browsable approaches when dealingwith a small screen (Jones et al. 1999).

One possible solution is the Accordion method, which enables a link toshrink and expand, based on an intelligent algorithm (Buyukkokten et al.2001).

The effect of the small display, as described above, affects the appropriateinformation length that should be displayed on the cellular screen and theinterface design. The cellular telephone was estimated as unsuitable to thenews’ length in the experiment (between 105 to 300 characters). Evidencesupporting this can be found in Dillon et al. (1990), which found two effectsof smaller displays. First, users reading from the small displays interactedwith the display window to a much higher degree than those with the largerwindow. Second, 75% of users who indicated that they would have liked tochange their screen size used a small screen display.

The conclusion reinforces the need to be aware of the cellular telephone’sphysical and functional limitations, and to adjust its usability to its purpose.In our experiment, we examined particular news’ length. It is possible thatwith shorter, more focused news, whose purpose is to fill a specific andimmediate need, the cellular telephone will be acknowledged as more suit-able. An example of focused news is ‘‘IBM stock is up by at least 3% morethan the change in the Dow Jones index.’’ Future research should examinewhat is the optimal length that fits the cellular telephone. However, it seemsthat adaptation can solve the problem of user negative attitude towards cellu-lar telephones for performing content tasks due to the physical limitations ofthe device. Subjects preferred the adaptive system in all three devices, neutra-lizing the negative attitude towards the cellular telephones for performingsearch tasks. This finding enhances the crucial role of adaptive applicationsfor cellular telephone interfaces in particular.

From the aspects of the influence of previous knowledge on subjects’interface fitness judgment, we saw a weak influence of previous knowledgeof the device to the interface fitness judgment, especially with the cellulartelephone. These results can be explained with the earlier experiments’

530 D. Goren-Bar

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

results, which showed that the cellular telephone is difficult to operate incontent tasks. Therefore, previous knowledge in that case helped the subjectsto perform the required tasks. An effect that can explain these results isknown in social psychology as ‘‘more familiar ¼ more likable,’’ meaning thatwhen a person is more familiar with the environment he works in, it is easierfor him to orient and operate. Therefore, he finds the environment morelikable.

CONCLUSIONS

Previous user studies evaluate the impact of adaptation on mobile devicesand the preferences of users over advanced devices such as cellular tele-phones, PDAs, and ITV. However, a combination of both, as done in thiswork, seems innovative and enlightening.

In general, all the results show a negative attitude to the cellulartelephone over the PDA and ITV. Previous knowledge with the deviceimproves this attitude.

The cellular telephone main usage should be showing the user focusednews, whose purpose is to answer a specific and immediate need.

The most important factor that may improve considerably user attitudestowards cellular telephones is the implementation of adaptation processes toenable easier and faster access to information content.

It was proved that adaptation process increases user satisfaction from thesearch results and even eliminates the preference of larger devices forperforming information search. Therefore, it is very important that futuresystems should have a personalization and adaptationmechanism as a default.

A very important finding is that the PDA is a most promising device. It ismobile and small in size, but perceived as good as ITV and, most important,users have a very positive attitude towards this device for performing contenttasks. Due to the fact that the subjects of this study were technology students,in order to validate the results, other user populations should be tested aswell.

A main limitation of the present study was the use of simulated devices inthe evaluation. Whether or not the reported results tell much about a com-parison conducted with real devices remains subject to another evaluation!

REFERENCES

Anderson, C.R.,P. Domingos, and D.S. Weld. 2001. Adaptive Web navigation for wireless devices, In

proceedings of the Seventeenth International Joint Conference on Artificial Intelligent (IJCAI-01),

pp. 879�884, Seattle, WA, August 4�10.

Bahattin, O., K. Ozgur, A. Mehmet, and D. Asuman. 2001. Highly personalized information delivery to

mobile clients. Proceedings of the 2nd ACM International Workshop on Data Engineering for Wireless

and Mobile Access, pages 35�42, Santa Barbara, CA, USA.

Adaptive Information Retrieval 531

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14

Balbanovic, M. and Y. Shoham. 1997. Fab: Content-based, collaborative recommendation.

Communications of the ACM 40(3):66�72.

Belkin, N.J. and W.B. Croft. 1992. Information filtering and information retrieval: Two sides of the same

coin? Communications of the ACM 35(12):29�38.

Billsus, D. and M.J. Pazzani. 1999a. A hybrid user model for news stories classification in UM99. In Pro-

ceedings of the Seventh International Conference on User Modeling, pages 99�108, Vienna,

Austria. New York: Springer.

Billsus, D. and M.J. Pazzani. 1999b. A personal news agent that talks, learns and explains. In Proceedings

of the Third International Conference on Autonomous Agents, pages 268�275 Seattle WA, USA.

Billsus, D., M.J. Pazzani, and J. Chen. 2000. A learning agent for wireless news access. In Proceedings of the

International Conference on Intelligent User Interfaces (IUI2000). pp. 33�36, New Orleans, LA, USA.

Brewester, K. and M. Art. 1991. An information system for corporation users: Wide Area Information

Servers (WAIS). Online 5(5):56�60.

BRM Technologies. 2000. BackWeb � Push Service. http:==www.backweb.com

Buyukkokten, O., H. Garcia-Molina, and A. Paepcke. 2001. Accordion summarization for end-game

browsing on PDAs and cellular phones. In Proceedings of the Human-Computer Interaction

Conference 2001 (CHI 2001), pp. 213�220, Seattle, WA, USA.

Card, S.K., T.P., Moran and A. Newell. 1983. The Psychology of Human-Computer Interaction. Hillsdale,

NJ: LEA Associates.

Dillon, A., J. Richardson, and C. McKnight. 1990. The effect of display size and text splitting on reading

lengthy text from the screen. Behavior and Information Technology 9(3):215�227.

Dix, A., J. Finlay G. Abowd and R. Beale. 1997.Human-Computer Interaction, 2nd ed., London: Prentice-Hall.

Goren-Bar, D. 2001. Designing model-based intelligent dialogue systems, in Rossi, M. & Siau, K. (eds.):

Information Modeling in the Next Millennium. pp. 271�287. Idea Group Publishing.

Goren-Bar, D. 2002. Adaptive information through mobile devices, does it help? Artificial Intelligence in

Mobile System Workshop 2002, July 22, 44�49, in conjunction with the 15th European Conference

on Artificial Intelligence (ECAI-2002), 22�26, July 2002, Lyon, France.

Hanani, U., B. Shapira, and P. Shoval. 2001. Information filtering: Overview of issues, research and

systems. User Modeling and User-Adapted Interaction (UMUAI) 11(3):203�259.

Jones, M., G. Marsden, N. Mohd-Nasir, K. Boone, and G. Buchanan. 1999. Improving Web interaction

on small displays. In Proceedings of the W8 Conference, Ontario Toronto. (Also reprinted in Inter-

national Journal of Computer and Telecommunications Networking 31:1129�1137)

Kamba, T., S.A. Elson, T. Harpold, T. Stamper, and P. Sukaviriya. 1996. Using small screen space more

efficiently. Proceedings of the Human-Computer Interaction Conference 1996 (CHI’96) Vancouver,

British Columbia, Canada, 383�390.

Kobsa, A. 1994. User-modeling and user adapted interaction. The Human-Computer Interaction Confer-

ence 1994 (CHI 1994), Conference Tutorials, Boston, MA, USA, April 24�28, 2002.

Konstan, J., B. Miller, D. Maltz, J. Herlocker, L. Gordon, and J. Riedl. 1997. GroupLens: Collaborative

filtering for Usenet news. Communications of the ACM 40(3):77�87.

Lieberman, H., B. Nardi, and D. Wright. 2001. Training agents to recognize text by example In

Proceedings of the ACM Conference on Autonomous Agents (Agents-99), Seattle, WA, USA pp.

116�122. Also reprinted in Journal of Autonomous Agents and Multi-Agent Systems and in Your

Wish is My Command ed. H. Lieberman, Morgan Kaufmann.

Magedanz, T., K. Rothermel, and S. Krause. 1996. Intelligent agents: An emerging technology for next

generation telecommunications. In Proceedings of the INFOCOM ‘96, pp. 464�472, San Francisco,

CA, USA.

McCleary, H. 1994. Filtered information services: A revolutionary new product or a new marketing strat-

egy? Online 4(18):35�42.

Stadnyk, I. and R. Kass. 1992. Modeling user’s interests in information filters. Communications of the

ACM 35(12):49�50. http:==www.cs. washington.edu=research=projects=ai=590i=bs=stadnyk.html

Robertson, S., C. Wharton, C. Ashworth, and M. Franzke. 1996. Dual device user interface design: PDAs

and interactive television. In Proceedings of the Conference on Human Factors in Computing Systems

(CHI’96), Vancouver, BC, Canada, pp. 79�86.

Wahlster, W. and A. Kobsa. 1989. User models in dialog systems. In User Models in Dialog Systems, eds.

W. Wahlster, and A. Kobsa. Berlin, 4�34. New York: Springer.

532 D. Goren-Bar

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

6:20

02

Dec

embe

r 20

14