1

Collaborative Recognition of Queuing Behavioron Mobile PhonesQiang Li∗†, Qi Han§, Limin Sun ∗ †

†Beijing Key Laboratory of IOT Information Security Technology∗ Institute of Information Engineering, Chinese Academy of Sciences

§Department of EECS, Colorado School of Mines, Golden, CO USA 80401{liqiang43, sunlimin}@iie.ac.cn, qhan@mines.edu

Abstract—Nowadays people spend a substantial amount of time waiting in different places such as supermarkets and amusementparks. Detecting the status of queuing may benefit both users and business. In this paper, we present QueueSense, a queuingrecognition system to assist in a queue management system. QueueSense consists of clients on smartphones that provide automatic,energy-efficient, and accurate queuing recognition, and a server in the cloud that collects data, identifies multi-queue lines, and provideswaiting time estimation. In order to be useful, QueueSense should be able to recognize queuing behavior in various queuing scenarioswithout greatly decreasing the battery life of mobile phones. We present features of queuing and build the classifier on smartphonesto automatically recognize queue classifier without human input. We investigate the complicated nature of energy consumption forqueue recognition on phones and design an effective algorithm to maximize energy savings while guaranteeing accuracy of queuerecognition. We evaluate QueueSense performance using the data set from real world queuing scenarios collected over a three-month period. Empirical results show that QueueSense is adaptive to various queuing scenarios with both high recognition accuracyand energy efficiency. We further implemented a prototype of QueueSense, the first queue detection system using smartphones. Weconducted real-world experiments in a dining hall and a supermarket near a university campus. Through implementation and evaluation,we demonstrate that QueueSense is capable of detecting waiting lines that occur in our daily lives.

Index Terms—Smartphone sensing, Queuing group behavior, Queue waiting time prediction.

✦

1 INTRODUCTION

Queuing is a pervasive phenomenon occurring in publicareas such as supermarkets, restaurants, banks, trans-portation terminals, and amusement parks. People spenda substantial amount of time waiting in lines, and longwaiting time brings about awful user experience. Beforedetermining whether to take a line or not, people usuallywant to know more about the waiting lines, e.g., thenumber of people ahead or waiting time. Providing suchqueuing information helps customers better spend timedoing something alternative rather than blindly waitingin line, thereby mitigating their anxiety and improvinguser experience. In addition, managing queues is im-portant for business since it can help reduce inefficientresource allocation and revenue loss. Therefore, there isa need for a better understanding of emerging trends ofthe queues in order to not only improve user experiencebut also benefit business.

Nowadays, mobile phones present an attractive plat-form for people-centric applications [1] as they integratea variety of sensors that may make similar observationsas human. Most smartphones [2] [3] [4] have built-in sensors that measure motion, orientation, locationand various environmental conditions. These sensors arecapable of providing raw data with high precision andaccuracy for sensing and inferring queuing behaviorrecognition. Besides, we observe that people often carry

their phones when they are not home and people arewilling to obtain queue information if it does not affectthe typical usage of their smartphones. This motivates usto use commodity mobile phones to detect queuing andobtain queuing information as a people-centric sensingapplication.

While the idea of using smartphones for queuingdetection seems simple, many challenges arise in prac-tice. First, queuing is a temporal group activity thatis different from individual human activity recognition(HAR) [5]. A group behavior cannot be fully understoodby piecemeal observations [6]. Second, queues are sensi-tive to and affected by environments. Queue recognitionshould be robustness to handle various queue scenar-ios. Last, continually monitoring queuing context incurssignificant overhead in terms of energy consumptionon mobile phones, which has impacted daily usage ofmobile phones.

In this paper, we propose a collaborative approachfor queuing recognition based on mobile phones byfollowing queuing rules (First Come First Server -FCFS)that users obey in the line. The key insight is that queuershave similar and relatively stable motions and directionsthan non-queuers in terms of their individual temporalactivities. This queuing behavior characteristic can op-erate in conjunction with machine learning tools [7] fordetermining whether the user is queuing or not. We alsorefer to this approach as low-infrastructure since it only

2

requires the collaboration from users’ phones and canbe more easily retrofitted in existing queuing applicationscenarios without any manual effort or the infrastructuresupport. We also study energy consumption for queuerecognition on smartphones at different levels – indi-vidual activity sensing, and collaborative detection ofqueues. An effective algorithmic solution is developedto maximize energy savings while guaranteeing accuracyof queue recognition.

We have implemented a prototype of QueueSensewith clients on smartphones based on Android platformsand a server in the cloud. Smartphones use widelyavailable sensors such as accelerometer, compass andbluetooth to sense individual activities. Queuing featuresare computed based on queuing characteristics in ter-m of individual activities and support vector machine(SVM) is used to automatically detect whether peopleare queuing or not on mobile phones. The cloud backendprocesses multi-lines scenarios and provides estimationof queue length and waiting time. Agglomerative hierar-chical clustering is implemented on the server to dividequeuers into different lines based on changing rate ofrelative positions of queuers.

We validate the effectiveness of our approach usingthe data set collected from real world settings over athree month period, which consists of 40 queuing exper-iments in three scenarios: dining hall, supermarket, andamusement park. Experimental results show that queu-ing behavior recognition has achieved high precision andrecall. We evaluate the energy consumption on queuingbehavior recognition of our adaptive algorithmic solu-tion. We further deploy the prototype of QueueSenseon the mobile phones of 10 people and these peopleparticipated in waiting in lines in a dining hall anda supermarket near a university campus. Experimentalresults show that its performance is acceptable for apeople-centric sensing application on mobile phones.

Briefly speaking, we make the following contributionsin this work.

1) We develop practical solutions to automaticallyrecognize queuing behaviors without manual inputand identify multiple concurrent lines.

2) We address energy consumption in queuing de-tection on mobile phones in three stages: sensing,communicating and computing. An adaptive al-gorithm is developed to adjustment collaborativesensing and inferring for queuing detection.

3) We implement a prototype application based onQueueSense on commodity phones.

4) We conduct empirical studies of QueueSense in realworld scenarios. Results show that QueueSense hasachieved high accuracy.

2 SYSTEM DESIGN CONSIDERATIONS

In this section, we discuss the technical considerationsthat underpin the design of QueueSense in two aspects:

0 20 40 60 80 1000

still

walking

still

walking

Time (s) Supermarket

Non−Queuing Queuing

Fig. 1: In a supermarket,people’s individual activitychanges over time.

Supermaket Amusement Park0

50

100

150

Av

erag

e se

rvic

e ti

me

(s)

Fig. 2: Average service timein two different queuing s-cenarios.

queuing recognition in various scenarios and energylimitations on mobile phones.

Directly using smartphones sensors to sample rawdata to recognize queuing behavior faces two challenges.Firstly, queuing is a group activity different from indi-vidual activities. When people are waiting in line, theyoften sequentially perform several simple activities suchas “standing”/“sitting” and “walking”. Even a personis not waiting in line, these activities may also occur.Therefore, it is hard to conceptually distinguish queuingfrom non-queuing purely based on individual activities.Secondly, queuing behavior of users are largely influ-enced by queuing style and environment. In differentenvironments, queuing behavior is depicted by differentcharacteristic of the activities. It leads to different waitingtime and movement distance for queuers at every serviceinterval. For instance, queuers waiting at an amusemen-t park move forward tens of meters each time, andqueuers at the checkout of a supermarket move forwardabout half of meter each time. The method of collect-ing queuing behavior in large annotated environmentsand then training generic classifiers for all the possiblemodels is simply not feasible nor, we argue, necessary.

To get a sense of the practical challenges that queuingrecognition presents to the design of QueueSense on amobile phone, we conducted a controlled experimentusing smartphones. Figure 1 shows the changes of in-dividual activities over time for one queuing user andone non-queuing user in a supermarket close to the In-formation Engineer Institute of Chinese Academy of Sci-ences (IIE-CAS). For simplicity, we use “walking/still” torepresent the changes of individual activities in queuingand non-queuing (such as eating at a table, shopping in astore, or walking at a amusement park). We observe thateven in the same queuing scenario, there is no differencein the snapshot of individual activities between queuersand non-queuers. Further, since service time may greatlyimpact queuing behavior of users, we investigated theservice time between different queuing scenarios. Figure2 shows that queuing service time in the supermarket isonly one third of that in the amusement park. We con-clude that for different queueing circumstances, changesin individual activities is inconsistent since differentscenarios have different service rate. In brief, the results

3

movement direction

displacement distance

peer-distance crossing

movement similarity

direction similarity

movement crossing

direction crossing

alignment distance

displacement deviation

Supervised Learning

Unsupervised Learning

Line 1

Line 2

Line n

Local Sensing and Inferring Collaborative Sensing and Inferring

Relative position changing rate (RPCR)

Non-Queuing

Queuing

Fig. 3: Software architecture of queue recognition

of individual activities changes during queuing clearlysuggest that individual activities is not an viable featurefor recognizing queuing behavior.

In this paper, we present a practical solution for recog-nizing queuing. Our solution is based on Kurt Lewin’s“Field Theory” [6] which states that group behavior isa function of the attributes, characteristics, and socialrelationships of all involved parties. We next presentthe general notion of a queue as the function of multi-queuers according to the overall characteristics [8]: ifthe front of a line has come to a stop, the back hasto stop too; if a person in a queue begins to moveforward, then people behind him will do the same. Thisnotion provides the insights necessary for distinguishingpeople who are queueing and who are not. Our formaldefinition of queueing is as follows.

Definition 1 (Queue Notion): Queuing is the behaviorthat people comply with the rule of first come firstserved (FCFS), where people have similar mobility andthe sequential relationship of the mobility is stable.

Another pressing challenge when designing a contin-uous sensing application such as QueueSense is thatmobile phones are resource limited. In particular, a bighurdle for context sesning is the limited battery capacityof mobile devices. In this paper, we explore energyconsumed in queuing recognition and then design aneffective algorithm to maximize energy savings whileguaranteeing accuracy of queue recognition.

3 QUEUE RECOGNITIONIn this section, we first present an overview of Queue-Sense and then discuss the detailed techniques to tacklethe challenges laid out previously, queuing recognitionon mobile phones.

3.1 OverviewQueueSense (Figure 3) consists of two stages: localsensing and inference, and collaborative sensing andinference. In the following, we describe the details ofeach stage and the interactions between them.

Local sensing and inferring. We mainly compute fourfeatures of individual activities: movement, direction,displacement, and distance. These features are used later

to extract queuing pattern features. The advantage ofusing these results instead of raw sensor data is thesignificant reduction of data that need to be commu-nicated and thus energy saved. The four individualfeatures including movement, direction, displacement,and distance are computed using multi-modality sensorsalready built in commodity mobile phones: accelerome-ter, compass, and bluetooth. As individual activity recog-nition is not the focus of this work, we defer the detailsof computing these four features to Section 5.

Collaborative sensing and inferring. We divide this stageinto two steps: queuing behavior recognition and queuepartition. i) The selection of queuing features is criticalin building a classifier. To capture the characteristic ofqueuing behavior, we use seven features derived fromindividual activities. These features are chosen becausethey are robust to various scenarios and they can beused to compute similarity between individuals. First,according to the queuing notion, queuers have similarsequential mobility. This is captured by four features in-cluding movement similarity, direction similarity, move-ment crossing rate, and direction crossing rate. Second,queuers follow the rule: FCFS. Hence, we adopt peer-distance crossing rate to capture whether they are fol-lowing the rule. Third, people in the front of a line moveforward first; and people in the back move later. Tocapture this, we use alignment distance to reflect tem-poral relationship between queuers. QueueSense adoptsa supervised learning algorithm for queuing activityrecognition. ii) After queuing behavior is recognizedon mobile phones, QueueSense adopts an unsupervisedlearning algorithm to partition queues. Specifically, weuse changing rate of relative positions as the distancemetric for hierarchical clustering.

In the rest of this section, we present the details ofqueuing activity recognition and queue partition, thecore of QueueSense.

3.2 Queuing Behavior Recognition

As shown in Figure 3, queuing behavior recognitionconsists of two steps: extraction of features that candistinguish queuing from non-queuing, and running aclassification model on the extracted features.

4

3.2.1 Queuing Feature ExtractionQueuing features are extracted from four features ofindividual activities including movement, direction, dis-placement, and distance computed by mobile phones inadvance.

a. Movement Similarity (MS). We use the activity over-lapping degree to measure movement similarity amongpeople. It is a time-domain feature that reflects twopersons’ movement similarity in a fixed time interval.Typically, queuers wait when the line stops and queuerssequentially move forwards when the line starts. Hence,queuers have similar movement. MS between persona and person b is computed as follows, assuming eachperson has a list of n movement activities si in the timeinterval.

MS(a, b) =1

n

n!

i=1

f(a, b)i (1)

wheref(a, b)i = 1 if sbi = sai (2)

MS(a, b) = 1 if and only if the movements of a and bare exactly the same in the given time interval.

b. Direction Similarity (DS). We also use activity over-lapping degree to measure direction similarity amongpeople, so DS is computed as follows in a way similarto how MS is computed, except that individual directionis typically continuous but movement is discrete.

DS(a, b) =1

n

n!

i=1

g(a, b)i, (3)

where

g(a, b)i = 1 if avg(hai )−avg(hb

i ) ≤std(ha

i ) + std(hbi )

2+G

(4)where hi is a personal direction reading, avg is theaverage and std is the standard deviation of individ-ual direction readings in the time interval, and G is aguard factor. Basically, two persons are considered tohave similar directions if the difference in their averagecompass readings is no higher than the average standarddeviations. When two persons have similar directions,they are more likely to be in the same line. This heuristicmethod is based on [9].

c. Movement Crossing Rate (MCR). Crossing rate isusually defined as the number of time-domain crossingswithin a time duration. We use it to represent the chang-ing rate of movement between people. When people arewaiting in line, they “move” forward sequentially as theline moves forward and they “stop” when the line stops.This implies that during the same time interval, thequeuers have the same movement. MCR is computedas follows:

MCR(a, b) =

"n−1i=1 |f(a, b)i − f(a, b)i+1|

n− 1(5)

where f(a, b)i is calculated using Equation (2). If twopersons a and b have the same movements in two

successive time steps, 0 is added to MCR; otherwise1 is added. Queues typically have regular intermittentmovement, namely stop to wait and then start to moveforward. If the time interval is long enough to cover thetime of the changing movement, MCR reflects at whatsimilarity degree the movements remain stable in thegiven time interval.

d. Direction Crossing Rate (DCR). Direction crossingrate reflects the stability of queuers’ directions duringqueuing. It is calculated similarly to Equation (5), exceptthat we replace MCR(a, b) with DCR(a, b), and replacethe function g using Equation (4) to determine whethertwo persons have similar directions.

e. Displacement Deviation (DD). It is observed that peo-ple who are queuing have similar displacement. We usefeature displacement deviation to capture displacementsimilarity among people. DD is computed as follows.

DD(a, b) =

"ni=1 f

′(a, b)in

(6)

where f ′(a, b)i is the difference between displacementof persons a and b at time step i. It is a time-domainfeature that reflects two persons’ average displacementdifference in a fixed time interval.

f. Peer-Distance Crossing Rate (PDCR). It is observedthat when people are queuing, their peer distancesremain stable. We use peer distance changing rate ofpeople to distinguish between queuing and non-queuingbehavior. PDCR between persons a and b is computedas follows, assuming each person has a list of n peer-distances d(a, b)i.

PDCR(a, b) =

"n−1i=1 (d(a, b)i − d(a, b)i+1)2

n− 1(7)

When people are queuing, their peer distances remainstable; thus PDCR is low.

g. Alignment Distance (AD). In time series analysis,alignment distance measures similarity between twotemporal sequences. It basically calculates the distancebetween two time-domain sequences in the fixed timeinterval by by counting the minimum number of opera-tions needed to transform one list into the other, wherean operation is defined as an insertion, a deletion, a sub-stitution, or a transposition. Similarity in two movementsequences can be represented by AD, even if one queuerin the front of the line moves forwards first and thequeuer in the back of the line move later. For queuingbehavior recognition, we compute AD for the three typesof temporal sequences: direction, movement and peer-distance. Specifically, we apply dynamic time warping(DTW) to measure AD.

3.2.2 Queuing ClassificationWe build a classification model for determining whetherpeople are queuing or not, using the seven queuingfeatures previously computed. As shown in Figure 3,the model is a binary classifier that its output takesonly two values, queuing or non-queuing. Given a data

5

LR LDA NB SVM DT0

10

20

30

40

50

60

70

80

90A

ccura

cy (

%)

Fig. 4: Performance of various clas-sification models for queuing be-havior recognition.

Fig. 5: The angle βi indicateswhether two persons have changedtheir relative position or not in twosuccessive time slots.

Fig. 6: Two straight lines (l1 and l2)have the same users at the turn, sothey are merged into one line.

point, the queuing classifier is to decide which class thepoint should be in, queuing or non-queuing. The sevenqueuing features are viewed as a 9-dimensional vector,including MS, DS, MCR, DCR, DD, PDCR, AD×3).Considering the limitations of mobile phones, a clas-sifier should only require lightweight processing andmeanwhile can work in various real world scenarios.Various machine learning algorithms [7] can be used,including logistic regression (LA), linear discriminan-t analysis (LDA), naive bayes classifier (NB), supportvector machine (SVM), and decision trees (DT). Ourexperiments show that these five classification modelsdo not have substantial performance differences (Fig. 4).We decide to use SVM since it can generate a maximummargin classifier.

3.3 Queue PartitionMultiple lines usually appear in public service areas.Even queuing behavior recognition achieves high accu-racy, inaccurate classification of queuers into differentlines will inevitably lead to inaccuracy in estimation ofqueue length and waiting time. Therefore, a fine-grainedpartitioning is required to divide queuers into the rightlines.

We apply agglomerative hierarchical clustering [7]. Itworks iteratively as follows. Two queuers with the small-est distance are joined by a branch satisfying a givendistance metric. These two queuers are then removedfrom the set of queuers, and the new branch is addedinto the set of queuers as a new queuer. We use theaverage linkage to compute the distance between a newqueuer and other remaining queuers. This process isrepeated until no queuers satisfy the distance criterion.Each cluster then represents one queue.

A key issue in using this hierarchical clustering tech-nique is the choice of an appropriate distance metric.Apparently Euclidean distance is inappropriate as it doesnot capture the disparity of queuers in different lines.We address this issue based on such an intuition. Whenqueuers are in the same line, their relative positions

would remain stable unless one of them leaves this line.This is because typically people are complying with therules of waiting in lines. When queuers are in differentlines, there is no guarantee that their relative positionsremain stable. People who are in different lines havea higher value of relative position changing rate thanpeople who are in the same line. Therefore, We usequeuers’ relative position changing rate (RPCR) as thedistance metric for clustering.

We calculate the RPCR feature as following. Let usconsider the example shown in Figure 5. Persons Aand B are at the locations marked as labels at time i,so their peer distance is |AB|. αi is the angle betweenthe vector AB and a fixed direction. At time i + 1, Bmoves to location B′, the angle becomes αi+1, the peerdistance becomes |AB′|, and |BB′| is the displacement.The difference angle βi (i.e., βi = |αi+1 − αi|) can becalculated using the Cosine theorem of triangle:

βi = arccos(|AB|2 + |AB′|2 − |BB′|2

2 · |AB| · |AB′| ) (8)

The change in the β can be used to determine changesin the relative position between A and B. We calculateRPCR two queuers A and B using β in a fixed timeinterval as follows.

RPCR(a, b) =

"n−1i=1 |βi − βi+1|

n− 1(9)

If two queuers are in the same line, their relative angleβ would remain stable and RPCR is small.

The feature RPCR used in hierarchical clustering canidentify straight lines. In practice, lines may not alwaysbe straight. They can be zigzagging or having a turn.To handle these cases, we use a merging technique.More specifically, if two straight lines have the sameusers at one end of the line and the two lines havedifferent directions, we merge them into one line (i.e.,one cluster). Figure 6 shows an example of this casewhere L1 and L2 are merged into one line. Further, whenthe coverage range of a queue is larger than the one-hop

6

TABLE 1: Energy consumption of different sensors onsmartphones.

Sensor Type EnergyAccelerometer 21 mW

Magnetometer (Compass) 31 mWGyroscope 70 mW

GPS sampling 210 mWMicrophone 180 mW

direct communication range of phones, merging linescan increase coverage range.

4 ENERGY EFFICIENCY MECHANISM

In this section, we first analyze energy consumptionof QueueSense on smartphones, then present an adap-tive scheme to reduce energy consumption. The energyefficiency scheme can be extended to general groupbehaviors recognition.

4.1 Energy Consumption Analysis of QueueSenseIn QueueSense, energy is consumed by two stages: localand collaborative recognition. Each stage has three tasks:raw data sampling, features extracting, and classificationmodel computing. Raw data captured is transformedinto features, allowing learning algorithms to recognizeeither individual activity or queue group activity. Wenext analyze energy consumption of these three tasks.

Raw data collection. In local sensing, energy con-sumed by raw data collection is impacted by two factors:sensing and sampling duration. The first factor variesby specific sensor type. Different sensors consume dif-ferent amount of energy in sensing. We have evaluat-ed power consumption using empirical power modelsconstructed from measurements in Power Tutor [10].Table 1 shows energy consumption of different sensorson smartphones. For instance, GPS costs more thanaccelerometer at an order of a magnitude. The secondfactor is sampling duty cycles, i.e., whether periodic sam-pling and sleeping, or continuous sampling. Energy sav-ings on raw data sampling can be achieved by carefullyselecting less energy consuming sensor type and theirduty cycles. Previous work such as [11] has investigatedhow to achieve energy efficiency on raw data sampling,so we can use their approach for QueueSense.

In collaborative sensing, energy consumed by rawdata collection is essentially communication cost forreceiving the individual activity list from neighbors.Similarly, there are also two factors impacting energyconsumption on raw data sampling: networking costand the time interval between two successive commu-nications. Detailed analysis of energy cost of differentcommunication methods may be found in [12]. Thefrequency of communication is an important part for en-ergy consumption of QueueSense. An adaptive schemefor adjusting intervals between communications should

0

20

40

60

8.7

42.2

4.20.5

3.2

15.2

37.4

3.10.5

En

erg

y (

mA

)

sam

pling

FFT

wav

elet

trans

form

decision

tree

mea

n & s

td

blue

toot

hwifi

queu

e fe

atur

e

SVM class

ifier

Fig. 7: The current drawn by each component in Queue-Sense.

not be limited to queuing recognition; instead, it shouldbe easily extended to other group behavior recognition.

Feature extraction. In QueueSense, the same proce-dure of feature extraction is used for local recognitionand collaborative recognition. They both adopt signalprocessing techniques to extract features for the classifi-cation model in the next stage. The energy cost dependsthe computing overhead of signal processing in terms ofsmartphone CPU usage.

Model classification. The same procedure of modelclassification is used for local recognition and collabora-tive recognition. In local recognition, the model outputsindividual activities based on features extracted fromsignal processing techniques and machine learning tools.For collaborative recognition, there are two types ofclassification models: one for queuing activity and onefor queue partition. For queuing activity recognition,the support vector machine (SVM) classifier does notconsume much energy; for queue partition, agglomera-tive hierarchical clustering is adopted to divide queuersinto different lines based on changing rate of relativepositions of queuers.

We use Power Tutor [10] to measure the current drawnby each component in QueueSense. Basically, we keepthe rest of the system and mobile phone at a nominalvoltage, measuring the current when a component isactive and inactive, then subtract the two and the resultis the current drawn by the component. Figure 7 showsthe current of each component in QueueSense duringqueuing. We observe that the first major component ofenergy consumption is individual activity recognition.Various techniques [13]–[16] have been developed tolower energy usage of this part, which is independent ofgroup behavior recognition. Second to that is communi-cation which includes propagation of individual activityfeatures to neighbors and communication of queuer datawith the remote server. This is what distinguishes ourwork from others. Therefore, we focus on the secondcomponent and design an energy efficiency mechanismthat dynamically adjusts the communication time in-terval for group behavior recognition. The last part ofenergy consumption is queuing features extraction andclassification model, which is close to 4% and has slightimpact on mobile phone usage compared with the firsttwo components.

7

4.2 Energy Efficiency Scheme

As mentioned previously, we focus on reducing energyconsumption of communication for queuing behaviorrecognition. There are two factors impacting energy con-sumption on raw data sampling: networking cost andthe time interval between two successive communica-tions. Networking cost depends on specific communica-tion methods, such as Cellular, WiFi, or Bluetooth. De-tailed analysis of energy cost of different communicationmethods may be found in [12]. In this work, instead,we focus on carefully selecting the time interval ofcommunication in order to reduce energy consumptionfor QueueSense. The larger the time interval, the moreenergy savings, however, the more delay in providingwaiting time estimation. For instance, if we use 60seconds as communication intervals, we process dataevery 60 seconds, so queue recognition would be de-layed for nearly 1 minute. This communication intervalis what distinguishes group behavior recognition fromindividual behavior recognition. Further, as pointed outby previous work [17], energy drains fast if applicationskeep smartphone in the awake state. In QueueSense,this issue boils down to the frequency of waking upthe phone for queue recognition. One naive approach isthat when customers need QueueSense, they turn it on.However, this will result in sparse queueing data, whichis essentially similar to requesting people to manuallylabel their queueing states. We, in contrast, design anadaptive approach (presented below) where the intervaltime for queuing recognition is adjusted to accommodatedynamic service time. This scheme is a unique aspecttowards energy efficient group behavior recognition thatdiffers from previous work.

We propose a delay-accuracy model to characterize theimpact of the interval time on the accuracy of queuewaiting time estimation.

ρ = Tinterval/(Terror + ξ) (10)

where Tinterval is the interval time between mobilephones communicating, Terror is the deviation betweenactual and estimated waiting time, and ξ is a compensa-tion coefficient to make sure that the denominator is notequal to zero. If the variable Tinterval is increased, Thelarger the variable ρ, the more energy savings.

Obtaining Terror, however, is non-trivial. This is be-cause that the deviation of waiting time estimation isaffected by the accuracy of local and collaborative recog-nition on mobil phones. When Terror becomes large, it isunclear whether the it is caused by the increased Tinterval

or the error of the classification model. We proposea benchmark for setting Tinterval, i.e., the service timeof queuing scenarios. Intuitively, a queuing line has aservice time, so each customer keeps the previous stateuntil the service of the previous customer is complete.Service time is used as a threshold value for the intervaltime between two persons waiting in lines. We set theTinterval value around the service time so that the mobile

1: Input: parameters of the normal distribution (µ,σ),queue line number N ;

2: Output: interval time Tinterval;3: Initialize mobile phone:4: Entry = False, Tinterval = µ;5: for (SVM.classifier ) do6: classification results for queue recognition keeps

stable7: if (Entry == False

#Count(N) < λ ) then

8: Entry = True9: Tinterval = µ - σ

10: else11: Tinterval + = λ ∗ σ12: end if13: end for

Fig. 8: An adaptive algorithm for adjusting the intervaltime of communication

phones can quickly reacts to changes in the queuingline. We adopt normal (or Gaussian) distribution toestimate the service time: X ∼ N(µ,σ), where the meanand variance are trained by the historical data of thequeuing scenarios. We broadcast the service time tomobile phones to preset Tinterval. However, the servicetime changes over time. For instance, in the supermarket,the queuing service time is dependent on the cashier andthe number of shopping items of customers.

We design an adaptive algorithm for adjustingTinterval to accommodate dynamic service times. Theprinciple is simple. Basically, when a person first entersa line, the interval time is set to be smaller than theservice time. This guarantees that the queue informa-tion is responsive to newer entries. The interval timeincreases for people already in lines. It is because thatwhen people are queuing, they have higher probabilityof queuing in the next time period. Figure 8 presentsthe pseudocode for adjusting the interval time. Mobilephones receive parameters (µ,σ), N of the queuingscenarios as inputs, then go through the adjustmentprocess before outputting Tinterval. Initially, the intervaltime is set to be the mean value and queuing state is setfalse (Line 4). SVM classifier then detects users’ queuingbehavior using the approach described in the previoussection (Line 6). If a user begins to queue, we set Tinterval

less than µ, which aims to provide quick responses tochanges in queue line information (Line 8). If the usercount of the line (Count(N)) is low, we also set smallvalue of Tinterval for prompt responses to changes inqueue line information. If a user has been queuing fora while, we increase the Tinterval by σ (Line 11). Weincrease the interval time in a heuristic manner basedon the intuition that queuing behavior would last untila user gets the service.

5 QUEUESENSE IMPLEMENTATIONWe implement a simplified prototype of QueueSense.In Figure 9a, the individual software components in

8

Bluetooth

Accelerometer

Compass Mean & Std.dev.

Regression model

Decision tree

Wavelet transform

Direction

Peer-distance

Movement

Displacement Step Count

Queue features

SVM classifier

The Mobile Phone Side The Server Side

Clustering

Relative Position Changing Rate

UPD multicast TCP

(a) System architecture of QueueSense implementation

!!!

!

(b) GUI

Fig. 9: (a) A prototype system implementation of QueueSense; (b) The queue information is shown in the GUI ona Nexus 4 Phone.

our implementation and the interactions between themare shown. On Android phones, we implement localsensing and inferring for individual activity recognition,collaborative sensing and inferring for queuing behaviorrecognition; and on the server, we implement queuepartition, which is a part of collaborative recognition.In addition, it also provides a GUI, presenting queueinformation from server to users and this informationincludes queue length and waiting time.

1) Movement. We collect raw acceleration data at therate of 30-50 Hz from the accelerometer, and store themin the buffer. Every acceleration frame (64 samples) isused to extract frequency-domain features via the FFTpackage. We only choose two types of movements: “still”and “moving”. We assume that in most queueing scenes,people are either still or moving. We use a decision tree[7] as the classifier for this purpose. We use J.48 learningalgorithm in WEKA [18] and train a tree with 7 nodesand a depth of 4 levels. We also count the number ofsteps a person has walked. An entire acceleration frameis more than one second (64 samples at 30-50 Hz rate) butthe frequency of people walking ranges from 0.6 Hz to2.5 Hz [19]. We apply discrete wavelet transform [20] toacceleration frame streaming to extract walk waveform.We can count steps by identifying the peaks of walkwaveform. if time between two consecutive local peaksis people walking frequency range and the differentbetween the maximum and minimum is larger than 1.7g,it is counted as a step.

2) Direction. Raw direction data is collected at the rateof 30-50 Hz from the compass. A compass frame (24samples) is used to calculate the average value and thestandard deviation. Using compass to compute directionis prone to large noise especially in indoor environments,so we suggest to combine multi-sensor to calibrate direc-tion data as proposed in [9], [21]. We first process thecompass readings to estimate the uses absolute direction.We then process the gyroscope readings to compute theangular changes to calibrate the readings from the com-pass, which actually are degraded by ambient magnetic

field fluctuations. If the compass data does not have thesimilar trend to the gyroscopes readings, gyroscope datais used for calibration of compass data; otherwise thecompass data is used as the direction. Also, the compassreadings are periodically used as the initial bias value ofthe gyroscope.

3) Displacement. We use the dead reckoning technol-ogy [21] to calculate a person’s moving distance. Deadreckoning is an established idea that uses the accelerom-eter and the compass to track a person’s displacement.Basically, it multiplies the number of steps a personhas walked by the step size, and then combine it withthe direction of each of these steps to infer specificdisplacement of people in every time step.

4) Peer distance. Peer-distance is a basic metric thathas been extensively applied for proximity detection.Here, we use Received Signal Strength Indicator (RSSI)of bluetooth radio to estimate peer-distance. We set upa controlled laboratory environment with four differentphones at known distances and measure their RSSI.We measure the RSSI value at 70 different pre-specifieddistances between two phones in different environments.A regression model is constructed that depicts the re-lationship between RSSI level and distance. Moreover,we estimate the uncertainty error when translating RSSIto distance. There are other location methods, such asusing GPS, WiFi, or audio. When queuing occurs out-doors, GPS location service [22] can be combined withBluetooth approach. WiFi localization, when available,is suitable for indoor environments, and it requiresfingerprint profiling [23]. The most accurate method isusing the sound with microphone on mobile phones [24].All these improvements can potentially help QueueSenseperform better in peer distance estimation for queuingrecognition.

We implement a UDP multicast client to allow mobilephones to broadcast messages to neighbors using WiFi.The exchanged message contains movement activities,direction activities, displacements and peer distances.One-hop direct communication range is sufficient to

9

0 50 100 150 200 250

still

moving

Time (s)

Movement states

Fig. 10: A queuer’s move-ment activities over time.

1 2 3 4 5 6−2

−1

0

1

2

Mag

nit

ude

of

Acc

(g)

Time (s)

Displacement

Fig. 11: A queuer’s walkcount over time.

0 50 100 150 2000

50

100

150

200

Time (s)

Av

g D

egre

e

Direction

Fig. 12: A queuer’s direc-tion over time.

−10 −5 0 5

2

4

6

8

10

12

14

RSSI (dB)

Dis

tance

(m

ete

r)

Truth distance

Estimated distance

Fig. 13: Distance estimat-ed by Bluetooth RSSI.

TABLE 2: Accuracy of local inference.

Controlled Normalmovement 90% 73%

displacement 85% 64%direction 92% 77%distance 72% 72%

communicate with neighbors, since our queuing recog-nition classifier is a local classifier that only needs infor-mation from neighbors. We implement queuing featureextraction on mobile phones, including MS, DS, MCR,DCR, DD, PDCR and AD. All feature functions arehandled at a fixed time interval (selection of the timeinterval size is discussed in Section 6). We use the SVMpackage to train the classifier to distinguish queuingbehavior from non-queuing behavior based on features.The SVM classifier runs on the local phone. It coupleswith queuing features to output the result of queuingrecognition.

To complement the local classifier, we implementqueue partition in the central server. All queuer infor-mation is sent to the server for queue partition wherea clustering algorithm in Weka [18] is used with theRPCR feature as the distance metric for clustering. Weimplement a TCP client to allow mobile phones to re-ceive the queuing information from the server via WiFi.Queue line partition can be implemented in a distributedmanner running on mobile phones. In a distributedimplementation, when the queue length is longer thanone-hop direct communication range, multi-hop commu-nication is required, issues to be addressed include timesynchronization and delay in obtaining information fromlocal propagation. The distributed implementation ofqueue partition will make QueueSense an independentthird party app running on mobile phones.

6 PERFORMANCE EVALUATION

In this section, we first evaluate the classification modelperformance of local sensing and inferring at mobilephones. We then present the detailed performance e-valuation of collaborative sensing and inferring, whichcontains two key components in QueueSense: queuingbehavior recognition and queue line partition. We reportCPU and memory footprints of different components of

QueueSense based on the prototype system and measureenergy consumption of the system. Finally, we evaluateQueueSense in two real-world scenarios: a dining halland a supermarket.

6.1 Local Inference PerformanceIn our prototype of QueueSense, we adopt four typesof individual activities using local sensing and inferringon mobile phones, including movement, direction, displace-ment, and distance. We examine the performance of localinference based on small scale supervised experiments.We discuss accuracy in two experimental setups, a con-trolled experiment and an uncontrolled experiment. Inthe controlled experiment, we ask participants to putphones at a fixed position of their body since phoneposition can greatly affect the accuracy of individual ac-tivity recognition on mobile phones; in the uncontrolledexperiments, we allow participants to decide where toput the phone. The results are based on 10 users whoannotate their actions as the ground truth for comparisonwith inference outputs. The ground truth is correlatedto the inference made by the local sensing and inferringon QueueSense. We carried out the scenario evaluationusing leave-one-out cross-validation. Table 2 summarizesthe accuracy of these four individual activity recognition.For movement, the accuracy is determined by the clas-sifier (i.e., decision tree used for movement states classi-fication); for the other three features including direction,displacement, and distance, the accuracy is defined asthe difference between the measurement inferred fromsensor readings and the ground truth.

For the movement class, the mean accuracy is closeto 90% in the controlled experiment and drops to 73%when phone position is unrestricted. For the displacementclass, its mean accuracy remains 85% in the controlledexperiment, but drops quickly to nearly 64% in the un-controlled experiments. This is because the accelerationdata of mobile phones is heavily affected by phoneposition. In the uncontrolled experiment, mobile phoneposition is a personal choice. Studies have found thatphone placement (e.g., on a hand, in pants pockets)canaffect the accuracy of activity inference. Furthermore,step counting is sometimes classified as errors becausein our approach displacement is based on countingthe different local peaks that damps the accelerometer

10

signatures that indicate running, compared to body po-sitions more rigidly affixed to the body. Figure 10 showsqueuers’ movements over time, where each status isdivided into “still” and “moving” by the decision treeclassification model. Persons’ walk waveform over timeunder the controlled environment is shown in Figure 11.

For the direction class, its mean accuracy is better incontrolled experiment than uncontrolled experiments.This is because using compass data, we measure di-rection via the built-in calibrated API and it is easilyaffected by the position of smartphones. Figure 12 showsqueuers’ directions over time, where each direction ismeasured by a compass frame.

For the distance class, its mean accuracy remainsthe same across controlled to uncontrolled experiments.For bluetooth measurement, the accuracy of the peer-distance is impacted by the environment rather thanby phone context or position. We do see an impactof the environment on distance estimation accuracy, asshown in Figure 13. Bluetooth RSSI of -10db maps to10meters in the empirical regression model, and actualmeasurements of distance range from 12m to 8m, wecalculate the uncertainty error as 0.4 at the RSSI levelof -10db. An improvement can be made when replacingBluetooth with a robust sensor such as acoustic sensor[24] or using sensor fusion for calibrating Bluetoothsignal.

The key role of local sensing and inferring is forextracting features for recognizing queuing behavior andalso dividing queuers into right lines. Since it doesrely on sensors built in commodity mobile phones,its performance inevitably is affected by the sensors.In our prototype, we utilize multi-modularity sensorsto sense and infer individual activities, which providebasic features for extracting queuing behavior features.However, built-in sensors on mobile phones have vari-ous interference problems, which impacts the accuracyof individual activity recognition, indirectly leading toerrors in queuing behavior recognition. For instance,accelerometer is sensitive to phone position on people,compass is affected by magnetic fields, and bluetoothRSSI is dependent on the surrounding environment.Calibration on acceleration data can reduce the effect ofmobile phone position. Sensor fusion such as the fusionof the compass and gyroscope data [21] can improve theaccuracy of direction. Distance estimation can be doneusing other methods such as the sound with microphoneon mobile phones [24]. These potential improvementswill help QueueSense perform better in a variety ofdifferent queuing scenarios.

6.2 Collaborative Classification PerformanceWe use the data set collected from real world settingsover a three month period. We conducted 40 queuingexperiments in three real-life scenarios: the dining hall inthe Chinese Academy of Sciences (CAS), supermarketsclose to the residence hall of CAS, and the Happy Valley

TABLE 3: Details of experiments for collecting trainingdata

Scenarios #experiments #lines #participants per lineDining hall 10 times 2 or 3 8 males

Supermarket 10 times 2 or 3 6 males and2 females

Amusement park 20 times 1 2 males and2 females

amusement park in Beijing. For each queue line, there areat least four people participating. During every queuingexperiment, participants carried their mobile phonesin fixed position of their body, such as trousers frontpocket. Participants were divided into different groupsto take queuing in batches. Table 3 provides the detailedinformation about the experiments for collecting trainingdata. The data recorded was synchronized and input intoan offline sensor replay mechanism in MATLAB for thisevaluation.

Queuing Behavior Recognition. We construct a SVMclassifier using the training data as shown in Table 3. Weare interested in how accurately people can be correctlyidentified as queuing. We measure the SVM classifier’saccuracy using F-score as applied in Pattern recognition[7]. It reaches its best value at 1 and worst score at 0.

As mentioned previously, queuing features (i.e., MS,DS, MCR, DCR, DD, PDCR, and AD) are calculated infixed time intervals. This time interval has a significantimpact on the performance of SVM classifier’s accuracy:with a small time interval, features cannot fully representqueuing characteristics; with a long time interval, delayis introduced into the classification. Figure 14 shows theimpact of window size on the accuracy of the SVMclassifier in three different scenarios: a dining hall, asupermarket, and an amusement park. With the increaseof the window size, the F-score is significantly increased.F-score reaches 0.9 when the windows length is closeto 20 seconds in the dining hall scenario, but it needsmore than 1 minute in the supermarket scenario. In theamusement park, to achieve an acceptable F-score forthe SVM classifier, the window size needs to be morethan 200 seconds. These results indicate that an appro-priate window size should be specific to the queuingenvironment, such as different places, times, or seasons.We suggest that a suitable window size should be equalto the time duration between a line stopping and aline moving again. Table 4 shows precision and recallof the SVM classifier in three queuing scenarios withthe window size chosen as suggested above. However,it is unnecessary to have a large training phase for allpossible window lengths, because automatic recognitionof queuing only needs an approximate window size. Theaverage service time of the specific queuing scenario isa good indicatin of the window size and prolongingit does not improve the accuracy of queue recognitionwhile introducing latency in providing estimated wait-ing time.

11

0 40 90 140 1900

0.2

0.4

0.6

0.8

1

Window Length (s)

F−

Sco

re M

ea

n &

Std

. D

ev.

Dining hallSupermarketAmusement Park

Fig. 14: Impact of time window sizeon accuracy of SVM classifier

0 0.2 0.4 0.6 0.8 1.00

0.2

0.4

0.6

0.8

1

RPCR feature Threshold

F−

Sco

re M

ea

n &

Std

. D

ev.

Dining hallSupermarketAmusement Park

Fig. 15: Impact of RPCR threshold onaccuracy of clustering

50 75 100 125 150 175 200 225 250 2750

0.2

0.4

0.6

0.8

1

People count

F−

sco

re

Queue lines clusteringQueuing behavior recognition

Fig. 16: Impact of people count onaccuracy of QueueSense

TABLE 4: Accuracy of queuing behavior recognition

Scenarios Precision Recall Window LengthDining hall 90% 92% 15s

Supermarket 82% 86% 35sAmusement park 91% 71% 120s

Queue Partition. We also use F-score as the measureof queue partition accuracy. As mentioned previously,we use relative position crossing rate (RPCR) as thedistance metric in the clustering technique for queuepartition. A threshold is used to determine whethertwo queuers should be joined. We study the impact ofdifferent thresholds on clustering performance in threescenarios and results are shown in Figure 15. In theseexperiments, we adopt the best window size shownin Table 4 to calculate RPCR values. When the RPCRthreshold is between 0.3 to 0.45, the clustering in thedinning hall achieves an F-score with an average of 80%with a small standard deviation. While in the amuse-ment park, the best F-score is achieved when the RPCRthreshold is close to 0.1. These results seem to indicatethat the RPCR threshold needs to vary as the queuingenvironment changes, and needs a large training phasefor each scenario. In fact, the threshold is independentof specific environment and it is the inaccurate peerdistance measurement that resulted in different RPCRthresholds in different scenarios. In this prototype, weestimate peer distance using Bluetooth signal strengththat is affected by the surrounding environment. Hence,three queuing scenarios have different threshold valuesas shown in Fig 15. If the peer-distance is measured at afiner granularity (e.g., centimeter level) using techniquessuch as BeepBeep [24], the threshold will not differ asmuch in different queuing scenarios. We believe thatpotential improvement in peer distance measurementwill avoid the necessity to train for each scenario.

Impact of People Count. Due to practical limitations,it is hard for us to recruit enough people to partici-pate in our experiments. However, the performance ofQueueSense will not degrade as people count increases.

In contrast, its performance will improve. This is becausegroup acclivities are featured by the overall charac-teristic of all participants, their physical activities andrelationships [6]. QueueSense adopts this field theoryin its design, so it will perform better if more peopleparticipate in queuing as the queuing characteristics aremore distinguished. To validate this claim, we used themulti-agent simulation toolbox in Matlab to demonstratethe impact of number of people on queuing behaviorrecognition. We set the area size as 50m x 50m. Everyminute 10 people joins the queue. There are 4 servicecounters. The entrance is the coordinate (x0 ,y0) at thebottom of the area and 4 service counters having co-ordinates (xi ,yi) are distributed at the top of area. Eachuser is modeled as an agent and starts from the entrance.Every second, At each point in time, each user takes arandom step away from his current position. When thetime of a user staying in the area exceeds 5 minutes,with 20% probability that he transitions to a queuer (i.e.,he moves to closet checkout counter directly). When theuser moves closer to one checkout counter, the First-Come-First-Serve rule is observed, so he stands at theend of the queue. We set the queuing service time as 30sand users are served one by one. We set optimal windowsize the same as service time for extracting queuingfeature and RPCR feature. As shown in Fig. 16, the F-score of queuing recognition with SVM classifier is closeto 80%, and the queue line partition has achieved nearly95%. The F-scores of queuing behavior recognition in thissimulation are lower than those using our data traces.The reason is that there are more randomly walkingpeople in the simulation and that has lowered SVMmodel classification accuracy. Queue line partition withclustering in simulation have better performance com-pared with our trace data performance. This is becauseRPCR feature is based on peer distance measurementand we use the (x,y) coordinates to calculate their dis-tance, which is more accurate than using Bluetooth inour prototype. In summary, performance of our queuerecognition algorithm does not degrade with the increaseof people count.

12

50 75 100 125 150 175 2000

0.2

0.4

0.6

0.8

1

energy (J)

CD

F

adaptive schemeservice time scheme

Fig. 17: Energy costComparison of twoschemes for the intervaltime Tinterval.

0 25% 50% 75% 100%0

0.2

0.4

0.6

0.8

1

Accuracy

CD

F

service time scheme

adaptive scheme

Fig. 18: Accuracy com-parison of two schemes .

TABLE 5: CPU usage and memory footprints.

status CPU usage Memory usageGUI only 2% 3MB

Non-queuing 9% 8.2MBQueuing 11% 8.5MB

6.3 Resource Usage

Impact of Energy Efficiency Mechanism on EnergyConsumption. We first only measure the impact of theinterval time Tinterval on energy consumption of Queue-Sense without taking into account energy consumed byother components. Then, we have performed a coarse-grained evaluation of the power consumption usingempirical power models that were constructed frommeasurements obtained using Power Tool software [10].CPU and memory usage is measured at last.

We compare two schemes for determining the intervaltime for QueueSense: one is our adaptive scheme and theother is to set the interval time to be the service time. Inthe service time scheme, we fix the time Tinterval as thethe service time. Figure 17 is the energy consumptiondistribution of networking cost of the two schemes. Itshows that our adaptive scheme achieves more than 25%energy savings compared with the service time scheme.This is because our adaptive scheme increases Tinterval

as waiting time increases. The longer a user spends onwaiting in line, the more energy savings our schemegets. We further compared the cumulative distributionfunction (CDF) of queue recognition accuracy of the twoschemes. As shown in Fig. 18, the accuracy achievedby the adaptive scheme is comparable to that achievedby the service time scheme. Combining the Figures 17and 18, we conclude that the adaptive scheme provideshigher energy efficiency for queuing recognition withoutlosing accuracy.

Further, we have performed experiments to evaluatethe impact of running QueueSense on smartphone bat-tery lifetime. As shown in Table 6, the overall energyconsumption of QueueSense has a significant impacton battery lifetime. We also want to point out thisbattery lifetime for the QueueSensing running case isconservative here, since it let QueueSense run for the

TABLE 6: Battery Duration of mobile phones.

Without QueueSense QueueSense RunningGalaxy Nexus 35 h 21 h

Moto Milestone 19 h 9 hHTC desire 27 h 20 hAirWe M19 30 h 18.5 h

entire time. In practice, this will not be necessary as auser will only need to start QueueSense when he/she isin a queue.

CPU and Memory Footprints. We measure the CPUand memory footprints of QueueSense based on theprototype system with Dalvik Debug Monitor Server(DDMS). Table 5 shows the average CPU and memo-ry usages of QueueSense. When only GUI is running,QueueSense consumes the lowest usage in CPU andmemory. The CPU usage is close to 9% when no queuingbehavior is recognized, and only reaches up to 11%when queuing is recognized. The memory usage isabout 8.2 MB during non-queuing and slightly increasesduring queuing (about 8.5MB). These results indicatethat QueueSense is appropriate as a third party app onmobile phones.

6.4 Real-World ValidationWe have conducted real-life experiments with 10 partic-ipants in two places: the dining hall and the Carrefoursupermarket close to the residence hall of CAS. In thedining hall, we conducted 10 trials during lunch ordinner time, and the service time ranged from 10 to 60seconds. In the supermarket, we conducted 3 trials, onein the morning, one in the afternoon, and one in theevening. The service time was from 20 seconds to 120seconds. In each scenario, there was one line and theuser was serviced one by one. Each user walked aroundin each testing location as usual. The only thing weasked all participants to do was to record the beginningand ending of their queuing behavior on their phonesusing the logging application we implemented. Theserecordings are needed for us to get the ground truth forcomparison. We have set the window size (Table 4) andthe RPCR threshold (Figure 15) most appropriate for thequeuing environment.

We first investigate the error in queue size estimation.We use false positive FP to denote the number of peopleQueueSense considers queueing when they are actuallynot queueing, and use false negative FN to represent thenumber of people QueueSense classifies as non-queuingwhile they are actually queueing. The error in queue sizeestimation is equal to the absolute difference betweenthe two values: | FP − FN |. Figure 19 shows the CDFof the error of queue size in both scenarios with tenparticipants queueing in each case. 60% of the queuesize estimation is accurate and 90% of the queue sizeestimation is off by less than 3 people. This result ispromising.

13

0 1 2 3 4 5 60.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Length estimation error

CD

F

SupermarketDining hall

Fig. 19: Error in queue length esti-mation

0 30 60 90 120 1800

0.2

0.4

0.6

0.8

1

Time estimation error (s)

CD

F

SupermarketDining hall

Fig. 20: Error in waiting time pre-diction

2 4 6 80

60

120

180

240

300

People Count

Seco

nds

estimated waiting timeactual waiting time

Fig. 21: Waiting time duringqueueing

We then estimate waiting time. The total time isequal to the number of people waiting multiplied bythe service time. Figure 20 plots the average error inwaiting time prediction. In dining hall, nearly 80% ofthe error is below 100 seconds; in supermarket, 60% ofthe error is below 150 seconds. Although two scenarioshave very similar queue sizes estimation as shown inFigure 19, there are differences in predicting waitingtime. It is because service time for each user is uncertainand purely relies on specific application scenarios. Thereis time deviation between estimated service time andactual service time. Furthermore, we explore the impactof people count on waiting time prediction. Figure 21depicts the actual waiting time deviation from the esti-mation when the actual number of queuers rather thanestimation is used.

It is unrealistic that all customers are willing to pre-install QueueSense on their phones. When not enoughpeople in a queue run QueueSense, the performance willdrop. There are two approaches to cope with this limita-tion. The first approach is to let QueueSense act as a soft-ware sensor on mobile phones. When customers enterthe queuing area, the queue management system sendsa broadcast message to ask mobile phones to turn on thissoftware sensor. It then continuously samples raw datajust like other hardware sensors and sends results to theserver. The second approach is to integrate an incentivemechanism to QueueSense to encourage the involvementof participants. A reward system can be used [25]. Forinstance, participants of QueueSense may be rewardedwith real-time queue information. An alternative is toattract people using a game [26]. For instance, we maysuggest an entertainment game to people waiting in line;when the game is being played, queuing informationis collected at background and queue information isthe reward. All these techniques can potentially helpQueueSense perform better in coping with the issue oflacking data.

7 RELATED WORK

There are two traditional approaches to monitor waitinglines. Video systems present an advantage as they areable to view local individual behavior and the resulting

group queuing lines behavior simultaneously, and arealso able to scale to larger groups. For instance, videocontent analysis has been used for pedestrian groupdetection and counting [27]. However, in these video-based approaches, occlusion where people block othersmay lead to detection inaccuracy. In addition, due to thelimited coverage of a camera, multiple cameras are oftendeployed to detect the whole group in crowds. The virtu-al queue system is another approach to monitor queues.A customer can leave and then come back later whenit is his turn, such as Disney’s Fastpass [28]. However,these approaches requires extra infrastructures support.In contrast, our work only relies on commodity mobilephones. Our approach can also be used to supplementthese two approaches.

More recently, Bulut et.al [29] presented LineKing,a crowdsourced line wait-time monitoring service. Theapproach utilized the WiFi or Location proximity toestimate queue lines. However, it is sensitive to peoplecount. When the count of people is large or queuingspace is small, the accuracy of this approach woulddecease quickly. In contrast, our approach relies on thesimilarity of individual activities, sensing and inferringby various sensors. As count of queuing people in-creases, the accuracy of our approach will be furtherimproved. Similar to LineKing, Wang et.al. [30] alsouse WiFi signals to track human queues. Our approach,instead, uses mobile phones to track human queues.

Monitoring location has also been shown to give in-sight into recognizing larger groups or crowds [31]. Heregroup behavior can be computed as a function of thelocation of multiple individuals and the properties of thespace in which they are located. Our work differs fromthe previous work in that we recognize queuing groupsusing a combination of of widely available sensors oncommodity phones including accelerometer, compass,and bluetooth. There is no need to localize each indi-vidual.

8 CONCLUSIONS

In this paper, we present QueueSense, a queuing activityrecognition system by only using existing sensors oncommodity mobile phones. We propose a collaborative

14

approach for recognizing queuing behavior and designan adaptive algorithm for adjusting the communicationinterval time for energy efficiency on smartphones. Ex-perimental results show that queuing behavior can berecognized with high precision and recall. In the future,in addition to address those issues mentioned in theprevious section, we plan to collect more data and testQueueSense in a larger scale.

ACKNOWLEDGMENTS

Limin Sun is corresponding author. This work was sup-ported in part by the National Natural Science Foun-dation of China (Grant No. 61472418) and StrategicPriority Research Program of the Chinese Academy ofSciencesGrant No. XDA06040101.

REFERENCES

[1] N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, andA. T. Campbell, “A survey of mobile phone sensing,” Comm. Mag.,vol. 48, no. 9, pp. 140–150, Sep. 2010.

[2] K. K. Rachuri and M. e. Musolesi, “Emotionsense: a mobilephones based adaptive platform for experimental social psychol-ogy research,” in Ubicomp, ACM, 2010, pp. 281–290.

[3] E. C. e. Larson, “Accurate and privacy preserving cough sensingusing a low-cost microphone,” in Ubicomp, ACM, 2011.

[4] Y. Jiang and K. e. Li, “Maqs: a mobile sensing system for indoorair quality,” in Ubicomp, ACM, 2011, pp. 493–494.

[5] J. Lester, T. Choudhury, and G. Borriello, “A practical approach torecognizing physical activities,” in Pervasive Computing. SpringerBerlin / Heidelberg, 2006, vol. 3968, pp. 1–16.

[6] K. Lewin and D. Cartwright, Field theory in social science: Selectedtheoretical papers. Tavistock London, 1952.

[7] C. M. Bishop and N. M. Nasrabadi, Pattern recognition and machinelearning. springer New York, 2006, vol. 1.

[8] R. Bennett, “Queues, customer characteristics and policies formanaging waiting-lines in supermarkets,” International Journal ofRetail & Distribution Management, vol. 26, no. 2, pp. 78–87, 1998.

[9] I. Constandache and X. Bao, “Did you see bob?: human localiza-tion using mobile phones,” in 16th Mobicom. ACM, pp. 149–160.

[10] M. Dong and L. Zhong, “Self-constructive high-rate system ener-gy modeling for battery-powered mobile systems,” in 9th Mobisys.ACM, 2011, pp. 335–348.

[11] D. Chu and N. D. e. Lane, “Balancing energy, latency and accuracyfor mobile sensor data classification,” in the 9th ACM Conferenceon Embedded Networked Sensor Systems. ACM, 2011, pp. 54–67.

[12] R. Friedman and e. Kogan, Alex, “On power and throughputtradeoffs of wifi and bluetooth in smartphones,” Mobile Comput-ing, IEEE Transactions on, vol. 12, no. 7, pp. 1363–1376, 2013.

[13] N. Roy, A. Misra, C. Julien, S. K. Das, and J. Biswas, “An energy-efficient quality adaptive framework for multi-modal sensor con-text recognition,” in PerCom, IEEE, 2011. IEEE, 2011, pp. 63–73.

[14] S. Nath, “Ace: exploiting correlation for energy-efficient andcontinuous context sensing,” in the 10th international conference onMobile systems, applications, and services. ACM, 2012, pp. 29–42.

[15] H. Lu, J. Yang, and L. et.al., “The jigsaw continuous sensingengine for mobile phone applications,” in Proceedings of the 8thACM Conference on Embedded Networked Sensor Systems, 2010.

[16] Z. Zhuang, K.-H. Kim, and J. P. Singh, “Improving energy effi-ciency of location sensing on smartphones,” in MobiSys, ACM,2010, pp. 315–330.

[17] A. Pathak and A. e. Jindal, “What is keeping my phone awake?:characterizing and detecting no-sleep energy bugs in smartphoneapps,” in Mobisys, ACM, 2012. ACM, 2012, pp. 267–280.

[18] I. Witten and E. Frank, Data Mining: Practical machine learning toolsand techniques. Morgan Kaufmann, 2005.

[19] M. Henriksen, H. Lund, R. Moe-Nilssen, and H. Bliddal, “Test–retest reliability of trunk accelerometric gait analysis,” Gait &posture, vol. 19, no. 3, pp. 288–297, 2004.

[20] P. Barralon, N. Vuillerme, and N. Noury, “Walk detection with akinematic sensor: Frequency and wavelet comparison,” in EMB-S’06. IEEE, 2006, pp. 1711–1714.

[21] H. Wang, S. Sen, A. Elgohary, M. Farid, M. Youssef, and R. R.Choudhury, “No need to war-drive: Unsupervised indoor local-ization,” in 10th Mobisys. ACM, 2012, pp. 197–210.

[22] N. Bulusu, J. Heidemann, and D. Estrin, “Gps-less low-cost out-door localization for very small devices,” Personal Communications,IEEE, vol. 7, no. 5, pp. 28–34, 2000.

[23] A. Haeberlen and E. e. Flannery, “Practical robust localizationover large-scale 802.11 wireless networks,” in 10th Mobicom.ACM, 2004, pp. 70–84.

[24] C. Peng, G. Shen, Y. Zhang, Y. Li, and K. Tan, “Beepbeep: a highaccuracy acoustic ranging system using cots mobile devices,” in5th Sensys. ACM, 2007, pp. 1–14.

[25] D. Yang, G. Xue, X. Fang, and J. Tang, “Crowdsourcing to smart-phones: incentive mechanism design for mobile phone sensing,”in Mobicom, ACM, 2012.

[26] J. Deng, J. Krause, and L. Fei-Fei, “Fine-grained crowdsourcing forfine-grained recognition,” in Computer Vision and Pattern Recogni-tion (CVPR), 2013 IEEE Conference on. IEEE, 2013, pp. 580–587.

[27] B. Leibe, E. Seemann, and B. Schiele, “Pedestrian detection incrowded scenes,” in CVPR 2005., vol. 1. IEEE, 2005, pp. 878–885.

[28] Disney’s fastpass. [Online]. Available: http://disneyworld.disney.go.com/guest-services/fast-pass/

[29] M. F. e. Bulut, “Lineking: Crowdsourced line wait-time estima-tion using smartphones,” in Mobile Computing, Applications, andServices. Springer, 2013, pp. 205–224.

[30] Y. Wang, J. Yang, Y. Chen, H. Liu, M. Gruteser, and R. P. Martin,“Tracking human queues using single-point signal monitoring,”Mobisys, ACM, 2014.

[31] M. B. Kjærgaard, M. Wirz, D. Roggen, and G. Troster, “MobileSensing of Pedestrian Flocks in Indoor Environments using WiFiSignals,” in Percom, IEEE, 2012, 2012.

Qiang Li received the BS degree in computerscience from Nankai University in August 2009.Currently, he is working toward the PhD de-gree in the Institute of Information EngineeringChinese Academy of Sciences and GraduatedUniversity of Chinese Academy of Sciences. Hisresearch interests revolve around Mobile com-puting and smartphone applications. He wouldlike to build prototype mobile systems based onwell-founded computational models and inter-esting smartphone applications.

Qi Han received the PhD degree in computerscience from the University of California, Irvinein 2005. Currently, she is an associate profes-sor in the Department of Electrical Engineer-ing and Computer Science, Colorado School ofMines. Her research interests include distribut-ed systems, middleware, mobile and pervasivecomputing, and dynamic data management, andcyber-physical systems. She is a member of theIEEE and the ACM.

Limin Sun received the BS, MS, and Ph.Ddegrees in College of Computer, National U-niversity of Defense Technology, in China, in1988, 1995, and 1998, respectively. Currently,he is working as a professor in the Institute ofInformation Engineering at Chinese Academyof Sciences (CAS). He is an editor of Jour-nal of Computer Science and Journal comput-er Applications. He is also a guest editor ofspecial issues in EURASIP, Journal of WirelessCommunications and Networking,and Journal of

Networks, respectively. His research interests include mobile vehiclenetworks, delay-tolerant networks, wireless sensor networks, mobile IP,and technologies for next generation of Internet.