모바일 추론 / 학습

연세대학교 컴퓨터과학과2006. 11. 이영설

Prediction, Expectation, and Surprise: Methods, Designs, and Study of a Deployed Traffic

Forecasting Service

Eric Horvitz, Johnson Apacible, Raman Sarin, Lin Liao

Outline

Introduction JamBayes: A Traffic Forecasting Service

Identifying Key BottlenecksCollecting a Case Library and Learning ModelsVisualization for Relaying InferencesStudy of Prediction QualityLearning Models of CompetencyRoute-Centric Alerting

Reasoning and Alerting about Surprises Surprise Forecasting

Learning and Using Models of Future SurprisesAccuracy of Models of Future Surprise

Summary and Future Work

Introduction

Developing a traffic forecasting serviceMonitoring traffic patternsPredictions about traffic forthcoming congestionsFlowing to users in mobile settingsUsed by over 2,500 people now

Predictive ModelTarget

the status and dynamics of traffic in Greater Seattle areaEvidence in learning and prediction

Incident report (The Washington Department of Transportation)the occurrence of major sporting eventsweather reportstime of daycalendar information

JamBayes: A Traffic Forecasting Service

Streaming IntelligenceServer-based learning and reasoning systems to portable devicesLearning and reasoning based on information from devices and other sources

Smartphlow Web serviceProviding users with current status and predictions about the future of traffic flow at key hotspots within the Seattle highway systemDisplaying color coded segments on major arteries to relay the speeds and densities of cars

Color : green → yellow → red → blackUsing information reported by a network of sensors operated by the Washington Department of Transportation(WDOT)JamBayes : Predictive component of Smartphlow

Identifying Key Bottlenecks

The identification of a set of hotspots (bottleneck)To focus the attention of modeling and alerting to a set of events and states that people care deeply aboutTo reduce the parameter space of learning and inference effortDeveloping an interactive tool that analyzes a large database of system-wide traffic flow data collected over many months

Collecting a Case Library and Learning Models

Collecting a case library

Visualization for Relaying Inferences

Lightweight navigation methodDepressing 1-9 dialing keys to access different regionsDepressing joystick to toggle between two levels of zoomDepressing 0 button to check constructed flyover or troublespots

VisualizationThe clock is filled with red proportional to the maximum likelihood time that the congestion will persist before it becomes a flowing thoroughfare

Study of Prediction Quality

Accuracy of predictions for time until jams will clear and will form (15 minute tolerance)

예측한 시간의 15 분 이내에 교통 상황이 예측대로 된다면 Success!

Learning Models of Competency

Reliability ModelPredict whether a base-level prediction will fail to be accurateJam 의 지속시간이나 너비 등 적은 수의 변수에만 영향을 받음

Bottleneck 1 과 11 에 대한 기본 모델의 신뢰도 모델 샘플

신뢰도 모델이 실시간으로 현재 위치에서 기본 모델의 정확도가 신뢰도 threshold 보다 낮다고 판단하면 시스템은 물음표를 보여주게 된다

Route-Centric Alerting

Means for users to set up time-dependent route-based alertingDeskflow

Provides JamBayes inferences in desktop settingsAllows for configuration of alerting on mobile deviceUsers receive audio and vibratory alerts when specific criteria are met, based on the time of day

Reasoning and Alerting about Surprises

To identify surprises, we compare the output of the marginal models with the real-time states to identify rare flows and congestion

If the likelihood of an observed open flow or congestion at a bottleneck occurs with a probability of 0.10 or less, we mark the situation as a surprising situation

Learning and Using Models of Future Surprises

Accuracy of Models of Future Surprise

Classification accuracy does not provide a valuable signal as the accuracy is invariably reported as high for the marginal model (which assumes no surprise) given the rarity of surprises

Summary and Future Work

The system has been made available within our organization and is now in active use by over 2,500 people

Our ongoing research includes investigating alternate machine learning modeling methods, such as exploring the value of boosting, and also considering extensions that explore other inference and modeling methodologies, including particle filtering, continuous time Bayesian networks, and queue-theoretic techniques.

Bayesphone: Precomputation of Context-Sensitive Policies for Inquiry and Action in Mobile

Devices

Eric Horvitz, Paul Joch, Raman Sarin, Johnson Apacible, and Muru SubramaniMicrosoft Research, One Microsoft Way

Outline

Introduction Learning Models of Interruptability and Attendance

Models of a User’s InterruptabilityModels of Meeting Attendance

Computing Expected Cost of Interruption Performing Cost-Benefit Analysis in Real Time Precomputing Ideal Interactions with Users Bayesphone Desktop and Mobile Applications Summary

Introduction

Inference and decision making with probabilistic user models may be infeasible on portable devices such as cell phones

The methods hinge on the learning of Bayesian user models for predicting whether users will attend meetings on their calendar and the cost of being interrupted by incoming calls should a meeting be attended

The precomputation of ideal decision-theoretic policies from probabilistic user model

The caching of the policies on a cell phone for decision making

Using probabilistic models to guide the handling of telephone calls

The cost of interruption VS. the cost of deferral of an incoming call

Learning Models of Interruptability and Attendance

A model that is used to infer a probability distribution over the cost of interruption of users

A model outputs the probability that users will attend meetings that appear on their electronic calendar

위 2 개의 모델로 부터 사용자가 방해받을 때 소요되는 기대값 계산

Models of a User’s Interruptability

Predictive models of the cost of interruption from evidence associated with a user’s context

Outlook application, including the time of day and day of week of the meeting, meeting duration, subject, location, organizer, the response status of the user (responded yes, responded as tentative, did not respond, or no response request was made), whether the meeting is recurrent or not, whether the time is marked as busy or free on the user’s calendar, whether the user was required or optional, the number of invitees, the organizational relationships of the invitees to the user, and the role of the user (user was the organizer versus a required or optional invitee)

수집된 데이터를 통해서 User 에게 미팅에 대한 평가를 위한 View 를 보여준다 .

A study of a model constructed from the same 559 appointments and tested on 100 hold out cases showed a classification accuracy of 0.81 for assigning interruptability

Models of Meeting Attendance

The model was trained with the same appointments as were used to train the model for the cost of interruption

The personalized attendance model generates, for previously untagged meetings, the likelihood that users will attend the meetings

A study of the accuracy on 100 cases held out for testing found that attendance was classified at an accuracy of 0.92

Computing Expected Cost of Interruption

ECI (expected cost of interruption)

: likelihood that users will attend a meeting, given evidential properties E associated with the meeting, obtained via Outlook appointment properties

: the probability that users will assign a cost Ci to the meeting, where i indexes the meeting as being either in low, medium, or high cost

: the background cost of being interrupted in the default situation S, representing the case where a user does not attend a meeting, as captured by the time of day and day of week

i

bii ScEApcEcpEApECI )())|(1( )|()|(

)|( EAp

)|( Ecp i

)(Scb

Performing Cost-Benefic Analysis in Real Time

A key piece of the decision is the cost of deferring calls from different callers

For such an assessment, we allow users to define groups of callers, based on properties of people, so as to provide a manageable set of classes

The tool allows users to create such organization-related groups as peers, direct reports, manager, position higher-up in the organizational chart, person within organization, and people identified in a user’s list of contactsCritial associates, close friends

Precomputing Ideal Interactions with Users

ECC (expected communication cost)The cost of deferral and cost of interruption for all incoming calls during a period of time(mornings, afternoons, evenings, and late night for weekdays and weekends)

t : periodfi : frequency of calls in each caller group i that has a cost of deferral lower than the cost of interruptionfj: the frequency of calls in each caller group j that has a cost of deferral higher than the cost of interruptioncdefer : the cost of deferral of each of these caller classescring : the cost of interruption of each of these caller classes

i j

ringjj

deferii tcfcftEECC )(),(

i j

jdeferii tEECIfcftEECC ))((),(

Precomputing Ideal Interactions with Users

EVI (the expected value of information)Decision-theoretic measure of the value of gathering additional information that considers the current uncertainties, the likelihood of different answers to a query for more information, and the ultimate influence of the different answers on ideal policies

Ca : the cost of asking the user before the meeting, just the ECI before the meeting begins

Attending meeting : the expected cost associated with being at the meeting

Not attending : the cost of interruption, the background cost associated with the time of day

i j k

kkjdeferii tcEcpfcftaEECC ))|(''(),,(

i j

bj

deferii tcfcftaEECC )''''(),,(

aCtaEECCEAptaEECCEAptEECCtEEVI )],,())|(1(),,()|([),(),(

Bayesphone Desktop and Mobile Applications

Desktop applicationRunning on Windows XPPerforms inference, cost-benefit analyses, and value of information precomputation of ideal real-time actions and inquiries

Application running on SmartphonesDownloads the precomputed policy file from the desktop via a device synchronization program

Summary

We have described a project highlighting the opportunity for precomputing inferences from Bayesian networks and coupling these inferences with cost-benefit policies for fielding policies for action and dialog with users on simple end-point devices like cell phones.

We reviewed the construction of probabilistic models that can infer the expected cost of interruption and the likelihood that users will attend meetings on their calendar

We are also working to extend the evidential considerations beyond meeting properties and time, to include such observations as local sensing of location, motion, and ambient acoustical signals, such as those representing a nearby conversation in progress