September 7, 2005

Massachusetts Institute of Technology

A Tractable Approach to Probabilistically Accurate Mode

Estimation

Oliver B. MartinSeung H. ChungBrian C. Williams

i-SAIRAS 2005Munich, Germany

2

ClosedClosed

ValveValve

OpenOpenStuckStuckopenopen

StuckStuckclosedclosed

OpenOpen CloseClose

0. 010. 01

0. 010. 01

0.010.01

0.010.01

inflow = outflow = 0

3

Estimation of PCCA(Probabilistic Concurrent Constraint Automata)

• Belief State Evolution visualized with a Trellis Diagram

• Complete history knowledge is captured in a single belief state by exploiting the Markov property

• Belief states are computed using the HMM belief state update equations

Compute the belief state for each estimation cycle

closed open stuck-open

cmd=openobs=flow

cmd=closedobs=no-flow

0.7

0.2

0.1

4

Hidden Markov ModelBelief State Update Equations

• Propagation step computes aprior probability

• Update step computes posterior probability

0, 0, 0, 0, 11 1, , ,t ti

t t t tt t t t tj j i is Ss o s s s o

P P P

1 1

0, 0,1 1 1

0, 1 0,1

0, 0,1 1 1

,,

,t ti

t tt t tj jt tt

jt tt t t

i is S

s o o ss o

s o o s

P PP

P P

t t+1

5

Approximations to PCCA Estimation

• k most likely estimates – The belief state can be accurately approximated by maintain the k most likely estimates

• Most likely trajectory – The probability of each state can be accurately approximated by the most likely trajectory to that state

• 1 or 0 observation probability – The observation probability can be reduced to 1.0 for observations consistent with the state, and 0.0 for observations inconsistent with the state

t+11.0 or 0.0

6

Previous Estimation Approach:Best-First Trajectory Estimation (BFTE)

• Livingstone– Williams and Nayak, AAAI-96

• Livingstone 2– Kurien and Nayak, AAAI-00

• Titan – Williams et al., IEEE ’03

• The belief state can be accurately approximated by maintain the k most likely estimates

• The probability of each state can be accurately approximated by the most likely trajectory to that state

• The observation probability can be reduced to– 1.0 for observations consistent

with the state,– 0.0 for observations inconsistent

with the state

0.50

0.15

0.35

Approximations to Estimation

Deep Space One Earth Observing One

7

Previous Estimation Approach: Best-First Belief State Enumeration (BFBSE)

• Diagnosis as Approximate Belief State Enumeration for Probabilistic Concurrent Constraint Automata– Martin et al., AAAI-05

• Increased estimator accuracy by maintaining a compact belief state encoding

• Minimal overhead over BFTE

• The belief state can be accurately approximated by maintain the k most likely estimates

• The probability of each state can be accurately approximated by the most likely trajectory to that state

• The observation probability can be reduced to– 1.0 for observations consistent

with the state,– 0.0 for observations inconsistent

with the state

Approximations to Estimation

0.65

0.05

0.30

8

New Approach: Best-First Belief State Update (BFBSU)

• Increased estimator accuracy by properly computing the observation probability

• Minimal overhead over BFBSE

• The belief state can be accurately approximated by maintain the k most likely estimates

• The probability of each state can be accurately approximated by the most likely trajectory to that state

• The observation probability can be reduced to– 1.0 for observations consistent

with the state,– 0.0 for observations inconsistent

with the state

Approximations to Estimation

0.65

0.30

0.05

1 1

1 1 1 1

1 if M ,

0 if M ,

1/ otherwise.

t tj

t t t tj j

s o

o s s o

m

P

‘

‘

9

BFBSE vs. BFBSU: The Consequence ofIncorrect Observation Probability

Sensor = High

Working

Broken

Observe (Sensor = High) at every time step

Assume: P(Sensor = High | Broken) = 1 P(Sensor = High | Broken) = 0.5

0.990

0.010

1

0

0

1

0.001

0.999

0.995

0.005

1

0

0.990

0.010

0.991

0.009

0.99

0.01

1

Best-First Belief State Estimation Best-First Belief State Update

Working State Estimate Probability

0

0.2

0.4

0.6

0.8

1

0 50 100 150 200 250 300 350 400 450 500

Estimation TimestepSta

te e

stim

ate

d p

roba

bili

ty

BFBSE

BFBSU

10

BSBSU ComputesHMM Belief State Update Equations

• Propagation Step

• Update Step

• Problem: Computing the observation probability for the “otherwise” case can be expensive.– Must compute total

number of consistentobservations for k states

– Model counting for kproblems

0, 0, 0, 0, 11 1, , ,t ti

t t t tt t t t tj j i is Ss o s s s o

P P P

1 1

0, 0,1 1 1

0, 1 0,1

0, 0,1 1 1

,,

,t ti

t tt t tj jt tt

jt tt t t

i is S

s o o ss o

s o o s

P PP

P P

1 1

1 1 1 1

1 if M ,

0 if M ,

1/ otherwise.

t tj

t t t tj j

s o

o s s o

m

P

‘

‘

11

Solution:Pre-Compute Observation Probability Rules

• Observation Probability Rule (OPR):– Compute OPR for a set of partial states– Compute OPRs using the dissents

– Ignore all OPRs with probability of 0 or 1

x P o x

M Q is inconsistent o x o x

number of inconsistent observationsi

iom o

o

D

1

mP o x

Model Max # of OPRs # OPRs Required Online

EO-1 1.77×108 64

MarsEDL 1.46×106 307

ST7-A 1.44×104 8

12

Solve Mode Estimation as anOptimal Constraint Satisfaction Problem

• Use Conflict-directed A* with a tight admissible heuristic– Use greedy approximation for the cost to go (BFBSE)– Include observation probability within the heuristic

(BFBSU)

1

1

1

1 1

0, 0, 1

, ,

max , ,

,

tg g

t t ti h h h h

t t t tg g g g ix v n

t t t t th h h h is S x v n v x

t tti

x v x v s

f n x v x v s o n

s o

D

P

P P

P

13

ST7A Model Results:State Estimate Accuracy

• Single-point Failure state estimate probability

• Nominal state estimate probability

14

ST7A Model Results:State Estimate Performance

• Space Performance– Best case: n·b

– Worst case: bn

• Time Performance– Best case: n2·b·k + n·b·C

– Worst case:• BFBSE: bn(n·k + C)• BFBSU: bn(n·k + bn·R + C)

0 5 10 15 20 25 3010

20

30

40

50

60

70

Ma

x n

um

ber

of

nod

es

in q

ue

ue

Estimate Enumeration Maximum Queue Size for ST7A Model

BFBSE, single estimateBFBSE, k estimatesBFBSU, single estimateBFBSU, k estimates

0 5 10 15 20 25 300

10

20

30

40

50

60

70

80

Tim

e (

ms)

Estimate Enumeration Time for ST7A Model

BFBSE, single estimateBFBSE, k estimatesBFBSU, single estimateBFBSU, k estimates

Number of initial states, k Number of initial states, k

b Average number of modesn Number of componentsk Number of estimates being trackedC Constant factor for data manipulationR OPR lookup time

15

EO1 Model Results:State Estimate Performance

• Space Performance • Time Performance

0 5 10 15 20 25 300

50

100

150

200

250

300

350

400

450

Number of initial states,k

Max

num

ber

of

nodes

in q

ueue

Estimate Enumeration Maximum Queue Size for EO1 Model

BFBSE, single estimateBFBSE, k estimatesBFBSU, single estimateBFBSU, k estimates

0 5 10 15 20 25 300

50

100

150

200

250

300

350

400

450

Number of initial states,k

Tim

e (

ms)

Estimate Enumeration Time for EO1 Model

BFBSE, single estimateBFBSE, k estimatesBFBSU, single estimateBFBSU, k estimates

16

Conclusion

• Best-First Belief State Update– Compute k most likely estimates– Remove most likely trajectory approximation by computing

the most likely belief state estimates – Remove 1 or 0 observation probability approximation by

computing the proper observation probabilities

• Increased PCCA estimator accuracy by computing the Optimal Constraint Satisfaction Problem (OCSP) utility function directly from the HMM propagation and update equations

• Maintaining the computational efficiency.