Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Neural Programs:Towards Adaptive Control in Cyber-Physical Systems

Konstantin Selyunin1, Denise Ratasich1

Ezio Bartocci1, M.A. Islam2

Scott A. Smolka2, Radu Grosu1

1Vienna University of TechnologyCyber-Physical Systems Group E182-1

Institut of Computer Engineering

2Stony Brook University, NY, USA

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Motivation

• Programs are not robust

• Case studies: neural circuit simulation & parallel parking

• Parameter synthesis: plateaus are bad for optimizations0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16

Time, s

0.030

0.028

0.026

0.024

0.022

0.020

0.018

0.016

Pote

ntia

l, V

AVAAVB

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Motivation II

This presentation:

• How to incorporate “smooth” decisions in CPS to makesystems more robust using neural circuits and GBN

• Technique to learn parameters of a model

• Application to two case studies and the relation between them

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Background

• Bayesian Networks• express probabilistic dependencies between variables• are represented as DAGs• allow compact representation using CPDs

• Gaussian Distributions• Univariate and Multivariate Gaussian distributions

• Step function vs. sigmoid

D I

G

L

S

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Background

• Bayesian Networks• express probabilistic dependencies between variables• are represented as DAGs• allow compact representation using CPDs

• Gaussian Distributions• Univariate and Multivariate Gaussian distributions

• Step function vs. sigmoid

D I

G

L

S

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Background

• Bayesian Networks• express probabilistic dependencies between variables• are represented as DAGs• allow compact representation using CPDs

• Gaussian Distributions• Univariate and Multivariate Gaussian distributions

• Step function vs. sigmoid

D I

G

L

S

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Background

• Passing random variables through conditions

if( q > 0.15) { q;}

if( p <= 3.0) { p;}

q ~ N(0 , 0.5)

p ~ N(4 , 0.7)

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Towards the nif statement

Our setting:

• Program operates random variables (RVs)

• RVs are mutually dependent Gaussians

Questions:

• How to incorporate uncertainty of making a decision andmake decisions “smooth”?

• How to avoid cutting distributions when passing a variablethrough a condition or a loop?

We propose to use nifs instead of traditional if statements.

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Towards the nif statement

Our setting:

• Program operates random variables (RVs)

• RVs are mutually dependent Gaussians

Questions:

• How to incorporate uncertainty of making a decision andmake decisions “smooth”?

• How to avoid cutting distributions when passing a variablethrough a condition or a loop?

We propose to use nifs instead of traditional if statements.

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Towards the nif statement

Our setting:

• Program operates random variables (RVs)

• RVs are mutually dependent Gaussians

Questions:

• How to incorporate uncertainty of making a decision andmake decisions “smooth”?

• How to avoid cutting distributions when passing a variablethrough a condition or a loop?

We propose to use nifs instead of traditional if statements.

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Neural if

The nif statement: nif( x # y, σ2 )

• Inequality relation {≥, >,<,≤}• Variance (represents our confidence of making a decision)

Example:

nif( x >= a, σ2) S1 else S2

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

nif( x # a, σ2 ) : Evaluation

1. Compute the difference between x, a

diff(x,a) =

x - a− ε if # is >,

x - a if # is ≥,a - x− ε if # is <,

a - x if # is ≤ .

2. Compute quantiles of the probability density function

q1(0,4)

q1(0,0.4) q2(0,0.4)q1(0, ) q2(0, )

q2(0,4)

0.0 0.0-5 0 5 -5 0 5

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

di�(x,a)

di�(1,0)

x

* **

3. Check if a random sample is within the interval

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

nif( x # a, σ2 ) : Evaluation

1. Compute the difference between x, a

diff(x,a) =

x - a− ε if # is >,

x - a if # is ≥,a - x− ε if # is <,

a - x if # is ≤ .

2. Compute quantiles of the probability density function

q1(0,4)

q1(0,0.4) q2(0,0.4)q1(0, ) q2(0, )

q2(0,4)

0.0 0.0-5 0 5 -5 0 5

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

di�(x,a)

di�(1,0)

x

* **

3. Check if a random sample is within the interval

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

nif( x # a, σ2 ) : Evaluation

1. Compute the difference between x, a

diff(x,a) =

x - a− ε if # is >,

x - a if # is ≥,a - x− ε if # is <,

a - x if # is ≤ .

2. Compute quantiles of the probability density function

q1(0,4)

q1(0,0.4) q2(0,0.4)q1(0, ) q2(0, )

q2(0,4)

0.0 0.0-5 0 5 -5 0 5

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

di�(x,a)

di�(1,0)

x

* **

3. Check if a random sample is within the interval

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

nif: Example

if( x > 0.15) { x;}

nif( x > 0.15, 0.1) { x;}

x ~ N(0 , 0.1)

x

No. o

f sam

ples

0.0 0.2-0.2-0.4 0.4

x

x0.0 0.2-0.2-0.4 0.4

0.0 0.2-0.2-0.4 0.4No

. of s

ampl

esNo

. of s

ampl

es

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Limit case σ2 → 0

• For the case with “no uncertainty” (σ2 → 0) the PDF isexpressed as the Dirac function:

• δ(x) = +∞ if x = 0 else 0•∫∞−∞ δ(x) dx = 1

• σ2 → 0 : the nif statement is equivalent to the if condition

nif( x >= a, σ2) S1 else S2

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Limit case σ2 → 0

• For the case with “no uncertainty” (σ2 → 0) the PDF isexpressed as the Dirac function:

• δ(x) = +∞ if x = 0 else 0•∫∞−∞ δ(x) dx = 1

• σ2 → 0 : the nif statement is equivalent to the if condition

nif( x >= a, σ2) S1 else S2

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

nwhile

Extension of a traditional while statement that incorporatesuncertainty

nwhile( x # a, σ2){P1}

Evaluation:

1. Compute diff(x,a), obtain quantiles q1 and q2

2. Check if a random sample is within the interval

3. If sample within the interval, execute P1 and go to 1, else exit

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

nwhile

Extension of a traditional while statement that incorporatesuncertainty

nwhile( x # a, σ2){P1}

Evaluation:

1. Compute diff(x,a), obtain quantiles q1 and q2

2. Check if a random sample is within the interval

3. If sample within the interval, execute P1 and go to 1, else exit

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Case study 1: C.elegans

C.elegans

• a 1-mm round worm

• each adult individual has exactly 302 neurons

• extensively studied in evolutional- and neurobiology

Tap withdrawal response

• apply stimulus to mechanosensory (input) neurons

• observe the behavior: forward / backward movement

Goal

• express the behavior using neural program

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Case study 1: C.elegans

C.elegans

• a 1-mm round worm

• each adult individual has exactly 302 neurons

• extensively studied in evolutional- and neurobiology

Tap withdrawal response

• apply stimulus to mechanosensory (input) neurons

• observe the behavior: forward / backward movement

Goal

• express the behavior using neural program

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Case study 1: C.elegans

C.elegans

• a 1-mm round worm

• each adult individual has exactly 302 neurons

• extensively studied in evolutional- and neurobiology

Tap withdrawal response

• apply stimulus to mechanosensory (input) neurons

• observe the behavior: forward / backward movement

Goal

• express the behavior using neural program

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Neural connections 101

V(i)

V(i) ggap

wgapV(j)

V(j)

V(i) V(j)

V(i)

E(ij)

gsyn(V(i))

wsyn

V(j)

Synaptic connection

• chemical nature

• either active or not

• synaptic weight wsyn

• use nif to model eachsynaptic connection

Gap junction connection

• instantaneous resistive connection

• linear combination of inputs

• gap junction weight wgap

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

C.elegans Tap withdrawal circuit

REV  

FWD  

2  2  

AVM  

PLM  

Why  not  as  simple  as  this?  

56  

REV  

28  

FWD  

AVB  

AVA  

2  2  

PVC  

AVD  

AVM  

PLM  

Why  gap  junc>ons  AVM-­‐AVD  and  PLM-­‐PVC?  

Why  the  synapses  and  neurons  AVD,  PVC?  

What  is  a  gap  junc>on  from  the  point  of  view  of  control?    

56  

REV  

10  28  

FWD  

4  

AVB  

AVA  

2  

1  

2  

PVC  

8  

1  AVD  

AVM  

PLM  

What  is  the  role  of  inhibitory  junc>ons  such  as  PLM-­‐AVA?  

56  

REV  

10  28  

FWD  

4  

6   2  

AVB  

10  17  

21  

AVA  4  

2  

13  

1  

2  

PVC  

8  

1  AVD  

1  3  

AVM  

PLM  

Why  is  the  en>re  circuit  so  complicated?  

Why  both  synapses  and  gap  junc>ons  between  PVC-­‐AVA?  

PLM

REV FWD

PVD ALM AVM

AVA

DVA

AVB

PVC

21

2

1

12

55 51

1

2827

9

10

12

227

28108

1

44

14

27270

AVD

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

C.elegans Tap withdrawal circuit

REV  

FWD  

2  2  

AVM  

PLM  

Why  not  as  simple  as  this?  

56  

REV  

28  

FWD  

AVB  

AVA  

2  2  

PVC  

AVD  

AVM  

PLM  

Why  gap  junc>ons  AVM-­‐AVD  and  PLM-­‐PVC?  

Why  the  synapses  and  neurons  AVD,  PVC?  

What  is  a  gap  junc>on  from  the  point  of  view  of  control?    

56  

REV  

10  28  

FWD  

4  

AVB  

AVA  

2  

1  

2  

PVC  

8  

1  AVD  

AVM  

PLM  

What  is  the  role  of  inhibitory  junc>ons  such  as  PLM-­‐AVA?  

56  

REV  

10  28  

FWD  

4  

6   2  

AVB  

10  17  

21  

AVA  4  

2  

13  

1  

2  

PVC  

8  

1  AVD  

1  3  

AVM  

PLM  

Why  is  the  en>re  circuit  so  complicated?  

Why  both  synapses  and  gap  junc>ons  between  PVC-­‐AVA?  

PLM

REV FWD

PVD ALM AVM

AVA

DVA

AVB

PVC

21

2

1

12

55 51

1

2827

9

10

12

227

28108

1

44

14

27270

AVD

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

C.elegans Tap withdrawal circuit

REV  

FWD  

2  2  

AVM  

PLM  

Why  not  as  simple  as  this?  

56  

REV  

28  

FWD  

AVB  

AVA  

2  2  

PVC  

AVD  

AVM  

PLM  

Why  gap  junc>ons  AVM-­‐AVD  and  PLM-­‐PVC?  

Why  the  synapses  and  neurons  AVD,  PVC?  

What  is  a  gap  junc>on  from  the  point  of  view  of  control?    

56  

REV  

10  28  

FWD  

4  

AVB  

AVA  

2  

1  

2  

PVC  

8  

1  AVD  

AVM  

PLM  

What  is  the  role  of  inhibitory  junc>ons  such  as  PLM-­‐AVA?  

56  

REV  

10  28  

FWD  

4  

6   2  

AVB  

10  17  

21  

AVA  4  

2  

13  

1  

2  

PVC  

8  

1  AVD  

1  3  

AVM  

PLM  

Why  is  the  en>re  circuit  so  complicated?  

Why  both  synapses  and  gap  junc>ons  between  PVC-­‐AVA?  

PLM

REV FWD

PVD ALM AVM

AVA

DVA

AVB

PVC

21

2

1

12

55 51

1

2827

9

10

12

227

28108

1

44

14

27270

AVD

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

C.elegans Tap withdrawal circuit

REV  

FWD  

2  2  

AVM  

PLM  

Why  not  as  simple  as  this?  

56  

REV  

28  

FWD  

AVB  

AVA  

2  2  

PVC  

AVD  

AVM  

PLM  

Why  gap  junc>ons  AVM-­‐AVD  and  PLM-­‐PVC?  

Why  the  synapses  and  neurons  AVD,  PVC?  

What  is  a  gap  junc>on  from  the  point  of  view  of  control?    

56  

REV  

10  28  

FWD  

4  

AVB  

AVA  

2  

1  

2  

PVC  

8  

1  AVD  

AVM  

PLM  

What  is  the  role  of  inhibitory  junc>ons  such  as  PLM-­‐AVA?  

56  

REV  

10  28  

FWD  

4  

6   2  

AVB  

10  17  

21  

AVA  4  

2  

13  

1  

2  

PVC  

8  

1  AVD  

1  3  

AVM  

PLM  

Why  is  the  en>re  circuit  so  complicated?  

Why  both  synapses  and  gap  junc>ons  between  PVC-­‐AVA?  

PLM

REV FWD

PVD ALM AVM

AVA

DVA

AVB

PVC

21

2

1

12

55 51

1

2827

9

10

12

227

28108

1

44

14

27270

AVD

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

C.elegans Tap withdrawal circuit

REV  

FWD  

2  2  

AVM  

PLM  

Why  not  as  simple  as  this?  

56  

REV  

28  

FWD  

AVB  

AVA  

2  2  

PVC  

AVD  

AVM  

PLM  

Why  gap  junc>ons  AVM-­‐AVD  and  PLM-­‐PVC?  

Why  the  synapses  and  neurons  AVD,  PVC?  

What  is  a  gap  junc>on  from  the  point  of  view  of  control?    

56  

REV  

10  28  

FWD  

4  

AVB  

AVA  

2  

1  

2  

PVC  

8  

1  AVD  

AVM  

PLM  

What  is  the  role  of  inhibitory  junc>ons  such  as  PLM-­‐AVA?  

56  

REV  

10  28  

FWD  

4  

6   2  

AVB  

10  17  

21  

AVA  4  

2  

13  

1  

2  

PVC  

8  

1  AVD  

1  3  

AVM  

PLM  

Why  is  the  en>re  circuit  so  complicated?  

Why  both  synapses  and  gap  junc>ons  between  PVC-­‐AVA?  

PLM

REV FWD

PVD ALM AVM

AVA

DVA

AVB

PVC

21

2

1

12

55 51

1

2827

9

10

12

227

28108

1

44

14

27270

AVD

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Tap Withdrawal Simulations as Neural Program

Biological Model Neural Program

dV (i)

dt=

VLeak − V (i)

R(i)m C

(i)m

+

∑Nj=1(I

(ij)syn + I

(ij)gap) + I

(i)stim

C(i)m

(1)

I(ij)gap = w

(ij)gap g

(ij)gap (Vj − Vi ) (2)

I(ij)syn = w

(ij)syn g

(ij)syn (E (ij) − V (j)) (3)

g(ij)syn (V (j)) =

gsyn

1 + eK

(V (j)−Veqj

Vrange

) (4)

1: nwhile ( t ≤ tdur , 0)

2: compute I(ij)gap using equation 2

3: nwhile ( k ≤ w(ij)syn , 0)

4: nif (V (j) ≤ Veq , K/Vrange)

5: g(ij)syn ← g

(ij)syn + gsyn

6: compute I(ij)syn using equation 3

7: compute dV (i) using equation 18: V (i) ← V (i) + dV (i)

9: t ← t + dt

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

C.elegans Tap withdrawal simulations

ODE Neural Program: One execution Neural Program: Average

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Case study 2: Parallel parking

Given:

• P3AT-SH Pioneer rover

• Carma Devkit

• ROS on Ubuntu 12.04

• Pi with Gertboard

Goal:Write a parallel parking controller as a neural program

Raspberry PI withaccelerometer and gyroscope

Carma Board runningUbuntu & ROS

Sonars

Bumbers

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Program skeleton

nwhile(currentDistance < targetLocation1 , sigma1){

moving ();

currentDistance = getPose ();

}

updateTargetLocations ();

nwhile(currentAngle < targetLocation2 , sigma2){

turning ();

currentAngle = getAngle ();

}

updateTargetLocations ();

nwhile(currentDistance < targetLocation3 , sigma3){

moving ();

currentDistance = getPose ();

}

...

Question: how to find the unknown parameters and how uncertainare we about each of them?

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Program skeleton

nwhile(currentDistance < targetLocation1 , sigma1){

moving ();

currentDistance = getPose ();

}

updateTargetLocations ();

nwhile(currentAngle < targetLocation2 , sigma2){

turning ();

currentAngle = getAngle ();

}

updateTargetLocations ();

nwhile(currentDistance < targetLocation3 , sigma3){

moving ();

currentDistance = getPose ();

}

...

Question: how to find the unknown parameters and how uncertainare we about each of them?

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning

Parking example:

• Sequence of moves and turns

• Each action depends on the previous one

• The dependence is probabilistic

• RVs are normally distributed (assumption)

Gaussian Bayesian Network:

l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning

Parking example:

• Sequence of moves and turns

• Each action depends on the previous one

• The dependence is probabilistic

• RVs are normally distributed (assumption)

Gaussian Bayesian Network:

l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Good traces

l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76

0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.61

0.5

0

0.5

Task: learn the parameters of the GBN from the good traces

1. Convert the GBN to the MGD[HG95]

2. Update the precision matrix T of the MGD[Nea03]

3. Extract σ2s and bijs from T

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Good traces

l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76

0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.61

0.5

0

0.5

Task: learn the parameters of the GBN from the good traces

1. Convert the GBN to the MGD[HG95]

2. Update the precision matrix T of the MGD[Nea03]

3. Extract σ2s and bijs from T

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Update step

• Iterative learning procedure• Incrementally update mean µ and covariance matrix β of the

prior• Mean update:

x =

∑Mh=1 x

(h)

M

µ∗ =vµ+ Mx

v + M

• Covariance matrix update:

s =M∑h=1

(x(h) − x

)(x(h) − x

)Tβ∗ = β + s +

vM

v + M

(x(h) − x

)(x(h) − x

)T(T∗)−1 ∼ β∗

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Update step

• Iterative learning procedure• Incrementally update mean µ and covariance matrix β of the

prior• Mean update:

x =

∑Mh=1 x

(h)

M

µ∗ =vµ+ Mx

v + M

• Covariance matrix update:

s =M∑h=1

(x(h) − x

)(x(h) − x

)Tβ∗ = β + s +

vM

v + M

(x(h) − x

)(x(h) − x

)T(T∗)−1 ∼ β∗

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Data

“Good trajectories”:

• *.bag files (collection of messagesthat are broadcasted in ROS)

• extract coordinates in the 2-Dspace and angle

• find important points

• obtain samples in the form:l1, α1, l2, α2, l3, α3, l4

.

.

.

.

.

.

.

.

.

1.204 -0.911 -1.221

1.207 -0.920 -1.221

1.209 -0.927 -1.221

1.211 -0.930 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.215

.

.

.

.

.

.

.

.

.

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Data

“Good trajectories”:

• *.bag files (collection of messagesthat are broadcasted in ROS)

• extract coordinates in the 2-Dspace and angle

• find important points

• obtain samples in the form:l1, α1, l2, α2, l3, α3, l4

.

.

.

.

.

.

.

.

.

1.204 -0.911 -1.221

1.207 -0.920 -1.221

1.209 -0.927 -1.221

1.211 -0.930 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.215

.

.

.

.

.

.

.

.

.

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Data

“Good trajectories”:

• *.bag files (collection of messagesthat are broadcasted in ROS)

• extract coordinates in the 2-Dspace and angle

• find important points

• obtain samples in the form:l1, α1, l2, α2, l3, α3, l4

.

.

.

.

.

.

.

.

.

1.204 -0.911 -1.221

1.207 -0.920 -1.221

1.209 -0.927 -1.221

1.211 -0.930 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.215

.

.

.

.

.

.

.

.

.

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Data

“Good trajectories”:

• *.bag files (collection of messagesthat are broadcasted in ROS)

• extract coordinates in the 2-Dspace and angle

• find important points

• obtain samples in the form:l1, α1, l2, α2, l3, α3, l4

.

.

.

.

.

.

.

.

.

1.204 -0.911 -1.221

1.207 -0.920 -1.221

1.209 -0.927 -1.221

1.211 -0.930 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.221

1.211 -0.931 -1.215

.

.

.

.

.

.

.

.

.

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Parking system architecture

Engine

nwhile ( . ) moving();nwhile ( . ) turning();...

initial motioncommands

velocitycommands

vl, vr, a, ω

x, y, θ

actualposition

resampledcommands

GBN(distributions)

Rover Interface

SensorFusion

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Conclusion and Future Work

Recap:

• use of smooth Probit distribution in conditional and loopstatements

• use of Gaussian Bayesian Network to capture dependenciesbetween Probit distributions

• Case studies: robust parking controller and tap withdrawalsimulation

Future work:

• Apply these techniques to monitoring

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

References I

David Heckerman and Dan Geiger.Learning bayesian networks: A unification for discrete andgaussian domains.In UAI, pages 274–284, 1995.

Richard E. Neapolitan.Learning Bayesian Networks.Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 2003.

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning Parameters in a Neural Program

Neural Program with Uncertain Parameters

Set of traces

Samplingfrom GBN

with learnedparameters

Learningprocedure

Learning phase

Execution phase

LearnedParameters

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Integration into ROS

• Rover Interface

• Sensor Fusion

• GBN and Engine

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Denotational Semantics

E ::= xi | c | bop(E1,E2) | uop(E1)

S ::= skip | xi := E | S1;S2 | nif( xi # c , σ2 ) S1 else S2 |nwhile( xi # c , σ2){ S1 }

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Denotational Semantics

JskipK(x) = x

Jxi := E K(x) = x[JEK(x) 7→ xi ]

J S1;S2 K(x) = JS2K(JS1K(x))

Jnif( xi # c , σ2) S1 else S2K(x) =

Jcheck(xi , a, σ2,#)K(x)JS1K(x) +

J¬check(xi , a, σ2,#)K(x)JS2K(x)

Jnwhile( xi # c, σ2){ S1 }K(x) =

xJ¬check(xi , a, σ2,# )K(x) +

Jcheck(xi , a, σ2,# )K(x)Jnwhile( xi # c , σ2){ S1 }K(JS1Kx)

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Conversion step

1. Define an ordering starting from initial node

t1 =1

σ21

bi =

bi1

...

bi,i−1

µ =

µ1

...

µn

2. Use iterative algorithm from[Heckerman and Geiger, 1995

]:

T1 = (t1);

for(i = 2; i ≤ n; i + +)

Ti =

Ti−1 + tibibTi −tibi

−tibTi ti

;

T = Tn;

l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Conversion step

1. Define an ordering starting from initial node

t1 =1

σ21

bi =

bi1

...

bi,i−1

µ =

µ1

...

µn

2. Use iterative algorithm from

[Heckerman and Geiger, 1995

]:

T1 = (t1);

for(i = 2; i ≤ n; i + +)

Ti =

Ti−1 + tibibTi −tibi

−tibTi ti

;

T = Tn;

l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Conversion step

• l1, α1:

T1 = (t1) =1

σ21

;

T2 =

1σ2

1+

b221

σ22−b21

σ22

−b21

σ22

1σ2

2

b2 =

(b21

)

l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Conversion step

• l2:

T3 =

1σ2

1+

b221

σ22

−b21

σ22

0

−b21

σ22

1σ2

2+

b232

σ23−b32

σ23

0 −b32

σ23

1σ2

3

b3 =

0

b21

l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Conversion step

• α2:

T4 =

1σ2

1+

b221

σ22

−b21

σ22

0 0

−b21

σ22

1σ2

2+

b232

σ23

−b32

σ23

0

0 −b32

σ23

1σ2

3+

b243

σ24−b43

σ24

0 0 −b43

σ24

1σ2

4

b4 =

0

0

b43

l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76

Motivation Background Neural Code C.elegans tap withdrawal simulation Parallel parking Conclusion

Learning: Conversion step

• l3:

T5 =

1σ2

1+

b221

σ22

−b21

σ22

0 0 0

−b21

σ22

1σ2

2+

b232

σ23

−b32

σ23

0 0

0 −b32

σ23

1σ2

3+

b243

σ24

−b43

σ24

0

0 0 −b43

σ24

1σ2

4+

b254

σ25−b54

σ25

0 0 0 −b54

σ25

1σ2

5

b5 =

0

0

0

b54

We can generalize T for arbitrary number of moves

l1 α1 l2 α2 l3 α3 l4b21 b32 b43 b54 b65 b76