Scaling Laws in Cognitive Science

Christopher KelloCognitive and Information Sciences

Thanks to NSF, DARPA, and the Keck Foundation

Background and Disclaimer

Cognitive Mechanics…

Fractional Order Mechanics?

Reasons for FC in Cogsci• Intrinsic Fluctuations

• Critical Branching

• Lévy-like Foraging

• Continuous-Time Random Walks

= Disabled synapse = Unblamed synapses

= Enabled synapse = Blamed synapses

1. Choose a disabled synapse

2. If , enable with probability ρ

3. Set to

B

?

?

itiV ,

~B~B

1. Choose an enabled synapse

2. If , disable with probability ρ

3. Set to

BB

Spike triggers axonal & dendritic processes

~B

Sequ

ence

dPo

isson

Poiss

on+S

TDP

Source Reservoir

6.9 6.902 6.904 6.906 6.908 6.91

x 105

0

10

20

30

40

50

60

70

80

90

100

Unit Time Interval

Sp

ike C

ou

nt

SequencePoissonPoisson+STDP

0 1 2 3 4 5

x 104Unit Time Interval

Sp

ike

Cou

nt

Sequence

Poisson

Poisson+STDP

CB on CB off

Intrinsic Fluctuations

• Neural activity is intrinsic and ever-present– Sleep, “wakeful rest”

• Behavioral activity also has intrinsic expressions– Postural sway, gait, any repetition

Lowen & Teich (1996), JASA

TN i

TN

TNTNTA

i

ii

2

21

TN i 1

Allan Factor Analyses Show Scaling Law Clustering

TTA

Intrinsic Fluctuations In Spike Trains

Intrinsic Fluctuations in LFPs

Beggs & Plenz (2003), J Neuroscience

Bursts of LFP Activity inRat Somatosensory Slice Preparations

Mazzoni et al. (2007), PLoS One

231 SSP

Burst Sizes Follow a 3/2 Inverse Scaling Law

Intrinsic Fluctuations in LFPs

Intact Leech Ganglia Dissociated Rat Hippocampus

Intrinsic Fluctuations in Speech

Am

pli

tud

e

Time

“Bucket” “Bucket” “Bucket” “Bucket”

12

9

3

6

3

0

Trial Number

Power (dB)-40

-20

0

20

40

60

Fre

qu

enc

y (K

Hz)

n n+1 n+2 n+3 n n+1 n+2 n+3

Buck Buck Buck Buck Ket Ket Ket Ket

Am

pli

tud

e

Time

“Bucket” “Bucket” “Bucket” “Bucket”

12

9

3

6

3

0

Trial Number

Power (dB)-40

-20

0

20

40

60

Fre

qu

enc

y (K

Hz)

n n+1 n+2 n+3 n n+1 n+2 n+3

Buck Buck Buck Buck Ket Ket Ket Ket

Intrinsic Fluctuations in Speech

0.15 kHz

6.05 kHz

13.15 kHz

Bucks KetsF

requ

ency

(K

Hz)

Inte

nsity

(st

anda

rdiz

ed)

Trial Number

0.15 kHz

6.05 kHz

13.15 kHz

Bucks KetsF

requ

ency

(K

Hz)

Inte

nsity

(st

anda

rdiz

ed)

Trial Number

Intrinsic Fluctuations in Speech

0.0 0.5 1.0 1.5 2.0

Alpha

0

30

60

90

120

150

Fre

qu

en

cy

M = 1.06SD = 0.26-0.85

Log f

Log

S(f)

S(f) ~ 1/fα

Scaling Laws in Brain and Behavior

• How can we model and simulate the pervasiveness of these scaling laws?

– Clustering in spike trains

– Burst distributions in local field potentials

– Fluctuations in repeated measures of behavior

Critical Branching• Critical branching is a critical point between

damped and runaway spike propagation

1~prepostc SN

1sub 1c 1super

Damped Runaway

pre

post

Spiking Network Model

PSPj,t : Ij,t = ωj

PSPk,t+τk

?, itiV

itiV ,

tjtt

titi IeVV i,

)'(',,

PSPk,t+τk

τk

Incoming PSP

Update Membrane(and floor at zero)

Crossed Threshold?(and not in refractory)

Reset Membrane

Outgoing PSPs forenabled synapses

ωkτk

ωk

LeakyIntegrate

&Fire

Neuron

Source

Sink

Rese

rvoir

Critical Branching Algorithm

= Disabled synapse = Unblamed synapses

= Enabled synapse = Blamed synapses

1. Choose a disabled synapse

2. If , enable with probability ρ

3. Set to

B

?

?

itiV ,

~B~B

1. Choose an enabled synapse

2. If , disable with probability ρ

3. Set to

BB

Spike triggers axonal & dendritic processes

~B

Critical Branching Tuning

0 1000 2000 3000 4000 5000 6000

Unit Time Interval X 10

Mea

n Lo

cal B

ranc

hing

Ra

tio

SequencePoissonPoisson+STDP

Tuning ON Tuning OFF

Spike TrainsSe

quen

ced

Poiss

onPo

isson

+STD

P

Source Reservoir

Allan Factor Results

100

101

102

103

100

101

102

Counting Time (T)

Alla

n F

acto

r A

(T)

Sequence

Poisson

Poisson+STDP

data5

data6

CB off

TN i

TN

TNTNTA

i

ii

2

21

TN i 1

TTA

Se

qu

en

ce

dP

ois

so

nP

ois

so

n+

ST

DP

Source Reservoir

Neuronal Bursts

6.9 6.902 6.904 6.906 6.908 6.91

x 105

0

10

20

30

40

50

60

70

80

90

100

Unit Time Interval

Spi

ke C

ount

SequencePoissonPoisson+STDP

Neuronal Avalanche Results

100

101

102

103

104

105

10-8

10-6

10-4

10-2

100

Size

P(S

ize)

SequencePoissonPoisson+STDPdata4data5

Simple Response Series

Predictable Cues Unpredictable Cues

Spik

e Co

unt

Sour

ceRe

serv

oir

Time

1/f Noise in Simple Responses

Response Times Response Durations

10-4

10-3

10-2

10-1

100

10-1

100

101

102

Frequency

Po

we

r

Evenly Timed CuesRandomly Timed Cues

10-4

10-3

10-2

10-1

100

10-1

100

101

102

Frequency

Po

we

r

Evenly Timed CuesRandomly Timed Cues

Memory Capacity of Spike Dynamics

0 5 10 15 20 25 300.5

0.6

0.7

0.8

0.9

1

Time Lag

% C

orre

ct

BR ~ 1BR < 1 (~0.8)BR > 1 (~1.1)Random

0.7 0.8 0.9 1 1.10.68

0.69

0.7

0.71

0.72

0.73

0.74

Branching Ratio Bias

Mea

n %

Cor

rect

*Random

Critical Branching and FC

• The critical branching algorithm produces pervasive scaling laws in its activity.

FC might serve to:

– Analyze and better understand the algorithm

– Formalize the capacity for spike computation

– Refine and optimize the algorithm

Lévy-like Foraging𝑃 (𝑙 ) 𝑙−𝜇

1<𝜇<3

Animal Foraging

𝑃 (𝑡𝑖 ) (𝑡𝑖+1 )−𝜇

𝜇 2

Memory Foraging

𝑃 (𝑡𝑖 ) (𝑡𝑖+1 )−𝜇

𝜇 2

Lévy-like Visual Search

Lévy-like Visual Search

100

101

102

103

104

105

100

101

102

103

104

105

106

Tile Size

Alla

n F

act

or V

aria

nce

Natural

Artificial

Natural

Artificial

Image

Eye

100

101

102

103

10-6

10-5

10-4

10-3

10-2

10-1

100

Saccade Length

P(S

acca

de L

eng

th)

Natural

Artificial

Lévy-like Foraging Games

.05 .15 .25 .50

-2.2

-2.1

-2

-1.9

-1.8

-1.7Number of Resources Averaged

Resource Clustering

Slo

pe

Top 20 ScoresMiddle 20 ScoresBottom 20 Scores

25 50 100 150

-2.2

-2.1

-2

-1.9

-1.8

-1.7Degree of Clustering Averaged

Resource Quantity

Top 20 ScoresMiddle 20 ScoresBottom 20 Scores

“Optimizing” Search with Levy Walks• Lévy walks with μ ~ 2 are maximally efficient

under certain assumptions

• How can these results be generalized and applied to more challenging search problems?

Continuous-Time Random WalksIn general, the CTRW probability density obeys

Mean waiting time:

Jump length variance:

Human-Robot Search Teams

• Wait times correspond to times for vertical movements

• Tradeoff between sensor accuracy and scope

• Human-controlled and algorithm-controlled search agents in virtual environments

Conclusions

• Neural and behavioral activities generally exhibit scaling laws

• Fractional calculus is a mathematics suited to scaling law phenomena

• Therefore, cognitive mechanics may be usefully formalized as fractional order mechanics

Collaborators

• Gregory Anderson• Brandon Beltz• Bryan Kerster• Jeff Rodny• Janelle Szary

• Marty Mayberry• Theo Rhodes

• John Beggs• Stefano Carpin• YangQuan Chen• Jay Holden• Guy Van Orden