A general framework for complex networks

Mark ChangiziSloan-Swartz Center for Theoretical Neuroscience

Caltech

Behavioral Non-behavioral

Selected Nervous systems Ant colonies Organisms (cell-networks) Businesses Electronic circuits Computer software

Legos Furniture Buildings

Non-selected Competitive: Ecosystems World-wide web Economies Acquaintances Non-competitive: Crystals/molecules Galaxies

Taxonomy

?

(1) Behavioral complexity

(2) Structural complexity

(3) Connectivity

(4) Parcellation

Four parts to the talk

Part 1

Building behaviors

Nodes and edges not shown

Structures in the network

etc

1

2

3

Behaviors of the network

Behaviors are built out of combinations of structures

structures behaviors

electronic devices basic actions device functions

computer software instructions runs

bird vocalization syllables songs

mammalian behavior muscle actions behaviorscell genes cell types

Legos connections ---------ecosystems food chains ---------

Examples

Universal language approach

Invariant-lengthapproach

?

1

2

3

4

1

2

3

4

5

6

7

8

1

2

3

4

5

6

7

8

How is behavioral repertoiresize increased?

y = 0.8076x + 0.4157R2 = 0.7273

-0.5

0.5

1.5

2.5

-0.5 0.5 1.5 2.5Log number of song types

Log

num

ber

of s

ylla

ble

type

s

Bird vocalization

structural repertoire versus behavioral repertoire

y = 0.8014x + 2.7017R2 = 0.8419

y = 0.2668x + 2.6997R2 = 0.5911

0.5

1.5

2.5

3.5

-2.0 -1.5 -1.0 -0.5

Log index encephalization

Log

## muscles types C (n=8)

Mean #ethobehavior types E (n=8)

Prim

ates

Prob

osci

dea

Rod

entia

Peris

soda

ctyl

aLa

gom

orph

a

Art

ioda

ctyl

a

Did

elph

imor

phia

Car

nivo

ra

Cet

acea

Car

nivo

raC

hiro

pter

a

Inse

ctiv

ora

Prim

ates

Rod

entia

Lago

mor

pha

Art

ioda

ctyl

a

Mammalian behavior

None are universal languages. I.e., none are flat.

Instead, behavior length is invariant.

Log

num

ber

of b

utto

ns

y = 0.114x + 1.4105R2 = 0.8752

1

1.5

2

0 1 2 3Log length of manual

Calculators

y = 0.631x + 0.6669R2 = 0.842

1

1.5

2

1 1.25 1.5 1.75Log length of manual

Televisions

y = 0.4835x + 0.7947R2 = 0.4896

1

1.5

2

1 1.25 1.5 1.75Log length of manual

CD players

y = 0.2529x + 1.1603R2 = 0.5078

1

1.5

2

1.35 1.6 1.85 2.1Log length of manual

VCRs

Electronic user-interface languages

Changizi, 2001, 2002, 2003

y = 0.3423x + 3.6712R2 = 0.8408

3

4

5

-0.25 0.25 0.75 1.25 1.75 2.25

log # cell types

log

# ge

nes

E. coli

Yeast

C. Elegans

Drosophila

Human

Genes and cell types

Computer software also tends to have invariant length behaviors, since programs must run within a feasible amount of time.

Instead of allowing running time to increase, programmers increase the number of instructions, or lines of code, in the program. [This is why, for example, quicksort has more lines of code than bubblesort.]

Computer software too

Part 2

Building structures

Nodes and edges not shown Edges not shown

...is actually...

Structures are built out of combinations of nodes

nodes structures behaviors

electronic circuits components basic functional circuit

computer software operators instructions (or lines of code)

businesses employees basic functional groups

organisms cells basic functional cell combinations

ant colonies ants basic functional ant combinations

nervous systems neurons basic functional neuron combinations

universities faculty teaching combinations -------------

Legos piece connections -------------

ecosystems organism food chains -------------

Examples

Universal language approach

Invariant-lengthapproach

?

How is structure repertoiresize increased?

None are universal

languages.

Instead, structure length is invariant.

(Also true in competitivenetworks)

y = 0.2191x + 1.081R2 = 0.5204

0

1

2

-1.5 -1 -0.5 0

Log index of neuron encephalization

Log

# ne

uron

type

s

Neocortex: Networks of neurons

y = 0.1225x - 0.1926R2 = 0.4827

-0.5

0.5

1.5

0 2 4 6 8

log colony size

log

# of

phy

sica

l cas

tes

Ant colonies: Networks of ants

y = 0.0564x + 0.6062R2 = 0.24890

1

2

3

0 5 10 15

log # cells

log

# ce

ll ty

pes

Organisms: Networks of cells

y = 0.4262x + 0.1379R2 = 0.7488

0

1

2

0 1 2 3

Log # of components

Log

# of

com

pone

nt ty

pes Circuits: Networks of electronic components

y = 0.5512x - 0.6548R2 = 0.6952

0

1

2

3

2 3 4 5

log # students (~ log # faculty)

log

# de

part

men

ts

(log

# fa

culty

type

s)

Universities: Networks of faculty

y = 0.7089x + 0.2707R2 = 0.9043

0

1

2

3

0 1 2 3 4

Log # Lego pieces

Log

# Le

go p

iece

type

s

Legos: Networks of connectable pieces

Changiz et al., 2002

# node types versus network size

y = 0.0137x + 2.3338R2 = 0.0024

1.5

2

2.5

3

-0.5 0 0.5 1 1.5 2 2.5

~ number of neurons

~ m

ean

# ne

uron

s ac

ross

y = 0.0841x + 0.8305R2 = 0.5334

0.5

1

1.5

0 0.5 1 1.5 2 2.5

Log N

Log

# ne

uron

type

s

Neocortex: # neuron types versus brain size

Brains thus appear to have invariant length structures

Cortical modules, barrels...: Number of neurons across versus network size

Invariant-length structures: Minicolumns and modules (below)

# neuron types increases in larger nervous networks: neocortex and retina

y = 0.1592x + 0.2311R2 = 0.8964

0.5

1

1.5

4.5 5 5.5 6 6.5

log # optic f ibers

log

# re

tinal

neu

ron

type

sgoldfish

cat

human

Retina: # neuron types versus brain size

Computer software also has invariant length structures

Invariant-length structures: Lines of code

# operator types increases in larger programs:

y = 1.021x - 0.5733R2 = 0.9368

-1

1

3

5

7

0 1 2 3 4 5 6 7

log N

log

# lin

es o

f co

de

n=144

Lines of code versus program size

y = 0.3937x + 0.4352R2 = 0.8415

0

1

2

3

4

5

0 1 2 3 4 5 6 7

log Nlo

g #

oper

ator

s

n=185

# operator types versus program size

Part 3

Connectivity and network diameterfor behaviors

Behavior is combinatorial, and thus the structures must all be “close”.

And this can only be accomplished via edges,and edges are between nodes.

Edges not shown

...is actually...

Keeping structures “close” with edges

nodes edges structures behaviors

electronic circuits components wires

computer software operators program flow edges

businesses employees communication

nervous systems neurons axons/dendrites

Legos piece linksecosystems organism trophic edges

Examples

=2

* * * * *

For behavioral networks, expect...network diameter1/v, for N.

Payoff: scales up very slowly, saving wire.Cost: Behaviors are longer (roughly v times longer).

~N1

1~N1/2

2

invariant

How is node-degree increased?

Behavior not redundant, but wire too costly

Wire cost low, but diameter too high and thus

behavior increasingly redundant

y = 0.5906x + 0.0593R2 = 0.9382

0

1

2

0 0.5 1 1.5 2 2.5 3log # edges

log

N

Electronic circuits

N ~ Vgray2/3

Nsyn ~ Vgray1

Neocortex

node-degree network diameter

electronic circuits ~N0.7 1.3

neocortex ~N0.5 2

How electronic circuits and neocortex scale

Some other consequences of a node-degree increase

• Node density decreases

- neocortex: ~Vgray-1/3.

- circuits

• Wires (and somas) thicken

- neocortex: R~Vgray1/9

- circuits

• White matter disproportionately increases

- neocortex: Vwh~Vgray4/3 [disproportionate due to wire thickening]

Also…node-degree increases in larger

software

Part 4

Parcellation

The partition problem

Broadly expect that # partitions scales up disproportionately slowly as network size increases

Theory for neocortex

y = 2.0219x - 0.4448R2 = 0.9835

1.0

2.0

3.0

0.75 1.25 1.75log # areas (A)

log

# ed

ges

(G)

# edges vs # areasin sensory subnetw orks

predicted slope = 2

total # area-edges ~ A2Well-connectedness

Economical well-connectednessimplies …

y = 0.4601x + 1.1997R2 = 0.5289

y = 0.5076x + 1.2097R2 = 0.6413

0

1

2

3

-1 0 1 2log # neurons

log

extr

apol

ated

tota

l num

ber

of

area

s

expected slope = 1/2

all animals:non-monotremes (dotted):

# areas ~ N1/2

y = 0.4575x + 0.6189R2 = 0.6777

0

1

2

0.0 0.5 1.0 1.5

log # neurons

log

D

degree vs # neuronssensory areasfrom all sources

predicted slope = 1/2

area degree ~ N1/2

Parcellation also increases disproportionately slowly in other behavioral networks

y = 0.7028x - 1.3932

R2 = 0.9097

-1

0

1

2

3

4

0 1 2 3 4 5 6 7log N

log

# m

odul

es

n=77

# program modules vs program size

Computer software

y = 0.2122x + 0.1896R2 = 0.3013

0

0.5

1

1.5

1 2 3 4

Log # employeesLo

g #

divi

sion

s

n=53

# divisions vs # employees

Businesses

Probably electronic circuits too (partition problem)

Conclusions

Behavioral Non-behavioral

Selected Nervous systems Ant colonies Organisms (cell-networks) Businesses Electronic circuits Computer software

Legos Furniture Buildings

Non-selected Competitive: Ecosystems World-wide web Economies Acquaintances Non-competitive: Crystals/molecules Galaxies

Summary

1. Invariant-length structures2. Invariant-length behaviors3. Invariant network diameter (via slow increase in degree)4. Parcellation increases

1. Invariant-length structures

1. Invariant-length structures

?

?

short partition(MATRIXELREC edgelist[MATNUM], int p, int r){ float pivot; short i, j; MATRIXELREC temp; pivot=edgelist[p].weight; i=p-1; j=r+1;

while (i<j) { do j--; while (edgelist[j].weight > pivot);

do i++; while (edgelist[i].weight < pivot);

if (i<j) { temp=edgelist[j]; edgelist[j]=edgelist[i]; edgelist[i]=temp; } else return j; }}

void quickedgesort(MATRIXELREC edgelist[MATNUM], int p, int r){ int q;

if (p<r) { q=partition(edgelist,p,r); quickedgesort(edgelist,p,q); quickedgesort(edgelist,q+1,r); }}

Software code,carved at its joints

temp; r} else r return j; while partition } q } quickedgesort(MATRIXELREC (edgelist[MATNUM], p, int r) { float; short i, j; temp; pivot=edgelist[p].weight; j i=p-1; j=r+1; int = (i<j) { do j--; quickedgesort edgelist pivot i while MATRIXELREC (edgelist[j].weight >); void do; while (edgelist[i].weight < pivot); great if (i<) pivot r {temp=edgelist[j]; =edgelist[i]; ++ edgelist[i]=<) { q partition(,p,r); short (edgelist,p,); (edgelist,q+,); }} quickedgesort 1edgelist[MATNUM], int p, MATRIXELREC edgelist[j] int r) { int q; if (p

Same software code,but with nodes scrambled

The long-term grand goal:

The ability to parse complex networks so as to reveal their underlying program.