new science of complexity?
DESCRIPTION
New science of complexity?. Much attention has been given to “complex adaptive systems” in the last decade. Popularization of not just information and entropy, but phase transitions, criticality, fractals, self-similarity, power laws, chaos, emergence, self-organization, etc. - PowerPoint PPT PresentationTRANSCRIPT
New science of complexity?
• Much attention has been given to “complex adaptive systems” in the last decade.
• Popularization of not just information and entropy, but phase transitions, criticality, fractals, self-similarity, power laws, chaos, emergence, self-organization, etc.
• Aim is to find universal characteristics of complexity.• From Physics: an emphasis on emergent complexity
via self-organization of a homogeneous substrate near a critical or bifurcation point (SOC/EOC).
Highly Optimized
Tolerance (HOT)
• Complex systems in biology, ecology, technology, sociology, economics, …
• are driven by design or evolution to high-performance states which are also tolerant to uncertainty in the environment and components.
• This leads to specialized, modular, hierarchical structures, often with enormous “hidden” complexity,
• with new sensitivities to unknown or neglected perturbations and design flaws.
• “Robust, yet fragile!”
“Robust, yet fragile”
• Robust to uncertainties – that are common,– the system was designed for, or – has evolved to handle,
• …yet fragile otherwise
• This is the most important feature of complex systems (the essence of HOT).
HOT features of ecosystems • Organisms are constantly challenged by environmental
uncertainties,• And have evolved a diversity of mechanisms to minimize
the consequences by exploiting the regularities in the uncertainty.
• The resulting specialization, modularity, structure, and redundancy leads to high densities and high throughputs,
• But increased sensitivity to novel perturbations not included in evolutionary history.
• Robust, yet fragile!• Complex engineering systems are
similar.
Example: Auto airbags
• Reduces risk in high-speed collisions
• Increases risk otherwise• Increases risk to small
occupants• Mitigated by new
designs with greater complexity
• Could just get a heavier vehicle
• Reduces risk without the increase!
• But shifts it elsewhere: occupants of other vehicles, pollution
Complexity
Robustness
Square site percolation or simplified “forest fire” model.
The simplest possible spatial model of HOT.
Carlson and Doyle,PRE, Aug. 1999
empty square lattice occupied sites
connectednot connected clusters
A “spark” that hits an empty site does
nothing.
Assume one “spark” hits the
lattice at a single site.
A “spark” that hits a cluster causes
loss of that cluster.
Yield = the density after one spark
yield density loss
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Y =(avg.)yield
= density
“critical point”
N=100
no sparks
sparks
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
limit N
“critical point”Y =(avg.)yield
= density
c = .5927
Cold
Fires don’t matter.
Y
Y
Burned
Everything burns.
Critical point
Y
critical
point c
Percolation
P( ) =
probability a site is on the cluster
P( )
0
1
1
no cluster
all sites occupied
Y = ( 1-P() ) + P() ( -P() )
P()
c
miss cluster
fulldensity
hit cluster
lose cluster
= - P()2
Y
100
101
102
103
104
10-1
100
101
102
Power laws
Criticality
cluster size
cumulativefrequency
Averagecumulativedistributions
clusters
fires
size
Power laws: only at the critical point
low density
high density
cluster size
cumulativefrequency
yiel
d
density
Tremendous attention has been given to this point.
Tremendous attention has been given to this point.
Edge-of-chaos, criticality, self-organized criticality
(EOC/SOC)
Claim: Life, networks, the
brain, the universe and everything are at
“criticality” or the “edge of chaos.”
yield
density
Edge-of-chaos, criticality, self-organized criticality
(EOC/SOC)
yieldyield
density
Essential claims:
• Nature is adequately described by generic configurations (with generic sensitivity).
• Interesting phenomena is at criticality (or near a bifurcation).
Forest Fires: An Example of Self-Organized Critical BehaviorBruce D. Malamud, Gleb Morein, Donald L. Turcotte
18 Sep 1998
4 data sets
10-6
10-4
10-2
100
102
100
102
104
106
Size of events
Cumulative
Frequency
Forest fires1000 km2
(Malamud)
WWW filesMbytes
(Crovella)-1
-1/2
10-6
10-4
10-2
100
102
100
102
104
106
FF
WWW
SOC FF .15
Exponents are way off.
-1/2
Size of events
Cumulative
Frequency
10-2
10-1
100
101
102
103
104
100
101
102
103
SOC FF
Additional 3 data sets
-1/2
What about high yield
configurations?
?
Forget random, generic
configurations.
Would you design a system this way?
Barriers
Barriers
What about high yield
configurations?
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
• Rare, nongeneric, measure zero.• Structured, stylized configurations.• Essentially ignored in stat. physics.• Ubiquitous in
• engineering• biology• geophysical phenomena?
What about high yield
configurations?
critical
Cold
Highly Optimized Tolerance (HOT)
Burned
Why power laws?Why power laws?
Almost any distribution
of sparks
OptimizeYield
Power law distribution
of events
both analytic and numerical results.
Special casesSpecial cases
Singleton(a priori
known spark)
Uniformspark
OptimizeYield
Uniformgrid
OptimizeYield
No fires
Special casesSpecial cases
No fires
Uniformgrid
In both cases, yields 1 as N .
Generally….Generally….
1. Gaussian2. Exponential3. Power law4. ….
OptimizeYield
Power law distribution
of events
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
random
“optimized”
density
yield
NumericalExample32x32 grid
x1=2.^(- ((1+ (1:n)/n)/.3 ).^2 );x2=2.^(- ((.5+ (1:n)/n)/.2 ).^2 );
Probability distribution (tail of normal)
nnyx
yp
xp
ypxpyxp
y
x
/),...,2,1(,
2)(
2)(
)()(),(
2
2
2./)5(.
3./)1(
5 10 15 20 25 30
5
10
15
20
25
30
0.1902 2.9529e-016
2.8655e-011 4.4486e-026
Probability distribution (tail of normal)
High probability region
• We expect barriers concentrated in the upper left.• Computing the global optimal is difficult, since
the number of configurations is
• But practically anything you do…• …gives high yields and power laws…• …at almost all densities.
102432 222
Grid design: optimize the position of “cuts.”
cuts = empty sites in an otherwise fully occupied lattice.
Compute the global optimum for this constraint.
Optimized grid
density = 0.8496yield = 0.7752
Small events likely
large events are unlikely
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
random
grid
High yields.
Optimized grid
density = 0.8496yield = 0.7752
“grow” one site at a time to maximize incremental (local) yield
“Evolutionary”algorithm
density= 0.8yield = 0.8
“grow” one site at a time to maximize incremental (local) yield
density= 0.9yield = 0.9
“grow” one site at a time to maximize incremental (local) yield
Optimal “evolved”density= 0.97yield = 0.96
“grow” one site at a time to maximize incremental (local) yield
Optimal “evolved”density= 0.9678yield = 0.9625
Several small events
A very large event.
Optimal “evolved”density= 0.9678yield = 0.9625Small events likely
large events are unlikely
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
random
“optimized”
density
High yields.
At almost all densities.
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
random
density
Very sharp “phase transition.”
Robust, yet fragile?
Robust, yet fragile?
Extreme robustness and extreme hypersensitivity.
Small flaws
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
If probability of sparks changes.
disaster
“Robust, yet fragile”
• Robust to uncertainties – that are common,– the system was designed for, or – has evolved to handle,
• …yet fragile otherwise
• This is the most important feature of complex systems (the essence of HOT).
Conserved?
Sensitivity to:
sparks
flawsassumed
p(i,j)
Can eliminate sensitivity to: assumed
p(i,j)
Uniformgrid
Eliminates power laws as well.
Can reduce impact of: flaws
Thick open barriers.
There is a yield penalty.
100
101
102
103
10-4
10-3
10-2
10-1
100
grid
“evolved”
“critical”
size
Cum.Prob.
All producePower laws
100
10110
210
310
-12
10-10
10-8
10-6
10-4
10-2
100Evolution
.9
.8
.7
optimaldensity= 0.9678yield = 0.9625
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
random
density
yield
This source of power laws is quite universal.
Almost any distribution
of sparks
OptimizeYield
Power law distribution
of events
10-6
10-4
10-2
100
102
100
102
104
106
Size of events
Cumulative
Frequency
Forest fires1000 km2
(Malamud)
WWW filesMbytes
(Crovella)
Data compression
(Huffman)
-1-1/2
Universal network behavior?
demand
throughputCongestion
induced “phase
transition.”
Similar for:• Power grid?• Freeway traffic?• Gene regulation?• Ecosystems?• Finance?
random networks
log(thru-put)
log(demand)
Networks Making a “random network:”• Remove protocols
– No IP routing
– No TCP congestion control
• Broadcast everything
Many orders of magnitude slower
BroadcastNetwork
Networks
random networks
real networks
HOTlog(thru-put)
log(demand)
BroadcastNetwork
random
designed
HOTYield,flow, …
Densities, pressure,…
The yield/density curve predicted using random ensembles is way off.
Similar for:• Power grid• Freeway traffic• Gene regulation• Ecosystems• Turbulence• Finance?
10-6
10-4
10-2
100
102
100
102
104
106
Size of events
Cumulative
Frequency
Forest fires1000 km2
(Malamud)
WWW filesMbytes
(Crovella)
Data compression
(Huffman)
-1-1/2
document
split into N files to minimize download time
A toy website model(= 1-d grid HOT design)
Forest fires?
Burnt regions are 2-d
Fire suppression mechanisms must stop a 1-d front.
pi = Probability
of event
drl
li = volume enclosed
ri = barrier density
d-dimensional
i
d
id
i lrl
1
,
Resource/loss relationship:
PLR optimization
1)(
r
crl
RrlpJ iiiMinimize
expected loss
P: uncertain events with probabilities pi L: with
loss li
R: limited resources ri
Resource/loss relationship
d
1)(
r
crl
RrlpJ iii
PLR optimization
d
1)( crrl
0)1( l Set amplitudes and small scale cutoff
1)(
r
crl
RrlpJ iii
PLR optimization
d
= 0 data compression = 1 web layout = 2 forest fires
= “dimension”
RrlpJ iii
PLR optimization
= 0 data compression
=0 is Shannon
source coding
0
0
1
)log(
)(
r
c
r
rl
0
0)log()log(
1
1
1
ii
iiii
pRc
ppRpp
J
p
01
0)log()(
r
cr
rl
1
)1/(1
)1/(1
j
j
ii
p
Rpcl
RrlpJ iii
11
ipcR
Minimize average cost using standard Lagrange multipliers
With optimal cost
Leads to optimal solutions for resource allocations and the relationship between the event probabilities and sizes.
01
0)log()(
r
cr
rl
1
)1/(1
)1/(1
j
j
ii
p
Rpcl
RrlpJ iii
11
ipcR
Minimize average cost using standard Lagrange multipliers
With optimal cost
Leads to optimal solutions for resource allocations and the relationship between the event probabilities and sizes.
011
0)log(
1
1
1
i
ii
p
pp
J
)/11(21ˆ clcp ii
1ˆˆ
iiik
ii llpP
1 ii llorder
ii Pl ˆ,plot
sizesfromdata
computeusingmodel
1
)1/(1
)1/(1
j
j
ii
p
Rpcl
Solve for p(l) gives
To compare with data.
Cumulative
10-6
10-4
10-2
100
102
100
102
104
106
HOT FF
HOT WWW
DC
10-6
10-4
10-2
100
102
100
102
104
106
HOT FF
HOT WWW
DC
SOC FF
10-6
10-4
10-2
100
102
100
101
102
103
104
105
106
1
4
3
3
4
4
5
5
4
HOT
SOC
HOT
SOC
d=1
d
d=1
d
10-6
10-4
10-2
100
102
100
102
104
106
HOT FF
HOT WWW
DC
SOC FF
Why are the HOT PRL predictions so good?
DC and WWW are prescriptive.
Forest Fires: An Example of Self-Organized Critical BehaviorBruce D. Malamud, Gleb Morein, Donald L. Turcotte
18 Sep 1998
4 data sets
10-2
10-1
100
101
102
103
104
100
101
102
103
SOC FF
HOT FF = 2
Additional 3 data sets
Forest fires?
Burnt regions are 2-d
Fire suppression mechanisms must stop a 1-d front.
Forest fires?
Geography could make <2.
California geography:further irresponsible speculation
• Rugged terrain, mountains, deserts• Fractal dimension 1?
100
101
102
103
104
105
106
100
101
102
103
104
105
= 1
California brushfires
d = 1
SOC FF
d = 2
Characteristic Critical HOT
Yield/density Low High
Robustness Generic Robust, yet fragile
Events/structure Generic, fractal Structured, modular
External behavior Complex Nominally simpleInternally Simple Complex
Power laws at criticality everywhere
Details Don’t matter Do matter
Characteristic Critical HOT
Yield/density Low High
Robustness Generic Robust, yet fragile
Events/structure Generic, fractal Structured, modular
External behavior Complex Nominally simpleInternally Simple Complex
Power laws at criticality everywhere
Details Don’t matter Do matter
Characteristics
Toy models
?• Power systems• Computers• Internet• Software • Ecosystems • Extinction• Turbulence
Examples/Applications
Characteristic Critical HOT
Yield/density Low High
Robustness Generic Robust, yet fragile
Events/structure Generic, fractal Structured, modular
External behavior Complex Nominally simpleInternally Simple Complex
Power laws at criticality everywhere
Details Don’t matter Do matter
• Power systems• Computers• Internet• Software • Ecosystems • Extinction• Turbulence
But when we look in detail at any of these examples...
…they have all the HOT features...
Robust
ComplexityRobustness
Aim: simplest possible story
Humans have exceptionally robust systems for vision and speech.
Yet fragile…but we’re not so good at surviving, say, large meteor impacts.
Yet fragile…but we’re not so good at surviving, say, large meteor impacts.
Robustness and uncertainty
Error,sensitivity
Types of uncertainty
Sensitive
Robust
Robustness and uncertainty
Error,sensitivity
Types of uncertainty
speech/vision
Meteor impact
Humans
Archaebacteria
Sensitive
Robust
Robust
yet fragile
Error,sensitivity
Types of uncertainty
speech/vision
Meteor impact
Humans
Archaebacteria
Sensitive
Robust
Robustness and uncertainty
Robust
yet fragile
Error,sensitivity
Types of uncertainty
“Complex” systems
Sensitive
Robust
Types of uncertainty
Automobile air bags
Error,sensitivity
high speedhead-oncollisions
children low-speed
Types of uncertainty
Sensitive details
Robust statistics
Multiscale modeling: Homogeneous systems
Sensitive
Error,sensitivity
Robust
Types of uncertainty
Sensitive details
Robust statistics
Multiscale, homogeneous example
Sensitive
Error,sensitivity
Robust
initial conditions
Gas molecules in this room
Temperature and pressure
StatisticalMechanics
Gas molecules in this room
Temperature and pressure
StatisticalMechanics
Types of uncertainty
Sensitive details
Robust statistics
Multiscale, homogeneous example
Sensitive
Error,sensitivity
Robust
initial conditions
The macroscopic statistics are uniformly and robustly
independent...
...of the microscopic details...
and can be obtainedby taking ensemble
averages
Robust
yet fragile
Error,sensitivity
Types of uncertainty
“Complex” systems
Sensitive
Robust
The macroscopic behavior has both extreme robustness
and hypersensitivity...
…to the microscopic details...
…and ensemble averages are of limited usefulness.
Robust
yet fragile
Error,sensitivity
Types of uncertainty
“Complex” systems
Sensitive
Robust
• Thus we are (forced into) redoing statistical physics from scratch for these systems. • The HOT results are among the first outcomes.•These are just steps toward...
Robust
yet fragile
Error,sensitivity
Types of uncertainty
“Complex” systems
Sensitive
RobustThe ultimate goal of
modeling, simulation, and analysis is to create this
plot for specific systems.
Types of uncertainty
Modeling “complex” systems
Required model
complexityError,sensitivity
Sensitive
Robust
May need great detail here
And much less detail here.
Error,sensitivity
Types of uncertainty
Why “scientific supercomputing” has disappointed.
Sensitive
Robust
averageteraflops
petaflops
googaflops
Still haven’t capturedimportant phenomena.
Error,sensitivity
Types of uncertainty
Why “scientific supercomputing” has disappointed.
Sensitive
Robust
averageteraflops
petaflops
googaflops
Still haven’t capturedimportant phenomena.• Relied too heavily on the
homogeneous view of multiscale phenomena from physics.• Deep misunderstanding of the “robust, yet fragile” character of much complex phenomena.