1

Tipping Points, Butterflies, and Black Swans: A Vision for Spatio-temporal Data Mining Analysis

Dr. James M. Kang and Daniel L. EdwardsInnoVision Basic and Applied Research OfficeNational Geospatial-Intelligence Agency

August 24, 2011

Approved for Public Release 11-412

2

VisionThe development of data mining and spatio-temporal analytical techniques to discover tipping-points, butterflies,

and black swans.

Approved for Public Release 11-412

3

What are Tipping Points?“the moment of critical mass, the threshold, the

boiling point” – M. Gladwell

Climate Tipping Point, Upsala Glacier, Patagonia, Argentina

(Source: http://www.changeclimate.org/)

Approved for Public Release 11-412

4

What is the Butterfly Effect?Behavior of dynamic systems

• Highly sensitive to initial conditions – J. Gleck• Involve topologically mixing – B. Hasselblatt

Source: http://www.guardian.co.uk/world/interactive/2011/mar/22/middle-east-protest-interactive-timeline

Approved for Public Release 11-412

5

What are Black Swans?• Unpredictable patterns that do not appear to

be Gaussian with an exponential diminishing tail, but a flatter curve with tails that are fatter

• Have the following characteristics:• The event is a surprise (to the observer).• The event has a major impact.• After its first recording, the event is rationalized

by hindsight, as if it could have been expected. – N. Taleb

Approved for Public Release 11-412

6

Challenges

Approved for Public Release 11-412

7

Tipping Point Challenges• Assumptions about

a dataset may change before and after a tipping point event

• Tobler’s Law vs. Teleconnections

Approved for Public Release 11-412

8

Butterfly Effect Challenges

1. Depth – sufficient data to mine vs. scope of problem?

2. Breadth - breadth of data sufficient to sample problem?

3. Missing – key data/meta data missing?

3. Stability - of mined patterns?

Bounding problem with sufficiency

Approved for Public Release 11-412

9

Black Swan Challenges• As a Black Swan unfolds,

• Mined patterns over populations and time may not become “interesting”

• May not be prevalent or anomalous

• After a Black Swan is recognized (hindsight),• Bounding sufficiency may be

too complex to overcome • May not generalize to other

known black swans

Approved for Public Release 11-412

10

First and Next Steps• Tipping Points

• Existing literature in abrupt changes, transitions, etc.

• Transient vs. Persistent

• Butterflies and Black Swans• Can these be generalized?• Are these even possible?• How can we begin quantifying

these events?

• Example Datasets• Guardian’s event dataset of

middle-east• CIA World Factbook dataset

Source: https://www.cia.gov/library/publications/the-world-factbook/

Approved for Public Release 11-412

11

www.nga.mil

Approved for Public Release 11-412