Identifying Repeated Patterns of Behavior in Time

Magnus S. MagnussonResearch ProfessorHuman Behavior LaboratoryUniversity of Icelandwww.hbl.hi.is

Life and Repetitionof Spatio/Temporal Patterns

Francis CRICK: “Another key feature of

biology is the existence of many identical examples of complex structures.”

(1989, p. 138.)

(Crick and Watson discovered the double helix structure of DNA.)

Some examples: DNA

– Base pairs, genes, chromosomes, genomes

Behavior– Words, gestures and

patterns of these Clothing

– Shoes, hats, coats,.. Urban environments

– Houses, streets, cars, shops, books, radios...

“Behavior consists of patterns in time. Investigations of behavior deal with sequences that, in contrast to bodily characteristics, are not always visible.”

Opening words of Eibl-Eibesfeldt’s Opening words of Eibl-Eibesfeldt’s Ethology: The Biology of Ethology: The Biology of

BehaviorBehavior, 1970, p. 1, 1970, p. 1; {Emphasis added.}

Behavior is Patterns- often hidden patterns

Self-organization –The Emergence of Patterns

Bénard cells

From Scott Kelso, 1997

Visible or hidden

“Emergence” often needs to be assisted

“It is rarely, if ever, the case that the appropriate notion of pattern is extracted from the phenomenon itself using minimally biased procedures. Briefly stated, in the realm of pattern formation ‘patterns’ are guessed and then verified.” Crutchfiled, J., 1993. (Here cited from Solé & Goodwin, 2000, p. 20).

New Research Directions

Much theoretical and methodological thinking within the behavioral sciences (and statistics) stems from the time before cheap powerful computers and advanced software development tools

Highly complex search patterns and algorithms can now be developed and applied

Behavioral scientists can aim for new kinds of discoveries

Basic Viewpoint and Task

Behavior is more structured than is perceived directly or through standard data analysis methods

To fully disclose its structure new pattern types and detection methods are needed to complement existing ones

The discovery of hidden patterns is of considerable importance for theoretical and practical reasons

Architechture vs. Structurea simple philosophy

Search algorithms should correspond to the structure of the phenomenon being studied

Even the most sophisticated and powerful square detection algorithm is not adequate for the detection of planetary orbits

An imperfect ellipse detection algorithm would be preferable

Some Basic Questions

What kinds of hidden significant structure exists in behavior?

How to characterize such structure? How to discover such hidden structure? How to discover effects of independent

variables on such structure?

Sequences and Patterns

A sequence: “1. an arrangement of two or

more things in successive order” “3. an action or event that

follows another or others” “Maths. a. an ordered set of

numbers or other mathematical entities in one-to-one correspondence with the integers 1 to n” - Collins.

A pattern: “1. an arrangement of

repeated or corresponding parts, decorative motives etc...”

“Most mathematicians define Mathematics as the science of patterns….”

A pattern (shape, form) may not be a sequence but may still include one. Detecting a pattern may thus mean detecting a sequence.

Behavior as Repeated Patterns

Linguistics: repeated hierarchical/syntactic patterns

Ethology: repeated hierarchical/syntactic patterns

Behaviorism: repeated real-time contingencies

Anthropology, Social Psychology and more: scripts, plans, routines,

strategies, rituals, ceremonies, etc.

The importance of repeated patterns in behavior is widely accepted

The recognition of the “hiddenness” of some such patterns is needed

New pattern definitions with corresponding detection algorithms and

tools (software) are required

Verbal and Nonverbal are One

“The activity of man constitutes a structural wholeThe activity of man constitutes a structural whole, in in

such a way that it cannot be subdivided into neat such a way that it cannot be subdivided into neat “parts” or “levels” or “compartments” insulated in “parts” or “levels” or “compartments” insulated in character, content, and organization from other character, content, and organization from other

behavior.behavior. Verbal and nonverbal activity is a unified Verbal and nonverbal activity is a unified

whole,whole, andand theory and methodology should be theory and methodology should be

organized or created to treat it as suchorganized or created to treat it as such.” Pike(1960, p. 2). {Emphasis added.}

Repetition versus Uniqueness

“.. a conversation, … a complex system of relationships which nonetheless may be understood in terms of general principles which are discoverable and generally applicable, even though the course of any specific encounter is unique (cf. Kendon 1963, Argyle and Kendon 1967).”

(Kendon, 1990, p. 4). (Emphasis added.)

Towards a General Pattern Type Different Time Scales and Content

These patterns of patterns have aspects in common:These patterns of patterns have aspects in common: How do you do? How do you do? Very well, thank you. Pass me the salt, Jack. Jack, passes the salt. If..then..else Dinner: Sit down..take an entrée..take a main

course..take dessert..drink coffee..stand up Rituals, ceremonies, routines, genes, poems,

hospital operations, conferences, classes, football matches, strikes, and melodies

Common Structural Aspects

Fixed order and “significantly invariant distances” between components

Hierarchical/syntactic structure

Self similarity / scale independence

Such Patterns are Often Hard to Spot

d ek w akb c d k w k d e wkakb ckd w

T

d ek w akb c d k w k d e wkakb ckd w

T

d ek w akb c d k w k d e wkakb ckd w d ek w akb c d k w k d e wkakb ckd w

1

2

1 and 2 are identical data sets

The six sub-patterns are separated by relatively freely structured intervals of relatively fixed length.

(From Magnusson, 2005.)

A Dinner...a Time-Flexible Pattern of Patterns

Patterns and Causation

Very well, thank you– an earlier word is usually not considered as a

cause of any word following it within such intra-individual patterns

How do you do? Very well, thank you– an earlier part of some inter-individual patterns

may be seen as a likely cause of a later part of the same pattern

The T-patterns

A t-pattern is a particular set of event-types recurring in the same order (and/or concurrently) with “significantly similar distances” between them on a single dimension.

T-patterns have a scale-independent and hierarchical structure (often syntactically constrained -- “grammar”)

T-patterns may occur randomly, but they often occur in cycles and even when their elements do not

An event-type may be an actor’s (agent’s) beginning or ending of a

particular behavior.

It may also be a base in a DNA molecule or an amino (recid.) in a

protein.

Sets of such series form multivariate point series to which all T-pattern definitions refer exclusively.

a a a a a a

The Data: Series of Points on 1-Dim.

Multivariate Point SeriesThe Basic Data Type

A . . . . . . . . ..B .. . . . .C . . .. .D . . E . . . . . . .... . . . .. F . .G . .. . . . . . _______________________________________ t1 t2

Recursive T-Pattern Definition

The T-pattern is an ordered set of T-patterns (X):

XX11 ≈≈dt1 X X22 ≈≈dt2 . . . . XXii-1-1 ≈≈dti-1 X Xii .. . . ≈≈dtm-1 X Xmm

that recurs with significantly similar time distances,

≈≈dt,dt, between its elements relative to the zero

hypothesis (fiction) of constant probability per unit

time for each Xi = NXi / observation time

with the event-type as the simplest T-pattern

Towards a Detection Algorithm Searching for Critical Intervals [d1, d2]

Repeatedly, an A is followed by a B within approximately the same distance

Comparing Series A and B

A

B

DetectedCritical (distance) Interval (window)

d1 d2

Critical Intervals and Binary Trees

Any T-pattern Q = X1 X2..Xm can be split into a

pair of shorter ones related by a critical

interval: QLeft [d1, d2] QRight

Recursively, QLeft and QRight can each be split

until the whole pattern X1..Xm is expressed as

the terminals of a binary-tree

Bottom-up Detection Patterns Grow & Compete

The bottom-up algorithm detects patterns gradually from event types, as pairs of pairs, i.e., as binary trees

It detects critical interval relations between the occurrence series of event types and/or already detected patterns and then connects these to form longer patterns (trees).

Many binary-trees may correspond fully or partly to the same pattern so all detected patterns are automatically compared and only the most complete (longest) patterns are kept.

Completeness Competition - partial and equivalent trees

(( A B) ((C D) (E F))) (( A ((B C) D)) (E F)) (( A B) (D (E F )) (( A C ) F ) (B E) (B D) (A F)

Behavior Record - ExampleTwo Children (Blue & Red) Play With One Toy for 13.5 Min; 81 Series

Time in 1/15 s

Event

Type O

ccurr

ence

Seri

es

New Pattern Presentationfor Complex Patterns

Nature’s Symmetries are Approximate

Different instances of the same t-pattern may have quite different internal intervals

Still the relationship is always the same, the Critical Interval Relationship.

Should this be called relative translation

symmetry?

Statistical Validation

Statistical Validation Types

Standard Statistical MethodsInadequate for T-pattern Detection

Multivariate statistical methods: look for clouds of points in n-dimensional space rather than for syntactic structures on one dimension

Time series analysis looks for trends or waves rather than hierarchical discontinuities occurring irregularly

Sequential analysis may look for a priori unlikely time sequences but involves no concept of complex repeated 1-D shapes or patterns. It may therefore detect a multitude of sequential relations without ever detecting such underlying patterns

25 min of Children’s Dyadic Problem Solving

Data from published studies by Beaudichon and Magnusson.

Univariate Bursts

Doctor-patient facial interaction data coded with FACS

Interindividual T-patterns in Doctor-Patient Facial Interactions

Data from the Psychiatric Hopitals in Geneva, V. Haynal et al.

Wild-type Male vs. wild-type Mated Female Wild-type Male vs. Mature wild-type Virgin Female

Wild-type Male vs. Immature wild-type Female

#P = 8

#P = 3 #P = 4

Drosophila interactionsB. Arthur’s data and patterns

The T-System or T-model A System of Mathematically Defined Terms

The t-pattern type is the basis of a growing system of terms for the description of temporal structure in complex behavioral processes

Corresponding detection algorithms have been developed and implemented in the THEME software

See patternvision.com & noldus.com

Extending the T-modelBuilding on the Critical Interval and T-pattern Concepts

Markers & Indicators Composition +/- Associates; Satellites & taboos Gravity- and repulsion zones Packets Packet markers Drifters T-kappa

T-markers

A t-marker of a t-pattern occurs almost exclusively as a part of that pattern

A t-marker’s occurrence thus indicates that a particular t-pattern is occurring

A marker that occurs early in a pattern predicts the rest of the pattern

A marker occurring late in the pattern retrodicts the earlier part of the pattern

T-associates

A positive or negative associate of a T-pattern is: some behavior that is not a part of that pattern, but occurs within or around its occurrences significantly more (or less) often than expected by chance

Associates may occur only, always, sometimes or even never within or near their corresponding T-pattern

The “only and always” case is called a t-satellite

The (almost) never case is called a t-taboo

The T-packet StructureA T-pattern with its Associates and Zones

An instance of a t-packet showing two t-associate instances

The gravity zone, [ t1, t2 ], of a t-pattern extends from the earliest to

the last positive associate The negative gravity or repulsion zone (not shown) is similarly the

interval within which any behaviors tend not to occur T-packets are simultaneously sequential and non-sequential

structures

Neurones as Interacting and Networking (Social) Organisms

Neuronal T-data Including Breathing(from Nicol, Kendrick, Magnusson, 2005)

A Breathing Related Neuronal T-pattern

N = 15 Len= 12 Dur = 266482 %Dur = 88 Log10( P(Template)) = -2,88 Log10( P(Template Occurrence)) = -43,18

T-Paths are T-patterns in Space -- A neuronal T-pattern on an Electrode Grid

Origins

Greener: higher probability of being the origin (first node)

Figure from A.J.F. Griffiths et al. 1999, p. 30.

Cycles, base-pairs, codons; DNA->tRNA->Protein->>>Phenotype->..

DNA and its “Ticking” Backbone Cyclical and Combinatorial

Genes and Genomes as T-patternsNon-coding or “Free” Segments Between and Within Genes

Figure from A.J.F. Griffiths et al. 1999, p. 33.

The six sub-patterns are separated by relatively freely structured intervals of relatively fixed length.

(From Magnusson, 2005.)

A Dinner...a Time-Flexible Pattern of Patterns

Thank You