modelling and knowledge

Modelling and Knowledge, Bruce Edmonds, Manchester Metropolitan University, Feb. 2016. slide 1

Modelling and Knowledge

Bruce EdmondsCentre for Policy Modelling

Manchester Metropolitan University


Models, Knowledge and FormalityPart 1


This talk looks at…

• How we use external representations to help use understand the world and make decisions about it

• In particular, at formal models and their uses– e.g. graphs, statistics, computer programs, visualisations,

logic, systems of equations, etc.– the different kinds of use they can be put– the different ways they can be judged– the different processes they can play a role within– the issues of their reliability and control

• It has a philosophical flavour but hopefully also has practical utility e.g. in judging models

• Looks at some examples


Knowledge

• Humans can experience one situation and apply that later in a similar (but not identical) situation

• They can also use language so someone else can apply the experience of another person

• We label these transferences between times or people as “knowledge”, “learning” etc.

• However, we don’t know how we know• Some knowledge is implicit (e.g. how to ride a

bicycle), knowledge we have but can’t explain• Other can be explained, it can be explicit and

hence shared remotely (without being there)


Representation

• We make ourselves cleverer by interacting with external systems (writing, drawing…)

• Sometimes these have a close relationship with our explicit knowledge

• That is, we can recreate our knowledge from them and encode our knowledge using them

• These we call representations• These can be direct or indirect, formal or informal,

useful or deceptive, clear or unclear etc.• One particular kind of representation we call a

model


Two dimensions of representational formality

Low Constrainton Manipulation

High Constrainton Manipulation

Low Constrainton Reference

High Constrainton Reference

SRSM informal

description

fixed-referentmodel

abstractsystem

Data

Graphs

Abstract Model

Law

Poetry

Regression Model

MacroeconomicModel


What is a model?

Something, A, that is used to understand or answer questions about something else, B

• e.g: A scale model to test in a wind tunnel• e.g: The official accounts of a business• e.g: The minutes of a meeting• e.g: A flow chart of a legal process• e.g: A memory of a past event• e.g: A computer simulation of the weather• e.g: The analogy of fashion as a virus

Models usually abstract certain features and have other features that are irrelevant to what is modelled

An Introduction to SS. By Bruce Edmonds, ISS Course, 2011, slide 7


The Model and its Target

A formal model is not a model at all without this mapping relation telling us the intended meaning of its parts, but this does not necessarily mean that the relationship is precise

Object System

Model

The mapping between formal

model and what the parts refer to


Analytic formal models

Where the model is expressed in terms that allow for formal inferences about its

general properties to be made• e.g. Mathematical formulae• Where you don’t have to compute the

consequences but can derive them logically• Usually requires numerical representation of

what is observed (but not always)Only fairly “simple” mathematical models can be

treated analytically – the rest have to be simulated/calculated


Statistical formal models

Where the collective properties of a group are modelled, eliminating some assumed

randomness between individuals• Descriptive statistics just summarise aspects

of a group that are assumed to be representative of that group

• Generative statistics are a model of some process done using the combination of a target trend plus additional randomness

Statistical models often rely on the “Law of Large Numbers” – that certain aspects are irrelevant

and can be treated as random


Computational formal models

Where a process is modelled in a series of precise instructions (the program) that can be

“run” on a computer• The same program always produces the same

results (essentially) but...• ...may use a “random seed” to randomise certain

aspects• Can be simple or very complex• Often tries to capture more “qualitative” aspects of

social phenomena


Modelling Purposes with ExamplesPart 2


Modelling Purposes

All modelling has a purpose (or several)Including:• Description• Prediction• Establishing/suggesting explanations• Illustration/communication• Exploration• Analogy

These are frequently conflated!


A Model used as an analogy, a way of thinking about a situation or purely formal exploration

Object Systemknown unknown

Modelinput

(parameters, initial conditions etc.)

output(results)

encoding(measurement)

decoding(interpretation)

Conceptual Model

Is suggestive

of


Example 1: Schelling’s Segregation Model

Schelling, Thomas C. 1971. Dynamic Models of Segregation. Journal of Mathematical Sociology 1:143-186.Rule: each iteration, each dot looks at its 8 neighbours and if less than 30% are the same colour as itself, it moves to a random empty squareSegregation can result from wanting only a few neighbours of a like colour


A trouble with such simulations

• Is that they are highly suggestive• Once you play with them a lot, you start to “see”

the world in terms of you model – a strong version of Kuhn’s theoretical spectacles

• They can help persuade beyond the limit of their reliability

• They may well not be directly related to any observations of social phenomena


Analogies

• Analogies are usually verbal, but can also be formal (equations, simulations, etc.)

• Their mapping to what is being considered is built “on the fly” for each situation

• Analogies seem to be very basic to the way humans think and communicate

• Their mapping to the situation is different for each context and each person (in contrast to a model where the mapping is defined)

• This is done automatically and largely unconsciously• This gives the illusion of generality


A Model used for prediction of unknown data


Modelinput


output(results)



Inference using model


Example 1: General Election Forecasting

• John Curtice (Strathclyde) and David Firth (Warwick) (+ input from others)

• Each constituency is statistically modelled as a three-way split (Lab, Con, LD) based on how much this swung with the general trend according to past data


Example 1: General Election Forecasting

• Each line is the 3-way vote share for each constituency in UK general elections,

• green spots show 2005 shares, tail is the 2001 shares


About Microsimulation

• Instead of having a generic process over all relevant situations one has a model for each situation

• This is limited and determined by available data for each of these situations

• Often these situations are geographical regions• Often each model is a population dynamics model with a different

distribution for each region, trained on available data (usually each distribution come from a family which encode assumptions about the processes)

• Thus variation is not handled by some generic “noise” but rather aggregation is put off to a post-hoc summary of the complex results retaining the context-specificity

• This approach is heavily data-driven• You have to look at each separate region to determine if the local

model is a good fit in each case


A Model used for explanation of known data in terms of mapping


Modelinput


output(results)



Inference using model

Explanation is the outcomes in

terms of the process and initial state

Model is adjusted until the outcomes

map to the results


A model of social influence and water demand

• Investigate the possible impact of social influence between households on patterns of water consumption

• Design and detailed behaviour from simulation validated against expert and stakeholder opinion at each stage

• Some of the inputs are real data• Characteristics of resulting aggregate time series

validated against similar real data


Simulation structure

• Activity

• Frequency

• Volume Households

Policy Agent

• Temperature

• Rainfall

• Daylight

Ground

Aggregate Demand

• Activity

• Frequency

• Volume Households

Policy Agent

• Temperature

• Rainfall

•

Ground

Aggregate Demand


Some of the household influence structure

- Global Biased- Locally Biased- Self Biased


Example results

Aggregate demand series scaled so 1973=100

0

20

40

60

80

100

120

140

160

180

200

J-73

J-74

J-75

J-76

J-77

J-78

J-79

J-80

J-81

J-82

J-83

J-84

J-85

J-86

J-87

J-88

J-89

J-90

J-91

J-92

J-93

J-94

J-95

J-96

J-97

Simulation Date

Rel

ativ

e D

eman

d


Conclusions from Example

• The use of a concrete descriptive simulation model allowed the detailed criticism and, hence, improvement of the model

• The inclusion of social influence resulted in aggregate water demand patterns with many of the characteristics of observed demand patterns

• The model established how it was possible that processes of mutual social influence could result in widely differing patterns of consumption that were self-reinforcing

• In other words, it supported that explanation of those patterns


Unravelling the Micro-Macro Link

Micro/ Individual data Qualitative, behavioural, social psychological data

Theory, narrative accounts

Social, economic surveys; Census Macro/ Social data

Simulation

Upw

ard

caus

atio

n –

emer

genc

e

Dow

nward causation –

imm

ergence


Some things that go wrongwith formal models

Part 3


The Whole Modelling Chain

• In both prediction and explanation, to get anything useful out…

• One has to traverse the whole modelling chain:1. From target system to model2. Inference using the model3. From model back to target system

• The “usefullness” of the model, roughly speaking, comes from the strength of the whole chain

• If one strengths one part only to critically weaken another part this does not help

• If one step is weak so is the whole chain


The “Simple is more General” Fallacy

• If one has a general model one can make it more specific (less general) by adding more processes/aspects…

• …in which case it can become more complex• However, the reverse is not true…• If one simplifies/abstracts then you don’t get a

more general model (well almost never)!– there may be no simpler model that is good enough for

your purpose– But, even if there is, you don’t know which aspects can

be safely omitted – if you remove an essential aspect if will be wrong everywhere (no generality)


KISS vs. KIDS as a search strategy

Simplest Possible

More Complex in Aspect 2

etc.

More Complex in Aspect 1

KISS

KIDS


Context-Dependency and Randomness

What appears to be random may in fact be due to variation of context


Using Randomness as a Proxy for Contextual Variation

• It is completely unsurprising that many factors will be significantly correlated with many outcomes in a multi-context situation

• It is also unsurprising that the explanatory level of such exercises are unimpressive

• The correlation may be due to a strong correlation in only one kind of context and, indeed, mask anti-correlations in others

• Using randomness as a proxy for contextual variation simply discards a lot of the information in the phenomena – it amounts to ignoring evidence


An analogy: An Ideal Gas

• The idea: although the motion of each particle in the gas is not predictable, taken together the gas obeys regular laws and is predictable

• This idea has seeped into the social sciences that many social phenomena become more predictable when dealt with in larger numbers

• (Asimov 1962, page 7): “Psycho-history dealt not with man, but with man-masses. It was the science of mobs; mobs in their billions ... The reaction of one man could be forecast by no known mathematics; the reaction of a billion is something else again”

http://jasss.soc.surrey.ac.uk/9/3/reviews/edmonds.html


Problems with this idea…

• This only “works” if there is a signal that is separable from noise and…– …the “noise” is essentially random (Law of Large

Numbers)…– …or can be safely ignored.

• But it is almost impossible to know either of these for sure!

• e.g. in stock markets, what seems to be random noise is rather the result of subtly linked social processes

• In other words, the context of modelling is inadequate and “leaky”


Judging ModelsPart 4


Some Criteria for Judging a Model

• Soundness of design– w.r.t. knowledge of how the object works– w.r.t. tradition in a field

• Accuracy (lack of error)• Simplicity (ease in communication, construction,

comprehension etc.)• Generality (when you can safely use it)• Sensitivity (relates to goals and object)• Plausibility (of design, process and results)• Cost (time, effort, etc.)


Some modelling trade-offs

simplicity

generality

Lack of error (accuracy of results)

realism(design reflects observations)


How one judge’s a model…

… depends, of course, on its purpose/goal.• A model for theoretical exploration or analogy just

has to be formally correct and have some meaning, judged on what new insights it gives

• A predictive model needs simply to predict sufficiently well on unknown data and under conditions one knows

• An explanatory model needs to be robust, with understood assumptions, targeted and sound so that we can know how to rely on the explanation it suggests

• See sheet


Some Philosophy of Knowledge and Modelling

Part 5


Popper’s Falsificationism

• Induction never proves anything• Hypotheses can only be disproved by observing a

counter-example (a black swan)• We rely on hypotheses more as they survive

attempts to disprove them• If there is constant innovation of hypotheses and

attempts to disprove them then knowledge will progress

• Hypotheses that are not amenable to being falsified (unfalsifiable hypotheses) are dubious


Comments on Popper’s Falsificationism

• History of science does not fully support it (e.g. Michelson-Morley experiment)

• How does one know whether the counter-example shows the main hypothesis is wrong or merely an auxiliary assumption?

• Marks a switch from the context of discovery to the context of justification

• Results in an evolutionary picture of the development of knowledge (evolutionary epistemology)


Lakatos’ Core and Protective Belt

• Research programs as key entities– These have a core of fundamental frameworks,

methods and assumptions that characterises them– And a belt of less fundamental hypotheses,

observations, techniques• In the face of counter-examples research

programs change things in the belt and preserve the core

• Some programs are more successful than others (the “degenerate programs”)


Instrumental vs. Realism

There are fundamentally two different ways of adapting a model, that is according to:1. how well it works (its return etc.)2. how well it reflects realityThe first is simpler and more immediate in its returnsThe second requires a separate process to then work out action from the best model


2 views of learning: (2) feedback via success when used (e.g. pain)

Choose one and put it into effect (work out what to do)

actionfeedback of success

Strategy 1

Strategy 2

etc.

Strategy 3

Evaluate how successful strategy was


2 views of learning: (1) feedback via predictive power

Choose one, work out predictions of effects of possible actions

actionperception of error

Model 1

Model 2

etc.

Model 3

Evaluate whether predicitons were accurate


Some ConclusionsPart 6


Multiple models

In many cases one has to use more than one model to adequately deal with a particular case• Parallel models

– e.g. different models gained by different approaches and simplifications, whose results are compared (e.g. Lasers)

• Clusters of models– e.g. use of analogical models alongside formal models

in atomic physics• Chains of models

– e.g. increasingly simple/abstract models

(What really happens)


An example of the layering of related models in chemistry from 1990

• Adapted from: Gunsteren, W. F; Berendsen, H. J. C. (1990) Computational Simulation of Molecular Dynamics: Methodology, Applications and Perspectives in Chemistry. Angewandte Chemie - International Edition in English, 29:992-1023.


Summary of three kinds of model

Type Criteria for their recognition Abilities Weaknesses

Theoretical Exploration - represent a theory or idea so that its consequences can be explored and understood

No well-defined relationship with any empirical data

Can be used to establish a theoretical possibilities or counter examples

Does not tell us how any observed system behaves

Establishing an Explanation – establishes an explanation of observed outcomes in terms of the simulation processes and structures

Outputs related to aspects of empirical data and has plausible processes and structures

Gives understanding why outcomes might have occurred allows suggestion of hypotheses for counter-factual possibilities that are close to what is observed.

Does not predict the unknown.

Prediction – reliably predicts the unknown

Predicts aspects of unknown data

Enables one to anticipate events before they happen, or look for phenomena we might not otherwise expect

Does not necessarily explain why predictions are successful


Conclusion – advantages of formal modelling

• Impressive • Little confusion about model• Formal model can be copied and tried by others –

a social “evolutionary” process• Can be easy to confront with evidence • Strong inference step• Helps unearth assumptions• Suggests new questions to investigate• Can be shown to be wrong (Popper) or better how

it is wrong


Conclusion – disadvantages of formal modelling

• Impressive • Poor in terms of meaning• Requires expertise• Easy to fool oneself into thinking the world is like

your model• Tempting to take short-cuts• Difficult to validate completely• Difficult to list all assumptions• Needs lots of evidence if one is going to rely on it

for anything critical


The End!

Bruce Edmonds: http://bruce.edmonds.nameCentre for Policy Modelling: http://cfpm.org

These slides available at:http://slideshare.net/BruceEdmonds

Funding Bodies:


Some Developing Work

• Institute for Social Change &Theoretical Physics Group,University of Manchester

• Centre for Policy Modelling,Manchester Metropolitan University


the SCID Modelling Approach

Data-Integration Simulation Model

Micro-Evidence Macro-Data

Abstract Simulation Model 1

Abstract Simulation Model 2

SNA Model Analytic Model


Roles of each kind of model

• Each is constrained by those “beneath” them, i.e. are consistent with them

• What each component should clearly represent something

• Models “above” analyse, check and explain what is happening in those below

• Models immediately “below” can be used to explore the safety of assumptions

• It might well happen that simpler, more abstract models have validity (w.r.t. a lower model) only under some settings


But why not just jump straight to simple models?• There are many possible models and you don’t know

why to choose one rather than another, this method provides the underlying reasons

• Much social behaviour is context-specific, and this approach allows one to check whether a particular simple model holds when background features/assumptions change

• The chain of reference to the evidence is explicit, allowing one to trace their effect and possibly better criticise/improve the model

• This approach facilitates the mapping onto qualitative stories/evidence


Data Integration Models

• Intended more as a computational description of a particular case than a theory (at least a general theory)

• Its aim is to represent as much of the relevant evidence as possible in one coherent and dynamic simulation

• Provides a precise target for abstraction (which are then checkable against it)

• Stages abstraction from data to theory• Separates representation and abstraction• Preserves chains of reference


Aims and Objectives of DIM

• To develop a simulation that integrates as much as possible of the relevant available evidence, both qualitative and statistical (a Data-Integration Model – a DIM)

• Regardless of how complex this makes it• A description of a specified kind of situation (not a

general theory) that represents the evidence in a single, consistent and dynamic simulation

• This simulation is then a fixed and formal target for later analysis and abstraction


DIM Development Method

• A relatively tight interactive “loop” between the social scientists who are experts in the subject matter and their data and the simulation developers...

• ...trying to give as much ownership and control to social scientists as possible.

• First target: What makes people vote (within the context of a diverse community)?

• Started with developing a fairly complete list of “causal stories” concerning the various processes that might contribute from

• Then initial model iteratively developed in NetLogo to enable maximum responsiveness and transparency

• To be reimplemented in Java/Repast when the target becomes more “settled” for more extensive simulation exploration and analysis


An overview of model structureThis effect the behaviours of individuals, which can then be extracted from the simulation as model results and compared with evidence etc.


Basic Elements

• 2D grid of locations each of which has either a: household, work place, school, activity 1 centre, activity 2 centre, or empty

• People in household going through lifecycle according to the timescale: 1945-2010 (birth, death, migration, partnering, separation, moving out. etc.)

• Social network made of: intra-household links, shared activity membership (schools, work, religion, etc.), “friendship” links

• Influence occurs over the social network contingent on the state of those involved


Population Model

• Agents are in households: parents, children etc. of different ages in one location

• Initialised from a sample of 1992 BHPS• Agents are born, age, make partnerships have

children, move house, separate, die• UK-based moving in/out of region, as well as

international immigration/emigration• Rates of all the above estimated from available

statistics


Agent Characteristics

• Age, Ethnicity, location, children, parent, partner, political leaning, date last moved, etc.

• The activities it participates in• Its social connections• Plus a memory of facts, e.g.:

– “talked about politics with” agent324 blue 1993– “got desired result from voting” red 1997– “I am a voter” 2003– “pissed off with my own party” 2004


Immigration and Movement

• No special rules for different ethnicities or kinds of people (e.g. class)

• Rather composition (household size, income, class, education, civic involvement etc.) derived from survey data

• Class and ethnicity come into effect via homophily – people have a tendency to make friends with those similar to themselves (including age, ethnicity, education level, class, location etc.)

• This effects the social networks that develop• Which, in turn, effect mutual influence,

communication and the spread of social norms


Activities Model

• As well as households there are activities: schools, places of work, and (currently 2) kinds of activity (church and canoe clubs)

• Kids (4-18) attend one of 2 local schools• Those employed (from 16-65) attend a place of

work randomly• Activities are joined probabilistically, with choice

related to homophily (similarity to existing members)


Social Network Model

• A “connection” is a relationship where a conversation about politics might occur (but only if the participants are inclined/receptive)

• All members of a household are connected; when someone moves out there is a chance of these being dropped as connections

• There is a probability of people attending the same activity to be connected (chance varying according to similarity)

• There is a chance of spatial neighbours who are most similar being connected

• There is a chance of a “Friend of a Friend” becoming a connection

• Connections can be dropped


Communication and its Effects

• Social norms transmitted in pimarily within households (if not contradictory)

• Interest in politics transmitted via contact network by interested/involved agents with those who are receptive

• Some discussants may be more influential than others

• Bias in terms of held beliefs and norms may evolve due to coherence / incoherence in the messages from others

• Interest & biases might convert to action if the situation the agent is in is appropriate


Demonstration Run

Parametersand

Controls

Pseudo-narrative log of eventshappening to a single agent

SimpleStatistics

concerningOutcomes

Pictureof World

IndicativeGraphs

andHistograms


Example Development of Social

• Three “snapshots” of the social network from a single run of the “Inner City” version

• Darker links are within-household, lighter are other social links

• Each link indicates a relationship where if the agents are so minded they might discuss or otherwise influence each other concerning politics, voting etc.

• The issue about initialisation is clearly visible here


Social Network at 1950


Effect of Immigration Rate on Voting in this Model (“Inner City” Settings)

30%

40%

50%

60%

70%

80%

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64

Turn

out

Simulation Ticks

0.005

0.01

0.015


What Use is this Model?

• Forces “causal stories” to be explicit• Suggests hypotheses to be investigated• Provides a benchmark for model variations• In particular, provides a base for more abstract

models, checking the scope of their assumptions • Allows for the principled integration of qualitative

and quantitative evidence• Allows ‘what if’ explorations

modelling and knowledge

Education