how can we rely upon social network measures? agent-base modelling as the next level test after face...

How can we check Social Network models and measures? Mitchell Seminar, Manchester, Feb. 2016. slide 1

How can we rely upon Social Network Measures?Agent-base modelling as the next level test after face validity

Bruce EdmondsCentre for Policy Modelling

Manchester Metropolitan UniversitySome of this work with Edmund Chattoe,

University of Leicester


This presentation….

1. Looks at the purposes of SN measures (whilst I focus on SN measures, much of this applies to other SN techniques)

2. It critiques some assumptions about such measures

3. It suggests that Agent-Based Modelling might be used as a check on their efficacy

4. It looks at some examples


Social Network Measures

• A Social Network is not real, but is only a model of what we observe

• It simplifies individuals and their complex relations and/or interactions into nodes&links, and then does some inference/calculation on this (e.g. using a measure) ...

• … to (presumably) learn something about what is observed

• Unless this measure is interpreted in terms of the observed the result is of formal interest only

• But if it is interpreted back to the observed, how can we rely on this?


Network Analysis for Illustration

A

B

A measure on the network, M(x)

Based on an already existing good

understanding of what is happening in

the target system

Choose and use a measure or

visualisation to illustrate that

understanding


Network Analysis for Hypothesis Testing

A

B


Based on an existing understanding of

what is happening in the target system form a hypothesis

about this

Choose the appropriate

measure to test this, discard

hypothesis if fail


Network Analysis for Pattern Exploration

A

B


Given a system that we don’t

really understand

Look for patterns using SN Analysis and

visualisations and hence inform

hypothesis formation


Network Analysis for Inference

A

B


Given a system that we don’t completely

understand

Use SN measure to infer something about what is happening in

the system


Summary of these four purposes

Purpose Prerequisites What it does What it does NOT do/allow

Illustration Existing good understanding of system

Illustrates this understanding

Aids communication, does not prove anything about the system

Testing An understanding to test, a measure that is a priori suitable

Can falsify the hypothesis, lack of falsification strengths it

Can not go back and look for a different measure that would support hypothesis

Pattern Discovery

Can reveal interesting patterns which might suggest hypotheses about the system

Suggestive, it does not prove anything about the system

Inference System with model

Could infer a new property about the system from other properties

Inference only as strong as model’s validity, does not prove anything from unvalidated model


The use of measures

• When it is used for the purpose of inference this is often unvalidated …

• ...which is unsurprising as independent validation of SN measures for each kind of situation is costly

• When an empirical conclusion is made using an SN measure this can not be a matter of formal proof but is necessarily contingent

• SN measures are often just a part of a complex sequence of steps, but this makes the question of their reliability even more important and tends to hide the contingent nature of SN measures


Agent-Based Modelling

• One has to decide what the agents are as and the rules that govern their behaviour(s)

• When run, the interactions between the agents determine the outcomes which might be abstracted in a variety of manners

• (Its agent-based if it is useful to consider the entities as having cognition in some sense)

• Vary from very abstract thought experiments to complex computational descriptions

• Since behavioural rules can include network constraints this can be seen as a generalisation of SNA – more general but also more complex (less analytically tractable)


Using ABS to Probe SNA Assumptions

• Agent-based simulations (ABS) are more complex representations – they ‘simplify less’ but are still only a model and only as strong as the evidence supporting them

• We can explore the robustness of SN analysis against plausible social simulations

• This can indicate the conditions under which a particular SN measure can be relied upon (or not)

• If the SN analysis does not work with a plausible ABS then how could we rely on it the observed phenomena that the ABS models?


Example I: An apparently simple case

• Here I consider a general abstract class of systems looking at the question of whether any measure can be relied upon to indicate eventual node importance.

• The class of systems considered all are:– Relatively simple– Deterministic– About which we have almost complete

information about behaviour, network, initial conditions etc. to help us chose our measure


The class of system considered (Giving Agents with Simple Plans)

• There are N agents, each of which– Can only give and receive units– Has an single store for these units– Gets one new unit from the environment each tick– Has a fixed number of behaviours, each consisting of a

number of “give one unit to agent X” instructions and one “jump to plan A if agent B has 0 else plan C” instruction

– All stores except one are initialised to 0• All plans are completely known in advance• The only thing NOT known is the value of agent-

1’s store at time 0


A Simple Type of Agent System


Thus the reformulated question is...

Given almost complete knowledge of a particular GASP system (except for the initial store of Agent-1), can you effectively find any measure, M, such that:

• If and only if M(A) ≥ M(B) then...• Eventually S(t,A) ≥ S(t,B) [ where S(t,x) is the

value of the store in agent x at time t ]• That is given a GASP is there an M so that:

M(A) ≥ M(B) ↔ Exists T; for t>T S(t,A) ≥ S(t,B)


And the answer is... No!

• In other words, there are GASP systems, where even though we know: their complete behaviour (comparable to detailed interviews of all participants); everything possible about their social network (who they can make transfers to); and almost all of the initial conditions (except one value)...

• ...there is no measure that will tell us from this initial structure which nodes will be more influential than others once running.


Proof Sketch

• The class of GASP systems are Turing Complete, in other words they can compute anything a Turing Machine (TM) can (shown by a mapping into an Unlimited Register Machine a know TM equivalent).

• If there were a such a measure, then we could use it to check (without computation) that the results of two GASP systems (the end value in the store of Agent-1) were equal by joining the two systems into one; finding the measure, M and then using it to see if the two output nodes would be equal. This is known to be uncomputable.


Example I: Conclusions

• Even with very simple, deterministic systems, where we know everything about the behaviour and the network structure of the system, there are no measures that would a priori inform us about eventual node importance.

• Most systems we observe are FAR more complex and less well understood than GASP systems

• Therefore one can not assume there is a ‘right’ measure that will reliably inform us about a system

• The burden of proof is on those that claim, with a largely unknown complex system, that any particular measure will tell us some particular thing reliably

• Effective measurement usually follows understanding


Example II: A Peer-to-Peer (P2P) File-sharing system

Collection of ‘servers’, each of which:– Is controlled by a user to some extent– ‘Knows’ a limited number of servers, with which it can

communicate (the network)– Makes some (or no) files available for download by

other servers– Search for files is by flood-fill: (i.e. send query to n

others who send it to n others… for a limited number of steps)

– If query matches an available file it is sent back to originator

• E.g. Bittorrent


A Simulation of a P2P System

• 50 servers, each can decide to share files (coop) or not (def) at any time

• Try collect ‘sets’ of related files stored (initially) randomly by sending queries

• Satisfaction is measured by success at collecting files – (small) cost of dealing with others’ queries (but decays over time)

• May look at and copy what a more satisfied server does, or may drop out and be replaced (especially if satisfaction is low)


Number of co-operators in a run of the simulation (out of 50)

• Key issue is number (and manner) of cooperation– Why does anyone cooperate?– How does position of nodes within the network

structure impact upon this?


Typical Emergent Network Structure

core partitition periphery

small isolated group


Suggests four types of node

• In-coop – those who share their files in core partition

• In-def – those who don’t share their files in core partition

• Out-coop – those who share their files but are outside the core partition

• Out-def – those who don’t share their files but are outside the core partition


Some General Statistics

Type Average utility Average number of

links

Average centrality

in-coop 0.79 3.0 0.41

out-coop 0.51 2.5 0.31

in-def 0.37 2.0 0.27

out-def 0.32 1.5 0.19

Stats. for each type, averaged over all the runs, for all nodes and times (after an initial period)


Regression coefficients with satisfaction levels of nodes

Type Number of links

Number of links

lagged 6 periods

Centrality Centrality lagged 6 periods

in-coop -0.058 0.13 -0.062 0.12out-coop 0.073 0.17 0.065 0.16in-def 0.039 0.074 0.067 0.087out-def -0.15 -0.053 0.066 0.13

Regression of satisfaction (number of files) of each node with number of links etc.

Other measures and lags had lower correlations, including those that just did these in aggregate


Checking the impact of lags (out-coops)

0.080.09

0.10.110.120.130.140.150.160.170.18

0 2 4 6 8 10 12

Lag (in cycles)

Cor

rela

tion

cent Rnum l R

Correlation of satisfaction to measures, all nodes of that kind, times and runs


Size of partitions during a run

Blue – size of largest partition

Green – 2nd largest (if there is one)

Red, orange, etc. – even smaller ones

Network is highly dynamic, with rapid churn of servers and changing structure


Example II: Conclusion

• The global SN measures were not very useful in providing understanding of leverage

• It would have been unsafe to assume that such would give a reliable picture of the role of such networks

• The structural analysis based on the detailed understanding of the dynamics created a more useful categorisation of node types (but this is precisely the kind of understanding difficult to obtain when the system is real rather than simulated)

• Given a good understanding it might be possible to choose better measures to illustrate this

• Highlights the distinction between demonstrating an existing understanding of a network and fishing for understanding using SNA measures


Example III: A Simulation of Ga-Selala (in the Limpopo Valley of South Africa)

• A Evidence-led Simulation of a particular village • Represents many aspects of life there, including:

sexual network and HIV/AIDS spread, friendship network, kinship network, employment, savings clubs, household structure, birth and death, government grants and health

• Purpose was to assess possible impacts of factors, in particular how fragile the social structure might be to these factors given the complex interplay of the various social structures and behaviours – suggesting hypotheses


Basic Methodology

• Repeated iterations of model development in response to stakeholder criticism, expert opinion, statistics, interviews etc.

• So that most aspects of the model had some (but varying) levels of justification from available evidence

• Result is a context-specific but dynamic “description” using a computer simulation

• Simulation is difficult to understand and slow to run, but open to unlimited experiment and inspection

• Changes in network structure can be studied in the simulation even though it is highly dynamic


Observations from running simulation experiments

• It seemed that (given the introduction of a new mining enterprise near the village) the social structure(s) collapsed in the model

• To try and show this, snapshots of the social network taken and their degree distribution compared using non-parametric statistics (Kolmogorov-Sinai) to see if there is evidence of significant change beyond our impression


Comparing the social network over time with that at time 0

Initialised with Watts-Strogatz Small-world network

Initialised with Erdös random network

P-scores of K-S test on the degree distributions of the social networks


Comparing the social network over time with the previous time

Initialised with Watts-Strogatz Small-world network

Initialised with Erdös random network

P-scores of K-S test on the degree distributions of the social networks


Example III: Conclusions

• It is inevitable that some ABS will be too complex to completely understand

• SN analyses can be really helpful for helping understand complex ABS

• Once we have a hypothesis about what is happening we can comprehensively check that any SN-derived analysis is correct

• Thus the SN analysis is indirectly useful for understanding what is observed…

• ...but limited by the validity of the ABS of course!


Staging abstraction using simulation models and SN models

Micro-Evidence Macro-Data

Agent-Based Simulation Model

SN Model 1 SN Model 2

SN Measures Other Analysis SN Measures


The End!

Bruce Edmonds: http://bruce.edmonds.nameCentre for Policy Modelling: http://cfpm.org

These slides available at:http://slideshare.net/BruceEdmonds

Funding Bodies:

how can we rely upon social network measures? agent-base modelling as the next level test after face...

Science