simulating civilization change with nexus cognitive agents

32
Simulating Civilization Change with Nexus Cognitive Agents Deborah Duong, Agent Based Learning Systems Gerald Pearman, Augustine Consulting AHFE 12

Upload: lajos

Post on 25-Feb-2016

48 views

Category:

Documents


2 download

DESCRIPTION

Simulating Civilization Change with Nexus Cognitive Agents. Deborah Duong, Agent Based Learning Systems Gerald Pearman , Augustine Consulting AHFE 12. Purpose and Agenda. Purpose: To show the benefits of Symbolic Interactionist Simulation for the Simulation of Social Aggregation. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Simulating Civilization Change  with  Nexus Cognitive  Agents

Simulating Civilization Change

with Nexus Cognitive Agents

Deborah Duong, Agent Based Learning SystemsGerald Pearman, Augustine Consulting

AHFE 12

Page 2: Simulating Civilization Change  with  Nexus Cognitive  Agents

2Social Impact Model

Purpose: To show the benefits of Symbolic Interactionist Simulation for the Simulation of Social Aggregation

Purpose and Agenda

Agenda

•Coevolution and Symbolic Interactionist Simulation

•The Sociological Dynamical System Simulation

•SISTER: Symbolic Interactionist Simulation of Trade and Emergent Roles

•Nexus Network Learner•Modeling Corruption

Page 3: Simulating Civilization Change  with  Nexus Cognitive  Agents

3Social Impact Model

Coevolving Agents

• Genetic Algorithms, Neural Networks, or Reinforcement Learning Algorithms in Agents can co-evolve

• Agents learn to optimize a function in an environment composed mostly of agents also learning to optimize a function.

• Moving fitness landscapes– Agents apply selective pressure upon each other.– Selective pressure causes the optimal way to achieve the goal

to change over time.• Evolutionary Stable Strategies

– Maynard-Smith’s theory that a co-evolutionary system converges when no species (or inducing agent) can make a change that will make it better off (Nash Equilibrium)

• Agents Differentiate into a System– Species (or inducing agents) can be both cooperative and

competitive

Page 4: Simulating Civilization Change  with  Nexus Cognitive  Agents

4Social Impact Model

Symbolic Interactionist Simulation

• Coevolutionary reinforcement learning algorithm.• Autonomous agents each have an inductive mechanism.

– For example, an entire Genetic Algorithm or Neural Network.

– Agents only experience through their senses (no direct knowledge transfers from other agent minds).

• Agents choose to interact with other agents based on signs that the other agents display.

– Agents induce both the signs to display and the signs that they read.

• Social system emerges– Signs come to mean behaviors. – Behaviors interlock into a system of expectations.– Value function becomes reward function as society

evolves (Adam Smith’s invisible hand).

Page 5: Simulating Civilization Change  with  Nexus Cognitive  Agents

5Social Impact Model

The First Symbolic Interactionist Simulation: The Sociological Dynamical System Simulation

• A System of IAC Networks as the Basis for Self Organization in a Sociological Dynamical System Simulation

- Duong, 1991; Duong and Reilly, Behavioral Science 1995.• Employer agents hire from a pool of employee agents, periodically laying

off employees.• Employees who are less talented are laid off in greater proportions than

those who are more talented. • Employees seek employment.• Employees can choose to wear three different kinds of signs

– One of two signs they can not change. This is their “skin color”.– One of three signs they have to pay for with money from

employment. This is their “suit”.– One of three signs they can change arbitrarily. This is their “fad”.

• Employees induce what sign they should wear based on what has gotten them employed in the past.

Page 6: Simulating Civilization Change  with  Nexus Cognitive  Agents

6Social Impact Model

The Sociological Dynamical System Simulation (SDSS)

• Employees have a hidden, unchanging, talent level that employers can not see until after the employee is employed.

• Employers seek talented employees.• Employers hire based on the signs the employees wear,

inducing how much talent they have from their signs.• The two “races” of employees are equally talented, but the

employers do not know this.• Employers and Employee Agents both have the same kind of

induction mechanism. – Each Agent has their own Interactive Activation and

Competition (IAC) Neural Network to induce the signs they read and display.

Page 7: Simulating Civilization Change  with  Nexus Cognitive  Agents

7Social Impact Model

Interactive Activation and Competition

• An Employer’s IAC

• The IAC is a Constraint Satisfaction (Hopfield) Neural Network.

• Nodes within each pool are mutually inhibited.

• A central pool contains memory instances.

• Each instance has positive connections to its features.

• To guess the talent of an applicant, the employer “turns on” the applicants features and sees which talent node turns on.

• Simulates schema, or mental groupings of features that go together

Page 8: Simulating Civilization Change  with  Nexus Cognitive  Agents

8Social Impact Model

SDSS: Emergent Phenomena

• Status Symbols– Employee agents learn to buy expensive suits and Employer

agents learn to seek expensive suits.– Because the less talented get laid off more, they have less

money, and talented employees learn to differentiate themselves.

• Racism and Social Class– One race gets into a vicious cycle: because of schema, by

accident one race gets associated with less talent (even though they are equally talented).

– Many talented in a race could not afford suits because they were never given opportunity.

– One race would have less money than the others as a result

• Meaning attributed to Fads

Page 9: Simulating Civilization Change  with  Nexus Cognitive  Agents

9Social Impact Model

SISTER: Symbolic Interactionist Simulation of Trade and Emergent Roles

“SISTER: A Symbolic Interactionist Simulation of Trade and Emergent Roles” - Duong 1995, Duong and Grefenstette, JASSS 1/2005.

• Trade is good for “farmer” agents.– Agents need each of four food groups, and as much as they

can get of each.– Agents can make more food if they concentrate their efforts on

fewer of them.• Agents have efforts to spend on making or trading food as

they wish.• Agents can trade if they have corresponding trade plans.• Agents have a sign to display to attract trade.• To learn to trade, agents induce what signs to display and

what signs to trade with based on whatever gets them the most of each food in the four food groups at the end of the day.

Page 10: Simulating Civilization Change  with  Nexus Cognitive  Agents

10Social Impact Model

Coevolving Genetic Algorithms

• Each agent has an entire genetic algorithm that tells it:

– Where to place efforts– What sign to trade with,

what and how much to trade

– What sign to display

Page 11: Simulating Civilization Change  with  Nexus Cognitive  Agents

11Social Impact Model

SISTER: Emergent Phenomena

• Division of Labor• As agents learn to trade,

their utility increases, and their sign comes to mean a role

• Price• Goods become valued at

standard ratios• Money

• In a third of the runs, one good is traded for the purpose of trading again

• Different types of stores• Central bargain stores

and local convenience stores

Page 12: Simulating Civilization Change  with  Nexus Cognitive  Agents

12Social Impact Model

SISTER: Results

Page 13: Simulating Civilization Change  with  Nexus Cognitive  Agents

13Social Impact Model

Nexus

• SISTER is a “theoretical” symbolic interactionist simulation

– SISTER embodies the formation of social patterns of behavior

– Its scenarios are not realistic

• Nexus is a “data-driven” symbolic interactionist simulation

– In order to do analysis, we must start from a scenario in the real world.

– Being realistic and theoretically correct at the same time is difficult.

– Nexus attempts to mirror the virtuous and vicious cycles of the real world that created its input data.

Page 14: Simulating Civilization Change  with  Nexus Cognitive  Agents

14

What is Nexus Network Learner ?

• One of the two Nexus Cognitive Agent models that Debbie Duong wrote at the OSD/CAPE/Simulation Analysis Center.

– Nexus Network Learner models Social Role Networks– Nexus Schema Learner models Cognitive Dissonance

• A Simulation of Social Role Networks in which Agents learn: – To choose role partners to perform transactions with:

- Choice based on signs, social markers and communications on past transaction behaviors.

– Transaction behaviors and signs.- Choice based on signs and social markers.

– Based on Cultural Values. • Social markers, roles, transaction behaviors, signs, role-

based communications and cultural values are all input to the program.

– Population data determines the initial population tendencies.– Utility and motivation determines how they change.

Page 15: Simulating Civilization Change  with  Nexus Cognitive  Agents

15

How Does Nexus Network Learner Work?

• Artificial Intelligence Technologies represent Cognition.– Rule Based.

- An ontology of roles with crisp rules for roles. – Represents general social structures, that can be used in many

scenarios.– Defines utility of transactions.

– Machine Learning.- Bayesian networks initialize social markers , signs/transaction

behaviors, and role choice behaviors.- The Bayesian Optimization Algorithm (BOA) changes those

behaviors based on the utility of transactions.– BOA can be seeded with initialization data and injected data.– A form of Evolutionary Computation using reinforcement Learning

optimizes (satisfices) utility.– As conditionals change, the equilibrium point moves (in accord with

the New Institutional Economics).

Page 16: Simulating Civilization Change  with  Nexus Cognitive  Agents

16

What Happens in Nexus Network Learner?

• Individual Agents Choose Network Partners.– Ontology tells who may choose and how many.

- Example: an “Employer” may choose an average of 5 employees with a standard deviation of three.

– Bayesian network tells how the choices are ranked.– Passive role may have an option to reject offer.

- Example: an “Employee” may reject an employer because a role relation has told her he steals paychecks.

– Ontology may include a chance occurrence of natural attrition.• Individual Agents engage in transactions.

– Account distributions send funds through networks according to rules in ontology and transaction behaviors in Bayesian networks.

– Probability of observing, reporting, and knowing about behaviors are role-based.– Agents may go to jail, and not be allowed to participate in transactions for a time.

• Every N cycles, they judge their learned strategies by utility based on transactions that their valued role partners engaged in.

– Ontology determines culturally valued individuals and transactions.– After testing all strategies agents recombine them.

Page 17: Simulating Civilization Change  with  Nexus Cognitive  Agents

17

Performing Tests with Nexus Network Learner

• A wide variety of tests relevant to Irregular Warfare (IW) may be performed.

• For example, new network formations and behaviors may be tested based on many different things…

– The effect of different utility functions.- For example, make agents care only for self rather than larger social

network.– The effect of different penalties.

- For example, a penalty attribute that encodes different jail terms or different chances of getting caught.

– The effect of different exogenous resources.- For example, test resource rents or foreign aid.

– The effect of different abilities to observe.- For example, the effect of a media agent.

– The effect of removing different agents.- For example, measure how long it takes to replace a terrorist leader

• Monte Carlo methods reveal if new structures are the result of different CONOPS.

– Bayesian Networks make Nexus Stochastic

Page 18: Simulating Civilization Change  with  Nexus Cognitive  Agents

18

How Nexus Agents Learn

• As each agent learns, all the agents coevolve, making them very adaptive.

– Every agent has its own private learning algorithm.– Their behaviors effect the larger social structure and the larger

social structure effects their behaviors.- Micro-Macro Integration is modeled.

– They can adapt to data from other simulations and to initial country data as well.

• The learning algorithm in each agent makes the adaptation to data flexible.

– BOA (Bayesian Optimization Algorithm) can start learning from initial data.

- In the calibration phase. agents to adapt to initial data, so that they generate it though their perceptions and motivations.

- Thus they “explain” the data, going from correlation to cause.– This greater ability to ingest data also allows them to meld with

other simulations in a composition.- Together, composed simulations create a coherent picture of

the social environment.- Conflicts are resolved through mutual adaptation.

Page 19: Simulating Civilization Change  with  Nexus Cognitive  Agents

19Social Impact Model

Use Case: Modeling Corruption with Nexus Network Learner

Page 20: Simulating Civilization Change  with  Nexus Cognitive  Agents

Interpretive Social Science Used in Nexus

• From economics: The New Institutional Economics (NIE) (North)– Institutions (Social and Legal Norms and Rules) underlie economic activity

and constitute economic incentive structures– Institutions come from the efforts of agents to understand their

environment, so as to reduce uncertainty, given their limited perception– When some uncertainties are reduced, others arise, causing economic

change– To find the leverages to corruption, NIE would look at actor’s definition of

their environment, and how this changes incentives and thus institutions• From sociology: Symbolic Interactionism (Mead)

– Roles and Role Relations (such as in trade roles and trade relations) are learned, created during the display and interpretation of signs (such as gender, ethnicity, and other demographic characteristics)

– Institutions (social and legal norms and rules) are commonly accepted interpretations of symbols, that start out as a subjective perception and engrained in society as an objective rule

[1] See http://coase.org/niereadinglist.htm for an extended reading list[2] See Duong, Deborah Vakas, “The Generative Power of Signs: Tags for Cultural. Reproduction” Handbook of Research on Agent-Based Societies: Social and Cultural Interactions, Goran Trajkovsky and Samuel Collins, eds., 2008. http://www.scs.gmu.edu/~dduong/GenerativePowerOfSigns.pdf and also Blumer, Herbert (1969). Symbolic Interactionism: Perspective and Method. Berkeley: University of California Press.[3] Duong, Deborah Vakas and John Grefenstette. “SISTER: A Symbolic Interactionist Simulation of Trade and Emergent Roles”. Journal of Artificial Societies and Social Simulation, January 2005. http://jasss.soc.surrey.ac.uk/8/1/1.html.

Page 21: Simulating Civilization Change  with  Nexus Cognitive  Agents

Nexus Main Components

• Nexus models individuals and their interactions.

• Individuals have various roles on the three different networks and dynamically interact with other agents through these roles.

– Retailer – Customer– Government Employer – Government

Employee– Head Of Household– Dependent

• Role Networks are Input to Nexus

• Only commodity is Money – Agents pass money to other agents’

accounts– External support is in the form of

injections of funds to certain individuals (that have certain roles)

– Utility of agents (their “happiness”) is raised when they spend the money on things they need

Trade Network

Bureaucratic Network

KinshipNetwork

Role Interactions

IndividualDetermined TraitsSituational TraitsBehavioral Traits

Transaction-based UtilityExperienceCognition

InstitutionsEmerged Learned Attributes

External ControlUser-defined Policies

ExternalSupport

Page 22: Simulating Civilization Change  with  Nexus Cognitive  Agents

Nexus Main Components: Individuals

• They want their kin to be happy, and can think about how to adjust their behaviors towards that goal, based on experience of what met that goal in the past

• They have the demographic characteristics, both determined and situational, of the modeled country

– Determined Traits: Gender, Ethnicity, Age, etc.– Behavioral Traits: Tendency to Steal or Bribe, based on other traits and on

learning during the run– Situational Traits: Employment, Are they under penalty, etc.

• They actively seek role relationships, following socio-cultural rules about who proposes the relationship, what sort of person is chosen

– For example, a husband chooses a wife, or an employer chooses an employee – They judge others based on characteristics they can see or they have heard

rumors about• There role responsibilities include the distribution of funds to accounts they are

responsible for• They are happy when funds flow through certain accounts, for example, from the

household budget to the grocery store income.• They have differing length legal penalties, as well as social stigma

Page 23: Simulating Civilization Change  with  Nexus Cognitive  Agents

Nexus Main Components: Role and Role Relations

• There are eight types of corruption relations possible in the three networks (example actions provided):

– Nepotism: Hiring Kin/ Trade Network– Commission for Illicit Services: Bribing/Government Network– Misappropriation: Stealing/ Trade or Gov Network– Rig Election: Elected Officials bribing for Employment– Gratuity: Bribing/ Trade Network– Unwarranted Payment: Accepting Bribes/Government Network– Levy Toll Sidelining: Stealing/Government Network – Scam: Stealing From Customer in Trade Sector.

• There are many other types of role and role relations (64) in the model:– Each role has a corresponding role– Roles are dynamic (such as an agent can move from a government employee to unemployed)

Bureaucratic Network

Gov Employee Gov EmployerAttempts to Bribe

Accepts/Rejects Bribe

Observer

Page 24: Simulating Civilization Change  with  Nexus Cognitive  Agents

Nexus Main Components: Kinship Network

Active Roles Corresponding Passive RoleFather Child

Head Of Household Home Receiver

Husband Wife

Brother

Dependent (Provider)

MaternalAunt

MaternalCousin

MaternalGrandparent

Mother

Parent

PaternalCousin

PaternalGrandparent

PaternalUncle

Sibling

Sister

Spouse

Provider (Dependent)

• Three main active roles, from which 14 more are derived

• Derived Roles are used to model Residence

– Utility (Satisfaction) calculated based on Residence in Anthropology

– Matrilineal, Patrilineal of Neolocal

• Support account goes from Provider to Dependent

Derived Roles

Page 25: Simulating Civilization Change  with  Nexus Cognitive  Agents

Nexus Main Components: Trade Network

• Derived Roles are ten different income levels• Accounts include personal salary, employee salaries, money for

office purchases

Active Roles Corresponding Passive RoleHeadOfCorporation CorporateReceiver

Customer Retailer

Employer Employee

Purchaser Vendor

Service Providee Service Provider

Page 26: Simulating Civilization Change  with  Nexus Cognitive  Agents

Nexus Main Components: Bureaucratic Network

Active Roles Corresponding Passive RoleTaxpayer Taxman

GovernmentEmployer GovernmentEmployee

GovernmentPurchaser GovernmentVendor

HeadOfGovernment GovernmentReceiver

Service Providee Service Provider

• Derived Roles are ten different pay grades• Accounts include corporate and income taxes, government salaries,

government office money, government money for purchases

Page 27: Simulating Civilization Change  with  Nexus Cognitive  Agents

27Social Impact Model

Experiments

Page 28: Simulating Civilization Change  with  Nexus Cognitive  Agents

28

Experiment: Stuck in Stealing Mode

• Comparison of the evolution of an African society which initiates in a strong vicious cycle of stealing to one with more moderate levels of stealing.

• If they started out stealing excessively, they never learned not to, never attempted to find service providers who wouldn’t steal from them

• If the stealing is in more moderate amounts, agents learn to find service providers that do not steal from them within two years

• Agents in a stealing vicious cycle never use bribing to accomplish goals

• Agents with moderate stealing used bribes, but after fifteen years, employers and service providers stopped bribing

• Convergence occurs at the fifteen year mark• After 15 years in both the excessive stealing scenario and the

‘normal’ stealing scenario, we see homogeneous responses across strategies. For instance, agents generally provided the same response for a particular parameter (say ‘Bribe-ForServices’) after 15 years as opposed to more heterogeneous responses for the same parameter after two years.

• Implication: Diversity of Behavior is needed for flexibility

Page 29: Simulating Civilization Change  with  Nexus Cognitive  Agents

29

Experiment: Reverse Engineer Terrorist Networks

• In the US Army 2011 Tactical Wargame Nexus Network Learner was used to simulate terrorist networks in Afghanistan.

• Added onto the above dynamic networks were a Pashtoon Tribal network, a terrorist network, and a coalition forces network.

• In these networks, if agents representing host nation citizens obtained a role they behaved in that role in the expected frequency. They also noted others behaviors according to that role, and had a chance of discussing that behavior with coalition forces according to the role and the popularity of coalition forces.

• Intelligence Community Wargame players were able to use the behavioral messages to reconstruct the terrorist networks.

• The dynamic and changing nature of networks made this task overly challenging to those who lack Intelligence training

Page 30: Simulating Civilization Change  with  Nexus Cognitive  Agents

30Social Impact Model

Summary

Page 31: Simulating Civilization Change  with  Nexus Cognitive  Agents

31

Symbolic Interactionist Simulation

• Symbolic Interactionist Simulation is a form of Reinforcement Learning by Coevolution.

• Agents learn associated rules in the form of actions to take with other agents based on signs displayed and read.

• Symbolic Interactionist Simulation can be Theoretical (SISTER) or Data-Driven (Nexus).

• Symbolic Interactionist Simulation can model the motivation based vicious and virtuous cycles of behavior that determine social structure

Page 32: Simulating Civilization Change  with  Nexus Cognitive  Agents

32Social Impact Model

Questions and Comments

POC: Deborah Duong [email protected]