scalable agent-based modelling with cloud hpc resources for social simulations
DESCRIPTION
New concepts like agent-based modelling are providing social scientists with new tools, more suited to their background than other simulation techniques. The success of this new trend will be strongly related to the existence of simulation tools capable of fulfilling the needs of these disciplines. Given the computational requirement of realistic agent-based models, high-performance computing infrastructure is often necessary to perform the calculations. At present, such resources are unlikely to be available to humanities researchers. Having developed Pandora, an open-source framework designed to create and execute large-scale social simulations in high-performance computing environments, this work presents an evaluation of the impact of cloud computing within this context. We find that the constraints of the cloud environment do not have a significant impact on the generic pattern of execution, providing a cost-effective solution for social scientists.TRANSCRIPT
1
Scalable Agent-based Modelling with Cloud HPC Resources for Social
Simulations
Peter Wittek – University of BorasXavier Rubio-Campillo – Barcelona Supercomputing Centre
Introduction
Computer simulation is increasingly used as a research tool in Humanities and Social Sciences projects
ABM is one the most promising tools, but its computing requirements can be high
HPC is the answer to this issue, but there are few applications of social simulation in these environments...
Why? What role plays Cloud Computing in the solution?
Agent-Based Modelling in Social Sciences
Inspired on SugarScape model
Two key concepts:Agent → individual entityEnvironment → cellular automata
The agent is defined by:Attributes defining its internal stateA “step” method containing predefined rules of behavior
ABM benefits & issues
ABM is attractive to social scientists because:Bottom-up approach to social phenomenaHeterogeneity and environment Advanced decision-making processesCloser to social science & humanities thinking
But we need to address some problems:Wrong understanding of what a model is (and is not)How to develop and understand a piece of software?How to analyze and share results?
ABM computational challenges
How can we distribute an execution in an efficient way?
Is possible to use more than 1 CPU/node?
How to serialize a constant rate of data without bottlenecks?
An example: distributing past societies
The requirements are diverse, given the broad scope of ABMWe need to address these challenges for different scenarios
i.e.: to model past societies we should take into account these properties:
Importance of environmentLow number of agentsIntensive local communication
How can we use this information to distribute the ABM?
The Pandora framework
Pandora is an open-source C++ framework developed to accomplish this task.Implementation & execution:
Automatic generation of code for parallel executionPython interfaceDistributed serialization (HDF5)
Analysis:GIS support (GRASS) & Statistical support (R package)Cassandra: an ABM visualization tool
Pandora workflow
Distributing the simulation
Order of execution
Breaking sequentiality
From:step()
To:updateKnowledge()selectAction()execute()updateState()
ABM in the cloud
HPC-CC is an interesting option because:ABM requires considerable computational powerSeveral Humanities/Social Sciences departments do not have access to HPC resourcesIt is difficult to share data
...but how this solution can impact the performance of an ABM framework like Pandora?
The experimental model
We will explore this problem with a model currently in use for archaeologists: gujaratSimThis ABM explores interactions between 2 populations competing for resources in a common environment:
hunter/gatherersPastoralists
Some details of gujaratSim are:Advanced decision making process (Markov Decision Process)Intensive use of enviromental data (biomass, water, etc.)Exploration of different population sizes
Infrastructures
The experiment will test 2 CC environments and a supercomputing infrastructure:
CC → Amazon Web ServicesEC2-regularEC2-clusterEC2-cluster unthrottled
Supercomputer → Mare NostrumEach node has 2 dual core IBM PowerPc 970 MP at 2.3 GhZMyrinet high-speed network
Performance experiments
We will test 2 different configurations with 2x2 and 3x3 nodes
CPU intensiveLow number of agentsIntensive MDP algorithm
Communication intensiveHigh number of agents'Wired' (rule-based) agent behavior
CPU intensive benchmark
Parallel efficiency
Load balance
CPU intensive MPI trace
Communication intensive benchmark
Parallel efficiency
Load balance
Communication intensive MPI trace
Conclusions
ABM is one of the most promising computational tools for investigating social phenomenaWe need to fulfill its computational requirements using HPCCloud-based HPC and multiplatform software are needed for this task
...can we develop GPU-based solutions?
...can we lower technical knowledge needed to develop distributed ABMs?
20
Thank you!
There is a need for cloud HPC in social simulationPandora allows to deploy ABM in a cloud-based solutionSupercomputers and Cloud-HPC are complementary, given the diversity of problems to solve
If you are interested you can download Pandora from: http://github.com/xrubio/pandora