An Open Source Platform for Social Science

Research

Sally KleinfeldtBarcelona

2017

The Program

Next Generation Social Sciences

(NGS2)

Funded by:

"The program aims to build and evaluate new methods and tools to advance rigorous,

reproducible social science studies at scales necessary to develop and validate causal

models of human social behaviors."

Benefits for

• Public health

• Economics

• National security

Initial Focus

Identify causal mechanisms of

collective identity formation

Dallinger

Grant Awarded To

Dallinger

• Crowdsourced experiments

• Abstracted into single function calls

• Can be inserted into higher-order algorithms, for example to progressively refine experiment

Fully Automated

• Recruits participants (Mechanical Turk)

• Obtains informed consent

• Arranges participants into a network

• Runs experiement (Heroku)

• Coordinates communication

Fully Automated

• Records the data they produce

• Pays participants

• Recruits new batches of participants contingent on the structure of the experiment

• Validates and manages the resulting data

How Does It Work?

• Experiments are modeled as directed graphs

• Experiments are like Plone add-ons being run by the Dallinger system

Games!

• All teams coalesced on using a public goods game

• Dallinger is the only team using a real-time multiplayer game: Grid Universe

Demo

Repository

• github.com/Dallinger/Dallinger

Jazkarta’s Contribution

Why Jazkarta?

• Expertise in Python, Flask, PostgreSQL, SQLAlchemy, Amazon Mechanical Turk, boto, tox, pytest, Redis, Selenium, PhantomJS, JavaScript, HTML, CSS…

• Expertise in project management

• Expertise in a mature open source community - Plone!

Our Process

• Discovery meeting fall 2017

• Developed user stories

• Estimated using planning poker

• Implementing in a series of iteration

Our Team

• Alec Mitchell

• David Glick

• Matthew Wilkes

• Carlos de la Guardia

• Jesse Snyder

Lessons Learned from Plone

Lessons

• Don’t over-engineer plugin architectures (like recruiters)

• Support live editing as much as possible

• Break backwards compatibility when needed

• Remove references to old ways of doing things

Lessons

• Ship lots of useful demos

• Be diligent about code reviews

• Make important approach decisions together

• People involved in decisions should know user needs intimately (we miss Joel Burton)

Plone vs. Dallinger Tech Stack

Tech Stack

• Web based, but Flask instead of Zope

• PostgreSQL instead of ZODB

• Real time websockets

• Built in deployment command

Writing Experiments

Helping Dallinger Users

• Documentation

• Slack channel

• Cookie cutter template

• Extendable base templates

• Javascript library

• Commands for local debugging

Ensuring Code Quality

Code Quality

• Automated lint checks

• Continuous integration with minimum code coverage requirement

• Code review

• Regression testing of an experiment (GU) against changes to core Dallinger

Fun Technical Challenges

Fun Challenges

• Scaling selenium-based bots

• Getting access to track interactions with 3rd-party sites (Chrome extension)

• Testing multiple participants in parallel without sharing cookies

Questions?