Slide 1

Amazon Mechanical Turk (Mturk) What is MTurk? Crowdsourcing Internet marketplace that utilizes human intelligence to perform tasks that computers are unable to do Mturk samples are Fast Cheap Reliable Diverse

Slide 2

First Things First Go to www.mturk.comwww.mturk.com Click here

Slide 3

First Things First Sign in with existing Amazon.com account or create one

Slide 4

Getting Started You are called a requester

Slide 5

Design The space that hosts your Qualtrics survey link is called a HIT Your HIT is published to the Mturk site for workers to see The first step is to design the layout of your HIT

Slide 6

Slide 7

Properties The title and description are seen by workers Keywords categorize your HIT and make it searchable

Slide 8

Properties Set time restrictions Chose Mturk Masters option Set qualifications Choose qualification requirement

Slide 9

Properties Choose payment amount per HIT Choose number of participants The next step will be to design the layout of the HIT

Slide 10

Layout Uses html code Give instructions and brief description of study to workers Provide survey link

Slide 11

Layout Template name should identify your study. This is not displayed to workers. Keep any of this you want and delete the rest.

Slide 12

Layout Code word and study instructions Qualtrics link Confirmation code User comments

Slide 13

Layout Use a code word to prevent workers from repeating similar surveys Lets say you create a survey about prejudice and use code word ALPHA for the HIT Inform workers before they begin that the HIT uses code word ALPHA After data collection you would like to re-run the study after making some changes on Qualtrics You dont want workers who completed the first round to do the second round also Again, inform workers that the HIT uses code word ALPHA and that if they had taken an ALPHA HIT before, they should not continue

Slide 14

Layout Other ways of controlling for repeats In Qualtrics, create an item at the beginning of the survey that asks the participant if they have completed a HIT from this requester before If no, okay If yes, ask if the HIT theyve done uses undesired code word (in this case, its ALPHA) If yes, send participant to end of survey message If no, okay Ask at end of survey if participant had completed a similar survey in the past Yes, no, I dont remember

Slide 15

Layout Confirmation code

Slide 16

Layout Confirmation Code Create a space for workers to enter the confirmation code you provide at the end of Qualtrics survey Code can be made up Used to check that workers completed entire survey

Slide 17

Layout

Slide 18

Space Link Space

Slide 19

Layout Space Instructions Text box

Slide 20

Layout Space Instructions Text box

Slide 21

Layout html code Click here to take the survey. !survey link !space At the end of the survey, you will be provided with a confirmation code. Please enter the code below to verify that you have completed the task: !code box !space Please provide any comments you may have below, we appreciate your input! !comment box

Slide 22

Preview and Finish

Slide 23

Publish

Slide 24

Publish HIT Name the batch Negative balance

Slide 25

Publish Note: If your Amazon account is funded, the HIT will be published right away If not, you will be notified of this and be able to click a link which will allow you to fund your account using a variety of methods

Slide 26

Manage Watch completion rate Accept or reject work Cancel batch

Slide 27

Manage Click here to accept work (pay participants)

Slide 28

Manage Confirmation code Marking worker and clicking approve will initiate payment Clicking worker ID allows you to block worker and pay bonuses Approved 1 HIT out of 1 for this worker

Slide 29

MTurk Things to consider Blocking workers is BAD Average pay anywhere from $.10 to $1.00 Command line tools Professionalism Turkernation.com SPSP Seminar on MTurk

Slide 30

Thank you! Matt Baldwin: mwbaldwin@ku.edumwbaldwin@ku.edu Lucas Keefer: lkeefer1@ku.edulkeefer1@ku.edu