That’s why I’m providing the (commented) code that I generated to create my MechTurk experiment. A short demo version of the experiment can be found here.
If you find this code helpful, please consider acknowledging it via the following URL in your paper/presentation to spread the word:
The structure of the experiment, which combines timed image presentation, playing audio via HTML5, recording naturalness ratings via radio buttons, and recording feedback via a text box:
- Participants first see an instructions page with a CONTINUE button. Upon clicking CONTINUE,
- they continue to a sound test. This is to make sure participants can actually hear sound via their speakers/headphones. They click PLAY and are asked to enter the first word of the phrase they hear. If the word is entered correctly,
- the main body of the experiment begins. Each trial consists of
- displaying an image of a full gumball machine, which after 1.5 seconds changes such that
- a second image is displayed where a certain number of gumballs has moved to the lower chamber. An audio file is played (statements of the form “You got X of the gumballs”). Next to the gumball machine, a 7-point scale of radio buttons is displayed on which participants are asked to rate the naturalness of the statements they heard as a description of the scene. Below the scale, there is a FALSE button that participants are asked to click if they think the statement was false. When a button is clicked, information about which button it was is recorded and the next trial begins.
- On the last trial, a feedback box is displayed alongside a SUBMIT button. Upon clicking SUBMIT, all the information about the experiment (worker ID, trial information, participants’ responses, etc) is submitted to Mechanical Turk. Note that in the demo version I’ve made available, the information is printed to a window that pops up when you click the SUBMIT button.
Good luck. Questions & comments welcome!