Tuesday, August 21, 2012

Random assingments to different surveys

Qualtrics can randomly assign participants to conditions or blocks within a given survey pretty well. But what happens if you have several surveys and you want to randomly assign participants across all surveys?

When I needed to do that, I decided to create another survey which I called "randomizer". In that survey's flow, I added a Randomizer element that contained several End of Survey elements. In each of the End of Survey elements, I customized the End of survey message to be a redirection to one of my actual surveys (see image below).

This way, I could direct all my participants to one url, the one for the "randomizer" survey, and from there they would be randomly re-directed to one of my surveys.


Thursday, July 26, 2012

Question's timestamp on Qualtrics

If you want Qualtrics to record the full timestamp (mm/dd/yyyy hh:mm:ss.ss) a participant viewed or clicked on a question, and not just the response time, you can use this javascript code:

Qualtrics.SurveyEngine.addOnload(function()
{  
  var timeStampOfOnLoad = new Date();
  Qualtrics.SurveyEngine.setEmbeddedData('timeStampOfOnLoad_QID1', timeStampOfOnLoad);
  //alert('QID1 '+timeStampOfOnLoad);
  this.questionclick = function(event,element){
    var timeStampOfAnswer = new Date();
    Qualtrics.SurveyEngine.setEmbeddedData('timeStampOfAnswer_QID1', timeStampOfAnswer);
    //alert('QID1 '+timeStampOfAnswer);
  }
}
);

Find your question on Qualtrics and click on the let-hand button and choose "Add Javascript". Copy and paste the above code (in whole) and change the "QID" parameter to your question's ID. 
You will also need to add an Embedded Data element to your survey flow with the same name as the value after the ".setEmbeddedData" function in the code (e.g., timeStampOfOnLoad_QID1 ). The first var in the code records the timestamp the question was viewable to the participants and the second one records the timestamp of the first click the participant made. If you want both, create two Embedded Data elements in your survey flow.

Bulk bonuses on MTurk

We (my RA Rob and I) found a way to give bonuses to multiple workers at a time. Here's how you can do it:

  1. Download and Install mTurk Command Line Tools (http://aws.amazon.com/developertools/694)
  2. Download your Results file from the Manage tab on MTurk
  3. On a new sheet, create the following columns that will read the info from the results sheet:
    1. Workerid
    2. Bonus (a value you should enter)
    3. Assignmentid
    4. Reason (a value you should enter)
  4. Then, create a formula that combines these values to the following format:
    grantBonus –workerId [WorkerID] –amount [Bonus Amount, in Dollars] ­–assignment [AssignmentID] –reason [Reason for Bonus, in quotation marks]
    The formula should be something like:
    ="grantbonus "&"-workerid "&A2&" -amount "&B2&" -assignment "&C2&" -reason "&D2

    The result of the formula should be like this, for example:
    grantBonus –workerId A3C4G8DMXFG5PQ –amount 0.25 ­–assignment ZYJZWSCAT0DZRFY5KYP00S0ZS8Y5H0NZR9YAMY1Z –reason "Good job"
  5. Populate the columns and formula with all the rows you need
  6. Copy and paste all of the populated formula rows into the Command Line Tool.The commands will run automatically, except for the last row. You will need to push enter one more time to run the last command.
  7. You should get a message saying "bonus assigned to worker X" after each row
  8. Note: If you get an error message that says you need to be on a secure connection for the commands to run, go to the “bin” folder within the mTurk program folder (which should be something like “mech-turk-tools-1.3.0” unless you named it something else). Open “mturk.properties” as a text file, and replace anything that says “http” with “https.”
  9. If you want to check that the workers got the bonuses, you should go the Manage HITs Individually on MTurk, find your HIT, and click on Review Submissions. The bonus should appear below each worker's row.
Good luck!

Screening AMT participants from previous studies

Sometimes, you want to make sure some AMT workers, who took one of your previous studies, cannot take your current study. We found a way to do that using a simple script on AMT and a pre-screening survey on Qualtrics. More details in this post on the Experimental Turk blog.

Welcome

Welcome to my new blog!

I like doing research using Amazon Mechanical Turk and Qualtrics. I picked up a few tips and tricks along the way and here's the place I will share them.