Thursday, July 26, 2012

Bulk bonuses on MTurk

We (my RA Rob and I) found a way to give bonuses to multiple workers at a time. Here's how you can do it:

  1. Download and Install mTurk Command Line Tools (https://requestersandbox.mturk.com/developer/tools/clt)
  2. Download your Results file from the Manage tab on MTurk
  3. On a new sheet, create the following columns that will read the info from the results sheet:
    1. Workerid
    2. Bonus (a value you should enter)
    3. Assignmentid
    4. Reason (a value you should enter)
  4. Then, create a formula that combines these values to the following format:
    grantBonus –workerId [WorkerID] –amount [Bonus Amount, in Dollars] ­–assignment [AssignmentID] –reason [Reason for Bonus, in quotation marks]
    The formula should be something like:
    ="grantbonus "&"-workerid "&A2&" -amount "&B2&" -assignment "&C2&" -reason "&D2

    The result of the formula should be like this, for example:
    grantBonus –workerId A3C4G8DMXFG5PQ –amount 0.25 ­–assignment ZYJZWSCAT0DZRFY5KYP00S0ZS8Y5H0NZR9YAMY1Z –reason "Good job"
  5. Populate the columns and formula with all the rows you need
  6. Copy and paste all of the populated formula rows into the Command Line Tool.The commands will run automatically, except for the last row. You will need to push enter one more time to run the last command.
  7. You should get a message saying "bonus assigned to worker X" after each row
  8. Note: If you get an error message that says you need to be on a secure connection for the commands to run, go to the “bin” folder within the mTurk program folder (which should be something like “mech-turk-tools-1.3.0” unless you named it something else). Open “mturk.properties” as a text file, and replace anything that says “http” with “https.”
  9. If you want to check that the workers got the bonuses, you should go the Manage HITs Individually on MTurk, find your HIT, and click on Review Submissions. The bonus should appear below each worker's row.
Good luck!

22 comments:

  1. Thanks. Quite simple and quite helpful.

    ReplyDelete
  2. Incredibly useful. Thanks a ton.

    ReplyDelete
  3. I am trying to use this. It just doesn't work! Can someone help me, please?

    ReplyDelete
    Replies
    1. I'll be happy to. Can you describe the error or problem that you are having?

      Delete
    2. Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version n
      umber in .class file
      at java.lang.ClassLoader.defineClass1(Native Method)
      at java.lang.ClassLoader.defineClass(Unknown Source)
      at java.security.SecureClassLoader.defineClass(Unknown Source)
      at java.net.URLClassLoader.defineClass(Unknown Source)
      at java.net.URLClassLoader.access$100(Unknown Source)
      at java.net.URLClassLoader$1.run(Unknown Source)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(Unknown Source)
      at java.lang.ClassLoader.loadClass(Unknown Source)
      at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
      at java.lang.ClassLoader.loadClass(Unknown Source)
      at java.lang.ClassLoader.loadClassInternal(Unknown Source)
      at com.amazonaws.mturk.service.axis.AWSService.(AWSService.java:
      104)
      at com.amazonaws.mturk.cmd.AbstractCmd.initService(AbstractCmd.java:119)

      at com.amazonaws.mturk.cmd.AbstractCmd.run(AbstractCmd.java:143)
      at com.amazonaws.mturk.cmd.GrantBonus.main(GrantBonus.java:30)


      This is the error I get!

      Delete
  4. This tip looks very useful, thank you! The title suggests this tip is specifically for MTurk linked to Qualtrics, but this would work if I was using SurveyMonkey, correct?

    ReplyDelete
    Replies
    1. Ah don't worry, I just realised that wasn't the title of the post. Thanks!

      Delete
  5. Hi, thanks for your tips but it doesn't work for me. I get this message after i copy and pasted the command in:

    C:\mech-turk-tools-1.3.1\bin>grantbonus -workerid A35L0V737W6DP4 -amount 0.06 -a
    ssignment 2CBENLVWJ5OPSL6PU5ZYTWB71YI1T3 -reason "Thank you! You have answered t
    he attention check questions correctly!"
    Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version n
    umber in .class file
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(Unknown Source)
    at java.security.SecureClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.access$100(Unknown Source)
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClassInternal(Unknown Source)
    at com.amazonaws.mturk.service.axis.AWSService.(AWSService.java:
    104)
    at com.amazonaws.mturk.cmd.AbstractCmd.initService(AbstractCmd.java:119)

    at com.amazonaws.mturk.cmd.AbstractCmd.run(AbstractCmd.java:143)
    at com.amazonaws.mturk.cmd.GrantBonus.main(GrantBonus.java:30)

    Can someone please help?

    ReplyDelete
    Replies
    1. Hey Anonymous,

      I had the same problem and found the solution in the thread here: https://forums.aws.amazon.com/message.jspa?messageID=388586 (the step-by-step instructions from uwviscoglab). Hope this helps!

      Delete
    2. I tried following those steps, but I do not have JAVA_HOME saved as a system variable!

      Delete
    3. Alia, what worked for me was create the variable as New..., and point JAVA_HOME to the JRE folder, in my case."C:\Program Files (x86)\Java\jre1.7.0_06" folder.

      Delete
    4. This solution worked. Just search for uwviscoglab's answer on the link provided above by maxim.

      One more thing, if you get the following error: "the system cannot find the path specified." -- check to make sure that mturk tools are installed in the Program Files (x86) folder [or, whichever folder has your Java files]. So, the path for mturk tools should be: "C:\Program Files (x86)\mech-turk-tools-1.3.1\"

      Mine first installed in "C:\mech-turk-tools-1.3.1\" which led to the "the system cannot find the path specified" error

      Delete
  6. Hi Eyal,

    Thanks for your informative blog post. I don't know why I didn't see it a year ago when I needed it, but I eventually wrote a script that did all that, at: https://github.com/desmond-ong/amtBonusScript

    I tried to make it self-contained with clear instructions, and all it requires is that you have Amazon Command Line Tools installed (and be on a Mac/Unix environment). I would greatly appreciate any feedback and/or help (e.g. porting it to Windows!). I hope this helps you and any readers of this blog out!

    ReplyDelete
  7. Does a HIT have to be created using CLT in order to use the CLT bonus approach described here? I downloaded a CSV file in excel from a HIT I had already posted. Each time I try to bonus a worker I get a return saying "workerid missing." Which it isnt of course. Anythoughts? This would make my life so much easier if I could get it to work. I personally find CLT so incredibly frustrating.

    ReplyDelete
    Replies
    1. Not it doesn't. Make sure the formula on excel is grabbing the correct field for worker id. Also make sure you're using the correct results file.

      Delete
    2. I had the same problem. It turned out to be the "-" in the command. This is the right one - and this is the wrong one –.

      Delete
  8. Hi Eyal and others,
    I really need to bonus AMT workers by batch but cannot work out the method presented here. I would be very grateful if anyone could share the material, code, Excel spreadsheet they have used to manage this. FYI I am a PC user (sorry!). Thank you very much in advance.
    Marie

    ReplyDelete
  9. This comment has been removed by the author.

    ReplyDelete
  10. This just saved me so much time. Thank you!

    ReplyDelete
  11. Hi,

    This method really saved me a lot of effort and time. However, I don't quite understand how I could check whether the workers have been granted bonus accordingly. I clicked on "Manage HITs individually" but all I could see are HITs for my other studies.

    I really appreciate if anyone could advise me on this.

    Thank you so much!

    ReplyDelete
  12. Great post! I am actually getting ready to across this information, It's very helpful for this blog.Also great with all of the valuable information you have Keep up the good work you are doing well.
    Embedded Training in Chennai

    ReplyDelete
  13. I guess AWS has changed the command. I'm using Windows Command Prompt to run these commands, and the formula right now should be:
    ="aws mturk "&"send-bonus "&"--worker-id "&A3&" --bonus-amount "&B3&" --assignment-id "&C3&" --reason "&D3

    ReplyDelete