Thursday, July 26, 2012

Bulk bonuses on MTurk

We (my RA Rob and I) found a way to give bonuses to multiple workers at a time. Here's how you can do it:

  1. Download and Install mTurk Command Line Tools (https://requestersandbox.mturk.com/developer/tools/clt)
  2. Download your Results file from the Manage tab on MTurk
  3. On a new sheet, create the following columns that will read the info from the results sheet:
    1. Workerid
    2. Bonus (a value you should enter)
    3. Assignmentid
    4. Reason (a value you should enter)
  4. Then, create a formula that combines these values to the following format:
    grantBonus –workerId [WorkerID] –amount [Bonus Amount, in Dollars] ­–assignment [AssignmentID] –reason [Reason for Bonus, in quotation marks]
    The formula should be something like:
    ="grantbonus "&"-workerid "&A2&" -amount "&B2&" -assignment "&C2&" -reason "&D2

    The result of the formula should be like this, for example:
    grantBonus –workerId A3C4G8DMXFG5PQ –amount 0.25 ­–assignment ZYJZWSCAT0DZRFY5KYP00S0ZS8Y5H0NZR9YAMY1Z –reason "Good job"
  5. Populate the columns and formula with all the rows you need
  6. Copy and paste all of the populated formula rows into the Command Line Tool.The commands will run automatically, except for the last row. You will need to push enter one more time to run the last command.
  7. You should get a message saying "bonus assigned to worker X" after each row
  8. Note: If you get an error message that says you need to be on a secure connection for the commands to run, go to the “bin” folder within the mTurk program folder (which should be something like “mech-turk-tools-1.3.0” unless you named it something else). Open “mturk.properties” as a text file, and replace anything that says “http” with “https.”
  9. If you want to check that the workers got the bonuses, you should go the Manage HITs Individually on MTurk, find your HIT, and click on Review Submissions. The bonus should appear below each worker's row.
Good luck!

97 comments:

  1. Thanks. Quite simple and quite helpful.

    ReplyDelete
  2. Incredibly useful. Thanks a ton.

    ReplyDelete
  3. I am trying to use this. It just doesn't work! Can someone help me, please?

    ReplyDelete
    Replies
    1. I'll be happy to. Can you describe the error or problem that you are having?

      Delete
    2. Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version n
      umber in .class file
      at java.lang.ClassLoader.defineClass1(Native Method)
      at java.lang.ClassLoader.defineClass(Unknown Source)
      at java.security.SecureClassLoader.defineClass(Unknown Source)
      at java.net.URLClassLoader.defineClass(Unknown Source)
      at java.net.URLClassLoader.access$100(Unknown Source)
      at java.net.URLClassLoader$1.run(Unknown Source)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(Unknown Source)
      at java.lang.ClassLoader.loadClass(Unknown Source)
      at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
      at java.lang.ClassLoader.loadClass(Unknown Source)
      at java.lang.ClassLoader.loadClassInternal(Unknown Source)
      at com.amazonaws.mturk.service.axis.AWSService.(AWSService.java:
      104)
      at com.amazonaws.mturk.cmd.AbstractCmd.initService(AbstractCmd.java:119)

      at com.amazonaws.mturk.cmd.AbstractCmd.run(AbstractCmd.java:143)
      at com.amazonaws.mturk.cmd.GrantBonus.main(GrantBonus.java:30)


      This is the error I get!

      Delete
  4. This tip looks very useful, thank you! The title suggests this tip is specifically for MTurk linked to Qualtrics, but this would work if I was using SurveyMonkey, correct?

    ReplyDelete
    Replies
    1. Ah don't worry, I just realised that wasn't the title of the post. Thanks!

      Delete
  5. Hi, thanks for your tips but it doesn't work for me. I get this message after i copy and pasted the command in:

    C:\mech-turk-tools-1.3.1\bin>grantbonus -workerid A35L0V737W6DP4 -amount 0.06 -a
    ssignment 2CBENLVWJ5OPSL6PU5ZYTWB71YI1T3 -reason "Thank you! You have answered t
    he attention check questions correctly!"
    Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version n
    umber in .class file
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(Unknown Source)
    at java.security.SecureClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.defineClass(Unknown Source)
    at java.net.URLClassLoader.access$100(Unknown Source)
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClassInternal(Unknown Source)
    at com.amazonaws.mturk.service.axis.AWSService.(AWSService.java:
    104)
    at com.amazonaws.mturk.cmd.AbstractCmd.initService(AbstractCmd.java:119)

    at com.amazonaws.mturk.cmd.AbstractCmd.run(AbstractCmd.java:143)
    at com.amazonaws.mturk.cmd.GrantBonus.main(GrantBonus.java:30)

    Can someone please help?

    ReplyDelete
    Replies
    1. Hey Anonymous,

      I had the same problem and found the solution in the thread here: https://forums.aws.amazon.com/message.jspa?messageID=388586 (the step-by-step instructions from uwviscoglab). Hope this helps!

      Delete
    2. I tried following those steps, but I do not have JAVA_HOME saved as a system variable!

      Delete
    3. Alia, what worked for me was create the variable as New..., and point JAVA_HOME to the JRE folder, in my case."C:\Program Files (x86)\Java\jre1.7.0_06" folder.

      Delete
    4. This solution worked. Just search for uwviscoglab's answer on the link provided above by maxim.

      One more thing, if you get the following error: "the system cannot find the path specified." -- check to make sure that mturk tools are installed in the Program Files (x86) folder [or, whichever folder has your Java files]. So, the path for mturk tools should be: "C:\Program Files (x86)\mech-turk-tools-1.3.1\"

      Mine first installed in "C:\mech-turk-tools-1.3.1\" which led to the "the system cannot find the path specified" error

      Delete
  6. Hi Eyal,

    Thanks for your informative blog post. I don't know why I didn't see it a year ago when I needed it, but I eventually wrote a script that did all that, at: https://github.com/desmond-ong/amtBonusScript

    I tried to make it self-contained with clear instructions, and all it requires is that you have Amazon Command Line Tools installed (and be on a Mac/Unix environment). I would greatly appreciate any feedback and/or help (e.g. porting it to Windows!). I hope this helps you and any readers of this blog out!

    ReplyDelete
  7. Does a HIT have to be created using CLT in order to use the CLT bonus approach described here? I downloaded a CSV file in excel from a HIT I had already posted. Each time I try to bonus a worker I get a return saying "workerid missing." Which it isnt of course. Anythoughts? This would make my life so much easier if I could get it to work. I personally find CLT so incredibly frustrating.

    ReplyDelete
    Replies
    1. Not it doesn't. Make sure the formula on excel is grabbing the correct field for worker id. Also make sure you're using the correct results file.

      Delete
    2. I had the same problem. It turned out to be the "-" in the command. This is the right one - and this is the wrong one –.

      Delete
  8. Hi Eyal and others,
    I really need to bonus AMT workers by batch but cannot work out the method presented here. I would be very grateful if anyone could share the material, code, Excel spreadsheet they have used to manage this. FYI I am a PC user (sorry!). Thank you very much in advance.
    Marie

    ReplyDelete
  9. This comment has been removed by the author.

    ReplyDelete
  10. This just saved me so much time. Thank you!

    ReplyDelete
  11. Hi,

    This method really saved me a lot of effort and time. However, I don't quite understand how I could check whether the workers have been granted bonus accordingly. I clicked on "Manage HITs individually" but all I could see are HITs for my other studies.

    I really appreciate if anyone could advise me on this.

    Thank you so much!

    ReplyDelete
  12. I guess AWS has changed the command. I'm using Windows Command Prompt to run these commands, and the formula right now should be:
    ="aws mturk "&"send-bonus "&"--worker-id "&A3&" --bonus-amount "&B3&" --assignment-id "&C3&" --reason "&D3

    ReplyDelete
    Replies
    1. can you confirm this is the way to do this now?

      Delete
  13. Which of these two should I download to start this process?

    AWS Command Line Interface
    AWS Tools for PowerShell

    ?

    Also, which of these two is the right command:

    ="aws mturk "&"send-bonus "&"--worker-id "&A3&" --bonus-amount "&B3&" --assignment-id "&C3&" --reason "&D3

    OR

    ="grantbonus "&"-workerid "&A2&" -amount "&B2&" -assignment "&C2&" -reason "&D2

    ReplyDelete
  14. Great tips! Picoworkers is another nice site for small jobs. The tasks there includ sign up jobs, Facebook/Twitter likes, YouTube subscribe/comment, app downloads, and more.

    Most tasks are easy to complete and there's always plenty of work available. They also pay for both employer and worker referrals.

    Sign up at the link below to claim your $1.50 sign up bonus:

    http://9nl.org/picoworkers

    ReplyDelete
  15. Those guidelines additionally worked to become a good way to recognize that other people online have the identical fervor like mine to grasp great deal more around this condition. . aws training in chennai

    ReplyDelete
  16. Thanks for this! Here's the Excel code that worked for me:
    ="aws mturk send-bonus "&"--worker-id "&A695&" --bonus-amount "&C695&" --assignment-id "&B695&" --reason "&E695&" --profile user2"

    Looks like this when pasted into Windows PowerShell
    aws mturk send-bonus --worker-id AZV8213XXXXXX --bonus-amount 0.5 --assignment-id 3Z2R0DQ0JHEWU7TQA5DE6U5XXXXXXX --reason "Donations Study" --profile user2

    Note that I connected my AWS CLI to two paired MTurk and AWS account sets. Appending "--profile user2" to each line targets the second profile manually specified in my credential/config files. To do this, follow these instructions: https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html

    ReplyDelete
    Replies
    1. Hi Bradley, Just a Q. Do you get a message like the one the author described (i.e.:You should get a message saying "bonus assigned to worker X" after each row) after each row of the code is executed? I did not get the messages.

      Delete
    2. Hi Joseph, no I don't get any message in Windows PowerShell, which is a sign to me that it worked :)

      Delete
    3. Thanks for posting the new formula syntax Bradley. I used this today and your formula is still good as of 02/2019.

      Delete
    4. Thank you so much Bradley. The formula works fine for me. It was great help what you shared it with us. Only one thing to note. Insert the reason of the bonus in "" in your excel column. Because it cannot read it as strings other wise. And one other thing: I spend 2 weeks on how to do the credentials right only because of a tiny problem. Log in to your Command Prompt as Administrator! and all will be fine.

      Delete
  17. I have read this post. collection of post is a nice one
    AWS Online Course

    ReplyDelete
  18. Existing without the answers to the difficulties you’ve sorted out through this guide is a critical case, as well as the kind which could have badly affected my entire career if I had not discovered your website.
    Digital Marketing online training

    full stack developer training in pune

    full stack developer training in annanagar

    full stack developer training in tambaram

    full stack developer training in velachery

    ReplyDelete
  19. hank you for benefiting from time to focus on this kind of, I feel firmly about it and also really like comprehending far more with this particular subject matter. In case doable, when you get know-how, is it possible to thoughts modernizing your site together with far more details? It’s extremely useful to me 
    Click here:
    python training in marathahalli
    Click here:
    python training in btm

    ReplyDelete
  20. From your discussion I have understood that which will be better for me and which is easy to use. Really, I have liked your brilliant discussion. I will comThis is great helping material for every one visitor. You have done a great responsible person. i want to say thanks owner of this blog.
    Python training in marathahalli
    AWS Training in chennai

    AWS Training in bangalore

    ReplyDelete
  21. This is ansuperior writing service point that doesn't always sink in within the context of the classroom. In the first superior writing service paragraph you either hook the reader's interest or lose it. Of course your teacher, who's getting paid to teach you how to write an good essay, 
    java training in jayanagar | java training in electronic city

    java training in chennai | java training in USA

    ReplyDelete
  22. I found your blog while searching for the updates, I am happy to be here. Very useful content and also easily understandable providing.. Believe me I did wrote an post about tutorials for beginners with reference of your blog. 

    angularjs Training in bangalore

    angularjs Training in btm

    angularjs Training in electronic-city

    angularjs online Training

    angularjs Training in marathahalli

    ReplyDelete
  23. When I initially commented, I clicked the “Notify me when new comments are added” checkbox and now each time a comment is added I get several emails with the same comment. Is there any way you can remove people from that service? Thanks.

    AWS Interview Questions And Answers

    AWS Training in Pune | Best Amazon Web Services Training in Pune

    AWS Training in Chennai |Best Amazon Web Services Training in Chennai


    AWS Training in Chennai | Best AWS Training in Chennai


    Amazon Web Services Training in Pune | Best AWS Training in Pune

    ReplyDelete
  24. Amazon Web Services (AWS) is the most popular and most widely used Infrastructure as a Service (IaaS) cloud in the world. AWS has four core feature buckets—Compute, Storage & Content Delivery, Databases, and Networking. At a high level, you can control all of these with extensive administrative controls accessible via a secure Web client.For more information visit.
    aws online training
    aws training in hyderabad
    amazon web services(AWS) online training
    amazon web sevrices(AWS) training online

    ReplyDelete
  25. This is beyond doubt a blog significant to follow. You’ve dig up a great deal to say about this topic, and so much awareness. I believe that you recognize how to construct people pay attention to what you have to pronounce, particularly with a concern that’s so vital. I am pleased to suggest this blog.
    Selenium training in Chennai

    Selenium training in Bangalore

    ReplyDelete
  26. Hello I am so delighted I found your blog, I really found you by mistake, while I was looking on Yahoo for something else, anyways I am here now and would just like to say thanks for a tremendous post. Please do keep up the great work.
    Microsoft Azure online training
    Selenium online training
    Java online training
    Java Script online training
    Share Point online training

    ReplyDelete
  27. Every one of the substance you said in post is too great and can be extremely helpful. I will remember it, much obliged for sharing the data
    Twitter accounts for sale

    ReplyDelete
  28. It is amazing and wonderful.Thank you for this post.
    Buy aged YouTube accounts

    ReplyDelete
  29. Thanks for sharing this, it was very informative, thank u so much…

    eTechno Soft Solutions is a leading training institute for all kind of the Oracle Training in Bangalore with real-time experienced trainers with 100% Placement Assistance.

    ReplyDelete
  30. I would be very grateful if anyone could share the material, code, Excel spreadsheet they have used to manage. You most absolutely have built this blog website into something special.
    python training in chennai

    python online training in chennai

    python training in bangalore

    python training in hyderabad

    python online training

    python flask training

    python flask online training

    python training in coimbatore




    ReplyDelete
  31. I know this is an old post, but since the command line tools have been replaced with the command line interface, I thought I'd add this in case anyone is still searching for this and would find it useful. Note this worked in Windows 10, not sure about other operating systems:
    1. Install the AWS Command Line Interface Version 2 https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-windows.html
    2. Install the latest version of Python https://www.python.org/downloads/
    3. Open the Command Prompt in Windows by searching for cmd
    4. To make sure the AWS CLI is working, at the first line in the Command Prompt, type aws –version. It should pop up with something like this: aws-cli/2.0.38 Python/3.7.7 Windows/10 exe/AMD64
    9. Configure the CLI to use your account. Type aws configure, then fill in the information:
    AWS Access Key ID: [your access key ID]
    AWS Secret Access key: [your secret access key]
    Default Region Name: us-east-1 (this is the only one that will work for MTurk)
    Default Output Format: (leave blank, just hit enter)
    10. To check that this has worked, type aws mturk get-account-balance. It should show you the correct account balance.
    11. To do any command, but in the Sandbox rather than the real site, just add --endpoint-url https://mturk-requester-sandbox.us-east-1.amazonaws.com to the end of the command. For example, get-account-balance in the Sandbox should return $10,000 every time.
    12. aws begins all commands you send to AWS, mturk gives you access to the sub-commands available for MTurk: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/mturk/index.html
    13. See https://blog.mturk.com/tutorial-managing-mturk-hits-with-the-aws-command-line-interface-56eaabb7fd4c for information on various things you can do, including assigning bonuses (note that this all assumes you’re using the AWS shell, so they don’t prefix their commands with aws, but if you do it seems to work the same way without setting up the shell).
    14. To send a bonus, type: aws mturk send-bonus --assignment-id "[fill assignment ID here – no brackets]" --worker-id "[fill worker id here – no brackets]" --bonus-amount "[bonus amount, 0.00 format]" --reason "[Note you want the worker to see]" --unique-request-token "[unique reason for bonus]"
    Example: aws mturk send-bonus --assignment-id "3U8YCDAGXPG0MT4U7MH13T0X16HQ0I" --worker-id "A25G8HPJE1QPA2" --bonus-amount "0.45" --reason "Bonus for game score. Thanks for playing!" --unique-request-token "Experiment 1"
    Note that the unique-request-token part makes it so that you can send a bonus to the same person for the same thing twice – it will throw an error. Just helps prevent you from double-paying by accident.
    You can also do this in the Sandbox, again by adding the Sandbox link to the end of the command. If you yourself have completed the HIT as a worker in the Sandbox, then you’ll get an email saying you got a bonus, so you’ll know it’s working.
    15. You can submit multiple bonuses at once by creating a text document that has the command above repeated on multiple lines, filled in with the correct information for each worker/assignment. As long as the document has carriage returns at the end of the line, when you copy/paste all the lines into the command prompt, it will execute them all at once.
    16. You won’t actually get any kind of confirmation message from the program if it works – just a lack of an error message. To see if the bonuses went through, you’ll have to check your transaction history from your MTurk Requestor account. It can take up to 12 hours for the bonuses to show up.

    ReplyDelete
  32. This article is a guest post on the significance of this post and how important it is for business to post it on their websites. In this post we are going to discuss why it is important for businesses to post a sign up sheet and other important things that one should consider when making this big investment. When making a big investment such as this one is very important to Twitter accounts for sale consider all aspects and that is why it is important for business to make sure that they post this on their websites. For those who are not able to make this post or understand its importance you can always check this out at my blog for more information

    ReplyDelete
  33. NDUW is a labor registration portal created by the Labor Department, Government of india. The registration of the unorganized workers (working class) of the state takes place on this portal.

    ReplyDelete
  34. Welcome to Rainbowpizza near me california| Rainbow Pizza at San Mateo California | Greek and Italian Cuisine | The best Pizza and Pasta at Cali

    ReplyDelete
  35. All type services start in Bihar Govt go to see this services and help for filling form RTPS Bihar

    ReplyDelete
  36. Benifit is a service offered by Yahoo that enables you to remove unused or obsolete Yahoo accounts. There are several ways that you can get your account deleted including: sending a mail to Yahoo asking them to terminate your account; blocking the account from accessing the Internet; and calling them on the telephone with a Verizon Fios modem or any other modem. To apply for your account to be terminated, you must email them a request to terminate your account with this information enclosed. Your request should include the following: your full name; current address; the account type; account passwords; and if applicable, a Verification Code. Buy gmail accounts

    ReplyDelete
  37. Buy google voice accounts There is a ton of information out there on the internet about CNNologist Profit Lance. One thing I want to point out before I go further is that not everyone will benefit from this system. This is a network marketing company that is geared towards helping those who want to create a passive income for themselves and their families. If you are looking to do just that, then by all means, check them out. But I warn people that if you are not ready to be a distributor or are not ready to make an income online, you might want to rethink your options.

    ReplyDelete
  38. Thank you so much for sharing this amazing information about SEO I really appreciate your hard work for writing this article thanks once again

    Please leave you comment on how to get short in height

    ReplyDelete
  39. These are very good post and i like your post...hindi skill

    ReplyDelete
  40. This is completely powerful software that helps it’s users in mainstream. The features of Windows 8 are totally amazing and they works very well. Crackeando Windows 8

    ReplyDelete
  41. This is a very well-written piece. Keep posting great things on your page. Your blog is wonderful.
    Scrivener

    ReplyDelete
  42. buy tinder accounts with standard price. 100% workable tinder account. Fully satisfied. Verified account. Backup available.

    ReplyDelete
  43. href="https://istanbulolala.biz/">https://istanbulolala.biz/
    NQMLY

    ReplyDelete