Forums » Campaigns » Call for comments: Turker-authored guidelines for research on AMT

careful_owl’s suggestion sounds good to me!

I also have a suggestion about consent forms. Right now it’s not really clear how to use them on MTurk, many people just leave them out and that makes it harder for researchers who want to use them. I had a short discussion with Stanford’s IRB manager about the consent form necessary for online research and it seems that we could come up with a template for MTurk (and other online) studies. If you are onboard with this idea I could schedule a meeting with him and see if we can create a template that we could host on Dynamo.

It could be something like: fill out this form about your study and when you submit it Dynamo will give you a Dynamo approved consent box to copy past to the top of your HIT. We could put in information that both Turkers care about and IRBs cares about. Such as: how long do you expect this task to take, how much are you paying, (we could use that to calculate an hourly rate and put that in automatically), what is the email address and phone number of your university’s IRB (we could also pull that in automatically based on their institution), etc.

What do you think?

I think that’s great, especially if it had a graphic on it that was like a Dynamo seal of approval (but just indicates the IC get the seal, not pay, etc.)

Sign in/Register to reply to this post.

light_dragonfly

says thanks for this post

There is one glitch with the survey. It allows a person to vote countless times. I just talked to TO user who said he voted 51 times! It would be great to see this limited to one vote per person so the results are not skewed.

Sign in/Register to reply to this post.

AllOurIdeas already corrects for this behind the scenes. All votes are not equal; each person is equal. I wish this were clearer in their interface!

Sign in/Register to reply to this post.

light_dragonfly and faithful_fly

say thanks for this post

AllOurIdeas already corrects for this behind the scenes. All votes are not equal; each person is equal. I wish this were clearer in their interface!

Agreed! I wondered why it just kept going… and going… and going… :)

Sign in/Register to reply to this post.

faithful_fly

says thanks for this post

I have a concern about one of the poll options not being accurate. One option says “Underpayment of crowd workers is anything less than an average of $9.00 hourly, which is the Federal minimum wage limit as of Jan 1, 2016.” Federal minimum wage in the United States is $7.25. http://www.dol.gov/whd/minimumwage.htm

Sign in/Register to reply to this post.

faithful_fly

says thanks for this post

That’s a good point, I can’t find any evidence for that either. It’s not possible to edit a current idea but perhaps you can add a new one?

Sign in/Register to reply to this post.

faithful_fly

says thanks for this post

This was my fault. I accidentally typed in my state minimum wage instead of the Federal standard of $7.25. I also misquoted the date from my state as well (Minnesota). Thank you for removing this from the poll.

It’s no excuse, but I have been juggling numerous TO projects along with dealing with personal matters all day, and my brain is rather jumbled. I guess after this I’d say it’s a good time to go and take a nap!

Sign in/Register to reply to this post.

light_dragonfly

says thanks for this post

We got 1000+ votes on the fair payments section!: http://www.allourideas.org/dynamo_fair_payment/results

I’ve opened the wiki pages so anyone with an account can now edit: guidelines.wearedynamo.org Make sure to document any change you make (and the reason for the change) in this page: http://wiki.wearedynamo.org/index.php?title=Guidelines_2.0_staging There is a section at the top “DONE”. I think we should document what we do there so that everyone can see the list of changes. There’s already a list of suggestions to implement on the page, we can start there.

Sign in/Register to reply to this post.

Could this “Basics” page be adapted to act as the one page resource for general requesters? It already has a lot of good content and seems very similar to what a general overview page would be. http://wiki.wearedynamo.org/index.php?title=Basics_of_how_to_be_a_good_requester

Sign in/Register to reply to this post.

light_dragonfly

says thanks for this post

Great! Last night and today I made some of the changes in the “to do” list and moved them to the “done” list. Someone with more requester-side knowledge than me should tackle the bullet point about the DoesNotExist thing!

Sign in/Register to reply to this post.

Thanks careful_owl!

About the basics page, I agree that a lot of it is generalizable. Should we change the name and maybe link to it at the top of the first page to signal that it’s for all requesters?

The only part that I see as specific to academics is the clearly identify yourself and the consent form section. We could perhaps move that to its own separate section on informed consent, and show the Dynamo consent template there.

Sign in/Register to reply to this post.

This all sounds like a good idea. I think that would make the basics page more approachable for all requesters and keep the academic/informed consent information on a special page for academic requesters.

Sign in/Register to reply to this post.

tl;dr: are there any further changes that you propose to the guidelines v2.0 before we freeze it?

Hi everyone! Hope you’re all well :)

MTurk just published a blog post about the Dynamo guidelines: https://blog.mturk.com/how-to-be-a-great-mturk-requester-3a714d7d7436 The problem is that they’ve used V1, we have a more recent version that’s still editable here: guidelines.wearedynamo.org The major difference is the section on fair wages. So I propose that we take a few days (perhaps till Tuesday night PDT) to freeze this version. Then I’ll save it as PDF and update the PDF file here: http://www.wearedynamo.org/Guidelines_for_Academic_Requesters.pdf I can also contact MTurk and ask that they update the text in their blog.

Sign in/Register to reply to this post.

careful_owl

says thanks for this post