Monday, May 23, 2011

NO TURK FOR YOU!

Well, this is helpful:

"Your Human Intelligence Tasks 'Search for an answer to 1 simple question' have been removed from Amazon Mechanical Turk because they violated the terms of our HIT listing policy."

I . . . what?  I went to look at the policies page for the list of "prohibited activities," which include the following:
  • collecting personal identifiable information
    • definitely did not do this
  • fraud
    • hahaha
  • disrupting or degrading the operation of any website or internet service
    • uh, does making Google searches count as being disruptive??
  • direct marketing
    • nope
  • spamming
    • nope
  • unsolicited contacting of users
    • nope
  • impermissible advertising or marketing activities
    • nope
  • infringing or misappropriating the rights of any third party
    • Although Zoogle is called Zoogle, I did that deliberately as well as explicitly stating that the whole point is that people were performing searches "on a modified Google search engine."  I cited my sources!
  • posting illegal or objectionable content
    • My logs do not show any inappropriate search terms, and I don't see how else the content could be illegal or objectionable?
  • disrupting operation of the Mechanical Turk website
    • nope
  • creating a security risk for Mechanical Turk, any Mechanical Turk user, or any third party
    • nope
 I don't even know if this happened because a random Turker flagged my HIT (why would they do that? :( ), or whether Amazon has people/algorithms that comb the HITs regularly to find "objectionable content," and they happened to stumble upon my apparently not-so-innocent little experiment.

Since I am not even entirely clear what policy I violated, I have not modified/reposted the HIT for obvious reasons.

Abruptly halted experiment is sad.  :(

Fortunately, I collected data from 104 Turkers before shutdown, so maybe I will be able to cull some interesting conclusions from that anyway.  I deliberately wanted to avoid polling people like my friends, because they would be a self-selected population of young adults who are probably reasonably computer savvy, which would not make for as interesting of an experiment group, I think.

Research is so exciting.

    1 comment:

    1. That sucks! It's also pretty weird considering AMT tends to be on the side of the people offering the jobs rather than the Turkers, it's pretty much a seller's market in my experience. Make sure you didn't withhold payment from someone who deserved it or something. Also sometimes people get a bit annoyed if you don't pay them in a timely fashion.

      There are a number of avenues you could go down. You mention you don't want your tech-savvy friends for the experiment; however I might be interested to see the results there nonetheless! Earlier in the quarter when we were reading about the paper that swapped Google search order I tried it out on myself and I found myself almost subconsciously clicking on the top link even though it very obviously wasn't the right link, so even a mildly tech-savvy person such as myself can be duped :)

      Besides your friends other things you can try are using a forum like reddit.com to post about your experiment and see if people are interested in participating. If you can couple it with some incentive (not necessarily monetary, like Turk, but if you can appeal to their sense of helping out in a research project or something) then you might be able to get some volunteers. However they are a somewhat tech-savvy crowd in general, so that also might not be ideal.

      Of course you could always just create another AMT account :)

      ReplyDelete