Setting Up Your Connection¶

The first thing you need to do is setup your connection to Mechanical Turk.

Turkleton uses a per-process global connection. You should always initialize it before you attempt to upload or download anything. You initialize it like so:

from turkleton import connection
connection.setup(AWS_ACCESS_KEY, AWS_SECRET_ACCESS_KEY)

That’s it!

Creating A Task And Uploading It¶

Once you’ve created your layout on Mechanical Turk you can create HITs by defining a task in Turkleton.

To define a HIT you create a Task representing the template of the assignment you want a worker to complete. For example:

import datetime

from turkleton.assignment import task

class MyTask(task.BaseTask):
    __layout_id__ = 'MY LAYOUT ID'
    __reward__ = 0.25
    __title__ = 'Guess How Old From Picture'
    __description__ = 'Look at a picture and guess how old the person is.'
    __keywords__ = ['image', 'categorization']
    __time_per_assignment__ = datetime.timedelta(minutes=5)

Now that you’ve defined your task you can easily upload HITs as follows:

task = MyTask(image_url='http://test.com/img.png', first_guess='29')
hit = task.upload(batch_id='1234')

This will create a new assignment from the task template and upload it to Mechanical Turk. The variables image_url and first_guess in your template will contain the values given. The optional batch_id parameter allows you to set the requester annotation for the task to an arbitrary string. This is useful when you’ve uploaded more than one task in a batch. In the evaluation phase you can filter which assignments are downloaded by a given batch id.

Uploading Multiple Tasks¶

Usually you want to upload more than a one task. Turkleton provides two methods for easily doing this.

The first method uses the create_and_upload method on your Task as follows:

for image_url in all_image_urls:
    MyTask.create_and_upload(
        image_url=image_url, first_guess='29', batch_id='1234'
    )

It is often convenient to only set the batch id once. The task.batched_upload context manager is providing to make this approach easy as well:

with task.batched_upload(batch_id='1234')
    for image_url in all_image_urls:
       MyTask.create_and_upload(image_url=image_url, first_guess='29')

Every task you upload within the context will be automatically given the specified batch id.

Downloading The Results¶

When you want to download your results you’ll need to define an assignment. The assignment defines the types of values you expect to get. These are used to automatically parse and type cast your answers so you can just deal with evaluating the results.

You can define a simple task for categorizing an image as follows:

from turkleton.assignment import assignment
from turkleton.assignment import answers

class MyAssignment(assignment.BaseAssignment):
    categories = answers.MultiChoiceAnswer(question_name='Categories')
    notes = answers.TextAnswer(question_name='AdditionalNotes', default='')
    does_not_match_any = answers.BooleanAnswer(
        question_name='DoesNotMatchAnyCategories', default=False
    )

You can then download all of the HITs in a given batch as follows:

from turkleton.assignment import hit
reviewable_hits = hit.get_reviewable_by_batch_id('1234')

Each HIT may have multiple assignments associated with it. This is the case if the __assignments_per_hit__ attribute in your task contains a number greater than 1.

Now that you have the HITs you can download all the assignments, review them, and dispose of the HIT as follows:

for each in MyAssignment.get_by_hit_id(hit.hit_id):
    print('{} - {} - {}'.format(each.categories, each.notes, each.does_not_match_any))
    if is_valid_assignment(each):
        each.accept('Good job!')
    else:
        each.reject('Assignment does not follow instructions.')
hit.dispose()

Report Bugs¶

Report bugs at https://github.com/etscrivner/turkleton/issues.

If you are reporting a bug, please include:

• Your operating system name and version.
• Any details about your local setup that might be helpful in troubleshooting.
• Detailed steps to reproduce the bug.

Fix Bugs¶

Look through the GitHub issues for bugs. Anything tagged with “bug” is open to whoever wants to implement it.

Implement Features¶

Look through the GitHub issues for features. Anything tagged with “feature” is open to whoever wants to implement it.

Write Documentation¶

turkleton could always use more documentation, whether as part of the official turkleton docs, in docstrings, or even on the web in blog posts, articles, and such.

Submit Feedback¶

The best way to send feedback is to file an issue at https://github.com/etscrivner/turkleton/issues.

If you are proposing a feature:

• Explain in detail how it would work.
• Keep the scope as narrow as possible, to make it easier to implement.
• Remember that this is a volunteer-driven project, and that contributions are welcome :)