UMTurk, a new way to collect data online and in real time using Amazon Mechanical Turk, was developed in a partnership with faculty from the Department of Psychology and staff from LSA Technology Services.
UMTurk is a simple interface that allows instructors to set up new tasks in a matter of seconds. Amazon Mechanical Turk (AMT) is used on the backend, and templates were created that allow faculty members to pre-fill survey descriptions. Faculty can also connect to their Qualtrics account to select a survey from a list of surveys in their account rather than having to open a website to look up the survey ID and URL. Previously, faculty had to do all of this manually, which was very slow and cumbersome, particularly in the middle of a lecture.
The tool was used in the Psychology Research Methods course which teaches psychology students how to perform quality scientific studies. The UMTurk tool allows faculty to have discussions about a topic and then explore it live, in-class. The class is also able to test and hypothesize how a change might affect a research study.
The new tool made student learning more interactive, engaging, and realistic. Before UMTurk, students would have to answer survey questions during class or collect prior to class. The UMTurk project allowed the class to collect responses from a diverse audience, all within a single class period.
UMTurk is a simple interface that allows instructors to set up new tasks in a matter of seconds.
The project was developed by Abbey Roelofs, research computing lead, and Chris Stockbridge, research software programmer, of LSA Technology Services in partnership with faculty members Julie Boland, Joshua Rabinowitz, and Colleen Seifert from the Department of Psychology. To ensure a successful project, the team regularly met with faculty to discuss the requirements and prioritize tasks.
“There weren’t any other options that met their exact needs. In order for this to be useful in the middle of a lecture, the process had to be as streamlined as possible. We created the UMTurk tool as a Chrome extension so that it would be close at hand during a lecture presented using Google Slides. This allowed us to easily create something that can work on either Mac or Windows computers. We were able to develop something that they’re really proud of,” said Chris.
Amazon Mechanical Turk (AMT) is a crowdsourcing platform that allows researchers or businesses to recruit and pay workers to complete online tasks. Anyone can sign up to be a worker, resulting in a very large and diverse potential worker pool which is a boon for researchers. During task creation, the researcher can specify qualifications required for workers to participate in the task, as well as a time limit for completing the task. However, specifying these options requires a lengthy configuration process. “The faculty members came to us asking for a better way to create AMT tasks quickly during lectures and also to create tasks with specific target demographic groups for comparison studies. AMT offered an API, and a related product, TurkPrime (now called Cloud Research), offered a way to further refine the demographics of the subject pool within AMT. Unfortunately, TurkPrime did not have an API at the time, and, thus wasn’t programmable for the in-lecture portion of the class. We stuck with the AMT API for the in-lecture portion, and we helped Julie, Colleen, and their group navigate TurkPrime for use in comparison studies during discussion sections,” said Abbey.
This effort was awarded a prestigious Provost Teaching Award that recognizes original, specific innovations that improve student learning in a uniquely meaningful way.
Learn more about the project’s beginnings by viewing the 2018 Michigan IT Symposium poster presentation (UMICH [Level-1] login required). The project was funded by an LSA New Initiatives/New Instruction (NINI) grant, and Carla Stellrecht served as a project facilitator.
If you’re interested in using UMTurk for adapting a similar tool, contact Research Software Programming in LSA Technology Services by emailing LSATechnologyServices@umich.edu.
