In the spring 2018 academic term, the U-M Medical School (UMMS) along with the rest of UM-Ann Arbor campus (18 schools and colleges), adopted Blue, a new course evaluation software.
Blue was first implemented by UMMS in fall 2017. The School of Information and the Taubman College of Architecture and Urban Planning piloted the application in the winter 2018 academic term, prior to its full adoption by central campus in the spring 2018 term.
The former course evaluation software, launched in 2008, was a collaboration between ITS and U-M’s former learning management system, CTools. The move from CTools to Canvas in 2015 precipitated the need for a vendor solution for course evaluation software.
There was a need for updated technologies and better reporting and analytics.
UMMS was using CurrEval, a homegrown system, that was hard-coded to the medical school’s old curriculum. But there was a need for updated technologies and better reporting and analytics. Because the Medical School curriculum was undergoing a dramatic transformation. There was also a looming accreditation cycle.
Health Information Technology and Services (HITS) and the Office of Medical School Education partnered with the Registrar’s Office in search of a new curriculum evaluation software. The team went to work looking for an answer. The solution had to provide a way to gauge academic experiences, evaluate teaching, and provide the data and reporting for curriculum management and overall program assessment.
“Blue offered the most flexibility for setting up evaluations and different types of questions.”
Lisa Emery, senior associate university registrar
“Blue offered the most flexibility for setting up evaluations and allowing for the variety of different types of questions,” said Lisa Emery, senior associate university registrar.
Part of this flexibility includes giving instructors the freedom to customize their course evaluations. With the former course evaluation software, instructors could only choose from a database of pre-selected questions which were to appear on their individual course evaluations. In contrast, Blue allows instructors to write their own questions and tailor the evaluations to their specific course.
The software gives students more options with their responses as well. Likert-scaling questions are no longer restricted to degrees of agreement; while that particular scale may still be used, instructors may now choose to have scales expressing frequency and expected grades in the course.
“The project team especially liked Blue’s custom reporting capabilities for courses and teaching evaluations.”
Nicolette Franck, HITS senior IT project manager
UMMS also appreciated Blue’s flexibility as it could also support the evaluation of its new curriculum. “The project team especially liked Blue’s custom reporting capabilities for courses and teaching evaluations,” said Nicolette Franck, HITS senior IT project manager on this project. The system provides summative reports at the end of an academic year and longitudinal reports spanning across various terms or years, for example. Sophisticated analytics and reporting on evaluation data is key for UMMS’ curriculum committees’ review processes, program accreditation and part of the faculty promotion and tenure process.
Blue gives instructors more nuanced feedback from students that can be used to improve their courses.
These unique affordances can have a significant impact on the academic experience of U-M students: the questions and responses become more applicable to a specific course, which gives instructors more nuanced feedback from students that can be used to improve their courses in specific ways in future semesters. In addition, regularly scheduled and custom reporting can lead to improved curricular experiences and positively impact the quality of teaching, which in turn benefits the student.
In the future, Blue seeks to support instructors of large courses by implementing a text analysis tool that will allow instructors to search through their open-ended comment questions. A tool like this will allow instructors to review the comments in their open-ended questions more efficiently, making the responses to these questions less time-consuming to review.
Nicolette Franck and Brian Simko from HITS contributed to this article.
