A new course evaluation process at the University of Michigan-Flint is boosting response rates and helping more students have their voices heard.
Last spring, the Office of Online and Digital Education conducted a pilot project connecting Canvas with Course Evaluations using custom pop-up notifications, an enhancement made possible with ChatGPT. The new process leverages the AI-powered chatbot’s advanced capabilities to generate JavaScript to improve end-of-semester evaluation reminders in Canvas.
Year-over-year response rates improved from 33.5 percent in spring 2022 to 55.6 percent in spring 2023 – the highest change recorded by the university in over a decade.
“The results of our pilot program in the spring marked a significant breakthrough in our challenge to improve response rates,” said Nick Gaspar, director at the Office of Online and Digital Education. “It really highlighted the potential of using generative AI to assist us and accelerate our work as we think of new methods to engage with students in a way that resonates with them.”
Chris Smith, help desk supervisor at the Office of Online and Digital Education, helped manage the initiative and said inspiration to utilize ChatGPT came after inquiring into the University of Michigan-Dearborn’s much higher response rates.
“Learning about Dearborn’s use of a Canvas pop-up alert inspired us to explore a similar approach for our course evaluations,” Smith said. “Although I’m not skilled in JavaScript, I had prior success with AI-assisted Python coding which led me to try ChatGPT. After a brief chat, I had exactly what I wanted.”
The initial course evaluation pop-up used a basic JavaScript code to create a pop-up message for students upon entering a Canvas course, notifying them only of the number of pending evaluations left to complete. Using ChatGPT, Smith said he was able to make the following improvements despite having only a basic understanding of writing JavaScript:
- Modified pop-up notification to display as HTML instead of plaintext
- Created new pop-up display list of the student’s evaluations with due dates
- Added link to launch students directly to the evaluation page
- Modified layout and style of the pop-up notification
The new process displays course evaluation reminders right on the student’s Courses page within Canvas–a system they spend a lot of time in and are used to navigating. In addition to providing due dates and direct access links, the notification appears only once per login to Canvas if they have an open or incomplete evaluation during the evaluation period. It can be dismissed and will not reappear until the student’s next Canvas login.
“All of these improvements were human reviewed and tested before it was put into production per the Safe Computing guidelines,” Gaspar said.
Aside from enhancing processes to improve course evaluation rates, utilizing ChatGPT offered additional advantages.
“On our side of this, ChatGPT not only made this project more feasible with limited resources, but it also dramatically reduced the development time,” Gaspar said.
Following the pilot phase, this new course evaluation process has maintained its success, showing a 27 percent growth in Fall 2023 response rates compared to the previous year.
