{"id":28452,"date":"2024-08-05T11:24:43","date_gmt":"2024-08-05T15:24:43","guid":{"rendered":"https:\/\/michigan.it.umich.edu\/news\/?p=28452"},"modified":"2024-08-05T11:24:43","modified_gmt":"2024-08-05T15:24:43","slug":"can-a-student-built-maizey-survive-a-blind-test","status":"publish","type":"post","link":"https:\/\/michigan.it.umich.edu\/news\/2024\/08\/05\/can-a-student-built-maizey-survive-a-blind-test\/","title":{"rendered":"Can a Student-Built Maizey Survive a Blind Test?"},"content":{"rendered":"\n

Last time<\/a>, I created a Maizey that helps researchers with menial tasks like summarizing literature reviews, identifying methods, and highlighting key findings. It works, but let\u2019s face it, I\u2019m no subject-matter expert. My one semester of experience in a college lab is measly compared to many of my research savant peers. In an effort to identify areas of improvement, I sought out my friend Lillian Shern, who has 48 months of research experience under her belt.<\/p>\n\n\n\n

\n
<\/div>\n<\/div>\n\n\n\n
\"\"<\/figure><\/div>\n\n\n\n
\n
\n

Shern\u2019s research experience primarily involves qualitative research, including extensive literature reviews, data organization, and survey creation. Shern has seen both sides of the coin: research pre- and post-AI. Before the incorporation of AI, she would manually gather research papers from university libraries and Google Scholar, then organize them in Excel by identifying key variables such as dependent, independent, mediating, and moderating variables. This process, while thorough, is time-consuming and labor-intensive, she noted.<\/p>\n<\/div>\n<\/div>\n\n\n\n

In her current workflow, Shern often utilized ChatGPT. She would input the abstracts of research papers into ChatGPT and ask it to identify the key variables from the abstract. This approach helped speed up the process of synthesizing information, though it had its limitations. Hallucinations<\/a> occurred, and sometimes the abstract was too abstract: it did not contain enough information, especially for mediating and moderating variables.<\/p>\n\n\n\n

When Shern tried out Maizey without any prior knowledge of how it works, she immediately gravitated toward her usual method: paste in the abstract and ask it to identify independent, dependent, moderating, and mediating variables. This was fascinating to observe and it was exciting to see experience at work. I will be shamelessly copying her method: I will modify the system prompt to let Maizey focus on the abstract instead of the entire paper, which can be dependent on graphics that are difficult for AI to index \u2014 at the time of writing, as this technology will outpace us all.<\/p>\n\n\n\n

Shern perceived Maizey\u2019s responses to be slower than ChatGPT\u2019s as it outputs the response only when it’s done as opposed to streaming the response in real time. On the flipside, she noted that Maizey offered the advantage of directly citing sources from Google Drive, making citations easier. She also observed that Maizey\u2019s identification of variables still involves some hallucination, often producing \u201cvariables that are not there,\u201d but appreciated its ability to suggest possible moderating and mediating variables, even when they were not explicitly mentioned in the abstracts.<\/p>\n\n\n\n

\n
\n
\n

For Shern, the biggest edge that Maizey has over ChatGPT is its security aspect. Maizey doesn\u2019t utilize any user input to train its model and no one at the University \u2014 not professors, not even ITS \u2014 can ever request to view the data. As she is taught by her Principal Investigator to not put sensitive research data into ChatGPT, she acknowledged that Maizey\u2019s secure handling of data could make it a preferable choice for more sensitive research tasks beyond literature review.<\/p>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n

Shern\u2019s experience underscores the promising capabilities of Maizey as a research assistant. At the moment, her lab\u2019s usage of generative AI is limited to preliminary tasks and much of the interpretation is still reserved for human researchers. <\/p>\n\n\n\n

\u201cWe can use generative AI in terms of prompting them with the existing information,\u201d Shern said, \u201cAnything of predictive nature \u2026 could go really weird so you don’t want that.\u201d<\/p>\n\n\n\n

While there are areas for improvement, the AI\u2019s ability to streamline literature reviews and provide secure, properly-sourced responses can support students navigating the oceans of text. As Maizey continues to develop, it holds the potential to transform the research experience at the University, making it more efficient and accessible for students like Shern.<\/p>\n","protected":false},"excerpt":{"rendered":"

Joey Lin, ITS intern, puts a research assistant chat bot to the test with the help of researcher, Lillian Shern<\/p>\n","protected":false},"author":210,"featured_media":28453,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":"","_umich_oidc_access":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_ef_editorial_meta_date_first-draft-date":"","_ef_editorial_meta_paragraph_assignment":"","_ef_editorial_meta_checkbox_needs-photo":"","_ef_editorial_meta_number_word-count":"","footnotes":""},"categories":[4],"tags":[1031,1081,1050,130],"class_list":["post-28452","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-features","tag-genai","tag-its-interns-2024","tag-maizey","tag-research"],"uagb_featured_image_src":{"full":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3.png",1024,1024,false],"thumbnail":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3-400x266.png",400,266,true],"medium":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3-300x300.png",300,300,true],"medium_large":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3-768x768.png",665,665,true],"large":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3-600x600.png",600,600,true],"1536x1536":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3.png",1024,1024,false],"2048x2048":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3.png",1024,1024,false],"excerpt-thumbnail":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3-200x140.png",200,140,true],"themonic-thumbnail":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3-60x42.png",60,42,true],"ioslider-thumbnail":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3-658x300.png",658,300,true],"post-thumbnail":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3-665x665.png",665,665,true],"400x250-crop":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2024\/08\/unnamed-3-400x250.png",400,250,true]},"uagb_author_info":{"display_name":"Joey Lin, ITS Marketing and Communications Intern","author_link":"https:\/\/michigan.it.umich.edu\/news\/author\/joeyylin\/"},"uagb_comment_info":0,"uagb_excerpt":"Joey Lin, ITS intern, puts a research assistant chat bot to the test with the help of researcher, Lillian Shern","_links":{"self":[{"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/posts\/28452","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/users\/210"}],"replies":[{"embeddable":true,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/comments?post=28452"}],"version-history":[{"count":7,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/posts\/28452\/revisions"}],"predecessor-version":[{"id":28465,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/posts\/28452\/revisions\/28465"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/media\/28453"}],"wp:attachment":[{"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/media?parent=28452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/categories?post=28452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/tags?post=28452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}