{"id":27061,"date":"2023-06-20T15:42:35","date_gmt":"2023-06-20T19:42:35","guid":{"rendered":"https:\/\/michigan.it.umich.edu\/news\/?p=27061"},"modified":"2024-07-08T06:04:16","modified_gmt":"2024-07-08T10:04:16","slug":"u-m-team-develops-two-new-apps-to-capture-interfaces-and-images-for-visually-impaired-users","status":"publish","type":"post","link":"https:\/\/michigan.it.umich.edu\/news\/2023\/06\/20\/u-m-team-develops-two-new-apps-to-capture-interfaces-and-images-for-visually-impaired-users\/","title":{"rendered":"U-M team develops two new apps to capture interfaces and images for visually impaired users"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"276\" src=\"https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-1-600x276.png\" alt=\"\" class=\"wp-image-27069\" srcset=\"https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-1-600x276.png 600w, https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-1-300x138.png 300w, https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-1-768x353.png 768w, https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-1-665x306.png 665w, https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-1.png 1000w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" \/><figcaption class=\"wp-element-caption\">ImageExplorer identifies the people, benches and bags in this photo. It correctly autocaptions the image as \u201ca couple of women walk down a sidewalk.\u201d The app accurately recognizes some clothing types, like skirts, while most tops are simplified to \u201cshirts.\u201d (Credit: Human-AI Lab, University of Michigan)<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">U-M assistant professor of computer science and engineering Anhong Guo led the development of the two apps for visually impaired users with a team based at the University of Michigan. The research has been funded by both U-M and Google.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One of these apps, <a href=\"https:\/\/vizlens.org\/\">VizLens<\/a>, is listed as an, \u201cappliance reader app for users with visual impairment.\u201d VizLens allows users to scan and access an existing image of an inaccessible interface\u2013 for example, a flat microwave panel \u2013and hear the button labels read aloud or, alternatively, use the camera to have labels read out in real-time in correspondence to where the user places their finger.<\/p>\n\n\n\n<iframe loading=\"lazy\" src=\"https:\/\/drive.google.com\/file\/d\/1sfg-qQ69UNJ-MVGBNZmvNCfs11cdTJbj\/preview\" width=\"640\" height=\"480\" allow=\"autoplay\"><\/iframe>\n\n\n\n<p class=\"wp-block-paragraph\">The other app, <a href=\"https:\/\/imageexplorer.org\/\">ImageExplorer<\/a>, is listed as an, \u201cimage exploration app for users with visual impairment.\u201d The alt text or caption may describe an image as, \u201cA boy on a skateboard.\u201d ImageExplorer can tell you that the man is wearing a black shirt, blue shorts, and blue and black shoes. It can also detect people and objects in the background and identify these objects in more detail with a quick double-tap.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The level of detail ImageExplorer provides, with tags recognizing objects from benches to bags, and its ability to identify where each object and person rests in an image make it more unique and more comprehensive.\u00a0<\/p>\n\n\n\n<iframe loading=\"lazy\" src=\"https:\/\/drive.google.com\/file\/d\/1gj3lLs7LtC4IIyWLIP4JYPYRUdE9IqrA\/preview\" width=\"640\" height=\"480\" allow=\"autoplay\"><\/iframe>\n\n\n\n<p class=\"wp-block-paragraph\">As of publication, ImageExplorer will occasionally provide conflicting information in its descriptions, and the app often simplifies certain details, such as identifying most tops as \u201cshirts.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cIn spite of these errors, the results we presented in 2022 show that ImageExplorer enables users to make more informed judgements of the accuracy of the AI-generated captions,\u201d Guo said.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The two apps were tested by hundreds of visually impaired, user-testing participants who provided feedback to the team, which is continuing to work to make improvements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u201cThe accuracy relies on the models we use, and as they improve, ImageExplorer will improve,\u201d Guo said. He hopes that the public release of the apps will bring forth feedback for the team.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">According to <a href=\"https:\/\/guoanhong.com\/\">Guo\u2019s website<\/a>, both apps were released and made publicly available on the iOS App Store in May of 2023. Both are available to download for free today.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Article summarized from Michigan Engineering: <a href=\"https:\/\/news.engin.umich.edu\/2023\/06\/new-apps-for-visually-impaired-users-provide-virtual-labels-for-controls-and-a-way-to-explore-images\/\">New apps for visually impaired users provide virtual labels for controls and a way to explore images<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>U-M assistant professor of computer science and engineering Anhong Guo led the development of the two apps for visually impaired users with a team based at the University of Michigan. The research has been funded by both U-M and Google.&nbsp; One of these apps, VizLens, is listed as an, \u201cappliance reader app for users with visual impairment.\u201d VizLens\u2026 <span class=\"read-more\"><a href=\"https:\/\/michigan.it.umich.edu\/news\/2023\/06\/20\/u-m-team-develops-two-new-apps-to-capture-interfaces-and-images-for-visually-impaired-users\/\">Read More &raquo;<\/a><\/span><\/p>\n","protected":false},"author":191,"featured_media":27062,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_uag_custom_page_level_css":"","_umich_oidc_access":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_ef_editorial_meta_date_first-draft-date":"","_ef_editorial_meta_paragraph_assignment":"","_ef_editorial_meta_checkbox_needs-photo":"","_ef_editorial_meta_number_word-count":"","footnotes":""},"categories":[5],"tags":[],"class_list":["post-27061","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-campus-news"],"uagb_featured_image_src":{"full":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer.png",1000,460,false],"thumbnail":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-400x266.png",400,266,true],"medium":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-300x138.png",300,138,true],"medium_large":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-768x353.png",665,306,true],"large":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-600x276.png",600,276,true],"1536x1536":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer.png",1000,460,false],"2048x2048":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer.png",1000,460,false],"excerpt-thumbnail":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-200x140.png",200,140,true],"themonic-thumbnail":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-60x42.png",60,42,true],"ioslider-thumbnail":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-658x300.png",658,300,true],"post-thumbnail":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-665x306.png",665,306,true],"400x250-crop":["https:\/\/michigan.it.umich.edu\/news\/wp-content\/uploads\/2023\/06\/ImageExplorer-400x250.png",400,250,true]},"uagb_author_info":{"display_name":"Josephine Anderson","author_link":"https:\/\/michigan.it.umich.edu\/news\/author\/jganders\/"},"uagb_comment_info":0,"uagb_excerpt":"U-M assistant professor of computer science and engineering Anhong Guo led the development of the two apps for visually impaired users with a team based at the University of Michigan. The research has been funded by both U-M and Google.&nbsp; One of these apps, VizLens, is listed as an, \u201cappliance reader app for users with&hellip;","_links":{"self":[{"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/posts\/27061","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/users\/191"}],"replies":[{"embeddable":true,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/comments?post=27061"}],"version-history":[{"count":5,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/posts\/27061\/revisions"}],"predecessor-version":[{"id":27072,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/posts\/27061\/revisions\/27072"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/media\/27062"}],"wp:attachment":[{"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/media?parent=27061"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/categories?post=27061"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/michigan.it.umich.edu\/news\/wp-json\/wp\/v2\/tags?post=27061"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}