Human-Centered Artificial Intelligence Lab

selected publications

2025

1. IUI

   AltAuthor: A Context-Aware Alt Text Authoring Tool with Image Classification and LMM-Powered Accessibility Compliance

   Hyungwoo Song, Minjeong Shin, Yeonjoon Kim, and 4 more authors

   In Proceedings of the 30th International Conference on Intelligent User Interfaces Companion, Mar 2025

   Abs HTML PDF

   Web accessibility remains a persistent challenge, particularly in providing appropriate alternative text (alt text) for images that meet the needs of people with visual impairments. Existing tools often generate generic descriptions without taking into account image types and webpage context, leading to alt text that fails to align with established accessibility guidelines. To overcome these limitations, we present AltAuthor, an authoring tool designed to assist developers in managing alt text on existing websites. AltAuthor consists of three key components: (1) Image Classifier, which identifies visual content and its type, (2) Alt Text Generator, leveraging a Large Multimodal Model to generate compliant descriptions by incorporating identified image type, webpage context, and Section 508 accessibility guidelines, and (3) Interactive Authoring Interface, allowing developers to customize automated suggestions. Through this integrated approach, AltAuthor provides developers with tools to enhance accessibility management.

2. AAAI

   Understanding How Paper Writers Use AI-Generated Captions in Figure Caption Writing

   Ho Yin Sam Ng, Ting-Yao Hsu, Jiyoo Min, and 5 more authors

   In AAAI 2025 Workshop: Towards a Knowledge-grounded Scientific Research Lifecycle, Mar 2025

   HTML PDF

2024

1. ASSETS

   Toward Closed-Domain Conversational Item Listing Assistant for Improvement of Experiences of Older Adults in Customer-to-Customer (C2C) Marketplaces

   Subin Park, Hyunggu Jung, Bektur Ryskeldiev, and 5 more authors

   In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility, Oct 2024

   Abs HTML PDF

   Due to the increased availability of internet-based services, the number of e-commerce users among older adults is steadily growing. However, little is known about how older adults would adapt to modern e-commerce services, especially in the case of customer-to-customer (C2C) marketplaces as buyers and sellers. To address that, we propose a closed-domain conversational chatbot assistant that guides older adults through the product listing process. We evaluated the usability of our tool with 10 participants, consisting of a pre-test survey and a post-task interview. We found that while older adults proficiently operated with familiar functions such as text entry and photo taking, they faced challenges in understanding the purpose of each task and how tasks were related to each other. We suggest design implications for creating conversational chatbot assistants that help older adults sell items in C2C marketplaces.

2. IJCAI

   SaGol: Using MiniGPT-4 to Generate Alt Text for Improving Image Accessibility

   Yunseo Moon, Hyunmin Lee, SeungYoung Oh, and 1 more author

   In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, Aug 2024

   Demo Track

   Abs HTML PDF

   SaGol is an AI-powered application to improve image accessibility for people with visual impairments (PVI) users. Alternative (alt) text, a general method of web accessibility for PVI users, is text or phrases that describe images on a website in an understandable way. SaGol generates alt text with the images on the user’s smartphone using a vision large language model called MiniGPT-4. SaGol searches for similar images based on the generated alt text. We evaluated the length of alt text and the search accuracy. This paper shows a potential opportunity to improve image accessibility for PVI users.

3. JMIR

   A Deep Learning–Based Rotten Food Recognition App for Older Adults: Development and Usability Study

   Minki Chun, Ha-Jin Yu, and Hyunggu Jung

   JMIR Form Res, Jul 2024

   Abs HTML PDF

   Background: Older adults are at greater risk of eating rotten fruits and of getting food poisoning because cognitive function declines as they age, making it difficult to distinguish rotten fruits. To address this problem, researchers have developed and evaluated various tools to detect rotten food items in various ways. Nevertheless, little is known about how to create an app to detect rotten food items to support older adults at a risk of health problems from eating rotten food items. Objective: This study aimed to (1) create a smartphone app that enables older adults to take a picture of food items with a camera and classifies the fruit as rotten or not rotten for older adults and (2) evaluate the usability of the app and the perceptions of older adults about the app. Methods: We developed a smartphone app that supports older adults in determining whether the 3 fruits selected for this study (apple, banana, and orange) were fresh enough to eat. We used several residual deep networks to check whether the fruit photos collected were of fresh fruit. We recruited healthy older adults aged over 65 years (n=15, 57.7%, males and n=11, 42.3%, females) as participants. We evaluated the usability of the app and the participants’ perceptions about the app through surveys and interviews. We analyzed the survey responses, including an after-scenario questionnaire, as evaluation indicators of the usability of the app and collected qualitative data from the interviewees for in-depth analysis of the survey responses. Results: The participants were satisfied with using an app to determine whether a fruit is fresh by taking a picture of the fruit but are reluctant to use the paid version of the app. The survey results revealed that the participants tended to use the app efficiently to take pictures of fruits and determine their freshness. The qualitative data analysis on app usability and participants’ perceptions about the app revealed that they found the app simple and easy to use, they had no difficulty taking pictures, and they found the app interface visually satisfactory. Conclusions: This study suggests the possibility of developing an app that supports older adults in identifying rotten food items effectively and efficiently. Future work to make the app distinguish the freshness of various food items other than the 3 fruits selected still remains.

4. DIB

   Creating a bias-free dataset with food delivery app reviews under data poisoning attack

   Hyunmin Lee, SeungYoung Oh, JinHyun Han, and 1 more author

   Data in Brief, Jun 2024

   Abs HTML

   In online food delivery apps, customers write reviews to reflect their experiences. However, certain restaurants use a “review event” strategy to solicit favorable reviews from customers and boost their revenue. Review event is a marketing strategy where a restaurant owner gives free services to customers in return for a promise to write a review. Nevertheless, current datasets of app reviews for food delivery services neglect this situation. Furthermore, there appears to be an absence of datasets with reviews written in Korean. To solve this gap, this paper presents a dataset that contains reviews obtained from restaurants on a Korean app which use a review event strategy. A total of 128,668 reviews were gathered from 136 restaurants through crawling reviews using the Selenium library in Python. The dataset consists of detailed information of each review which contains information about ordered dishes, each review’s written time, whether the food image is included in the review or not, and various star ratings such as total, taste, quantity, and delivery ratings. This dataset supports an innovative process of preparing AI training data for achieving fairness AI by proposing a bias-free dataset of food delivery app reviews with data poisoning attacks as an example Additionally, the dataset is beneficial for researchers who are examining review events or analyzing the sentiment of food delivery app reviews.

2023

1. CSCW

   Toward Value Scenario Generation Through Large Language Models

   Hyunggu Jung, Woosuk Seo, Seokwoo Song, and 1 more author

   In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing, Oct 2023

   Abs HTML PDF

   We propose a method of generating value scenarios for design research by leveraging ChatGPT, an AI-powered chatbot based on large language models. Identifying the needs of a vulnerable population, such as North Korean defectors, is challenging for researchers. To address this, we introduce ChatGPT-generated value scenarios, an extension of scenario-based design that supports critical, systemic, long-term thinking in current design practice, technology development, and deployment. Using our proposed method, we created a prompt to generate value scenarios on ChatGPT. Based on our analysis of the generated scenarios, we identified that ChatGPT could generate plausible information about Value Implications. However, it lacks details on Pervasiveness and Systemic Effects. After discussing the limitations and opportunities of ChatGPT in generating value scenarios, we conclude with suggestions for how ChatGPT might be better used to generate value scenarios.

2. CSCW

   Visualizing the Carbon Intensity of Machine Learning Inference for Image Analysis on TensorFlow Hub

   Taewon Yoo, Hyunmin Lee, SeungYoung Oh, and 2 more authors

   In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing, Oct 2023

   Abs HTML PDF

   The increasing performance of machine learning (ML) models necessitates greater computing resources, contributing to rising carbon intensity in ML computing and raising concerns about computational equity. Previous studies focused on developing tools that enable model developers to view the carbon intensity of the ML models in the training process. Still, little is known about how to support ML developers in online communities to explore the carbon intensity of ML models during inference. We developed MIEV, a model inference emission visualizer, that supports ML developers on TensorFlow Hub to explore the carbon intensity of image domain models during the model Inference phase. We also provide insights into designing technologies that promote collaborative work among ML developers to drive sustainable AI development processes. To the best of our knowledge, this is the first attempt to interactively visualize the carbon intensity of ML models in online communities during the Inference phase.

3. IUI

   WATAA: Web Alternative Text Authoring Assistant for Improving Web Content Accessibility

   Hyeonhak Jeong, Minki Chun, Hyunmin Lee, and 2 more authors

   In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, Mar 2023

   Abs HTML PDF

   Alternative (alt) text is essential for people with visual impairments to acquire information about image content through a screen reader. However, collecting images and creating alt text requires time and effort. To deal with this problem, automatically collecting images that have no alt text is essential to ensure that all image content in a web page has their alt text. Additionally, automatic alt text has limitations in accuracy and quality compared to human-created alt text despite the improvements in image recognition and natural language process technology. We present WATAA, a web alt text authoring assistant that collects images containing no alt text and suggests automatic alt text to help human alt text authors improve a web page’s accessibility.

4. IUI

   PORDE: Explaining Data Poisoning Attacks Through Visual Analytics with Food Delivery App Reviews

   Hyunmin Lee, Minki Chun, and Hyunggu Jung

   In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, Mar 2023

   Abs HTML PDF

   Artificial intelligence (AI) gives many benefits to our lives. However, biased AI models created by receiving data poisoning attacks may induce social problems. Therefore, developers must consider carefully whether the training data received a poison attack when developing an AI model. Data visualization is one of the methods to facilitate the analysis of the data required for checking if the training data received a poisoning attack. However, prior studies did not visualize real-world AI training data. Restaurant reviews in delivery apps are one of the cases of a poisoned dataset. Restaurants hold review events on delivery apps to encourage customers to write a positive review in return for certain rewards, thereby creating reviews with bias. In this study, we propose POisoned Real-world Data Explainer (PORDE) that explains data poisoning attacks through visual analytics with food delivery app reviews. The findings of this study suggest implications for securing safe training data and developing less biased AI models.

5. IUI

   Toward Keyword Generation Through Large Language Models

   Wanhae Lee, Minki Chun, Hyeonhak Jeong, and 1 more author

   In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, Mar 2023

   Abs HTML PDF

   It is essential to understand research trends for researchers, decision-makers, and investors. One way to analyze research trends is to collect and analyze author-defined keywords in scientific papers. Unfortunately, while author-defined keywords are beneficial to researchers aiming to figure out the trends of their research fields, 45% of scientific papers in Microsoft Academic Graph did not contain their author-defined keywords. Additionally, six of the top seven AI conferences neither collect nor disclose keywords. This paper proposes a method for generating the keywords using Galactica, a pre-trained large language model published by Meta. We evaluate this method’s performance by comparing the keywords provided by authors in the CoRL’22 and report characteristics of the generated keywords. Our study shows the F1 score of our proposed method was ten times better than that of previous studies, and 42.7% of the generated keywords are relevant to author-defined keywords.

2022

1. CSCW

   Exploring the Community of Model Publishers on TensorFlow Hub

   Taewon Yoo, Minki Chun, Yunjung Bae, and 2 more authors

   In Companion Publication of the 2022 Conference on Computer Supported Cooperative Work and Social Computing, Nov 2022

   Abs HTML PDF

   We explore the community of AI model publishers on TensorFlow Hub (TF Hub). While researchers identified the challenges AI model publishers and AI model users faced, little is known about how they interact with each other in an online community. The analysis of the metadata recorded on TF Hub revealed the models that the AI model publishers uploaded. Also, we found out how the models published by the AI model publishers were shared with other people on TF Hub. To our knowledge, this is the first attempt to explore the online community of AI model publishers sharing their models with each other.

2. ECSCW

   Toward an AI-assisted Assessment Tool to Support Online Art Therapy Practices: A Pilot Study

   Woosuk Seo, Joonyoung Jun, Minki Chun, and 5 more authors

   In Proceedings of 20th European Conference on Computer-Supported Cooperative Work, Nov 2022

   Abs HTML PDF

   Artificial intelligence (AI) has been widely used to assist art therapists with artwork assessments by providing objective information. While prior studies showed that AI-assisted tools are feasible to improve drawing analysis in in-person art therapy practices, the use of those tools in online art therapy is still under-examined. To fill the gap, we created a prototype of an AI-assisted tool for online therapy in a House-Tree-Person (HTP) test scenario and ran lab-based usability sessions with 10 art therapists in which they used our proposed prototype to complete predefined tasks. We then conducted semi-structured interviews with the participants to understand their acceptance and concerns about the prototype. The findings revealed the unique needs of art therapists and opportunities of using AI-assisted tools to improve online art therapy practices. Based on these findings, we suggest implications for creating AI-assisted tools that meet specific needs of art therapists in online therapy sessions, and further discuss future directions of research about AI-assisted tools for art therapists in online settings.

2021

1. CSCW

   Exploring the Experiences of Student Volunteer and Student Volunteer Chair Communities at Academic Conferences

   Subin Park, Heejae Jung, Jae Won Choi, and 4 more authors

   Proceedings of the ACM on Human-Computer Interaction, Oct 2021

   HTML
2. CSCW

   Exploring the Experiences of Streamers with Visual Impairments

   Joonyoung Jun, Woosuk Seo, Jihyeon Park, and 2 more authors

   Proceedings of the ACM on Human-Computer Interaction, Oct 2021

   HTML
3. IUI

   MonoPass: A Password Manager without Master Password Authentication

   Hyeonhak Jeong, and Hyunggu Jung

   In 26th International Conference on Intelligent User Interfaces-Companion, Apr 2021

   HTML PDF
4. IUI

   LectYS: A System for Summarizing Lecture Videos on YouTube

   Taewon Yoo, Hyewon Jeong, Donghwan Lee, and 1 more author

   In 26th International Conference on Intelligent User Interfaces-Companion, Apr 2021

   HTML PDF
5. JMIR

   A novel food record app for dietary assessments among older adults with type 2 diabetes: development and usability study

   Hyunggu Jung, George Demiris, Peter Tarczy-Hornoch, and 1 more author

   JMIR Formative Research, Feb 2021

   HTML

2017

1. ASSETS

   Exploring the community of blind or visually impaired people on YouTube

   Woosuk Seo, and Hyunggu Jung

   In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, Oct 2017

   HTML PDF
2. CSCW

   Personas and scenarios to design technologies for North Korean defectors with depression

   Hyunggu Jung, Woosuk Seo, and Michelle Cha

   In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, Feb 2017

   HTML PDF

2016

1. AMIA

   Development of a novel markov chain model for the prediction of head and neck squamous cell carcinoma dissemination

   Hyunggu Jung, Anthony Law, Eli Grunblatt, and 4 more authors

   In AMIA Annual Symposium Proceedings, Nov 2016

   HTML PDF
2. AMIA

   Nurse informaticians report low satisfaction and multi-level concerns with electronic health records: results from an international survey

   Maxim Topaz, Charlene Ronquillo, Laura-Maria Peltonen, and 8 more authors

   In AMIA Annual Symposium Proceedings, Nov 2016

   HTML PDF
3. CHI

   ’MASTerful’Matchmaking in Service Transactions: Inferred Abilities, Needs and Interests versus Activity Histories

   Hyunggu Jung, Victoria Bellotti, Afsaneh Doryab, and 7 more authors

   In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, May 2016

   HTML PDF

2015

1. AMIA

   Biological Model Development as an Opportunity to Provide Content Auditing for the Foundational Model of Anatomy Ontology

   Lucy L Wang, Eli Grunblatt, Hyunggu Jung, and 2 more authors

   In AMIA Annual Symposium Proceedings, Nov 2015

   HTML PDF
2. SIGDOC

   Designing tools to support advanced users in new forms of social media interaction

   Hyunggu Jung, Sungsoo Hong, Perry Meas, and 1 more author

   In Proceedings of the 33rd Annual International Conference on the Design of Communication, Jul 2015

   HTML PDF