Human-Centered Artificial Intelligence Lab

HCAIL at the IJCAI 2024 conference HCAIL group photo at IJCAI’24 in August 2024!

Welcome to the Human-Centered Artificial Intelligence Lab (HCAIL)! We are a research group in the College of Nursing at Seoul National University. We enjoy working on the intersection of multiple domains: Human-centered AI, Digital Health, Social Computing, and Accessibility & Aging. We are pursing research in the combinations of the following directions:

Artificial Intelligence
Health Informatics
Human-Computer Interaction

Our mission is to advance AI research through design and engineering to support individuals with special needs.

Interested in joining HCAIL as a MS or PhD student?

We have an opening position for a fully-funded MS and PhD student starting Spring 2025. Visit [석사과정/박사과정/박사후연구연구원 모집공고]. Send your resume to me so that we can discuss about how to join HCAIL an MS or PhD student. For Spring 2025, we aim to admit up to 2-3 new MS or PhD students.

HCAIL 연구실 자료 참고바랍니다: [슬라이드] [영상]

Interested in a research opportunity for undergraduate students?

Visit [학부연구생 인턴 모집공고] and send your resume to me if you are interested in a unique research intern opportunity. HCAIL is ready for maximizing your research potential during your internship.

news

Aug 2024	Our paper has been accepted to appear at ASSETS’24, St. John’s, Newfoundland and Labrador, Canada (October 2024).
Aug 2024	Hyunmin and Yunseo served as a student volunteer at IJCAI’24 successfully!
Aug 2024	Our paper was presented at IJCAI 2024 in Seogwipo, Korea.
Aug 2024	Started a research project on Development of a large-scale multimodal data framework for supporting periodontal disease care through improving nutrition and microbiome environment supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) as a co-investigator.
Jul 2024	Hyunggu served as an co-organizer, and Hyunmin, Jiyoo, & Yunseo as a student volunteer at the 2024 SIGCHI Korea Local Chapter Summer Event successfully!

selected publications

2024

ASSETS

Toward Closed-Domain Conversational Item Listing Assistant for Improvement of Experiences of Older Adults in Customer-to-Customer (C2C) Marketplaces

Subin Park, Hyunggu Jung, Bektur Ryskeldiev, and 5 more authors

In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility, Oct 2024
IJCAI

SaGol: Using MiniGPT-4 to Generate Alt Text for Improving Image Accessibility

Yunseo Moon, Hyunmin Lee, SeungYoung Oh, and 1 more author

In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, Aug 2024

Demo Track

Abs HTML PDF

SaGol is an AI-powered application to improve image accessibility for people with visual impairments (PVI) users. Alternative (alt) text, a general method of web accessibility for PVI users, is text or phrases that describe images on a website in an understandable way. SaGol generates alt text with the images on the user’s smartphone using a vision large language model called MiniGPT-4. SaGol searches for similar images based on the generated alt text. We evaluated the length of alt text and the search accuracy. This paper shows a potential opportunity to improve image accessibility for PVI users.
JMIR

A Deep Learning–Based Rotten Food Recognition App for Older Adults: Development and Usability Study

Minki Chun, Ha-Jin Yu, and Hyunggu Jung

JMIR Form Res, Jul 2024

Abs HTML PDF

Background: Older adults are at greater risk of eating rotten fruits and of getting food poisoning because cognitive function declines as they age, making it difficult to distinguish rotten fruits. To address this problem, researchers have developed and evaluated various tools to detect rotten food items in various ways. Nevertheless, little is known about how to create an app to detect rotten food items to support older adults at a risk of health problems from eating rotten food items. Objective: This study aimed to (1) create a smartphone app that enables older adults to take a picture of food items with a camera and classifies the fruit as rotten or not rotten for older adults and (2) evaluate the usability of the app and the perceptions of older adults about the app. Methods: We developed a smartphone app that supports older adults in determining whether the 3 fruits selected for this study (apple, banana, and orange) were fresh enough to eat. We used several residual deep networks to check whether the fruit photos collected were of fresh fruit. We recruited healthy older adults aged over 65 years (n=15, 57.7%, males and n=11, 42.3%, females) as participants. We evaluated the usability of the app and the participants’ perceptions about the app through surveys and interviews. We analyzed the survey responses, including an after-scenario questionnaire, as evaluation indicators of the usability of the app and collected qualitative data from the interviewees for in-depth analysis of the survey responses. Results: The participants were satisfied with using an app to determine whether a fruit is fresh by taking a picture of the fruit but are reluctant to use the paid version of the app. The survey results revealed that the participants tended to use the app efficiently to take pictures of fruits and determine their freshness. The qualitative data analysis on app usability and participants’ perceptions about the app revealed that they found the app simple and easy to use, they had no difficulty taking pictures, and they found the app interface visually satisfactory. Conclusions: This study suggests the possibility of developing an app that supports older adults in identifying rotten food items effectively and efficiently. Future work to make the app distinguish the freshness of various food items other than the 3 fruits selected still remains.
DIB

Creating a bias-free dataset with food delivery app reviews under data poisoning attack

Hyunmin Lee, SeungYoung Oh, JinHyun Han, and 1 more author

Data in Brief, Jun 2024

Abs HTML

In online food delivery apps, customers write reviews to reflect their experiences. However, certain restaurants use a “review event” strategy to solicit favorable reviews from customers and boost their revenue. Review event is a marketing strategy where a restaurant owner gives free services to customers in return for a promise to write a review. Nevertheless, current datasets of app reviews for food delivery services neglect this situation. Furthermore, there appears to be an absence of datasets with reviews written in Korean. To solve this gap, this paper presents a dataset that contains reviews obtained from restaurants on a Korean app which use a review event strategy. A total of 128,668 reviews were gathered from 136 restaurants through crawling reviews using the Selenium library in Python. The dataset consists of detailed information of each review which contains information about ordered dishes, each review’s written time, whether the food image is included in the review or not, and various star ratings such as total, taste, quantity, and delivery ratings. This dataset supports an innovative process of preparing AI training data for achieving fairness AI by proposing a bias-free dataset of food delivery app reviews with data poisoning attacks as an example Additionally, the dataset is beneficial for researchers who are examining review events or analyzing the sentiment of food delivery app reviews.

2023

CSCW

Toward Value Scenario Generation Through Large Language Models

Hyunggu Jung, Woosuk Seo, Seokwoo Song, and 1 more author

In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing, Oct 2023

Abs HTML PDF

We propose a method of generating value scenarios for design research by leveraging ChatGPT, an AI-powered chatbot based on large language models. Identifying the needs of a vulnerable population, such as North Korean defectors, is challenging for researchers. To address this, we introduce ChatGPT-generated value scenarios, an extension of scenario-based design that supports critical, systemic, long-term thinking in current design practice, technology development, and deployment. Using our proposed method, we created a prompt to generate value scenarios on ChatGPT. Based on our analysis of the generated scenarios, we identified that ChatGPT could generate plausible information about Value Implications. However, it lacks details on Pervasiveness and Systemic Effects. After discussing the limitations and opportunities of ChatGPT in generating value scenarios, we conclude with suggestions for how ChatGPT might be better used to generate value scenarios.
CSCW

Visualizing the Carbon Intensity of Machine Learning Inference for Image Analysis on TensorFlow Hub

Taewon Yoo, Hyunmin Lee, SeungYoung Oh, and 2 more authors

In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing, Oct 2023

Abs HTML PDF

The increasing performance of machine learning (ML) models necessitates greater computing resources, contributing to rising carbon intensity in ML computing and raising concerns about computational equity. Previous studies focused on developing tools that enable model developers to view the carbon intensity of the ML models in the training process. Still, little is known about how to support ML developers in online communities to explore the carbon intensity of ML models during inference. We developed MIEV, a model inference emission visualizer, that supports ML developers on TensorFlow Hub to explore the carbon intensity of image domain models during the model Inference phase. We also provide insights into designing technologies that promote collaborative work among ML developers to drive sustainable AI development processes. To the best of our knowledge, this is the first attempt to interactively visualize the carbon intensity of ML models in online communities during the Inference phase.
IUI

WATAA: Web Alternative Text Authoring Assistant for Improving Web Content Accessibility

Hyeonhak Jeong, Minki Chun, Hyunmin Lee, and 2 more authors

In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, Mar 2023

Abs HTML PDF

Alternative (alt) text is essential for people with visual impairments to acquire information about image content through a screen reader. However, collecting images and creating alt text requires time and effort. To deal with this problem, automatically collecting images that have no alt text is essential to ensure that all image content in a web page has their alt text. Additionally, automatic alt text has limitations in accuracy and quality compared to human-created alt text despite the improvements in image recognition and natural language process technology. We present WATAA, a web alt text authoring assistant that collects images containing no alt text and suggests automatic alt text to help human alt text authors improve a web page’s accessibility.
IUI

PORDE: Explaining Data Poisoning Attacks Through Visual Analytics with Food Delivery App Reviews

Hyunmin Lee, Minki Chun, and Hyunggu Jung

In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, Mar 2023

Abs HTML PDF

Artificial intelligence (AI) gives many benefits to our lives. However, biased AI models created by receiving data poisoning attacks may induce social problems. Therefore, developers must consider carefully whether the training data received a poison attack when developing an AI model. Data visualization is one of the methods to facilitate the analysis of the data required for checking if the training data received a poisoning attack. However, prior studies did not visualize real-world AI training data. Restaurant reviews in delivery apps are one of the cases of a poisoned dataset. Restaurants hold review events on delivery apps to encourage customers to write a positive review in return for certain rewards, thereby creating reviews with bias. In this study, we propose POisoned Real-world Data Explainer (PORDE) that explains data poisoning attacks through visual analytics with food delivery app reviews. The findings of this study suggest implications for securing safe training data and developing less biased AI models.
IUI

Toward Keyword Generation Through Large Language Models

Wanhae Lee, Minki Chun, Hyeonhak Jeong, and 1 more author

In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, Mar 2023

Abs HTML PDF

It is essential to understand research trends for researchers, decision-makers, and investors. One way to analyze research trends is to collect and analyze author-defined keywords in scientific papers. Unfortunately, while author-defined keywords are beneficial to researchers aiming to figure out the trends of their research fields, 45% of scientific papers in Microsoft Academic Graph did not contain their author-defined keywords. Additionally, six of the top seven AI conferences neither collect nor disclose keywords. This paper proposes a method for generating the keywords using Galactica, a pre-trained large language model published by Meta. We evaluate this method’s performance by comparing the keywords provided by authors in the CoRL’22 and report characteristics of the generated keywords. Our study shows the F1 score of our proposed method was ten times better than that of previous studies, and 42.7% of the generated keywords are relevant to author-defined keywords.

2022

CSCW

Exploring the Community of Model Publishers on TensorFlow Hub

Taewon Yoo, Minki Chun, Yunjung Bae, and 2 more authors

In Companion Publication of the 2022 Conference on Computer Supported Cooperative Work and Social Computing, Nov 2022

Abs HTML PDF

We explore the community of AI model publishers on TensorFlow Hub (TF Hub). While researchers identified the challenges AI model publishers and AI model users faced, little is known about how they interact with each other in an online community. The analysis of the metadata recorded on TF Hub revealed the models that the AI model publishers uploaded. Also, we found out how the models published by the AI model publishers were shared with other people on TF Hub. To our knowledge, this is the first attempt to explore the online community of AI model publishers sharing their models with each other.
ECSCW

Toward an AI-assisted Assessment Tool to Support Online Art Therapy Practices: A Pilot Study

Woosuk Seo, Joonyoung Jun, Minki Chun, and 5 more authors

In Proceedings of 20th European Conference on Computer-Supported Cooperative Work, Nov 2022

Abs HTML PDF

Artificial intelligence (AI) has been widely used to assist art therapists with artwork assessments by providing objective information. While prior studies showed that AI-assisted tools are feasible to improve drawing analysis in in-person art therapy practices, the use of those tools in online art therapy is still under-examined. To fill the gap, we created a prototype of an AI-assisted tool for online therapy in a House-Tree-Person (HTP) test scenario and ran lab-based usability sessions with 10 art therapists in which they used our proposed prototype to complete predefined tasks. We then conducted semi-structured interviews with the participants to understand their acceptance and concerns about the prototype. The findings revealed the unique needs of art therapists and opportunities of using AI-assisted tools to improve online art therapy practices. Based on these findings, we suggest implications for creating AI-assisted tools that meet specific needs of art therapists in online therapy sessions, and further discuss future directions of research about AI-assisted tools for art therapists in online settings.

2021

CSCW

Exploring the Experiences of Student Volunteer and Student Volunteer Chair Communities at Academic Conferences

Subin Park, Heejae Jung, Jae Won Choi, and 4 more authors

Proceedings of the ACM on Human-Computer Interaction, Oct 2021

HTML
CSCW

Exploring the Experiences of Streamers with Visual Impairments

Joonyoung Jun, Woosuk Seo, Jihyeon Park, and 2 more authors

Proceedings of the ACM on Human-Computer Interaction, Oct 2021

HTML
IUI

MonoPass: A Password Manager without Master Password Authentication

Hyeonhak Jeong, and Hyunggu Jung

In 26th International Conference on Intelligent User Interfaces-Companion, Apr 2021

HTML PDF
IUI

LectYS: A System for Summarizing Lecture Videos on YouTube

Taewon Yoo, Hyewon Jeong, Donghwan Lee, and 1 more author

In 26th International Conference on Intelligent User Interfaces-Companion, Apr 2021

HTML PDF
JMIR

A novel food record app for dietary assessments among older adults with type 2 diabetes: development and usability study

Hyunggu Jung, George Demiris, Peter Tarczy-Hornoch, and 1 more author

JMIR Formative Research, Feb 2021

HTML

2017

ASSETS

Exploring the community of blind or visually impaired people on YouTube

Woosuk Seo, and Hyunggu Jung

In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, Oct 2017

HTML PDF
CSCW

Personas and scenarios to design technologies for North Korean defectors with depression

Hyunggu Jung, Woosuk Seo, and Michelle Cha

In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, Feb 2017

HTML PDF

2016

AMIA

Development of a novel markov chain model for the prediction of head and neck squamous cell carcinoma dissemination

Hyunggu Jung, Anthony Law, Eli Grunblatt, and 4 more authors

In AMIA Annual Symposium Proceedings, Nov 2016

HTML PDF
AMIA

Nurse informaticians report low satisfaction and multi-level concerns with electronic health records: results from an international survey

Maxim Topaz, Charlene Ronquillo, Laura-Maria Peltonen, and 8 more authors

In AMIA Annual Symposium Proceedings, Nov 2016

HTML PDF
CHI

’MASTerful’Matchmaking in Service Transactions: Inferred Abilities, Needs and Interests versus Activity Histories

Hyunggu Jung, Victoria Bellotti, Afsaneh Doryab, and 7 more authors

In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, May 2016

HTML PDF

2015

AMIA

Biological Model Development as an Opportunity to Provide Content Auditing for the Foundational Model of Anatomy Ontology

Lucy L Wang, Eli Grunblatt, Hyunggu Jung, and 2 more authors

In AMIA Annual Symposium Proceedings, Nov 2015

HTML PDF
SIGDOC

Designing tools to support advanced users in new forms of social media interaction

Hyunggu Jung, Sungsoo Hong, Perry Meas, and 1 more author

In Proceedings of the 33rd Annual International Conference on the Design of Communication, Jul 2015

HTML PDF