Research
Below is a summary of my current research projects. For abstracts of and links to published manuscripts, see Publications.
Topic 1: Human-AI Social Interaction in Extended Reality (XR)
It is becoming evident that interactions with artificial intelligence (AI) based agents can positively influence human behavior and judgment. However, studies to date focus on text-based conversational agents (CA) with limited embodiment, restricting our understanding of how social influence principles apply to AI agents. We address this gap by leveraging the latest advances in AI (language models) and combining them with immersive virtual reality (VR) to build AI-driven embodied agents that can naturally converse with humans.
The first study using this research paradigm examined the effect of interpersonal similarity (via gender matching) on social, behavioral, and biobehavioral outcomes:
Lim, S., Schmälzle, R, & Bente, G. (2024). Artificial social influence via human-embodied AI agent interaction in immersive virtual reality (VR): Effects of similarity-matching during health conversations. arXiv. https://doi.org/10.48550/arXiv.2406.05486
Lim, S., Schmälzle, R., & Bente, G. (in press). Artificial social influence: Rapport-building, LLM-based embodied conversational agents for health coaching. Proceedings of CONNECT Workshop at the 24th ACM International conference on Intelligent virtual agents (IVA). [Link to Paper]
I was nominated for the Google PhD Fellowship for this work.
Topic 2: Designing Emerging Technology-Based Interventions
Recent technological advancements in AI, XR, and social robots allow us to create effective interventions for specific communities and topics. My current work in this area, funded by the Strosacker Research Grant, involves designing an immersive virtual reality (VR) and social robot well-being support tools for an immigrant young adult community. Three papers from this project are either under review or in preparation.
Previous work examined how to design conversational agents to help older adults combat misinformation:
Peng, W., Lee, H., & Lim, S. (2024). Leverage chatbots to combat health misinformation for older adults: A participatory design approach. JMIR Formative Research. https://doi.org/10.2196/60712
Using Generative AI for Persuasive Communication
My other projects used generative AI to develop informative and persuasive health messages.
Lim, S., & Schmälzle, R. (2024). The effect of source disclosure on evaluation of AI-generated messages. Computers in Human Behavior: Artificial Human, 2(1), 100058. https://doi.org/10.1016/j.chbah.2024.100058
Lim, S., & Schmälzle, R. (2023). Artificial intelligence for health message generation: An empirical study using a large language model (LLM) and prompt engineering. Frontiers in Communication, 8, 1129082. https://doi.org/10.1371/journal.pone.0291924
Topic 3: Using XR to Examine Psychological and Communication Processes
My third research area capitalizes on XR platforms, integrated with biobehavioral and psychophysiological measures, to examine individuals’ information processing in simulated real-life scenarios.
NSF VR Public Speaking Study
Our current NSF-funded study in the CARISMA lab investigates public speaking behavior in immersive VR using neurophysiological and self-report measurements. Specifically, participants gave presentations in front of a supportive and unsupportive audience while immersed in a simulated virtual conference venue. Manuscripts from this project are in preparation.
Examination of Message Effects via Immersive VR
We are constantly exposed to persuasive messaging in our day-to-day lives, yet it is difficult for researchers to precisely measure how much attention people pay to these messages:
- With stationary eye-trackers, it is difficult to study how messages attract attention in a natural-real-world setting.
- In controlled laboratory settings, it is difficult to achieve ecological validity and account for real-life situational factors.
Immersive virtual reality (VR), with integrated eye-tracking capabilities, is a promising platform that allows us to examine message effects in simulated real-life contexts. In addition, the advancements in generative AI tools allow us to augment our message development processes.
The most recent study had participants drive through a virtual city with 40 billboards distributed throughout. We then examined how the participants’ task condition (sale sign counting vs. free driving), billboard location (start vs. middle vs. end), and visual message design (less vs. more attention-grabbing) influenced their viewing behavior (i.e., fixation count and gaze duration) and memory (recall and recognition).
Lim, S., Cho, H. J., Jeon, M., Cui, X., & Schmälzle, R. (2024). Using VR and eye-tracking to study attention to and retention of AI-generated ads in outdoor advertising environments. bioRxiv. https://doi.org/10.1101/2024.08.15.607684 *Second/third authors contributed equally.
This study received an honorable mention (best poster) at the 2024 Kentucky Conference on Health Communication (KCHC).
Other works in this area are below: Schmälzle, R., Lim, S., Cho, H. J., Wu, J., & Bente, G. (2023). Examining the exposure-reception-retention link in realistic communication environments via VR and eye-tracking: The VR billboard paradigm. PLOS One, 18(11), e0291924. https://doi.org/10.1371/journal.pone.0291924
Jeon, M., Lim, S., Lapinski, M. K., Bente, G., Spates, S., & Schmälzle, R. (2024). Attention and retention effects of culturally targeted billboard messages: An eye-tracking study in a realistic virtual reality environment. bioRxiv. https://doi.org/10.1101/2024.09.10.610975
Cho, H. J., Lim, S., Turner, M., Bente, G., & Schmälzle, R. (2024). Eyes on VR: Unpacking the causal chain between exposure, reception, and retention for emotional billboard messages. bioRxiv. https://doi.org/10.1101/2024.07.19.604208
Schmälzle, R.1, Wu, J.1, Lim, S., & Bente, G. (2024). The eyes have it: Inter-subject correlations of pupillary responses for audience response measurement in VR. bioRxiv. https://doi.org/10.1101/2024.01.22.576685