Designing Social Robots with LLMs for Engaging Human Interaction
Large Language Models (LLMs), particularly those enhanced through Reinforcement Learning from Human Feedback, such as ChatGPT, have opened up new possibilities for natural and open-ended spoken interaction in social robotics. However, these models are not inherently designed for embodied, multimodal...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/11/6377 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850160506293190656 |
|---|---|
| author | Maria Pinto-Bernal Matthijs Biondina Tony Belpaeme |
| author_facet | Maria Pinto-Bernal Matthijs Biondina Tony Belpaeme |
| author_sort | Maria Pinto-Bernal |
| collection | DOAJ |
| description | Large Language Models (LLMs), particularly those enhanced through Reinforcement Learning from Human Feedback, such as ChatGPT, have opened up new possibilities for natural and open-ended spoken interaction in social robotics. However, these models are not inherently designed for embodied, multimodal contexts. This paper presents a user-centred approach to integrating an LLM into a humanoid robot, designed to engage in fluid, context-aware conversation with socially isolated older adults. We describe our system architecture, which combines real-time speech processing, layered memory summarisation, persona conditioning, and multilingual voice adaptation to support personalised, socially appropriate interactions. Through iterative development and evaluation, including in-home exploratory trials with older adults (<i>n</i> = 7) and a preliminary study with young adults (<i>n</i> = 43), we investigated the technical and experiential challenges of deploying LLMs in real-world human–robot dialogue. Our findings show that memory continuity, adaptive turn-taking, and culturally attuned voice design enhance user perceptions of trust, naturalness, and social presence. We also identify persistent limitations related to response latency, hallucinations, and expectation management. This work contributes design insights and architectural strategies for future LLM-integrated robots that aim to support meaningful, emotionally resonant companionship in socially assistive settings. |
| format | Article |
| id | doaj-art-aa33b517e73a4396b07f7f27979dbfc9 |
| institution | OA Journals |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-aa33b517e73a4396b07f7f27979dbfc92025-08-20T02:23:08ZengMDPI AGApplied Sciences2076-34172025-06-011511637710.3390/app15116377Designing Social Robots with LLMs for Engaging Human InteractionMaria Pinto-Bernal0Matthijs Biondina1Tony Belpaeme2IDLab-Airo, Ghent University—imec, Technologiepark-Zwijnaarde 126, 9052 Ghent, BelgiumIDLab-Airo, Ghent University—imec, Technologiepark-Zwijnaarde 126, 9052 Ghent, BelgiumIDLab-Airo, Ghent University—imec, Technologiepark-Zwijnaarde 126, 9052 Ghent, BelgiumLarge Language Models (LLMs), particularly those enhanced through Reinforcement Learning from Human Feedback, such as ChatGPT, have opened up new possibilities for natural and open-ended spoken interaction in social robotics. However, these models are not inherently designed for embodied, multimodal contexts. This paper presents a user-centred approach to integrating an LLM into a humanoid robot, designed to engage in fluid, context-aware conversation with socially isolated older adults. We describe our system architecture, which combines real-time speech processing, layered memory summarisation, persona conditioning, and multilingual voice adaptation to support personalised, socially appropriate interactions. Through iterative development and evaluation, including in-home exploratory trials with older adults (<i>n</i> = 7) and a preliminary study with young adults (<i>n</i> = 43), we investigated the technical and experiential challenges of deploying LLMs in real-world human–robot dialogue. Our findings show that memory continuity, adaptive turn-taking, and culturally attuned voice design enhance user perceptions of trust, naturalness, and social presence. We also identify persistent limitations related to response latency, hallucinations, and expectation management. This work contributes design insights and architectural strategies for future LLM-integrated robots that aim to support meaningful, emotionally resonant companionship in socially assistive settings.https://www.mdpi.com/2076-3417/15/11/6377Large Language Modelsspoken language interactionsocial robotselderly users |
| spellingShingle | Maria Pinto-Bernal Matthijs Biondina Tony Belpaeme Designing Social Robots with LLMs for Engaging Human Interaction Applied Sciences Large Language Models spoken language interaction social robots elderly users |
| title | Designing Social Robots with LLMs for Engaging Human Interaction |
| title_full | Designing Social Robots with LLMs for Engaging Human Interaction |
| title_fullStr | Designing Social Robots with LLMs for Engaging Human Interaction |
| title_full_unstemmed | Designing Social Robots with LLMs for Engaging Human Interaction |
| title_short | Designing Social Robots with LLMs for Engaging Human Interaction |
| title_sort | designing social robots with llms for engaging human interaction |
| topic | Large Language Models spoken language interaction social robots elderly users |
| url | https://www.mdpi.com/2076-3417/15/11/6377 |
| work_keys_str_mv | AT mariapintobernal designingsocialrobotswithllmsforengaginghumaninteraction AT matthijsbiondina designingsocialrobotswithllmsforengaginghumaninteraction AT tonybelpaeme designingsocialrobotswithllmsforengaginghumaninteraction |