Online Health Information–Seeking in the Era of Large Language Models: Cross-Sectional Web-Based Survey Study
BackgroundAs large language model (LLM)–based chatbots such as ChatGPT (OpenAI) grow in popularity, it is essential to understand their role in delivering online health information compared to other resources. These chatbots often generate inaccurate content, posing potential...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-03-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e68560 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850275463418609664 |
|---|---|
| author | Hye Sun Yun Timothy Bickmore |
| author_facet | Hye Sun Yun Timothy Bickmore |
| author_sort | Hye Sun Yun |
| collection | DOAJ |
| description |
BackgroundAs large language model (LLM)–based chatbots such as ChatGPT (OpenAI) grow in popularity, it is essential to understand their role in delivering online health information compared to other resources. These chatbots often generate inaccurate content, posing potential safety risks. This motivates the need to examine how users perceive and act on health information provided by LLM-based chatbots.
ObjectiveThis study investigates the patterns, perceptions, and actions of users seeking health information online, including LLM-based chatbots. The relationships between online health information–seeking behaviors and important sociodemographic characteristics are examined as well.
MethodsA web-based survey of crowd workers was conducted via Prolific. The questionnaire covered sociodemographic information, trust in health care providers, eHealth literacy, artificial intelligence (AI) attitudes, chronic health condition status, online health information source types, perceptions, and actions, such as cross-checking or adherence. Quantitative and qualitative analyses were applied.
ResultsMost participants consulted search engines (291/297, 98%) and health-related websites (203/297, 68.4%) for their health information, while 21.2% (63/297) used LLM-based chatbots, with ChatGPT and Microsoft Copilot being the most popular. Most participants (268/297, 90.2%) sought information on health conditions, with fewer seeking advice on medication (179/297, 60.3%), treatments (137/297, 46.1%), and self-diagnosis (62/297, 23.2%). Perceived information quality and trust varied little across source types. The preferred source for validating information from the internet was consulting health care professionals (40/132, 30.3%), while only a very small percentage of participants (5/214, 2.3%) consulted AI tools to cross-check information from search engines and health-related websites. For information obtained from LLM-based chatbots, 19.4% (12/63) of participants cross-checked the information, while 48.4% (30/63) of participants followed the advice. Both of these rates were lower than information from search engines, health-related websites, forums, or social media. Furthermore, use of LLM-based chatbots for health information was negatively correlated with age (ρ=–0.16, P=.006). In contrast, attitudes surrounding AI for medicine had significant positive correlations with the number of source types consulted for health advice (ρ=0.14, P=.01), use of LLM-based chatbots for health information (ρ=0.31, P<.001), and number of health topics searched (ρ=0.19, P<.001).
ConclusionsAlthough traditional online sources remain dominant, LLM-based chatbots are emerging as a resource for health information for some users, specifically those who are younger and have a higher trust in AI. The perceived quality and trustworthiness of health information varied little across source types. However, the adherence to health information from LLM-based chatbots seemed more cautious compared to search engines or health-related websites. As LLMs continue to evolve, enhancing their accuracy and transparency will be essential in mitigating any potential risks by supporting responsible information-seeking while maximizing the potential of AI in health contexts. |
| format | Article |
| id | doaj-art-97626ea06dbe47f6a37aefb7fe0bef36 |
| institution | OA Journals |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-97626ea06dbe47f6a37aefb7fe0bef362025-08-20T01:50:44ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-03-0127e6856010.2196/68560Online Health Information–Seeking in the Era of Large Language Models: Cross-Sectional Web-Based Survey StudyHye Sun Yunhttps://orcid.org/0000-0001-6405-8082Timothy Bickmorehttps://orcid.org/0000-0002-2676-9788 BackgroundAs large language model (LLM)–based chatbots such as ChatGPT (OpenAI) grow in popularity, it is essential to understand their role in delivering online health information compared to other resources. These chatbots often generate inaccurate content, posing potential safety risks. This motivates the need to examine how users perceive and act on health information provided by LLM-based chatbots. ObjectiveThis study investigates the patterns, perceptions, and actions of users seeking health information online, including LLM-based chatbots. The relationships between online health information–seeking behaviors and important sociodemographic characteristics are examined as well. MethodsA web-based survey of crowd workers was conducted via Prolific. The questionnaire covered sociodemographic information, trust in health care providers, eHealth literacy, artificial intelligence (AI) attitudes, chronic health condition status, online health information source types, perceptions, and actions, such as cross-checking or adherence. Quantitative and qualitative analyses were applied. ResultsMost participants consulted search engines (291/297, 98%) and health-related websites (203/297, 68.4%) for their health information, while 21.2% (63/297) used LLM-based chatbots, with ChatGPT and Microsoft Copilot being the most popular. Most participants (268/297, 90.2%) sought information on health conditions, with fewer seeking advice on medication (179/297, 60.3%), treatments (137/297, 46.1%), and self-diagnosis (62/297, 23.2%). Perceived information quality and trust varied little across source types. The preferred source for validating information from the internet was consulting health care professionals (40/132, 30.3%), while only a very small percentage of participants (5/214, 2.3%) consulted AI tools to cross-check information from search engines and health-related websites. For information obtained from LLM-based chatbots, 19.4% (12/63) of participants cross-checked the information, while 48.4% (30/63) of participants followed the advice. Both of these rates were lower than information from search engines, health-related websites, forums, or social media. Furthermore, use of LLM-based chatbots for health information was negatively correlated with age (ρ=–0.16, P=.006). In contrast, attitudes surrounding AI for medicine had significant positive correlations with the number of source types consulted for health advice (ρ=0.14, P=.01), use of LLM-based chatbots for health information (ρ=0.31, P<.001), and number of health topics searched (ρ=0.19, P<.001). ConclusionsAlthough traditional online sources remain dominant, LLM-based chatbots are emerging as a resource for health information for some users, specifically those who are younger and have a higher trust in AI. The perceived quality and trustworthiness of health information varied little across source types. However, the adherence to health information from LLM-based chatbots seemed more cautious compared to search engines or health-related websites. As LLMs continue to evolve, enhancing their accuracy and transparency will be essential in mitigating any potential risks by supporting responsible information-seeking while maximizing the potential of AI in health contexts.https://www.jmir.org/2025/1/e68560 |
| spellingShingle | Hye Sun Yun Timothy Bickmore Online Health Information–Seeking in the Era of Large Language Models: Cross-Sectional Web-Based Survey Study Journal of Medical Internet Research |
| title | Online Health Information–Seeking in the Era of Large Language Models: Cross-Sectional Web-Based Survey Study |
| title_full | Online Health Information–Seeking in the Era of Large Language Models: Cross-Sectional Web-Based Survey Study |
| title_fullStr | Online Health Information–Seeking in the Era of Large Language Models: Cross-Sectional Web-Based Survey Study |
| title_full_unstemmed | Online Health Information–Seeking in the Era of Large Language Models: Cross-Sectional Web-Based Survey Study |
| title_short | Online Health Information–Seeking in the Era of Large Language Models: Cross-Sectional Web-Based Survey Study |
| title_sort | online health information seeking in the era of large language models cross sectional web based survey study |
| url | https://www.jmir.org/2025/1/e68560 |
| work_keys_str_mv | AT hyesunyun onlinehealthinformationseekingintheeraoflargelanguagemodelscrosssectionalwebbasedsurveystudy AT timothybickmore onlinehealthinformationseekingintheeraoflargelanguagemodelscrosssectionalwebbasedsurveystudy |