Facilitating university admission using a chatbot based on large language models with retrieval-augmented generation
University admission consultation is a professional service that assists students with the university application process. Typically, accessing this service entails exploring university websites, directly contacting faculty members and officers via phone calls or emails, and engaging educational int...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
International Forum of Educational Technology & Society
2024-10-01
|
| Series: | Educational Technology & Society |
| Subjects: | |
| Online Access: | https://www.j-ets.net/collection/published-issues/27_4#h.31gs75qaf1rj |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850203453904650240 |
|---|---|
| author | Zheng Chen Di Zou Haoran Xie Huajie Lou Zhiyuan Pang |
| author_facet | Zheng Chen Di Zou Haoran Xie Huajie Lou Zhiyuan Pang |
| author_sort | Zheng Chen |
| collection | DOAJ |
| description | University admission consultation is a professional service that assists students with the university application process. Typically, accessing this service entails exploring university websites, directly contacting faculty members and officers via phone calls or emails, and engaging educational intermediaries. University admission consultation is crucial for both students and institutions. However, conventional consultation methods face challenges such as time and spatial constraints, leading to a growing interest in utilizing chatbots for university admission consultation. In this study, we propose a novel approach that leverages generative pretrained transformer (ChatGPT 3.5) models and implements the retrieval-augmented generation technique using the LlamaIndex framework. To evaluate the effectiveness of this approach, we applied it to undergraduate admission data from three universities: a science and technology university in the United States, a comprehensive university in Kenya, and a comprehensive university in Hong Kong. We also gathered feedback from 53 high school students who tested the chatbot. The results demonstrated a significant improvement in average accuracy, from 41.4% with the ChatGPT 3.5 model to 89.5% with the proposed chatbot, with peak accuracy reaching 94.7%. User reviews also indicated a generally positive perception of the admission chatbot. This methodology has the potential to revolutionize university admissions by utilizing chatbots based on large language models with retrieval-augmented generation. |
| format | Article |
| id | doaj-art-1def23f23c554d11a962e11b2399a49e |
| institution | OA Journals |
| issn | 1176-3647 1436-4522 |
| language | English |
| publishDate | 2024-10-01 |
| publisher | International Forum of Educational Technology & Society |
| record_format | Article |
| series | Educational Technology & Society |
| spelling | doaj-art-1def23f23c554d11a962e11b2399a49e2025-08-20T02:11:30ZengInternational Forum of Educational Technology & SocietyEducational Technology & Society1176-36471436-45222024-10-01274454470https://doi.org/10.30191/ETS.202410_27(4).TP02Facilitating university admission using a chatbot based on large language models with retrieval-augmented generationZheng Chen0Di Zou1Haoran Xie2Huajie Lou3Zhiyuan Pang4Department of Statistics and Actuarial Science, The University of Hong Kong, Hong KongDepartment of English and Communication, The Hong Kong Polytechnic University, Hong KongSchool of Data Science, Lingnan University, Hong KongDepartment of Statistics and Actuarial Science, The University of Hong Kong, Hong KongDepartment of Statistics and Actuarial Science, The University of Hong Kong, Hong KongUniversity admission consultation is a professional service that assists students with the university application process. Typically, accessing this service entails exploring university websites, directly contacting faculty members and officers via phone calls or emails, and engaging educational intermediaries. University admission consultation is crucial for both students and institutions. However, conventional consultation methods face challenges such as time and spatial constraints, leading to a growing interest in utilizing chatbots for university admission consultation. In this study, we propose a novel approach that leverages generative pretrained transformer (ChatGPT 3.5) models and implements the retrieval-augmented generation technique using the LlamaIndex framework. To evaluate the effectiveness of this approach, we applied it to undergraduate admission data from three universities: a science and technology university in the United States, a comprehensive university in Kenya, and a comprehensive university in Hong Kong. We also gathered feedback from 53 high school students who tested the chatbot. The results demonstrated a significant improvement in average accuracy, from 41.4% with the ChatGPT 3.5 model to 89.5% with the proposed chatbot, with peak accuracy reaching 94.7%. User reviews also indicated a generally positive perception of the admission chatbot. This methodology has the potential to revolutionize university admissions by utilizing chatbots based on large language models with retrieval-augmented generation.https://www.j-ets.net/collection/published-issues/27_4#h.31gs75qaf1rjuniversity admissionschatbotgptlarge language modelsretrieval-augmented generation |
| spellingShingle | Zheng Chen Di Zou Haoran Xie Huajie Lou Zhiyuan Pang Facilitating university admission using a chatbot based on large language models with retrieval-augmented generation Educational Technology & Society university admissions chatbot gpt large language models retrieval-augmented generation |
| title | Facilitating university admission using a chatbot based on large language models with retrieval-augmented generation |
| title_full | Facilitating university admission using a chatbot based on large language models with retrieval-augmented generation |
| title_fullStr | Facilitating university admission using a chatbot based on large language models with retrieval-augmented generation |
| title_full_unstemmed | Facilitating university admission using a chatbot based on large language models with retrieval-augmented generation |
| title_short | Facilitating university admission using a chatbot based on large language models with retrieval-augmented generation |
| title_sort | facilitating university admission using a chatbot based on large language models with retrieval augmented generation |
| topic | university admissions chatbot gpt large language models retrieval-augmented generation |
| url | https://www.j-ets.net/collection/published-issues/27_4#h.31gs75qaf1rj |
| work_keys_str_mv | AT zhengchen facilitatinguniversityadmissionusingachatbotbasedonlargelanguagemodelswithretrievalaugmentedgeneration AT dizou facilitatinguniversityadmissionusingachatbotbasedonlargelanguagemodelswithretrievalaugmentedgeneration AT haoranxie facilitatinguniversityadmissionusingachatbotbasedonlargelanguagemodelswithretrievalaugmentedgeneration AT huajielou facilitatinguniversityadmissionusingachatbotbasedonlargelanguagemodelswithretrievalaugmentedgeneration AT zhiyuanpang facilitatinguniversityadmissionusingachatbotbasedonlargelanguagemodelswithretrievalaugmentedgeneration |