A Comparative Performance Analysis of Locally Deployed Large Language Models Through a Retrieval-Augmented Generation Educational Assistant Application for Textual Data Extraction
<b>Background:</b> Rapid advancements in large language models (LLMs) have significantly enhanced Retrieval-Augmented Generation (RAG) techniques, leading to more accurate and context-aware information retrieval systems. <b>Methods:</b> This article presents the creation of a...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | AI |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2673-2688/6/6/119 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849423955756056576 |
|---|---|
| author | Amitabh Mishra Nagaraju Brahmanapally |
| author_facet | Amitabh Mishra Nagaraju Brahmanapally |
| author_sort | Amitabh Mishra |
| collection | DOAJ |
| description | <b>Background:</b> Rapid advancements in large language models (LLMs) have significantly enhanced Retrieval-Augmented Generation (RAG) techniques, leading to more accurate and context-aware information retrieval systems. <b>Methods:</b> This article presents the creation of a RAG-based chatbot tailored for university course catalogs, aimed at answering queries related to course details and other essential academic information, and investigates its performance by testing it on several locally deployed large language models. By leveraging multiple LLM architectures, we evaluate performance of the models under test in terms of context length, embedding size, computational efficiency, and relevance of responses. <b>Results:</b> The experimental analysis obtained by this research, which builds on recent comparative studies, reveals that while larger models achieve higher relevance scores, they incur greater response times than smaller, more efficient models. <b>Conclusions:</b> The findings underscore the importance of balancing accuracy and efficiency for real-time educational applications. Overall, this work contributes to the field by offering insights into optimal RAG configurations and practical guidelines for deploying AI-powered educational assistants. |
| format | Article |
| id | doaj-art-8925f2ac2ddb43bda54785b9f9a5cf87 |
| institution | Kabale University |
| issn | 2673-2688 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | AI |
| spelling | doaj-art-8925f2ac2ddb43bda54785b9f9a5cf872025-08-20T03:30:24ZengMDPI AGAI2673-26882025-06-016611910.3390/ai6060119A Comparative Performance Analysis of Locally Deployed Large Language Models Through a Retrieval-Augmented Generation Educational Assistant Application for Textual Data ExtractionAmitabh Mishra0Nagaraju Brahmanapally1Department of Computer Science, University of West Florida, 11000 University Parkway, Pensacola, FL 32514, USADepartment of Computer Science, University of West Florida, 11000 University Parkway, Pensacola, FL 32514, USA<b>Background:</b> Rapid advancements in large language models (LLMs) have significantly enhanced Retrieval-Augmented Generation (RAG) techniques, leading to more accurate and context-aware information retrieval systems. <b>Methods:</b> This article presents the creation of a RAG-based chatbot tailored for university course catalogs, aimed at answering queries related to course details and other essential academic information, and investigates its performance by testing it on several locally deployed large language models. By leveraging multiple LLM architectures, we evaluate performance of the models under test in terms of context length, embedding size, computational efficiency, and relevance of responses. <b>Results:</b> The experimental analysis obtained by this research, which builds on recent comparative studies, reveals that while larger models achieve higher relevance scores, they incur greater response times than smaller, more efficient models. <b>Conclusions:</b> The findings underscore the importance of balancing accuracy and efficiency for real-time educational applications. Overall, this work contributes to the field by offering insights into optimal RAG configurations and practical guidelines for deploying AI-powered educational assistants.https://www.mdpi.com/2673-2688/6/6/119large language modelsGenerative Artificial IntelligenceRetrieval-Augmented Generationnatural language processing |
| spellingShingle | Amitabh Mishra Nagaraju Brahmanapally A Comparative Performance Analysis of Locally Deployed Large Language Models Through a Retrieval-Augmented Generation Educational Assistant Application for Textual Data Extraction AI large language models Generative Artificial Intelligence Retrieval-Augmented Generation natural language processing |
| title | A Comparative Performance Analysis of Locally Deployed Large Language Models Through a Retrieval-Augmented Generation Educational Assistant Application for Textual Data Extraction |
| title_full | A Comparative Performance Analysis of Locally Deployed Large Language Models Through a Retrieval-Augmented Generation Educational Assistant Application for Textual Data Extraction |
| title_fullStr | A Comparative Performance Analysis of Locally Deployed Large Language Models Through a Retrieval-Augmented Generation Educational Assistant Application for Textual Data Extraction |
| title_full_unstemmed | A Comparative Performance Analysis of Locally Deployed Large Language Models Through a Retrieval-Augmented Generation Educational Assistant Application for Textual Data Extraction |
| title_short | A Comparative Performance Analysis of Locally Deployed Large Language Models Through a Retrieval-Augmented Generation Educational Assistant Application for Textual Data Extraction |
| title_sort | comparative performance analysis of locally deployed large language models through a retrieval augmented generation educational assistant application for textual data extraction |
| topic | large language models Generative Artificial Intelligence Retrieval-Augmented Generation natural language processing |
| url | https://www.mdpi.com/2673-2688/6/6/119 |
| work_keys_str_mv | AT amitabhmishra acomparativeperformanceanalysisoflocallydeployedlargelanguagemodelsthrougharetrievalaugmentedgenerationeducationalassistantapplicationfortextualdataextraction AT nagarajubrahmanapally acomparativeperformanceanalysisoflocallydeployedlargelanguagemodelsthrougharetrievalaugmentedgenerationeducationalassistantapplicationfortextualdataextraction AT amitabhmishra comparativeperformanceanalysisoflocallydeployedlargelanguagemodelsthrougharetrievalaugmentedgenerationeducationalassistantapplicationfortextualdataextraction AT nagarajubrahmanapally comparativeperformanceanalysisoflocallydeployedlargelanguagemodelsthrougharetrievalaugmentedgenerationeducationalassistantapplicationfortextualdataextraction |