Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection
Large Language Models (LLMs) have found increasing application in tasks requiring multi-step reasoning, yet challenges such as hallucinations and inconsistencies in the generated responses persist. This study presents an innovative methodology to enhance the reasoning capabilities of LLMs by brokeri...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10966887/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850179265737261056 |
|---|---|
| author | Tejasvee Sheokand Garveet Jain Arshdeep Bahga Vijay K. Madisetti |
| author_facet | Tejasvee Sheokand Garveet Jain Arshdeep Bahga Vijay K. Madisetti |
| author_sort | Tejasvee Sheokand |
| collection | DOAJ |
| description | Large Language Models (LLMs) have found increasing application in tasks requiring multi-step reasoning, yet challenges such as hallucinations and inconsistencies in the generated responses persist. This study presents an innovative methodology to enhance the reasoning capabilities of LLMs by brokering and integrating multiple expert LLMs within a reflection layer to provide targeted feedback on the reasoning trajectories of the base LLM. The approach employs a foundational pre-trained LLM as the base model, which is further supported by agents to promote cognitive assistance for specific task types. In instances where conclusions are deemed incorrect or reasoning is interrupted, these instances are forwarded to the expert LLM layer, which includes systems such as Claude-3 haiku for intricate contexts and MedAlpaca for medical reasoning, to deliver feedback on the base model’s reasoning paths. This feedback forms a ‘reflection pool,’ enabling the base LLM to amend and enhance its reasoning trajectories in subsequent iterations. The experiments conducted across diverse datasets, including HotPotQA, SimpleQA, and PubmedQA, underscore the proposed architecture’s efficacy in augmenting success signals, Rouge-L scores (indicative of quality and precision), and CTRLEval Consistency Scores (indicative of coherence and consistency). The architecture effectively addresses the issues of hallucinations and inconsistencies that frequently occur in multi-step reasoning. Importantly, the approach exhibits considerable potential in tackling domain-specific tasks, underscoring the importance of achieving correct and reliable conclusions. To facilitate further investigation and validation of our proposed brokered multi-expert reflection framework for non-commercial use, the source code of our system is available at <uri>https://github.com/WiZY936/Brokered-Multi-Expert-Reflection</uri> |
| format | Article |
| id | doaj-art-a02f6d434a934623a18a158ec24461fd |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-a02f6d434a934623a18a158ec24461fd2025-08-20T02:18:32ZengIEEEIEEE Access2169-35362025-01-0113679936801910.1109/ACCESS.2025.356169310966887Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert ReflectionTejasvee Sheokand0https://orcid.org/0009-0000-8855-9738Garveet Jain1https://orcid.org/0009-0000-0578-3145Arshdeep Bahga2Vijay K. Madisetti3https://orcid.org/0000-0002-6539-6769Bennett University, Greater Noida, Uttar Pradesh, IndiaBennett University, Greater Noida, Uttar Pradesh, IndiaCloudemy Technology Labs, Chandigarh, IndiaGeorgia Institute of Technology, Atlanta, GA, USALarge Language Models (LLMs) have found increasing application in tasks requiring multi-step reasoning, yet challenges such as hallucinations and inconsistencies in the generated responses persist. This study presents an innovative methodology to enhance the reasoning capabilities of LLMs by brokering and integrating multiple expert LLMs within a reflection layer to provide targeted feedback on the reasoning trajectories of the base LLM. The approach employs a foundational pre-trained LLM as the base model, which is further supported by agents to promote cognitive assistance for specific task types. In instances where conclusions are deemed incorrect or reasoning is interrupted, these instances are forwarded to the expert LLM layer, which includes systems such as Claude-3 haiku for intricate contexts and MedAlpaca for medical reasoning, to deliver feedback on the base model’s reasoning paths. This feedback forms a ‘reflection pool,’ enabling the base LLM to amend and enhance its reasoning trajectories in subsequent iterations. The experiments conducted across diverse datasets, including HotPotQA, SimpleQA, and PubmedQA, underscore the proposed architecture’s efficacy in augmenting success signals, Rouge-L scores (indicative of quality and precision), and CTRLEval Consistency Scores (indicative of coherence and consistency). The architecture effectively addresses the issues of hallucinations and inconsistencies that frequently occur in multi-step reasoning. Importantly, the approach exhibits considerable potential in tackling domain-specific tasks, underscoring the importance of achieving correct and reliable conclusions. To facilitate further investigation and validation of our proposed brokered multi-expert reflection framework for non-commercial use, the source code of our system is available at <uri>https://github.com/WiZY936/Brokered-Multi-Expert-Reflection</uri>https://ieeexplore.ieee.org/document/10966887/LLMself-reflectionchain-of-thoughtmulti-hop inferencefeedback-driven refinement |
| spellingShingle | Tejasvee Sheokand Garveet Jain Arshdeep Bahga Vijay K. Madisetti Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection IEEE Access LLM self-reflection chain-of-thought multi-hop inference feedback-driven refinement |
| title | Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection |
| title_full | Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection |
| title_fullStr | Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection |
| title_full_unstemmed | Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection |
| title_short | Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection |
| title_sort | enhancing llm reasoning capabilities through brokered multi expert reflection |
| topic | LLM self-reflection chain-of-thought multi-hop inference feedback-driven refinement |
| url | https://ieeexplore.ieee.org/document/10966887/ |
| work_keys_str_mv | AT tejasveesheokand enhancingllmreasoningcapabilitiesthroughbrokeredmultiexpertreflection AT garveetjain enhancingllmreasoningcapabilitiesthroughbrokeredmultiexpertreflection AT arshdeepbahga enhancingllmreasoningcapabilitiesthroughbrokeredmultiexpertreflection AT vijaykmadisetti enhancingllmreasoningcapabilitiesthroughbrokeredmultiexpertreflection |