Towards a large-language-model-based chatbot system to automatically monitor student goal setting and planning in online learning
Despite the prevalence of online learning, the lack of student self-regulated learning (SRL) skills continues to be persistent issue. To support students’ SRL, teachers can prompt with SRL-related questions and provide timely, personalized feedback. Providing timely, personalized feedback to each st...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
International Forum of Educational Technology & Society
2025-07-01
|
| Series: | Educational Technology & Society |
| Subjects: | |
| Online Access: | https://www.j-ets.net/collection/published-issues/28_3#h.jm43iwcjozgo |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849429416496594944 |
|---|---|
| author | Khe Foon Hew, Weijiao Huang, Sikai Wang, Xinyi Luo and Donn Emmanuel Gonda |
| author_facet | Khe Foon Hew, Weijiao Huang, Sikai Wang, Xinyi Luo and Donn Emmanuel Gonda |
| author_sort | Khe Foon Hew, Weijiao Huang, Sikai Wang, Xinyi Luo and Donn Emmanuel Gonda |
| collection | DOAJ |
| description | Despite the prevalence of online learning, the lack of student self-regulated learning (SRL) skills continues to be persistent issue. To support students’ SRL, teachers can prompt with SRL-related questions and provide timely, personalized feedback. Providing timely, personalized feedback to each student in large classes, however, can be labor-intensive for teachers. This 2-stage study offers a novel contribution by developing a Large Language Model (LLM)-based chatbot system that can automatically monitor students’ goal setting and planning in online learning. Goal setting and planning are two important skills that can occur in all SRL phases. In stage 1, we developed the Goal-And-Plan-Mentor ChatGPT system (GoalPlanMentor) by creating an SRL knowledge base with goal and plan indicators, using Memory-Augmented-Prompts to automatically detect student goals and plans, and providing personalized feedback. In stage 2, we compared the accuracy of GoalPlanMentor’s detection (coding) of students’ goals and plans with human coders, examined the quality of GoalPlanMentor’s feedback, and students’ perceptions about the usefulness of GoalPlanMentor. Results show substantial to near perfect agreement between GoalPlanMentor’s and human’s coding, and high quality of GoalPlanMentor’s feedback in terms of providing clear directions for improvement. Overall, students perceived GoalPlanMentor to be useful in setting their goals and plans, with average values being significantly higher than the midpoint of the scale. Students who highly perceived the system’s usefulness for goal-setting exhibited significantly greater learning achievements compared to those with a low perception of its usefulness. Implications for future research are discussed. |
| format | Article |
| id | doaj-art-9f60cd8a644b4e6cb8ed79a725d60b66 |
| institution | Kabale University |
| issn | 1176-3647 1436-4522 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | International Forum of Educational Technology & Society |
| record_format | Article |
| series | Educational Technology & Society |
| spelling | doaj-art-9f60cd8a644b4e6cb8ed79a725d60b662025-08-20T03:28:22ZengInternational Forum of Educational Technology & SocietyEducational Technology & Society1176-36471436-45222025-07-01283112132https://doi.org/10.30191/ETS.202507_28(3).SP08Towards a large-language-model-based chatbot system to automatically monitor student goal setting and planning in online learningKhe Foon Hew, Weijiao Huang, Sikai Wang, Xinyi Luo and Donn Emmanuel GondaDespite the prevalence of online learning, the lack of student self-regulated learning (SRL) skills continues to be persistent issue. To support students’ SRL, teachers can prompt with SRL-related questions and provide timely, personalized feedback. Providing timely, personalized feedback to each student in large classes, however, can be labor-intensive for teachers. This 2-stage study offers a novel contribution by developing a Large Language Model (LLM)-based chatbot system that can automatically monitor students’ goal setting and planning in online learning. Goal setting and planning are two important skills that can occur in all SRL phases. In stage 1, we developed the Goal-And-Plan-Mentor ChatGPT system (GoalPlanMentor) by creating an SRL knowledge base with goal and plan indicators, using Memory-Augmented-Prompts to automatically detect student goals and plans, and providing personalized feedback. In stage 2, we compared the accuracy of GoalPlanMentor’s detection (coding) of students’ goals and plans with human coders, examined the quality of GoalPlanMentor’s feedback, and students’ perceptions about the usefulness of GoalPlanMentor. Results show substantial to near perfect agreement between GoalPlanMentor’s and human’s coding, and high quality of GoalPlanMentor’s feedback in terms of providing clear directions for improvement. Overall, students perceived GoalPlanMentor to be useful in setting their goals and plans, with average values being significantly higher than the midpoint of the scale. Students who highly perceived the system’s usefulness for goal-setting exhibited significantly greater learning achievements compared to those with a low perception of its usefulness. Implications for future research are discussed.https://www.j-ets.net/collection/published-issues/28_3#h.jm43iwcjozgogenerative artificial intelligencechatbotself-regulated learningonline learninglarge language models |
| spellingShingle | Khe Foon Hew, Weijiao Huang, Sikai Wang, Xinyi Luo and Donn Emmanuel Gonda Towards a large-language-model-based chatbot system to automatically monitor student goal setting and planning in online learning Educational Technology & Society generative artificial intelligence chatbot self-regulated learning online learning large language models |
| title | Towards a large-language-model-based chatbot system to automatically monitor student goal setting and planning in online learning |
| title_full | Towards a large-language-model-based chatbot system to automatically monitor student goal setting and planning in online learning |
| title_fullStr | Towards a large-language-model-based chatbot system to automatically monitor student goal setting and planning in online learning |
| title_full_unstemmed | Towards a large-language-model-based chatbot system to automatically monitor student goal setting and planning in online learning |
| title_short | Towards a large-language-model-based chatbot system to automatically monitor student goal setting and planning in online learning |
| title_sort | towards a large language model based chatbot system to automatically monitor student goal setting and planning in online learning |
| topic | generative artificial intelligence chatbot self-regulated learning online learning large language models |
| url | https://www.j-ets.net/collection/published-issues/28_3#h.jm43iwcjozgo |
| work_keys_str_mv | AT khefoonhewweijiaohuangsikaiwangxinyiluoanddonnemmanuelgonda towardsalargelanguagemodelbasedchatbotsystemtoautomaticallymonitorstudentgoalsettingandplanninginonlinelearning |