Evaluating the effects of Generative AI on student learning outcomes: Insights from a meta-analysis
The emergence of Generative AI technologies, represented by ChatGPT, has triggered extensive discussions among scholars in the education sector. While relevant research continues to grow, there is a lack of comprehensive understanding that systematically measures the effects of Generative AI on stud...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
International Forum of Educational Technology & Society
2025-07-01
|
| Series: | Educational Technology & Society |
| Subjects: | |
| Online Access: | https://www.j-ets.net/collection/published-issues/28_3#h.n2fu3tdas6sq |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849319918389952512 |
|---|---|
| author | De-Xin Hu, Dan-Dan Pang and Zhe Xing |
| author_facet | De-Xin Hu, Dan-Dan Pang and Zhe Xing |
| author_sort | De-Xin Hu, Dan-Dan Pang and Zhe Xing |
| collection | DOAJ |
| description | The emergence of Generative AI technologies, represented by ChatGPT, has triggered extensive discussions among scholars in the education sector. While relevant research continues to grow, there is a lack of comprehensive understanding that systematically measures the effects of Generative AI on student learning outcomes. This study employs meta-analysis to integrate findings from previous experimental and quasi-experimental research to evaluate the impact of Generative AI on student learning outcomes. The analysis of 44 effect sizes from 21 independent studies indicates that Generative AI tools, compared to traditional AI tools or no intervention, moderately enhance student learning outcomes (g = 0.572). These tools significantly improve the cognitive (g = 0.604), behavioral (g = 0.698), and affective (g = 0.478) dimensions of learning outcomes. In addition, the study identifies and examines 6 potential moderating variables: educational level, sample size, subject area, teaching model, intervention duration, and assessment instrument. The results of the moderating effects test reveal that sample size and assessment instrument significantly influence the effectiveness of Generative AI. For sample size, the effect of Generative AI on small samples (g = 1.216) is greater than that on medium (g = 0.476) and large samples (g = 0.547). For assessment instrument, the effect of Generative AI on self-developed tests (g = 0.984) is greater than that on standardized tests (g = 0.557). The meta-analysis result indicated that the use of Generative AI should be supplemented with detailed guidance and flexible strategies. Specific recommendations for future research and practical implementations of Generative AI in education are discussed. |
| format | Article |
| id | doaj-art-74e4241c1d154787bc45c4f3683f799e |
| institution | Kabale University |
| issn | 1176-3647 1436-4522 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | International Forum of Educational Technology & Society |
| record_format | Article |
| series | Educational Technology & Society |
| spelling | doaj-art-74e4241c1d154787bc45c4f3683f799e2025-08-20T03:50:16ZengInternational Forum of Educational Technology & SocietyEducational Technology & Society1176-36471436-45222025-07-01283226240https://doi.org/10.30191/ETS.202507_28(3).TP02Evaluating the effects of Generative AI on student learning outcomes: Insights from a meta-analysisDe-Xin Hu, Dan-Dan Pang and Zhe XingThe emergence of Generative AI technologies, represented by ChatGPT, has triggered extensive discussions among scholars in the education sector. While relevant research continues to grow, there is a lack of comprehensive understanding that systematically measures the effects of Generative AI on student learning outcomes. This study employs meta-analysis to integrate findings from previous experimental and quasi-experimental research to evaluate the impact of Generative AI on student learning outcomes. The analysis of 44 effect sizes from 21 independent studies indicates that Generative AI tools, compared to traditional AI tools or no intervention, moderately enhance student learning outcomes (g = 0.572). These tools significantly improve the cognitive (g = 0.604), behavioral (g = 0.698), and affective (g = 0.478) dimensions of learning outcomes. In addition, the study identifies and examines 6 potential moderating variables: educational level, sample size, subject area, teaching model, intervention duration, and assessment instrument. The results of the moderating effects test reveal that sample size and assessment instrument significantly influence the effectiveness of Generative AI. For sample size, the effect of Generative AI on small samples (g = 1.216) is greater than that on medium (g = 0.476) and large samples (g = 0.547). For assessment instrument, the effect of Generative AI on self-developed tests (g = 0.984) is greater than that on standardized tests (g = 0.557). The meta-analysis result indicated that the use of Generative AI should be supplemented with detailed guidance and flexible strategies. Specific recommendations for future research and practical implementations of Generative AI in education are discussed.https://www.j-ets.net/collection/published-issues/28_3#h.n2fu3tdas6sqeducationgenerative artificial intelligencegenerative ailearning outcomesmeta-analysis |
| spellingShingle | De-Xin Hu, Dan-Dan Pang and Zhe Xing Evaluating the effects of Generative AI on student learning outcomes: Insights from a meta-analysis Educational Technology & Society education generative artificial intelligence generative ai learning outcomes meta-analysis |
| title | Evaluating the effects of Generative AI on student learning outcomes: Insights from a meta-analysis |
| title_full | Evaluating the effects of Generative AI on student learning outcomes: Insights from a meta-analysis |
| title_fullStr | Evaluating the effects of Generative AI on student learning outcomes: Insights from a meta-analysis |
| title_full_unstemmed | Evaluating the effects of Generative AI on student learning outcomes: Insights from a meta-analysis |
| title_short | Evaluating the effects of Generative AI on student learning outcomes: Insights from a meta-analysis |
| title_sort | evaluating the effects of generative ai on student learning outcomes insights from a meta analysis |
| topic | education generative artificial intelligence generative ai learning outcomes meta-analysis |
| url | https://www.j-ets.net/collection/published-issues/28_3#h.n2fu3tdas6sq |
| work_keys_str_mv | AT dexinhudandanpangandzhexing evaluatingtheeffectsofgenerativeaionstudentlearningoutcomesinsightsfromametaanalysis |