The Influence of Artificial Intelligence Tools on Learning Outcomes in Computer Programming: A Systematic Review and Meta-Analysis
This systematic review and meta-analysis investigates the impact of artificial intelligence (AI) tools, including ChatGPT 3.5 and GitHub Copilot, on learning outcomes in computer programming courses. A total of 35 controlled studies published between 2020 and 2024 were analysed to assess the effecti...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-05-01
|
| Series: | Computers |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-431X/14/5/185 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849327398938476544 |
|---|---|
| author | Manal Alanazi Ben Soh Halima Samra Alice Li |
| author_facet | Manal Alanazi Ben Soh Halima Samra Alice Li |
| author_sort | Manal Alanazi |
| collection | DOAJ |
| description | This systematic review and meta-analysis investigates the impact of artificial intelligence (AI) tools, including ChatGPT 3.5 and GitHub Copilot, on learning outcomes in computer programming courses. A total of 35 controlled studies published between 2020 and 2024 were analysed to assess the effectiveness of AI-assisted learning. The results indicate that students using AI tools outperformed those without such aids. The meta-analysis findings revealed that AI-assisted learning significantly reduced task completion time (SMD = −0.69, 95% CI [−2.13, −0.74], I<sup>2</sup> = 95%, <i>p</i> = 0.34) and improved student performance scores (SMD = 0.86, 95% CI [0.36, 1.37], <i>p</i> = 0.0008, I<sup>2</sup> = 54%). However, AI tools did not provide a statistically significant advantage in learning success or ease of understanding (SMD = 0.16, 95% CI [−0.23, 0.55], <i>p</i> = 0.41, I<sup>2</sup> = 55%), with sensitivity analysis suggesting result variability. Student perceptions of AI tools were overwhelmingly positive, with a pooled estimate of 1.0 (95% CI [0.92, 1.00], I<sup>2</sup> = 0%). While AI tools enhance computer programming proficiency and efficiency, their effectiveness depends on factors such as tool functionality and course design. To maximise benefits and mitigate over-reliance, tailored pedagogical strategies are essential. This study underscores the transformative role of AI in computer programming education and provides evidence-based insights for optimising AI-assisted learning. |
| format | Article |
| id | doaj-art-8cb0538b53004f75910fadbc4e098fa1 |
| institution | Kabale University |
| issn | 2073-431X |
| language | English |
| publishDate | 2025-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Computers |
| spelling | doaj-art-8cb0538b53004f75910fadbc4e098fa12025-08-20T03:47:53ZengMDPI AGComputers2073-431X2025-05-0114518510.3390/computers14050185The Influence of Artificial Intelligence Tools on Learning Outcomes in Computer Programming: A Systematic Review and Meta-AnalysisManal Alanazi0Ben Soh1Halima Samra2Alice Li3Department of Computer Science and Information Technology, La Trobe University, Melbourne 3086, AustraliaDepartment of Computer Science and Information Technology, La Trobe University, Melbourne 3086, AustraliaComputer Science & Information Technology, King Abdulaziz University, Jeddah 21589, Saudi ArabiaLa Trobe Business School, La Trobe University, Melbourne 3086, AustraliaThis systematic review and meta-analysis investigates the impact of artificial intelligence (AI) tools, including ChatGPT 3.5 and GitHub Copilot, on learning outcomes in computer programming courses. A total of 35 controlled studies published between 2020 and 2024 were analysed to assess the effectiveness of AI-assisted learning. The results indicate that students using AI tools outperformed those without such aids. The meta-analysis findings revealed that AI-assisted learning significantly reduced task completion time (SMD = −0.69, 95% CI [−2.13, −0.74], I<sup>2</sup> = 95%, <i>p</i> = 0.34) and improved student performance scores (SMD = 0.86, 95% CI [0.36, 1.37], <i>p</i> = 0.0008, I<sup>2</sup> = 54%). However, AI tools did not provide a statistically significant advantage in learning success or ease of understanding (SMD = 0.16, 95% CI [−0.23, 0.55], <i>p</i> = 0.41, I<sup>2</sup> = 55%), with sensitivity analysis suggesting result variability. Student perceptions of AI tools were overwhelmingly positive, with a pooled estimate of 1.0 (95% CI [0.92, 1.00], I<sup>2</sup> = 0%). While AI tools enhance computer programming proficiency and efficiency, their effectiveness depends on factors such as tool functionality and course design. To maximise benefits and mitigate over-reliance, tailored pedagogical strategies are essential. This study underscores the transformative role of AI in computer programming education and provides evidence-based insights for optimising AI-assisted learning.https://www.mdpi.com/2073-431X/14/5/185artificial intelligence (AI)ChatGPTcomputer programming educationlearning outcomesGitHub Copilottask efficiency |
| spellingShingle | Manal Alanazi Ben Soh Halima Samra Alice Li The Influence of Artificial Intelligence Tools on Learning Outcomes in Computer Programming: A Systematic Review and Meta-Analysis Computers artificial intelligence (AI) ChatGPT computer programming education learning outcomes GitHub Copilot task efficiency |
| title | The Influence of Artificial Intelligence Tools on Learning Outcomes in Computer Programming: A Systematic Review and Meta-Analysis |
| title_full | The Influence of Artificial Intelligence Tools on Learning Outcomes in Computer Programming: A Systematic Review and Meta-Analysis |
| title_fullStr | The Influence of Artificial Intelligence Tools on Learning Outcomes in Computer Programming: A Systematic Review and Meta-Analysis |
| title_full_unstemmed | The Influence of Artificial Intelligence Tools on Learning Outcomes in Computer Programming: A Systematic Review and Meta-Analysis |
| title_short | The Influence of Artificial Intelligence Tools on Learning Outcomes in Computer Programming: A Systematic Review and Meta-Analysis |
| title_sort | influence of artificial intelligence tools on learning outcomes in computer programming a systematic review and meta analysis |
| topic | artificial intelligence (AI) ChatGPT computer programming education learning outcomes GitHub Copilot task efficiency |
| url | https://www.mdpi.com/2073-431X/14/5/185 |
| work_keys_str_mv | AT manalalanazi theinfluenceofartificialintelligencetoolsonlearningoutcomesincomputerprogrammingasystematicreviewandmetaanalysis AT bensoh theinfluenceofartificialintelligencetoolsonlearningoutcomesincomputerprogrammingasystematicreviewandmetaanalysis AT halimasamra theinfluenceofartificialintelligencetoolsonlearningoutcomesincomputerprogrammingasystematicreviewandmetaanalysis AT aliceli theinfluenceofartificialintelligencetoolsonlearningoutcomesincomputerprogrammingasystematicreviewandmetaanalysis AT manalalanazi influenceofartificialintelligencetoolsonlearningoutcomesincomputerprogrammingasystematicreviewandmetaanalysis AT bensoh influenceofartificialintelligencetoolsonlearningoutcomesincomputerprogrammingasystematicreviewandmetaanalysis AT halimasamra influenceofartificialintelligencetoolsonlearningoutcomesincomputerprogrammingasystematicreviewandmetaanalysis AT aliceli influenceofartificialintelligencetoolsonlearningoutcomesincomputerprogrammingasystematicreviewandmetaanalysis |