Automated grading using natural language processing and semantic analysis
Educational institutions' grading systems have evolved significantly toward automation, propelled by advances in Natural Language Processing (NLP) and Artificial Intelligence (AI). This research comprehensively explores automated grading systems, analyzing traditional methods alongside contempo...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-06-01
|
| Series: | MethodsX |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2215016125002419 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849471770295271424 |
|---|---|
| author | Ahmad Ayaan Kok-Why Ng |
| author_facet | Ahmad Ayaan Kok-Why Ng |
| author_sort | Ahmad Ayaan |
| collection | DOAJ |
| description | Educational institutions' grading systems have evolved significantly toward automation, propelled by advances in Natural Language Processing (NLP) and Artificial Intelligence (AI). This research comprehensively explores automated grading systems, analyzing traditional methods alongside contemporary approaches and research on exam grading. Through extensive literature review, we examine the advantages and disadvantages of keyword-centric techniques, NLP-based systems, and hybrid models. We implement a novel NLP-based automatic grading system that combines multiple similarity measures with semantic analysis using TensorFlow's Universal Sentence Encoder. The system evaluates student responses by comparing them to reference answers using a weighted combination of edit similarity, cosine similarity, Jaccard similarity, normalized word count, and semantic similarity. Experiments conducted on 14 student responses demonstrate the system's ability to provide consistent and accurate grading while identifying cases requiring further human review. This study contributes to understanding automated grading systems, offering insights into their efficacy, limitations, and prospects in educational assessment practices. • Hybrid evaluation: The proposed methodology combines traditional NLP techniques with advanced semantic analysis to provide comprehensive evaluation of student answers; the system integrates both surface-level textual similarity and deep semantic analysis to evaluate open-ended student responses. • Weighted scoring: The system computes a weighted base score by combining four NLP metrics (Jaccard, edit distance, cosine similarity, normalized word count) and then blends this with a semantic similarity score from the Universal Sentence Encoder to assign marks. • Rule-based final scoring: The final scoring layer applies threshold logic to assign zero, partial, or full marks based on semantic score and word count, and flags responses that fall into ambiguous ranges for teacher review. |
| format | Article |
| id | doaj-art-ae7c059cab4e48a09b820a0c81dd0faf |
| institution | Kabale University |
| issn | 2215-0161 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Elsevier |
| record_format | Article |
| series | MethodsX |
| spelling | doaj-art-ae7c059cab4e48a09b820a0c81dd0faf2025-08-20T03:24:43ZengElsevierMethodsX2215-01612025-06-011410339510.1016/j.mex.2025.103395Automated grading using natural language processing and semantic analysisAhmad Ayaan0Kok-Why Ng1Faculty of Computing and Informatics, Multimedia University, Cyberjaya, 63100, Selangor, MalaysiaCorresponding author.; Faculty of Computing and Informatics, Multimedia University, Cyberjaya, 63100, Selangor, MalaysiaEducational institutions' grading systems have evolved significantly toward automation, propelled by advances in Natural Language Processing (NLP) and Artificial Intelligence (AI). This research comprehensively explores automated grading systems, analyzing traditional methods alongside contemporary approaches and research on exam grading. Through extensive literature review, we examine the advantages and disadvantages of keyword-centric techniques, NLP-based systems, and hybrid models. We implement a novel NLP-based automatic grading system that combines multiple similarity measures with semantic analysis using TensorFlow's Universal Sentence Encoder. The system evaluates student responses by comparing them to reference answers using a weighted combination of edit similarity, cosine similarity, Jaccard similarity, normalized word count, and semantic similarity. Experiments conducted on 14 student responses demonstrate the system's ability to provide consistent and accurate grading while identifying cases requiring further human review. This study contributes to understanding automated grading systems, offering insights into their efficacy, limitations, and prospects in educational assessment practices. • Hybrid evaluation: The proposed methodology combines traditional NLP techniques with advanced semantic analysis to provide comprehensive evaluation of student answers; the system integrates both surface-level textual similarity and deep semantic analysis to evaluate open-ended student responses. • Weighted scoring: The system computes a weighted base score by combining four NLP metrics (Jaccard, edit distance, cosine similarity, normalized word count) and then blends this with a semantic similarity score from the Universal Sentence Encoder to assign marks. • Rule-based final scoring: The final scoring layer applies threshold logic to assign zero, partial, or full marks based on semantic score and word count, and flags responses that fall into ambiguous ranges for teacher review.http://www.sciencedirect.com/science/article/pii/S2215016125002419Automated Grading using Natural Language Processing and Semantic Analysis |
| spellingShingle | Ahmad Ayaan Kok-Why Ng Automated grading using natural language processing and semantic analysis MethodsX Automated Grading using Natural Language Processing and Semantic Analysis |
| title | Automated grading using natural language processing and semantic analysis |
| title_full | Automated grading using natural language processing and semantic analysis |
| title_fullStr | Automated grading using natural language processing and semantic analysis |
| title_full_unstemmed | Automated grading using natural language processing and semantic analysis |
| title_short | Automated grading using natural language processing and semantic analysis |
| title_sort | automated grading using natural language processing and semantic analysis |
| topic | Automated Grading using Natural Language Processing and Semantic Analysis |
| url | http://www.sciencedirect.com/science/article/pii/S2215016125002419 |
| work_keys_str_mv | AT ahmadayaan automatedgradingusingnaturallanguageprocessingandsemanticanalysis AT kokwhyng automatedgradingusingnaturallanguageprocessingandsemanticanalysis |