ET-GNN: Ensemble Transformer-Based Graph Neural Networks for Holistic Automated Essay Scoring
Essay writing tasks are crucial for assessing students’ writing skills, but manual evaluation can be time-consuming and prone to inconsistencies. Automated Essay Scoring (AES) offers a solution by automatically evaluating essays, reducing the need for human intervention. This paper presen...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10945775/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849702076535275520 |
|---|---|
| author | Hind Aljuaid Areej Alhothali Ohoud Alzamzami Hussein Assalahi Tahani Aldosemani |
| author_facet | Hind Aljuaid Areej Alhothali Ohoud Alzamzami Hussein Assalahi Tahani Aldosemani |
| author_sort | Hind Aljuaid |
| collection | DOAJ |
| description | Essay writing tasks are crucial for assessing students’ writing skills, but manual evaluation can be time-consuming and prone to inconsistencies. Automated Essay Scoring (AES) offers a solution by automatically evaluating essays, reducing the need for human intervention. This paper presents a hybrid method, called Ensemble Transformer-Based Graph Neural Networks (ET-GNN), which integrates Transformer-based models with Graph Convolutional Networks (GCNs) for holistic AES. Three Transformer models, DistilBERT, RoBERTa, and DeBERTaV3, were individually fine-tuned to generate contextual embeddings for each essay. The GCNs process these embeddings, effectively capturing relevant semantic information and inter-essay similarities. Additionally, ensemble methods are used to combine the DistilBERT-GCN, RoBERTa-GCN, and DeBERTaV3-GCN models employing averaging for regression tasks, majority voting for classification tasks, and a weighted ensemble method for both types of tasks. The proposed ET-GNN method enhances the performance and robustness of AES systems, achieving Quadratic Weighted Kappa (QWK) scores of 0.835 and 0.841 on the ASAP and AES 2.0 datasets, respectively. These results outperform other state-of-the-art models based on Transformer or GCNs architectures for the AES task. |
| format | Article |
| id | doaj-art-d225be0b239341d08adf46e2a512c417 |
| institution | DOAJ |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-d225be0b239341d08adf46e2a512c4172025-08-20T03:17:46ZengIEEEIEEE Access2169-35362025-01-0113587465875810.1109/ACCESS.2025.355635210945775ET-GNN: Ensemble Transformer-Based Graph Neural Networks for Holistic Automated Essay ScoringHind Aljuaid0https://orcid.org/0009-0000-6451-1238Areej Alhothali1https://orcid.org/0000-0001-9727-0178Ohoud Alzamzami2Hussein Assalahi3Tahani Aldosemani4https://orcid.org/0000-0002-2347-1564Department of Computer Science, King Abdulaziz University, Jeddah, Saudi ArabiaDepartment of Computer Science, King Abdulaziz University, Jeddah, Saudi ArabiaDepartment of Computer Science, King Abdulaziz University, Jeddah, Saudi ArabiaEnglish Language Institute, King Abdulaziz University, Jeddah, Saudi ArabiaCollege of Education, Prince Sattam bin Abdulaziz University, Al-Kharj, Saudi ArabiaEssay writing tasks are crucial for assessing students’ writing skills, but manual evaluation can be time-consuming and prone to inconsistencies. Automated Essay Scoring (AES) offers a solution by automatically evaluating essays, reducing the need for human intervention. This paper presents a hybrid method, called Ensemble Transformer-Based Graph Neural Networks (ET-GNN), which integrates Transformer-based models with Graph Convolutional Networks (GCNs) for holistic AES. Three Transformer models, DistilBERT, RoBERTa, and DeBERTaV3, were individually fine-tuned to generate contextual embeddings for each essay. The GCNs process these embeddings, effectively capturing relevant semantic information and inter-essay similarities. Additionally, ensemble methods are used to combine the DistilBERT-GCN, RoBERTa-GCN, and DeBERTaV3-GCN models employing averaging for regression tasks, majority voting for classification tasks, and a weighted ensemble method for both types of tasks. The proposed ET-GNN method enhances the performance and robustness of AES systems, achieving Quadratic Weighted Kappa (QWK) scores of 0.835 and 0.841 on the ASAP and AES 2.0 datasets, respectively. These results outperform other state-of-the-art models based on Transformer or GCNs architectures for the AES task.https://ieeexplore.ieee.org/document/10945775/Automated essay scoring (AES)transformer modelsgraph convolutional networks (GCNs)ensemble methodsnatural language processing (NLP) |
| spellingShingle | Hind Aljuaid Areej Alhothali Ohoud Alzamzami Hussein Assalahi Tahani Aldosemani ET-GNN: Ensemble Transformer-Based Graph Neural Networks for Holistic Automated Essay Scoring IEEE Access Automated essay scoring (AES) transformer models graph convolutional networks (GCNs) ensemble methods natural language processing (NLP) |
| title | ET-GNN: Ensemble Transformer-Based Graph Neural Networks for Holistic Automated Essay Scoring |
| title_full | ET-GNN: Ensemble Transformer-Based Graph Neural Networks for Holistic Automated Essay Scoring |
| title_fullStr | ET-GNN: Ensemble Transformer-Based Graph Neural Networks for Holistic Automated Essay Scoring |
| title_full_unstemmed | ET-GNN: Ensemble Transformer-Based Graph Neural Networks for Holistic Automated Essay Scoring |
| title_short | ET-GNN: Ensemble Transformer-Based Graph Neural Networks for Holistic Automated Essay Scoring |
| title_sort | et gnn ensemble transformer based graph neural networks for holistic automated essay scoring |
| topic | Automated essay scoring (AES) transformer models graph convolutional networks (GCNs) ensemble methods natural language processing (NLP) |
| url | https://ieeexplore.ieee.org/document/10945775/ |
| work_keys_str_mv | AT hindaljuaid etgnnensembletransformerbasedgraphneuralnetworksforholisticautomatedessayscoring AT areejalhothali etgnnensembletransformerbasedgraphneuralnetworksforholisticautomatedessayscoring AT ohoudalzamzami etgnnensembletransformerbasedgraphneuralnetworksforholisticautomatedessayscoring AT husseinassalahi etgnnensembletransformerbasedgraphneuralnetworksforholisticautomatedessayscoring AT tahanialdosemani etgnnensembletransformerbasedgraphneuralnetworksforholisticautomatedessayscoring |