Syntactic And Lexical Comparison Between Ai-generated Reading Passages And Japanese Universities' National Test
This research aimed to develop a methodology for creating mock English reading tests using AI-generated passages to help decrease the workload of EFL teachers who struggle in creating the high-quality mock English reading test for universities’ entrance examinations. To achieve this goal, this pape...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
University of Bejaia Abderrahmane Mira
2025-05-01
|
| Series: | The Journal of Studies in Language, Culture and Society |
| Subjects: | |
| Online Access: | https://univ-bejaia.dz/revue/jslcs/article/view/590 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850072294644252672 |
|---|---|
| author | Satoshi Kurokawa Maelys Salingre |
| author_facet | Satoshi Kurokawa Maelys Salingre |
| author_sort | Satoshi Kurokawa |
| collection | DOAJ |
| description |
This research aimed to develop a methodology for creating mock English reading tests using AI-generated passages to help decrease the workload of EFL teachers who struggle in creating the high-quality mock English reading test for universities’ entrance examinations. To achieve this goal, this paper examined the lexical and syntactical differences between English Subject of the Center Test at Japanese universities’ entrance exams (ESCT) and AI-generated passages. Three different generative AI were used: OpenAI ChatGPT version 4, Google Gemini version 1.5 Flash, and DeepSeek-V3. To make the vocabulary coverage between AI-generated passages and ESCTs meaningful, topics based on ESCTs were used to create 11 prompts for AI-generated passages. This paper examined text coverage using CEFR-based wordlist and syntactic complexity using the Python library spaCy. Findings revealed that the proportion of A2 level tokens does not differ greatly between AI-generated passages (ChatGPT: 19.1%; Gemini: 17.4%; DeepSeek: 17.3%) and ESCT (15.6%); ESCT had more complex but shorter sentences comparing to AI-generated passages, and personal pronouns accounted for 1.3% of ESCT tokens, while they accounted for less than 1% of generative AIs tokens (ChatGPT, 0.39% ; Gemini, 0.71% ; DeepSeek 0.45%). Few wh-pronouns and the existential there were found in AI-generated passages. This study concluded that when the EFL teachers convert AI-generative text to the mock ESCT, they should (a) add a wider variety of A1 level lemmas, (b) rewrite more complex and shorter sentences, (c) increase personal pronouns and determiners, and (d) reduce adjective modifiers, conjuncts, and coordination.
|
| format | Article |
| id | doaj-art-fd5544f06d6d4615ab6184fbfd787225 |
| institution | DOAJ |
| issn | 2716-9189 2676-1750 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | University of Bejaia Abderrahmane Mira |
| record_format | Article |
| series | The Journal of Studies in Language, Culture and Society |
| spelling | doaj-art-fd5544f06d6d4615ab6184fbfd7872252025-08-20T02:47:06ZengUniversity of Bejaia Abderrahmane MiraThe Journal of Studies in Language, Culture and Society2716-91892676-17502025-05-0181Syntactic And Lexical Comparison Between Ai-generated Reading Passages And Japanese Universities' National TestSatoshi Kurokawa0Maelys Salingre1Nagoya UniversityShizuoka University This research aimed to develop a methodology for creating mock English reading tests using AI-generated passages to help decrease the workload of EFL teachers who struggle in creating the high-quality mock English reading test for universities’ entrance examinations. To achieve this goal, this paper examined the lexical and syntactical differences between English Subject of the Center Test at Japanese universities’ entrance exams (ESCT) and AI-generated passages. Three different generative AI were used: OpenAI ChatGPT version 4, Google Gemini version 1.5 Flash, and DeepSeek-V3. To make the vocabulary coverage between AI-generated passages and ESCTs meaningful, topics based on ESCTs were used to create 11 prompts for AI-generated passages. This paper examined text coverage using CEFR-based wordlist and syntactic complexity using the Python library spaCy. Findings revealed that the proportion of A2 level tokens does not differ greatly between AI-generated passages (ChatGPT: 19.1%; Gemini: 17.4%; DeepSeek: 17.3%) and ESCT (15.6%); ESCT had more complex but shorter sentences comparing to AI-generated passages, and personal pronouns accounted for 1.3% of ESCT tokens, while they accounted for less than 1% of generative AIs tokens (ChatGPT, 0.39% ; Gemini, 0.71% ; DeepSeek 0.45%). Few wh-pronouns and the existential there were found in AI-generated passages. This study concluded that when the EFL teachers convert AI-generative text to the mock ESCT, they should (a) add a wider variety of A1 level lemmas, (b) rewrite more complex and shorter sentences, (c) increase personal pronouns and determiners, and (d) reduce adjective modifiers, conjuncts, and coordination. https://univ-bejaia.dz/revue/jslcs/article/view/590Generative AINatural language processinText coverageSyntactic complexityUniversity entrance examination |
| spellingShingle | Satoshi Kurokawa Maelys Salingre Syntactic And Lexical Comparison Between Ai-generated Reading Passages And Japanese Universities' National Test The Journal of Studies in Language, Culture and Society Generative AI Natural language processin Text coverage Syntactic complexity University entrance examination |
| title | Syntactic And Lexical Comparison Between Ai-generated Reading Passages And Japanese Universities' National Test |
| title_full | Syntactic And Lexical Comparison Between Ai-generated Reading Passages And Japanese Universities' National Test |
| title_fullStr | Syntactic And Lexical Comparison Between Ai-generated Reading Passages And Japanese Universities' National Test |
| title_full_unstemmed | Syntactic And Lexical Comparison Between Ai-generated Reading Passages And Japanese Universities' National Test |
| title_short | Syntactic And Lexical Comparison Between Ai-generated Reading Passages And Japanese Universities' National Test |
| title_sort | syntactic and lexical comparison between ai generated reading passages and japanese universities national test |
| topic | Generative AI Natural language processin Text coverage Syntactic complexity University entrance examination |
| url | https://univ-bejaia.dz/revue/jslcs/article/view/590 |
| work_keys_str_mv | AT satoshikurokawa syntacticandlexicalcomparisonbetweenaigeneratedreadingpassagesandjapaneseuniversitiesnationaltest AT maelyssalingre syntacticandlexicalcomparisonbetweenaigeneratedreadingpassagesandjapaneseuniversitiesnationaltest |