Validating the ChatGPT Usage Scale: psychometric properties and factor structures among postgraduate students
Abstract Background The rapid adoption of ChatGPT in academic settings has raised concerns about its impact on learning, research, and academic integrity. This study aimed to develop and validate a comprehensive ChatGPT Usage Scale specifically tailored to postgraduate students, addressing the need...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2024-09-01
|
| Series: | BMC Psychology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40359-024-01983-4 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850136700946219008 |
|---|---|
| author | Mohamed Nemt-allah Waleed Khalifa Mahmoud Badawy Yasser Elbably Ashraf Ibrahim |
| author_facet | Mohamed Nemt-allah Waleed Khalifa Mahmoud Badawy Yasser Elbably Ashraf Ibrahim |
| author_sort | Mohamed Nemt-allah |
| collection | DOAJ |
| description | Abstract Background The rapid adoption of ChatGPT in academic settings has raised concerns about its impact on learning, research, and academic integrity. This study aimed to develop and validate a comprehensive ChatGPT Usage Scale specifically tailored to postgraduate students, addressing the need for a psychometrically sound instrument to assess the multidimensional nature of ChatGPT usage in higher education. Methods A cross-sectional survey design was employed, involving 443 postgraduate students from two Egyptian universities. The initial 39-item scale underwent Exploratory Factor Analysis (EFA) using principal component analysis with Varimax rotation. Confirmatory Factor Analysis (CFA) was conducted to assess the model fit and psychometric properties of the final 15-item measure. Internal consistency reliability was evaluated using Cronbach’s alpha and McDonald’s omega. Results EFA revealed a three-factor structure explaining 49.186% of the total variance: Academic Writing Aid (20.438%), Academic Task Support (14.410%), and Reliance and Trust (14.338%). CFA confirmed the three-factor structure with acceptable fit indices (χ2(87) = 223.604, p < .001; CMIN/DF = 2.570; CFI = 0.917; TLI = 0.900; RMSEA = 0.060). All standardized factor loadings were statistically significant (p < .001), ranging from 0.434 to 0.728. The scale demonstrated good internal consistency (Cronbach’s α = 0.848, McDonald’s ω = 0.849) and composite reliability (CR = 0.855). The average variance extracted (AVE) was 0.664, supporting convergent validity. Conclusions The validated ChatGPT Usage Scale provides a reliable and valid instrument for assessing postgraduate students’ engagement with ChatGPT across multiple dimensions. This tool offers valuable insights into AI-assisted academic practices, enabling more nuanced investigations into the effects of ChatGPT on postgraduate education. |
| format | Article |
| id | doaj-art-0f8d9a7d87c543d392f74bfce6819954 |
| institution | OA Journals |
| issn | 2050-7283 |
| language | English |
| publishDate | 2024-09-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Psychology |
| spelling | doaj-art-0f8d9a7d87c543d392f74bfce68199542025-08-20T02:31:03ZengBMCBMC Psychology2050-72832024-09-011211710.1186/s40359-024-01983-4Validating the ChatGPT Usage Scale: psychometric properties and factor structures among postgraduate studentsMohamed Nemt-allah0Waleed Khalifa1Mahmoud Badawy2Yasser Elbably3Ashraf Ibrahim4Faculty of Education, Al-Azhar UniversityFaculty of Education, Al-Azhar UniversityFaculty of Education, Al-Azhar UniversityFaculty of Education, Al-Azhar UniversityFaculty of Education, Al-Azhar UniversityAbstract Background The rapid adoption of ChatGPT in academic settings has raised concerns about its impact on learning, research, and academic integrity. This study aimed to develop and validate a comprehensive ChatGPT Usage Scale specifically tailored to postgraduate students, addressing the need for a psychometrically sound instrument to assess the multidimensional nature of ChatGPT usage in higher education. Methods A cross-sectional survey design was employed, involving 443 postgraduate students from two Egyptian universities. The initial 39-item scale underwent Exploratory Factor Analysis (EFA) using principal component analysis with Varimax rotation. Confirmatory Factor Analysis (CFA) was conducted to assess the model fit and psychometric properties of the final 15-item measure. Internal consistency reliability was evaluated using Cronbach’s alpha and McDonald’s omega. Results EFA revealed a three-factor structure explaining 49.186% of the total variance: Academic Writing Aid (20.438%), Academic Task Support (14.410%), and Reliance and Trust (14.338%). CFA confirmed the three-factor structure with acceptable fit indices (χ2(87) = 223.604, p < .001; CMIN/DF = 2.570; CFI = 0.917; TLI = 0.900; RMSEA = 0.060). All standardized factor loadings were statistically significant (p < .001), ranging from 0.434 to 0.728. The scale demonstrated good internal consistency (Cronbach’s α = 0.848, McDonald’s ω = 0.849) and composite reliability (CR = 0.855). The average variance extracted (AVE) was 0.664, supporting convergent validity. Conclusions The validated ChatGPT Usage Scale provides a reliable and valid instrument for assessing postgraduate students’ engagement with ChatGPT across multiple dimensions. This tool offers valuable insights into AI-assisted academic practices, enabling more nuanced investigations into the effects of ChatGPT on postgraduate education.https://doi.org/10.1186/s40359-024-01983-4ChatGPTArtificial intelligencePostgraduate educationAcademic writingPsychometricsFactor analysis |
| spellingShingle | Mohamed Nemt-allah Waleed Khalifa Mahmoud Badawy Yasser Elbably Ashraf Ibrahim Validating the ChatGPT Usage Scale: psychometric properties and factor structures among postgraduate students BMC Psychology ChatGPT Artificial intelligence Postgraduate education Academic writing Psychometrics Factor analysis |
| title | Validating the ChatGPT Usage Scale: psychometric properties and factor structures among postgraduate students |
| title_full | Validating the ChatGPT Usage Scale: psychometric properties and factor structures among postgraduate students |
| title_fullStr | Validating the ChatGPT Usage Scale: psychometric properties and factor structures among postgraduate students |
| title_full_unstemmed | Validating the ChatGPT Usage Scale: psychometric properties and factor structures among postgraduate students |
| title_short | Validating the ChatGPT Usage Scale: psychometric properties and factor structures among postgraduate students |
| title_sort | validating the chatgpt usage scale psychometric properties and factor structures among postgraduate students |
| topic | ChatGPT Artificial intelligence Postgraduate education Academic writing Psychometrics Factor analysis |
| url | https://doi.org/10.1186/s40359-024-01983-4 |
| work_keys_str_mv | AT mohamednemtallah validatingthechatgptusagescalepsychometricpropertiesandfactorstructuresamongpostgraduatestudents AT waleedkhalifa validatingthechatgptusagescalepsychometricpropertiesandfactorstructuresamongpostgraduatestudents AT mahmoudbadawy validatingthechatgptusagescalepsychometricpropertiesandfactorstructuresamongpostgraduatestudents AT yasserelbably validatingthechatgptusagescalepsychometricpropertiesandfactorstructuresamongpostgraduatestudents AT ashrafibrahim validatingthechatgptusagescalepsychometricpropertiesandfactorstructuresamongpostgraduatestudents |