How Well Do Simulated Population Samples with GPT-4 Align with Real Ones? The Case of the Eysenck Personality Questionnaire Revised-Abbreviated Personality Test
Background: Advances in artificial intelligence have enabled the simulation of human-like behaviors, raising the possibility of using large language models (LLMs) to generate synthetic population samples for research purposes, which may be particularly useful in health and social sciences. Methods:...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
American Association for the Advancement of Science (AAAS)
2025-01-01
|
| Series: | Health Data Science |
| Online Access: | https://spj.science.org/doi/10.34133/hds.0284 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849427473169645568 |
|---|---|
| author | Gregorio Ferreira Jacopo Amidei Rubén Nieto Andreas Kaltenbrunner |
| author_facet | Gregorio Ferreira Jacopo Amidei Rubén Nieto Andreas Kaltenbrunner |
| author_sort | Gregorio Ferreira |
| collection | DOAJ |
| description | Background: Advances in artificial intelligence have enabled the simulation of human-like behaviors, raising the possibility of using large language models (LLMs) to generate synthetic population samples for research purposes, which may be particularly useful in health and social sciences. Methods: This paper explores the potential of LLMs to simulate population samples mirroring real ones, as well as the feasibility of using personality questionnaires to assess the personality of LLMs. To advance in that direction, 2 experiments were conducted with GPT-4o using the Eysenck Personality Questionnaire Revised-Abbreviated (EPQR-A) in 6 languages: Spanish, English, Slovak, Hebrew, Portuguese, and Turkish. Results: We find that GPT-4o exhibits distinct personality traits, which vary based on parameter settings and the language of the questionnaire. While the model shows promising trends in reflecting certain personality traits and differences across gender and academic fields, discrepancies between the synthetic populations’ responses and those from real populations remain. Conclusions: These inconsistencies suggest that creating fully reliable synthetic population samples for questionnaire testing is still an open challenge. Further research is required to better align synthetic and real population behaviors. |
| format | Article |
| id | doaj-art-17b1471bdf344b6cbc7e4b43a1295140 |
| institution | Kabale University |
| issn | 2765-8783 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | American Association for the Advancement of Science (AAAS) |
| record_format | Article |
| series | Health Data Science |
| spelling | doaj-art-17b1471bdf344b6cbc7e4b43a12951402025-08-20T03:29:02ZengAmerican Association for the Advancement of Science (AAAS)Health Data Science2765-87832025-01-01510.34133/hds.0284How Well Do Simulated Population Samples with GPT-4 Align with Real Ones? The Case of the Eysenck Personality Questionnaire Revised-Abbreviated Personality TestGregorio Ferreira0Jacopo Amidei1Rubén Nieto2Andreas Kaltenbrunner3IN3, Universitat Oberta de Catalunya, Barcelona, Spain.IN3, Universitat Oberta de Catalunya, Barcelona, Spain.eHealth Research Lab, Universitat Oberta de Catalunya, Barcelona, Spain.IN3, Universitat Oberta de Catalunya, Barcelona, Spain.Background: Advances in artificial intelligence have enabled the simulation of human-like behaviors, raising the possibility of using large language models (LLMs) to generate synthetic population samples for research purposes, which may be particularly useful in health and social sciences. Methods: This paper explores the potential of LLMs to simulate population samples mirroring real ones, as well as the feasibility of using personality questionnaires to assess the personality of LLMs. To advance in that direction, 2 experiments were conducted with GPT-4o using the Eysenck Personality Questionnaire Revised-Abbreviated (EPQR-A) in 6 languages: Spanish, English, Slovak, Hebrew, Portuguese, and Turkish. Results: We find that GPT-4o exhibits distinct personality traits, which vary based on parameter settings and the language of the questionnaire. While the model shows promising trends in reflecting certain personality traits and differences across gender and academic fields, discrepancies between the synthetic populations’ responses and those from real populations remain. Conclusions: These inconsistencies suggest that creating fully reliable synthetic population samples for questionnaire testing is still an open challenge. Further research is required to better align synthetic and real population behaviors.https://spj.science.org/doi/10.34133/hds.0284 |
| spellingShingle | Gregorio Ferreira Jacopo Amidei Rubén Nieto Andreas Kaltenbrunner How Well Do Simulated Population Samples with GPT-4 Align with Real Ones? The Case of the Eysenck Personality Questionnaire Revised-Abbreviated Personality Test Health Data Science |
| title | How Well Do Simulated Population Samples with GPT-4 Align with Real Ones? The Case of the Eysenck Personality Questionnaire Revised-Abbreviated Personality Test |
| title_full | How Well Do Simulated Population Samples with GPT-4 Align with Real Ones? The Case of the Eysenck Personality Questionnaire Revised-Abbreviated Personality Test |
| title_fullStr | How Well Do Simulated Population Samples with GPT-4 Align with Real Ones? The Case of the Eysenck Personality Questionnaire Revised-Abbreviated Personality Test |
| title_full_unstemmed | How Well Do Simulated Population Samples with GPT-4 Align with Real Ones? The Case of the Eysenck Personality Questionnaire Revised-Abbreviated Personality Test |
| title_short | How Well Do Simulated Population Samples with GPT-4 Align with Real Ones? The Case of the Eysenck Personality Questionnaire Revised-Abbreviated Personality Test |
| title_sort | how well do simulated population samples with gpt 4 align with real ones the case of the eysenck personality questionnaire revised abbreviated personality test |
| url | https://spj.science.org/doi/10.34133/hds.0284 |
| work_keys_str_mv | AT gregorioferreira howwelldosimulatedpopulationsampleswithgpt4alignwithrealonesthecaseoftheeysenckpersonalityquestionnairerevisedabbreviatedpersonalitytest AT jacopoamidei howwelldosimulatedpopulationsampleswithgpt4alignwithrealonesthecaseoftheeysenckpersonalityquestionnairerevisedabbreviatedpersonalitytest AT rubennieto howwelldosimulatedpopulationsampleswithgpt4alignwithrealonesthecaseoftheeysenckpersonalityquestionnairerevisedabbreviatedpersonalitytest AT andreaskaltenbrunner howwelldosimulatedpopulationsampleswithgpt4alignwithrealonesthecaseoftheeysenckpersonalityquestionnairerevisedabbreviatedpersonalitytest |