Collectivism and individualism political bias in large language models: A two-step approach
In this paper, we investigate the political biases of large language models concerning collectivism and individualism through a combined analysis of their value judgments and factual assessments. We propose a two-step approach to evaluate the patterns of bias in the outputs of large language models...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2025-06-01
|
| Series: | Big Data & Society |
| Online Access: | https://doi.org/10.1177/20539517251343861 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850140011984322560 |
|---|---|
| author | Xiaobo Shan Yan Teng Yixu Wang Haiquan Zhao Yingchun Wang |
| author_facet | Xiaobo Shan Yan Teng Yixu Wang Haiquan Zhao Yingchun Wang |
| author_sort | Xiaobo Shan |
| collection | DOAJ |
| description | In this paper, we investigate the political biases of large language models concerning collectivism and individualism through a combined analysis of their value judgments and factual assessments. We propose a two-step approach to evaluate the patterns of bias in the outputs of large language models as well as a specific set of questions to examine large language model outputs’ political bias on collectivism and individualism. Our methodology involves two main phases. (a) Value assessment: we initiate the first phase by prompting large language models with questions from our set to identify patterns of political bias in their generated content. (b) Factual assessment: we refine the questions in our set and conduct a second round of prompting to verify the accuracy of the models’ responses regarding “ collectivism ” and “ individualism .” This step aims to assess whether the models can accurately discern these concepts in a factual context. Our experiments reveal varying degrees of political bias in the outputs of different large language models. While some models demonstrate proficiency in distinguishing between collectivism and individualism, they display outputs that are not neutral on political matters. Conversely, other models face challenges in accurately differentiating between these concepts and generating unbiased content. The latter indicates that many large language models not only fail to accurately distinguish between collectivism and individualism but also exhibit significant political biases in their outputs. We argue that a reliable large language model should achieve accuracy in factual assessments while generating unbiased content in value judgments, thus avoiding guiding users’ opinions. |
| format | Article |
| id | doaj-art-7422fe067b1b4d58b71fd1e9f2c4eb50 |
| institution | OA Journals |
| issn | 2053-9517 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | Big Data & Society |
| spelling | doaj-art-7422fe067b1b4d58b71fd1e9f2c4eb502025-08-20T02:30:00ZengSAGE PublishingBig Data & Society2053-95172025-06-011210.1177/20539517251343861Collectivism and individualism political bias in large language models: A two-step approachXiaobo Shan0Yan Teng1Yixu Wang2Haiquan Zhao3Yingchun Wang4 KoGuan School of Law, , Shanghai, China , Shanghai, China School of CS, , Shanghai, China School of CS, , Shanghai, China , Shanghai, ChinaIn this paper, we investigate the political biases of large language models concerning collectivism and individualism through a combined analysis of their value judgments and factual assessments. We propose a two-step approach to evaluate the patterns of bias in the outputs of large language models as well as a specific set of questions to examine large language model outputs’ political bias on collectivism and individualism. Our methodology involves two main phases. (a) Value assessment: we initiate the first phase by prompting large language models with questions from our set to identify patterns of political bias in their generated content. (b) Factual assessment: we refine the questions in our set and conduct a second round of prompting to verify the accuracy of the models’ responses regarding “ collectivism ” and “ individualism .” This step aims to assess whether the models can accurately discern these concepts in a factual context. Our experiments reveal varying degrees of political bias in the outputs of different large language models. While some models demonstrate proficiency in distinguishing between collectivism and individualism, they display outputs that are not neutral on political matters. Conversely, other models face challenges in accurately differentiating between these concepts and generating unbiased content. The latter indicates that many large language models not only fail to accurately distinguish between collectivism and individualism but also exhibit significant political biases in their outputs. We argue that a reliable large language model should achieve accuracy in factual assessments while generating unbiased content in value judgments, thus avoiding guiding users’ opinions.https://doi.org/10.1177/20539517251343861 |
| spellingShingle | Xiaobo Shan Yan Teng Yixu Wang Haiquan Zhao Yingchun Wang Collectivism and individualism political bias in large language models: A two-step approach Big Data & Society |
| title | Collectivism and individualism political bias in large language models: A two-step approach |
| title_full | Collectivism and individualism political bias in large language models: A two-step approach |
| title_fullStr | Collectivism and individualism political bias in large language models: A two-step approach |
| title_full_unstemmed | Collectivism and individualism political bias in large language models: A two-step approach |
| title_short | Collectivism and individualism political bias in large language models: A two-step approach |
| title_sort | collectivism and individualism political bias in large language models a two step approach |
| url | https://doi.org/10.1177/20539517251343861 |
| work_keys_str_mv | AT xiaoboshan collectivismandindividualismpoliticalbiasinlargelanguagemodelsatwostepapproach AT yanteng collectivismandindividualismpoliticalbiasinlargelanguagemodelsatwostepapproach AT yixuwang collectivismandindividualismpoliticalbiasinlargelanguagemodelsatwostepapproach AT haiquanzhao collectivismandindividualismpoliticalbiasinlargelanguagemodelsatwostepapproach AT yingchunwang collectivismandindividualismpoliticalbiasinlargelanguagemodelsatwostepapproach |