Does word knowledge account for the effect of world knowledge on pronoun interpretation?
To what extent can statistical language knowledge account for the effects of world knowledge in language comprehension? We address this question by focusing on a core aspect of language understanding: pronoun resolution. While existing studies suggest that comprehenders use world knowledge to resolv...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Cambridge University Press
2024-12-01
|
| Series: | Language and Cognition |
| Subjects: | |
| Online Access: | https://www.cambridge.org/core/product/identifier/S1866980824000024/type/journal_article |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850174606658240512 |
|---|---|
| author | Cameron R. Jones Benjamin Bergen |
| author_facet | Cameron R. Jones Benjamin Bergen |
| author_sort | Cameron R. Jones |
| collection | DOAJ |
| description | To what extent can statistical language knowledge account for the effects of world knowledge in language comprehension? We address this question by focusing on a core aspect of language understanding: pronoun resolution. While existing studies suggest that comprehenders use world knowledge to resolve pronouns, the distributional hypothesis and its operationalization in large language models (LLMs) provide an alternative account of how purely linguistic information could drive apparent world knowledge effects. We addressed these confounds in two experiments. In Experiment 1, we found a strong effect of world knowledge plausibility (measured using a norming study) on responses to comprehension questions that probed pronoun interpretation. In experiment 2, participants were slower to read continuations that contradicted world knowledge-consistent interpretations of a pronoun, implying that comprehenders deploy world knowledge spontaneously. Both effects persisted when controlling for the predictions of GPT-3, an LLM, suggesting that pronoun interpretation is at least partly driven by knowledge about the world and not the word. We propose two potential mechanisms by which knowledge-driven pronoun resolution occurs, based on validation- and expectation-driven discourse processes. The results suggest that while distributional information may capture some aspects of world knowledge, human comprehenders likely draw on other sources unavailable to LLMs. |
| format | Article |
| id | doaj-art-df32a40045f548c8aa98898015120836 |
| institution | OA Journals |
| issn | 1866-9808 1866-9859 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Cambridge University Press |
| record_format | Article |
| series | Language and Cognition |
| spelling | doaj-art-df32a40045f548c8aa988980151208362025-08-20T02:19:37ZengCambridge University PressLanguage and Cognition1866-98081866-98592024-12-01161182121310.1017/langcog.2024.2Does word knowledge account for the effect of world knowledge on pronoun interpretation?Cameron R. Jones0https://orcid.org/0000-0002-6609-8966Benjamin Bergen1Department of Cognitive Science, UC San Diego, San Diego, CA, USADepartment of Cognitive Science, UC San Diego, San Diego, CA, USATo what extent can statistical language knowledge account for the effects of world knowledge in language comprehension? We address this question by focusing on a core aspect of language understanding: pronoun resolution. While existing studies suggest that comprehenders use world knowledge to resolve pronouns, the distributional hypothesis and its operationalization in large language models (LLMs) provide an alternative account of how purely linguistic information could drive apparent world knowledge effects. We addressed these confounds in two experiments. In Experiment 1, we found a strong effect of world knowledge plausibility (measured using a norming study) on responses to comprehension questions that probed pronoun interpretation. In experiment 2, participants were slower to read continuations that contradicted world knowledge-consistent interpretations of a pronoun, implying that comprehenders deploy world knowledge spontaneously. Both effects persisted when controlling for the predictions of GPT-3, an LLM, suggesting that pronoun interpretation is at least partly driven by knowledge about the world and not the word. We propose two potential mechanisms by which knowledge-driven pronoun resolution occurs, based on validation- and expectation-driven discourse processes. The results suggest that while distributional information may capture some aspects of world knowledge, human comprehenders likely draw on other sources unavailable to LLMs.https://www.cambridge.org/core/product/identifier/S1866980824000024/type/journal_articledistributional baselinedistributional hypothesislarge language modelspronoun resolutionworld knowledge |
| spellingShingle | Cameron R. Jones Benjamin Bergen Does word knowledge account for the effect of world knowledge on pronoun interpretation? Language and Cognition distributional baseline distributional hypothesis large language models pronoun resolution world knowledge |
| title | Does word knowledge account for the effect of world knowledge on pronoun interpretation? |
| title_full | Does word knowledge account for the effect of world knowledge on pronoun interpretation? |
| title_fullStr | Does word knowledge account for the effect of world knowledge on pronoun interpretation? |
| title_full_unstemmed | Does word knowledge account for the effect of world knowledge on pronoun interpretation? |
| title_short | Does word knowledge account for the effect of world knowledge on pronoun interpretation? |
| title_sort | does word knowledge account for the effect of world knowledge on pronoun interpretation |
| topic | distributional baseline distributional hypothesis large language models pronoun resolution world knowledge |
| url | https://www.cambridge.org/core/product/identifier/S1866980824000024/type/journal_article |
| work_keys_str_mv | AT cameronrjones doeswordknowledgeaccountfortheeffectofworldknowledgeonpronouninterpretation AT benjaminbergen doeswordknowledgeaccountfortheeffectofworldknowledgeonpronouninterpretation |