Multimodal Communication in Virtual and Face-to-Face Settings: Gesture Production and Speech Disfluency
The COVID-19 pandemic has made online data collection a popular choice. It is important to evaluate how comparable online studies are to face-to-face studies, particularly in multimodal language research where modes of communication significantly impact the results. In this study, we examined indivi...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Istanbul University Press
2024-12-01
|
| Series: | Psikoloji Çalışmaları |
| Subjects: | |
| Online Access: | https://cdn.istanbul.edu.tr/file/JTA6CLJ8T5/64BBA2F80DB34CFDADEDB5843EBA68E0 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850187854791049216 |
|---|---|
| author | Burcu Arslan Can Avcı Demet Özer |
| author_facet | Burcu Arslan Can Avcı Demet Özer |
| author_sort | Burcu Arslan |
| collection | DOAJ |
| description | The COVID-19 pandemic has made online data collection a popular choice. It is important to evaluate how comparable online studies are to face-to-face studies, particularly in multimodal language research where modes of communication significantly impact the results. In this study, we examined individuals’ rates and patterns of speech disfluency and gesture use across face-to-face and online videoconferencing settings as they described their daily routines (N = 64). We asked whether and how multimodal language is affected across different communication settings and gesture use, particularly iconic gestures, is associated with speech fluency regardless of the context. Our results have showed that the participants’ overall disfluency rate was higher for the speech communicated via videoconferencing than the speech communicated face-to-face. However, the type of disfluencies changed across contexts, such that filled pauses and repairs were more common in online communication, whereas silent pauses were more common in face-to-face communication. These f indings signal an interplay between the cognitive functions of different disfluency types and communicative strategies. Results indicate that the overall gesture frequency and iconic gesture use were similar in both settings. Furthermore, the use of iconic gestures was found to negatively predict the overall disfluency rate, regardless of the setting. This finding suggests that using iconic gestures might facilitate cognitive processes, paving the way for a more fluent speech. This study demonstrates that multimodal language and communication strategies may vary across different communication settings and nuanced understanding of the differences in multimodal language between online and face-to-face communication can be gained using different contexts. The findings contribute to understanding the impact of increasingly widespread online communication on multimodal language production processes and provide foundation for future research. |
| format | Article |
| id | doaj-art-1817a5ce66ca4244ad0b40a1142ac3fa |
| institution | OA Journals |
| issn | 2602-2982 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Istanbul University Press |
| record_format | Article |
| series | Psikoloji Çalışmaları |
| spelling | doaj-art-1817a5ce66ca4244ad0b40a1142ac3fa2025-08-20T02:16:01ZengIstanbul University PressPsikoloji Çalışmaları2602-29822024-12-0144334936310.26650/SP2024-1443983123456Multimodal Communication in Virtual and Face-to-Face Settings: Gesture Production and Speech DisfluencyBurcu Arslan0https://orcid.org/0000-0002-2465-360XCan Avcı1https://orcid.org/0009-0008-9204-3150Demet Özer2https://orcid.org/0000-0003-3230-2874Koç Üniversitesi, Istanbul, TurkiyeKoç Üniversitesi, Istanbul, TurkiyeKadir Has Üniversitesi, Istanbul, TurkiyeThe COVID-19 pandemic has made online data collection a popular choice. It is important to evaluate how comparable online studies are to face-to-face studies, particularly in multimodal language research where modes of communication significantly impact the results. In this study, we examined individuals’ rates and patterns of speech disfluency and gesture use across face-to-face and online videoconferencing settings as they described their daily routines (N = 64). We asked whether and how multimodal language is affected across different communication settings and gesture use, particularly iconic gestures, is associated with speech fluency regardless of the context. Our results have showed that the participants’ overall disfluency rate was higher for the speech communicated via videoconferencing than the speech communicated face-to-face. However, the type of disfluencies changed across contexts, such that filled pauses and repairs were more common in online communication, whereas silent pauses were more common in face-to-face communication. These f indings signal an interplay between the cognitive functions of different disfluency types and communicative strategies. Results indicate that the overall gesture frequency and iconic gesture use were similar in both settings. Furthermore, the use of iconic gestures was found to negatively predict the overall disfluency rate, regardless of the setting. This finding suggests that using iconic gestures might facilitate cognitive processes, paving the way for a more fluent speech. This study demonstrates that multimodal language and communication strategies may vary across different communication settings and nuanced understanding of the differences in multimodal language between online and face-to-face communication can be gained using different contexts. The findings contribute to understanding the impact of increasingly widespread online communication on multimodal language production processes and provide foundation for future research.https://cdn.istanbul.edu.tr/file/JTA6CLJ8T5/64BBA2F80DB34CFDADEDB5843EBA68E0gesture productionspeech disfluencyvirtual communicationface-to-face communication |
| spellingShingle | Burcu Arslan Can Avcı Demet Özer Multimodal Communication in Virtual and Face-to-Face Settings: Gesture Production and Speech Disfluency Psikoloji Çalışmaları gesture production speech disfluency virtual communication face-to-face communication |
| title | Multimodal Communication in Virtual and Face-to-Face Settings: Gesture Production and Speech Disfluency |
| title_full | Multimodal Communication in Virtual and Face-to-Face Settings: Gesture Production and Speech Disfluency |
| title_fullStr | Multimodal Communication in Virtual and Face-to-Face Settings: Gesture Production and Speech Disfluency |
| title_full_unstemmed | Multimodal Communication in Virtual and Face-to-Face Settings: Gesture Production and Speech Disfluency |
| title_short | Multimodal Communication in Virtual and Face-to-Face Settings: Gesture Production and Speech Disfluency |
| title_sort | multimodal communication in virtual and face to face settings gesture production and speech disfluency |
| topic | gesture production speech disfluency virtual communication face-to-face communication |
| url | https://cdn.istanbul.edu.tr/file/JTA6CLJ8T5/64BBA2F80DB34CFDADEDB5843EBA68E0 |
| work_keys_str_mv | AT burcuarslan multimodalcommunicationinvirtualandfacetofacesettingsgestureproductionandspeechdisfluency AT canavcı multimodalcommunicationinvirtualandfacetofacesettingsgestureproductionandspeechdisfluency AT demetozer multimodalcommunicationinvirtualandfacetofacesettingsgestureproductionandspeechdisfluency |