Self-Created Film as a Resource in a Multimodal Conversational Narrative
When access to natural speech is limited or challenging, as is the case for people with complex communication needs, self-created digital film can be practical to use as one of the resources within a multimodal conversation about a personal experience. The detailed and contextual information such au...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Multimodal Technologies and Interaction |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2414-4088/9/3/25 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850090971606286336 |
|---|---|
| author | Mascha Legel Stijn R. J. M. Deckers Gloria Soto Nicola Grove Annalu Waller Hans van Balkom Ronald Spanjers Christopher S. Norrie Bert Steenbergen |
| author_facet | Mascha Legel Stijn R. J. M. Deckers Gloria Soto Nicola Grove Annalu Waller Hans van Balkom Ronald Spanjers Christopher S. Norrie Bert Steenbergen |
| author_sort | Mascha Legel |
| collection | DOAJ |
| description | When access to natural speech is limited or challenging, as is the case for people with complex communication needs, self-created digital film can be practical to use as one of the resources within a multimodal conversation about a personal experience. The detailed and contextual information such audiovisual media offers with today’s available technology may assist the utility of other communication modes, such as (computerized) spoken, written or signed language, to foster mutual understanding and story growth. To promote the use of self-created film, here named a personal-video-scene (PVS), in the practice of augmentative and alternative communication (AAC), a greater understanding is required of how such media can operate as a resource within social interactions, such as daily conversations. This study therefore introduces a multimodal coding model developed to study the employment of a PVS within a film-elicited conversational narrative, relating to four aspects of conversational control: (a) topic development, (b) conversational structure, (c) conversational repair and (d) conversational maintenance. A case study illustrates how the use of a PVS in story-sharing was instrumental in establishing common ground between narrators, boosting the frequency of comments and questions, mitigating instances of conversational repair and expanding topic development. |
| format | Article |
| id | doaj-art-f05bffe428274ffd810255b99bc0ed93 |
| institution | DOAJ |
| issn | 2414-4088 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Multimodal Technologies and Interaction |
| spelling | doaj-art-f05bffe428274ffd810255b99bc0ed932025-08-20T02:42:28ZengMDPI AGMultimodal Technologies and Interaction2414-40882025-03-01932510.3390/mti9030025Self-Created Film as a Resource in a Multimodal Conversational NarrativeMascha Legel0Stijn R. J. M. Deckers1Gloria Soto2Nicola Grove3Annalu Waller4Hans van Balkom5Ronald Spanjers6Christopher S. Norrie7Bert Steenbergen8Behavioural Science Institute, Radboud University, 6525 GD Nijmegen, The NetherlandsStichting Milo, 5482 JH Schijndel, The NetherlandsDepartment of Speech, Language and Hearing Sciences, San Francisco State University, San Francisco, CA 94132, USARix Centre, University of East London, London E16 2RD, UKAssistive Technology and AAC, University of Dundee, Dundee DD1 4HB, UKBehavioural Science Institute, Radboud University, 6525 GD Nijmegen, The NetherlandsRoessingh Rehabilitation, 7522 AH Enschede, The NetherlandsStichting OOK-OC, Research Center for AAC, 5482 JH Schijndel, The NetherlandsBehavioural Science Institute, Radboud University, 6525 GD Nijmegen, The NetherlandsWhen access to natural speech is limited or challenging, as is the case for people with complex communication needs, self-created digital film can be practical to use as one of the resources within a multimodal conversation about a personal experience. The detailed and contextual information such audiovisual media offers with today’s available technology may assist the utility of other communication modes, such as (computerized) spoken, written or signed language, to foster mutual understanding and story growth. To promote the use of self-created film, here named a personal-video-scene (PVS), in the practice of augmentative and alternative communication (AAC), a greater understanding is required of how such media can operate as a resource within social interactions, such as daily conversations. This study therefore introduces a multimodal coding model developed to study the employment of a PVS within a film-elicited conversational narrative, relating to four aspects of conversational control: (a) topic development, (b) conversational structure, (c) conversational repair and (d) conversational maintenance. A case study illustrates how the use of a PVS in story-sharing was instrumental in establishing common ground between narrators, boosting the frequency of comments and questions, mitigating instances of conversational repair and expanding topic development.https://www.mdpi.com/2414-4088/9/3/25self-created film mediamultimodal coding modelconversational narrativecomplex communication needsaugmentative and alternative communication |
| spellingShingle | Mascha Legel Stijn R. J. M. Deckers Gloria Soto Nicola Grove Annalu Waller Hans van Balkom Ronald Spanjers Christopher S. Norrie Bert Steenbergen Self-Created Film as a Resource in a Multimodal Conversational Narrative Multimodal Technologies and Interaction self-created film media multimodal coding model conversational narrative complex communication needs augmentative and alternative communication |
| title | Self-Created Film as a Resource in a Multimodal Conversational Narrative |
| title_full | Self-Created Film as a Resource in a Multimodal Conversational Narrative |
| title_fullStr | Self-Created Film as a Resource in a Multimodal Conversational Narrative |
| title_full_unstemmed | Self-Created Film as a Resource in a Multimodal Conversational Narrative |
| title_short | Self-Created Film as a Resource in a Multimodal Conversational Narrative |
| title_sort | self created film as a resource in a multimodal conversational narrative |
| topic | self-created film media multimodal coding model conversational narrative complex communication needs augmentative and alternative communication |
| url | https://www.mdpi.com/2414-4088/9/3/25 |
| work_keys_str_mv | AT maschalegel selfcreatedfilmasaresourceinamultimodalconversationalnarrative AT stijnrjmdeckers selfcreatedfilmasaresourceinamultimodalconversationalnarrative AT gloriasoto selfcreatedfilmasaresourceinamultimodalconversationalnarrative AT nicolagrove selfcreatedfilmasaresourceinamultimodalconversationalnarrative AT annaluwaller selfcreatedfilmasaresourceinamultimodalconversationalnarrative AT hansvanbalkom selfcreatedfilmasaresourceinamultimodalconversationalnarrative AT ronaldspanjers selfcreatedfilmasaresourceinamultimodalconversationalnarrative AT christophersnorrie selfcreatedfilmasaresourceinamultimodalconversationalnarrative AT bertsteenbergen selfcreatedfilmasaresourceinamultimodalconversationalnarrative |