Unraveling temporally entangled multimodal interactions: investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledge
Abstract In various technology-enhanced learning (TEL) environments, knowledge co-creation progresses through multimodal interactions that integrate verbal and nonverbal modalities, such as speech and gestures. This study investigated two distinct analytical approaches for analyzing multimodal inter...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2025-02-01
|
| Series: | International Journal of Educational Technology in Higher Education |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s41239-025-00504-6 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850197933015695360 |
|---|---|
| author | Hanall Sung Mitchell J. Nathan |
| author_facet | Hanall Sung Mitchell J. Nathan |
| author_sort | Hanall Sung |
| collection | DOAJ |
| description | Abstract In various technology-enhanced learning (TEL) environments, knowledge co-creation progresses through multimodal interactions that integrate verbal and nonverbal modalities, such as speech and gestures. This study investigated two distinct analytical approaches for analyzing multimodal interactions—triangulating and interleaving—by applying them to collaborative learning processes during an online embodied mathematics intervention. The findings demonstrate that the interleaving approach captures the temporal dynamics and nuanced interplay between multimodal events, providing deeper insights into how shared meaning-making evolves over time. In contrast, the triangulating approach effectively identifies cumulative interaction patterns but does not account for their temporal structure. Specifically, the interleaving approach, employing epistemic network analysis, revealed statistically significant differences in discourse patterns between learners with larger and smaller variances in upper body movements during the co-design activity. These findings underscore the complementary value of the interleaving approach in analyzing multimodal interactions and offer practical implications for advancing understanding of collaborative learning processes in TEL environments. |
| format | Article |
| id | doaj-art-e770a2f114f9413ba7c5bdeb1cec133f |
| institution | OA Journals |
| issn | 2365-9440 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | International Journal of Educational Technology in Higher Education |
| spelling | doaj-art-e770a2f114f9413ba7c5bdeb1cec133f2025-08-20T02:12:59ZengSpringerOpenInternational Journal of Educational Technology in Higher Education2365-94402025-02-0122112810.1186/s41239-025-00504-6Unraveling temporally entangled multimodal interactions: investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledgeHanall Sung0Mitchell J. Nathan1Department of Theory & Practice in Teacher Education, University of TennesseeDepartment of Educational Psychology, University of Wisconsin-MadisonAbstract In various technology-enhanced learning (TEL) environments, knowledge co-creation progresses through multimodal interactions that integrate verbal and nonverbal modalities, such as speech and gestures. This study investigated two distinct analytical approaches for analyzing multimodal interactions—triangulating and interleaving—by applying them to collaborative learning processes during an online embodied mathematics intervention. The findings demonstrate that the interleaving approach captures the temporal dynamics and nuanced interplay between multimodal events, providing deeper insights into how shared meaning-making evolves over time. In contrast, the triangulating approach effectively identifies cumulative interaction patterns but does not account for their temporal structure. Specifically, the interleaving approach, employing epistemic network analysis, revealed statistically significant differences in discourse patterns between learners with larger and smaller variances in upper body movements during the co-design activity. These findings underscore the complementary value of the interleaving approach in analyzing multimodal interactions and offer practical implications for advancing understanding of collaborative learning processes in TEL environments.https://doi.org/10.1186/s41239-025-00504-6Multimodal learning analyticsTemporalLearning processBody movementGestureMotion sensing |
| spellingShingle | Hanall Sung Mitchell J. Nathan Unraveling temporally entangled multimodal interactions: investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledge International Journal of Educational Technology in Higher Education Multimodal learning analytics Temporal Learning process Body movement Gesture Motion sensing |
| title | Unraveling temporally entangled multimodal interactions: investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledge |
| title_full | Unraveling temporally entangled multimodal interactions: investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledge |
| title_fullStr | Unraveling temporally entangled multimodal interactions: investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledge |
| title_full_unstemmed | Unraveling temporally entangled multimodal interactions: investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledge |
| title_short | Unraveling temporally entangled multimodal interactions: investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledge |
| title_sort | unraveling temporally entangled multimodal interactions investigating verbal and nonverbal contributions to collaborative construction of embodied math knowledge |
| topic | Multimodal learning analytics Temporal Learning process Body movement Gesture Motion sensing |
| url | https://doi.org/10.1186/s41239-025-00504-6 |
| work_keys_str_mv | AT hanallsung unravelingtemporallyentangledmultimodalinteractionsinvestigatingverbalandnonverbalcontributionstocollaborativeconstructionofembodiedmathknowledge AT mitchelljnathan unravelingtemporallyentangledmultimodalinteractionsinvestigatingverbalandnonverbalcontributionstocollaborativeconstructionofembodiedmathknowledge |