Keypoint-based modeling reveals fine-grained body pose tuning in superior temporal sulcus neurons
Abstract Body pose and orientation serve as vital visual signals in primate non-verbal social communication. Leveraging deep learning algorithms that extract body poses from videos of behaving monkeys, applied to a monkey avatar, we investigated neural tuning for pose and viewpoint, targeting fMRI-d...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-07-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-025-60945-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849334380530499584 |
|---|---|
| author | Rajani Raman Anna Bognár Ghazaleh Ghamkhari Nejad Albert Mukovskiy Lucas Martini Martin Giese Rufin Vogels |
| author_facet | Rajani Raman Anna Bognár Ghazaleh Ghamkhari Nejad Albert Mukovskiy Lucas Martini Martin Giese Rufin Vogels |
| author_sort | Rajani Raman |
| collection | DOAJ |
| description | Abstract Body pose and orientation serve as vital visual signals in primate non-verbal social communication. Leveraging deep learning algorithms that extract body poses from videos of behaving monkeys, applied to a monkey avatar, we investigated neural tuning for pose and viewpoint, targeting fMRI-defined mid and anterior Superior Temporal Sulcus (STS) body patches. We modeled the pose and viewpoint selectivity of the units with keypoint-based principal component regression with cross-validation and applied model inversion as a key approach to identify effective body parts and views. Mid STS units were effectively modeled using view-dependent 2D keypoint representations, revealing that their responses were driven by specific body parts that differed among neurons. Some anterior STS units exhibited better predictive performances with a view-dependent 3D model. On average, anterior STS units were better fitted by a keypoint-based model incorporating mirror-symmetric viewpoint tuning than by view-dependent 2D and 3D keypoint models. However, in both regions, a view-independent keypoint model resulted in worse predictive performance. This keypoint-based approach provides insights into how the primate visual system encodes socially relevant body cues, deepening our understanding of body pose representation in the STS. |
| format | Article |
| id | doaj-art-18b9e50b411941aa9696eff302fa92a3 |
| institution | Kabale University |
| issn | 2041-1723 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj-art-18b9e50b411941aa9696eff302fa92a32025-08-20T03:45:35ZengNature PortfolioNature Communications2041-17232025-07-0116111610.1038/s41467-025-60945-5Keypoint-based modeling reveals fine-grained body pose tuning in superior temporal sulcus neuronsRajani Raman0Anna Bognár1Ghazaleh Ghamkhari Nejad2Albert Mukovskiy3Lucas Martini4Martin Giese5Rufin Vogels6Department of Neurosciences, KU LeuvenDepartment of Neurosciences, KU LeuvenDepartment of Neurosciences, KU LeuvenSection Computational Sensomotorics, Department N3, Hertie Institute for Clinical Brain Research & Centre for Integrative Neurocience, University Clinic TübingenSection Computational Sensomotorics, Department N3, Hertie Institute for Clinical Brain Research & Centre for Integrative Neurocience, University Clinic TübingenSection Computational Sensomotorics, Department N3, Hertie Institute for Clinical Brain Research & Centre for Integrative Neurocience, University Clinic TübingenDepartment of Neurosciences, KU LeuvenAbstract Body pose and orientation serve as vital visual signals in primate non-verbal social communication. Leveraging deep learning algorithms that extract body poses from videos of behaving monkeys, applied to a monkey avatar, we investigated neural tuning for pose and viewpoint, targeting fMRI-defined mid and anterior Superior Temporal Sulcus (STS) body patches. We modeled the pose and viewpoint selectivity of the units with keypoint-based principal component regression with cross-validation and applied model inversion as a key approach to identify effective body parts and views. Mid STS units were effectively modeled using view-dependent 2D keypoint representations, revealing that their responses were driven by specific body parts that differed among neurons. Some anterior STS units exhibited better predictive performances with a view-dependent 3D model. On average, anterior STS units were better fitted by a keypoint-based model incorporating mirror-symmetric viewpoint tuning than by view-dependent 2D and 3D keypoint models. However, in both regions, a view-independent keypoint model resulted in worse predictive performance. This keypoint-based approach provides insights into how the primate visual system encodes socially relevant body cues, deepening our understanding of body pose representation in the STS.https://doi.org/10.1038/s41467-025-60945-5 |
| spellingShingle | Rajani Raman Anna Bognár Ghazaleh Ghamkhari Nejad Albert Mukovskiy Lucas Martini Martin Giese Rufin Vogels Keypoint-based modeling reveals fine-grained body pose tuning in superior temporal sulcus neurons Nature Communications |
| title | Keypoint-based modeling reveals fine-grained body pose tuning in superior temporal sulcus neurons |
| title_full | Keypoint-based modeling reveals fine-grained body pose tuning in superior temporal sulcus neurons |
| title_fullStr | Keypoint-based modeling reveals fine-grained body pose tuning in superior temporal sulcus neurons |
| title_full_unstemmed | Keypoint-based modeling reveals fine-grained body pose tuning in superior temporal sulcus neurons |
| title_short | Keypoint-based modeling reveals fine-grained body pose tuning in superior temporal sulcus neurons |
| title_sort | keypoint based modeling reveals fine grained body pose tuning in superior temporal sulcus neurons |
| url | https://doi.org/10.1038/s41467-025-60945-5 |
| work_keys_str_mv | AT rajaniraman keypointbasedmodelingrevealsfinegrainedbodyposetuninginsuperiortemporalsulcusneurons AT annabognar keypointbasedmodelingrevealsfinegrainedbodyposetuninginsuperiortemporalsulcusneurons AT ghazalehghamkharinejad keypointbasedmodelingrevealsfinegrainedbodyposetuninginsuperiortemporalsulcusneurons AT albertmukovskiy keypointbasedmodelingrevealsfinegrainedbodyposetuninginsuperiortemporalsulcusneurons AT lucasmartini keypointbasedmodelingrevealsfinegrainedbodyposetuninginsuperiortemporalsulcusneurons AT martingiese keypointbasedmodelingrevealsfinegrainedbodyposetuninginsuperiortemporalsulcusneurons AT rufinvogels keypointbasedmodelingrevealsfinegrainedbodyposetuninginsuperiortemporalsulcusneurons |