Multimodal Fusion Image Stabilization Algorithm for Bio-Inspired Flapping-Wing Aircraft
This paper presents FWStab, a specialized video stabilization dataset tailored for flapping-wing platforms. The dataset encompasses five typical flight scenarios, featuring 48 video clips with intense dynamic jitter. The corresponding Inertial Measurement Unit (IMU) sensor data are synchronously col...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-07-01
|
| Series: | Biomimetics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-7673/10/7/448 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849303528403632128 |
|---|---|
| author | Zhikai Wang Sen Wang Yiwen Hu Yangfan Zhou Na Li Xiaofeng Zhang |
| author_facet | Zhikai Wang Sen Wang Yiwen Hu Yangfan Zhou Na Li Xiaofeng Zhang |
| author_sort | Zhikai Wang |
| collection | DOAJ |
| description | This paper presents FWStab, a specialized video stabilization dataset tailored for flapping-wing platforms. The dataset encompasses five typical flight scenarios, featuring 48 video clips with intense dynamic jitter. The corresponding Inertial Measurement Unit (IMU) sensor data are synchronously collected, which jointly provide reliable support for multimodal modeling. Based on this, to address the issue of poor image acquisition quality due to severe vibrations in aerial vehicles, this paper proposes a multi-modal signal fusion video stabilization framework. This framework effectively integrates image features and inertial sensor features to predict smooth and stable camera poses. During the video stabilization process, the true camera motion originally estimated based on sensors is warped to the smooth trajectory predicted by the network, thereby optimizing the inter-frame stability. This approach maintains the global rigidity of scene motion, avoids visual artifacts caused by traditional dense optical flow-based spatiotemporal warping, and rectifies rolling shutter-induced distortions. Furthermore, the network is trained in an unsupervised manner by leveraging a joint loss function that integrates camera pose smoothness and optical flow residuals. When coupled with a multi-stage training strategy, this framework demonstrates remarkable stabilization adaptability across a wide range of scenarios. The entire framework employs Long Short-Term Memory (LSTM) to model the temporal characteristics of camera trajectories, enabling high-precision prediction of smooth trajectories. |
| format | Article |
| id | doaj-art-b02eee7875cc4394bc2d14ecbc2993d5 |
| institution | Kabale University |
| issn | 2313-7673 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Biomimetics |
| spelling | doaj-art-b02eee7875cc4394bc2d14ecbc2993d52025-08-20T03:58:26ZengMDPI AGBiomimetics2313-76732025-07-0110744810.3390/biomimetics10070448Multimodal Fusion Image Stabilization Algorithm for Bio-Inspired Flapping-Wing AircraftZhikai Wang0Sen Wang1Yiwen Hu2Yangfan Zhou3Na Li4Xiaofeng Zhang5College of Information Engineering, Henan University of Science and Technology, Luoyang 471023, ChinaCollege of Information Engineering, Henan University of Science and Technology, Luoyang 471023, ChinaNational Key Laboratory of Digital and Agile Aircraft Design, Chengdu 610091, ChinaSuzhou Institute of Nano-Tech and Nano-Bionics (SINANO), Chinese Academy of Sciences, Suzhou 215123, ChinaCollege of Information Engineering, Henan University of Science and Technology, Luoyang 471023, ChinaSchool of Electrical Engineering and Automation, Harbin Institute of Technology, Harbin 150001, ChinaThis paper presents FWStab, a specialized video stabilization dataset tailored for flapping-wing platforms. The dataset encompasses five typical flight scenarios, featuring 48 video clips with intense dynamic jitter. The corresponding Inertial Measurement Unit (IMU) sensor data are synchronously collected, which jointly provide reliable support for multimodal modeling. Based on this, to address the issue of poor image acquisition quality due to severe vibrations in aerial vehicles, this paper proposes a multi-modal signal fusion video stabilization framework. This framework effectively integrates image features and inertial sensor features to predict smooth and stable camera poses. During the video stabilization process, the true camera motion originally estimated based on sensors is warped to the smooth trajectory predicted by the network, thereby optimizing the inter-frame stability. This approach maintains the global rigidity of scene motion, avoids visual artifacts caused by traditional dense optical flow-based spatiotemporal warping, and rectifies rolling shutter-induced distortions. Furthermore, the network is trained in an unsupervised manner by leveraging a joint loss function that integrates camera pose smoothness and optical flow residuals. When coupled with a multi-stage training strategy, this framework demonstrates remarkable stabilization adaptability across a wide range of scenarios. The entire framework employs Long Short-Term Memory (LSTM) to model the temporal characteristics of camera trajectories, enabling high-precision prediction of smooth trajectories.https://www.mdpi.com/2313-7673/10/7/448biomimetic aircraftcross-attention mechanismoptical flowSEA-RAFTimage enhancement |
| spellingShingle | Zhikai Wang Sen Wang Yiwen Hu Yangfan Zhou Na Li Xiaofeng Zhang Multimodal Fusion Image Stabilization Algorithm for Bio-Inspired Flapping-Wing Aircraft Biomimetics biomimetic aircraft cross-attention mechanism optical flow SEA-RAFT image enhancement |
| title | Multimodal Fusion Image Stabilization Algorithm for Bio-Inspired Flapping-Wing Aircraft |
| title_full | Multimodal Fusion Image Stabilization Algorithm for Bio-Inspired Flapping-Wing Aircraft |
| title_fullStr | Multimodal Fusion Image Stabilization Algorithm for Bio-Inspired Flapping-Wing Aircraft |
| title_full_unstemmed | Multimodal Fusion Image Stabilization Algorithm for Bio-Inspired Flapping-Wing Aircraft |
| title_short | Multimodal Fusion Image Stabilization Algorithm for Bio-Inspired Flapping-Wing Aircraft |
| title_sort | multimodal fusion image stabilization algorithm for bio inspired flapping wing aircraft |
| topic | biomimetic aircraft cross-attention mechanism optical flow SEA-RAFT image enhancement |
| url | https://www.mdpi.com/2313-7673/10/7/448 |
| work_keys_str_mv | AT zhikaiwang multimodalfusionimagestabilizationalgorithmforbioinspiredflappingwingaircraft AT senwang multimodalfusionimagestabilizationalgorithmforbioinspiredflappingwingaircraft AT yiwenhu multimodalfusionimagestabilizationalgorithmforbioinspiredflappingwingaircraft AT yangfanzhou multimodalfusionimagestabilizationalgorithmforbioinspiredflappingwingaircraft AT nali multimodalfusionimagestabilizationalgorithmforbioinspiredflappingwingaircraft AT xiaofengzhang multimodalfusionimagestabilizationalgorithmforbioinspiredflappingwingaircraft |