Single-shot super-resolved fringe projection profilometry (SSSR-FPP): 100,000 frames-per-second 3D imaging with deep learning
Abstract To reveal the fundamental aspects hidden behind a variety of transient events in mechanics, physics, and biology, the highly desired ability to acquire three-dimensional (3D) images with ultrafast temporal resolution has been long sought. As one of the most commonly employed 3D sensing tech...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Publishing Group
2025-02-01
|
| Series: | Light: Science & Applications |
| Online Access: | https://doi.org/10.1038/s41377-024-01721-w |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1823861566581243904 |
|---|---|
| author | Bowen Wang Wenwu Chen Jiaming Qian Shijie Feng Qian Chen Chao Zuo |
| author_facet | Bowen Wang Wenwu Chen Jiaming Qian Shijie Feng Qian Chen Chao Zuo |
| author_sort | Bowen Wang |
| collection | DOAJ |
| description | Abstract To reveal the fundamental aspects hidden behind a variety of transient events in mechanics, physics, and biology, the highly desired ability to acquire three-dimensional (3D) images with ultrafast temporal resolution has been long sought. As one of the most commonly employed 3D sensing techniques, fringe projection profilometry (FPP) reconstructs the depth of a scene from stereo images taken with sequentially structured illuminations. However, the imaging speed of current FPP methods is generally capped at several kHz, which is limited by the projector-camera hardware and the number of fringe patterns required for phase retrieval and unwrapping. Here we report a novel learning-based ultrafast 3D imaging technique, termed single-shot super-resolved FPP (SSSR-FPP), which enables ultrafast 3D imaging at 100,000 Hz. SSSR-FPP uses only one pair of low signal-to-noise ratio (SNR), low-resolution, and pixelated fringe patterns as input, while the high-resolution unwrapped phase and fringe orders can be deciphered with a specific trained deep neural network. Our approach exploits the significant speed gain achieved by reducing the imaging window of conventional high-speed cameras, while “regenerating” the lost spatial resolution through deep learning. To demonstrate the high spatio-temporal resolution of SSSR-FPP, we present 3D videography of several transient scenes, including rotating turbofan blades, exploding building blocks, and the reciprocating motion of a steam engine, etc., which were previously challenging or even impossible to capture with conventional methods. Experimental results establish SSSR-FPP as a significant step forward in the field of 3D optical sensing, offering new insights into a broad spectrum of dynamic processes across various scientific disciplines. |
| format | Article |
| id | doaj-art-6af33c75233b41f694246cea263159b5 |
| institution | Kabale University |
| issn | 2047-7538 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Nature Publishing Group |
| record_format | Article |
| series | Light: Science & Applications |
| spelling | doaj-art-6af33c75233b41f694246cea263159b52025-02-09T12:55:03ZengNature Publishing GroupLight: Science & Applications2047-75382025-02-0114111310.1038/s41377-024-01721-wSingle-shot super-resolved fringe projection profilometry (SSSR-FPP): 100,000 frames-per-second 3D imaging with deep learningBowen Wang0Wenwu Chen1Jiaming Qian2Shijie Feng3Qian Chen4Chao Zuo5Smart Computational Imaging Laboratory (SCILab), Nanjing University of Science and TechnologySmart Computational Imaging Laboratory (SCILab), Nanjing University of Science and TechnologySmart Computational Imaging Laboratory (SCILab), Nanjing University of Science and TechnologySmart Computational Imaging Laboratory (SCILab), Nanjing University of Science and TechnologyJiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and TechnologySmart Computational Imaging Laboratory (SCILab), Nanjing University of Science and TechnologyAbstract To reveal the fundamental aspects hidden behind a variety of transient events in mechanics, physics, and biology, the highly desired ability to acquire three-dimensional (3D) images with ultrafast temporal resolution has been long sought. As one of the most commonly employed 3D sensing techniques, fringe projection profilometry (FPP) reconstructs the depth of a scene from stereo images taken with sequentially structured illuminations. However, the imaging speed of current FPP methods is generally capped at several kHz, which is limited by the projector-camera hardware and the number of fringe patterns required for phase retrieval and unwrapping. Here we report a novel learning-based ultrafast 3D imaging technique, termed single-shot super-resolved FPP (SSSR-FPP), which enables ultrafast 3D imaging at 100,000 Hz. SSSR-FPP uses only one pair of low signal-to-noise ratio (SNR), low-resolution, and pixelated fringe patterns as input, while the high-resolution unwrapped phase and fringe orders can be deciphered with a specific trained deep neural network. Our approach exploits the significant speed gain achieved by reducing the imaging window of conventional high-speed cameras, while “regenerating” the lost spatial resolution through deep learning. To demonstrate the high spatio-temporal resolution of SSSR-FPP, we present 3D videography of several transient scenes, including rotating turbofan blades, exploding building blocks, and the reciprocating motion of a steam engine, etc., which were previously challenging or even impossible to capture with conventional methods. Experimental results establish SSSR-FPP as a significant step forward in the field of 3D optical sensing, offering new insights into a broad spectrum of dynamic processes across various scientific disciplines.https://doi.org/10.1038/s41377-024-01721-w |
| spellingShingle | Bowen Wang Wenwu Chen Jiaming Qian Shijie Feng Qian Chen Chao Zuo Single-shot super-resolved fringe projection profilometry (SSSR-FPP): 100,000 frames-per-second 3D imaging with deep learning Light: Science & Applications |
| title | Single-shot super-resolved fringe projection profilometry (SSSR-FPP): 100,000 frames-per-second 3D imaging with deep learning |
| title_full | Single-shot super-resolved fringe projection profilometry (SSSR-FPP): 100,000 frames-per-second 3D imaging with deep learning |
| title_fullStr | Single-shot super-resolved fringe projection profilometry (SSSR-FPP): 100,000 frames-per-second 3D imaging with deep learning |
| title_full_unstemmed | Single-shot super-resolved fringe projection profilometry (SSSR-FPP): 100,000 frames-per-second 3D imaging with deep learning |
| title_short | Single-shot super-resolved fringe projection profilometry (SSSR-FPP): 100,000 frames-per-second 3D imaging with deep learning |
| title_sort | single shot super resolved fringe projection profilometry sssr fpp 100 000 frames per second 3d imaging with deep learning |
| url | https://doi.org/10.1038/s41377-024-01721-w |
| work_keys_str_mv | AT bowenwang singleshotsuperresolvedfringeprojectionprofilometrysssrfpp100000framespersecond3dimagingwithdeeplearning AT wenwuchen singleshotsuperresolvedfringeprojectionprofilometrysssrfpp100000framespersecond3dimagingwithdeeplearning AT jiamingqian singleshotsuperresolvedfringeprojectionprofilometrysssrfpp100000framespersecond3dimagingwithdeeplearning AT shijiefeng singleshotsuperresolvedfringeprojectionprofilometrysssrfpp100000framespersecond3dimagingwithdeeplearning AT qianchen singleshotsuperresolvedfringeprojectionprofilometrysssrfpp100000framespersecond3dimagingwithdeeplearning AT chaozuo singleshotsuperresolvedfringeprojectionprofilometrysssrfpp100000framespersecond3dimagingwithdeeplearning |