Hybrid transformer-CNN network-driven optical-scanning undersampling for photoacoustic remote sensing microscopy
Imaging speed is critical for photoacoustic microscopy as it affects the capability to capture dynamic biological processes and support real-time clinical applications. Conventional approaches for increasing imaging speed typically involve high-repetition-rate lasers, which pose a risk of thermal da...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-04-01
|
| Series: | Photoacoustics |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2213597925000163 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850046230896312320 |
|---|---|
| author | Yihan Pi Jijing Chen Kaixuan Ding Tongyan Zhang Hao Zhang Bingxue Zhang Junhao Guo Zhen Tian Jiao Li |
| author_facet | Yihan Pi Jijing Chen Kaixuan Ding Tongyan Zhang Hao Zhang Bingxue Zhang Junhao Guo Zhen Tian Jiao Li |
| author_sort | Yihan Pi |
| collection | DOAJ |
| description | Imaging speed is critical for photoacoustic microscopy as it affects the capability to capture dynamic biological processes and support real-time clinical applications. Conventional approaches for increasing imaging speed typically involve high-repetition-rate lasers, which pose a risk of thermal damage to samples. Here, we propose a deep-learning-driven optical-scanning undersampling method for photoacoustic remote sensing (PARS) microscopy, accelerating imaging acquisition while maintaining a constant laser repetition rate and reducing laser dosage. We develop a hybrid Transformer-Convolutional Neural Network, HTC-GAN, to address the challenges of both nonuniform sampling and motion misalignment inherent in optical-scanning undersampling. A mouse ear vasculature image dataset is created through our customized galvanometer-scanned PARS system to train and validate HTC-GAN. The network successfully restores high-quality images from 1/2-undersampled and 1/4-undersampled data, closely approximating the ground truth images. A series of performance experiments demonstrate that HTC-GAN surpasses the basic misalignment compensation algorithm, and standalone CNN or Transformer networks in terms of perceptual quality and quantitative metrics. Moreover, three-dimensional imaging results validate the robustness and versatility of the proposed optical-scanning undersampling imaging method across multiscale scanning modes. Our method achieves a fourfold improvement in PARS imaging speed without hardware upgrades, offering an available solution for enhancing imaging speed in other optical-scanning microscopic systems. |
| format | Article |
| id | doaj-art-af7ca5620d2f4f0e8ef65cfed1181690 |
| institution | DOAJ |
| issn | 2213-5979 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Photoacoustics |
| spelling | doaj-art-af7ca5620d2f4f0e8ef65cfed11816902025-08-20T02:54:29ZengElsevierPhotoacoustics2213-59792025-04-014210069710.1016/j.pacs.2025.100697Hybrid transformer-CNN network-driven optical-scanning undersampling for photoacoustic remote sensing microscopyYihan Pi0Jijing Chen1Kaixuan Ding2Tongyan Zhang3Hao Zhang4Bingxue Zhang5Junhao Guo6Zhen Tian7Jiao Li8College of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, China; Center for Terahertz Waves, Tianjin University, Tianjin 300072, ChinaCollege of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, China; Center for Terahertz Waves, Tianjin University, Tianjin 300072, ChinaCollege of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, ChinaInfectious Diseases Department, Second Affiliated Hospital of Tianjin University of Traditional Chinese Medicine, Tianjin, ChinaCollege of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, China; Center for Terahertz Waves, Tianjin University, Tianjin 300072, ChinaCollege of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, ChinaCollege of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, China; Center for Terahertz Waves, Tianjin University, Tianjin 300072, ChinaCollege of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, China; Center for Terahertz Waves, Tianjin University, Tianjin 300072, China; State Key Laboratory of Precision Measurement Technology and Instruments, Tianjin University, Tianjin 300072, China; Georgia Tech Shenzhen Institute (GTSI), Tianjin University, Shenzhen 518067, China; Corresponding authors at: College of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, ChinaCollege of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, China; State Key Laboratory of Precision Measurement Technology and Instruments, Tianjin University, Tianjin 300072, China; Georgia Tech Shenzhen Institute (GTSI), Tianjin University, Shenzhen 518067, China; Corresponding authors at: College of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin 300072, ChinaImaging speed is critical for photoacoustic microscopy as it affects the capability to capture dynamic biological processes and support real-time clinical applications. Conventional approaches for increasing imaging speed typically involve high-repetition-rate lasers, which pose a risk of thermal damage to samples. Here, we propose a deep-learning-driven optical-scanning undersampling method for photoacoustic remote sensing (PARS) microscopy, accelerating imaging acquisition while maintaining a constant laser repetition rate and reducing laser dosage. We develop a hybrid Transformer-Convolutional Neural Network, HTC-GAN, to address the challenges of both nonuniform sampling and motion misalignment inherent in optical-scanning undersampling. A mouse ear vasculature image dataset is created through our customized galvanometer-scanned PARS system to train and validate HTC-GAN. The network successfully restores high-quality images from 1/2-undersampled and 1/4-undersampled data, closely approximating the ground truth images. A series of performance experiments demonstrate that HTC-GAN surpasses the basic misalignment compensation algorithm, and standalone CNN or Transformer networks in terms of perceptual quality and quantitative metrics. Moreover, three-dimensional imaging results validate the robustness and versatility of the proposed optical-scanning undersampling imaging method across multiscale scanning modes. Our method achieves a fourfold improvement in PARS imaging speed without hardware upgrades, offering an available solution for enhancing imaging speed in other optical-scanning microscopic systems.http://www.sciencedirect.com/science/article/pii/S2213597925000163PhotoacousticDeep learningOptical scanningUndersampling |
| spellingShingle | Yihan Pi Jijing Chen Kaixuan Ding Tongyan Zhang Hao Zhang Bingxue Zhang Junhao Guo Zhen Tian Jiao Li Hybrid transformer-CNN network-driven optical-scanning undersampling for photoacoustic remote sensing microscopy Photoacoustics Photoacoustic Deep learning Optical scanning Undersampling |
| title | Hybrid transformer-CNN network-driven optical-scanning undersampling for photoacoustic remote sensing microscopy |
| title_full | Hybrid transformer-CNN network-driven optical-scanning undersampling for photoacoustic remote sensing microscopy |
| title_fullStr | Hybrid transformer-CNN network-driven optical-scanning undersampling for photoacoustic remote sensing microscopy |
| title_full_unstemmed | Hybrid transformer-CNN network-driven optical-scanning undersampling for photoacoustic remote sensing microscopy |
| title_short | Hybrid transformer-CNN network-driven optical-scanning undersampling for photoacoustic remote sensing microscopy |
| title_sort | hybrid transformer cnn network driven optical scanning undersampling for photoacoustic remote sensing microscopy |
| topic | Photoacoustic Deep learning Optical scanning Undersampling |
| url | http://www.sciencedirect.com/science/article/pii/S2213597925000163 |
| work_keys_str_mv | AT yihanpi hybridtransformercnnnetworkdrivenopticalscanningundersamplingforphotoacousticremotesensingmicroscopy AT jijingchen hybridtransformercnnnetworkdrivenopticalscanningundersamplingforphotoacousticremotesensingmicroscopy AT kaixuanding hybridtransformercnnnetworkdrivenopticalscanningundersamplingforphotoacousticremotesensingmicroscopy AT tongyanzhang hybridtransformercnnnetworkdrivenopticalscanningundersamplingforphotoacousticremotesensingmicroscopy AT haozhang hybridtransformercnnnetworkdrivenopticalscanningundersamplingforphotoacousticremotesensingmicroscopy AT bingxuezhang hybridtransformercnnnetworkdrivenopticalscanningundersamplingforphotoacousticremotesensingmicroscopy AT junhaoguo hybridtransformercnnnetworkdrivenopticalscanningundersamplingforphotoacousticremotesensingmicroscopy AT zhentian hybridtransformercnnnetworkdrivenopticalscanningundersamplingforphotoacousticremotesensingmicroscopy AT jiaoli hybridtransformercnnnetworkdrivenopticalscanningundersamplingforphotoacousticremotesensingmicroscopy |