Streamlined photoacoustic image processing with foundation models: A training-free solution
Foundation models (FMs) have rapidly evolved and have achieved significant accomplishments in computer vision tasks. Specifically, the prompt mechanism conveniently allows users to integrate image prior information into the model, making it possible to apply models without any training. Therefore, w...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
World Scientific Publishing
2025-01-01
|
| Series: | Journal of Innovative Optical Health Sciences |
| Subjects: | |
| Online Access: | https://www.worldscientific.com/doi/10.1142/S1793545824500196 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832585051747909632 |
|---|---|
| author | Handi Deng Yucheng Zhou Jiaxuan Xiang Liujie Gu Yan Luo Hai Feng Mingyuan Liu Cheng Ma |
| author_facet | Handi Deng Yucheng Zhou Jiaxuan Xiang Liujie Gu Yan Luo Hai Feng Mingyuan Liu Cheng Ma |
| author_sort | Handi Deng |
| collection | DOAJ |
| description | Foundation models (FMs) have rapidly evolved and have achieved significant accomplishments in computer vision tasks. Specifically, the prompt mechanism conveniently allows users to integrate image prior information into the model, making it possible to apply models without any training. Therefore, we proposed a workflow based on foundation models and zero training to solve the tasks of photoacoustic (PA) image processing. We employed the Segment Anything Model (SAM) by setting simple prompts and integrating the model’s outputs with prior knowledge of the imaged objects to accomplish various tasks, including: (1) removing the skin signal in three-dimensional PA image rendering; (2) dual speed-of-sound reconstruction, and (3) segmentation of finger blood vessels. Through these demonstrations, we have concluded that FMs can be directly applied in PA imaging without the requirement for network design and training. This potentially allows for a hands-on, convenient approach to achieving efficient and accurate segmentation of PA images. This paper serves as a comprehensive tutorial, facilitating the mastery of the technique through the provision of code and sample datasets. |
| format | Article |
| id | doaj-art-2218390580d14198b0db03dc5ad16471 |
| institution | Kabale University |
| issn | 1793-5458 1793-7205 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | World Scientific Publishing |
| record_format | Article |
| series | Journal of Innovative Optical Health Sciences |
| spelling | doaj-art-2218390580d14198b0db03dc5ad164712025-01-27T05:49:52ZengWorld Scientific PublishingJournal of Innovative Optical Health Sciences1793-54581793-72052025-01-01180110.1142/S1793545824500196Streamlined photoacoustic image processing with foundation models: A training-free solutionHandi Deng0Yucheng Zhou1Jiaxuan Xiang2Liujie Gu3Yan Luo4Hai Feng5Mingyuan Liu6Cheng Ma7Beijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University 30 Shuangqing Road, Haidian, Beijing 100084, P. R. ChinaSchool of Biological Science and Medical Engineering, Beihang University, 37 XueYuan Road, Haidian, Beijing 100191, P. R. ChinaTsingPAI Technology Co., Ltd., 27 Jiancaicheng Middle Road, Haidian, Beijing 100096, P. R. ChinaBeijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University 30 Shuangqing Road, Haidian, Beijing 100084, P. R. ChinaBeijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University 30 Shuangqing Road, Haidian, Beijing 100084, P. R. ChinaDepartment of Vascular Surgery, Beijing Friendship Hospital, Capital Medical University, 95 Yongan Road, Haidian, Beijing 100050, P. R. ChinaDepartment of Vascular Surgery, Beijing Friendship Hospital, Capital Medical University, 95 Yongan Road, Haidian, Beijing 100050, P. R. ChinaBeijing National Research Center for Information Science and Technology, Department of Electronic Engineering, Tsinghua University 30 Shuangqing Road, Haidian, Beijing 100084, P. R. ChinaFoundation models (FMs) have rapidly evolved and have achieved significant accomplishments in computer vision tasks. Specifically, the prompt mechanism conveniently allows users to integrate image prior information into the model, making it possible to apply models without any training. Therefore, we proposed a workflow based on foundation models and zero training to solve the tasks of photoacoustic (PA) image processing. We employed the Segment Anything Model (SAM) by setting simple prompts and integrating the model’s outputs with prior knowledge of the imaged objects to accomplish various tasks, including: (1) removing the skin signal in three-dimensional PA image rendering; (2) dual speed-of-sound reconstruction, and (3) segmentation of finger blood vessels. Through these demonstrations, we have concluded that FMs can be directly applied in PA imaging without the requirement for network design and training. This potentially allows for a hands-on, convenient approach to achieving efficient and accurate segmentation of PA images. This paper serves as a comprehensive tutorial, facilitating the mastery of the technique through the provision of code and sample datasets.https://www.worldscientific.com/doi/10.1142/S1793545824500196Foundation modelsphotoacoustic imagingimage segmentationlarge model |
| spellingShingle | Handi Deng Yucheng Zhou Jiaxuan Xiang Liujie Gu Yan Luo Hai Feng Mingyuan Liu Cheng Ma Streamlined photoacoustic image processing with foundation models: A training-free solution Journal of Innovative Optical Health Sciences Foundation models photoacoustic imaging image segmentation large model |
| title | Streamlined photoacoustic image processing with foundation models: A training-free solution |
| title_full | Streamlined photoacoustic image processing with foundation models: A training-free solution |
| title_fullStr | Streamlined photoacoustic image processing with foundation models: A training-free solution |
| title_full_unstemmed | Streamlined photoacoustic image processing with foundation models: A training-free solution |
| title_short | Streamlined photoacoustic image processing with foundation models: A training-free solution |
| title_sort | streamlined photoacoustic image processing with foundation models a training free solution |
| topic | Foundation models photoacoustic imaging image segmentation large model |
| url | https://www.worldscientific.com/doi/10.1142/S1793545824500196 |
| work_keys_str_mv | AT handideng streamlinedphotoacousticimageprocessingwithfoundationmodelsatrainingfreesolution AT yuchengzhou streamlinedphotoacousticimageprocessingwithfoundationmodelsatrainingfreesolution AT jiaxuanxiang streamlinedphotoacousticimageprocessingwithfoundationmodelsatrainingfreesolution AT liujiegu streamlinedphotoacousticimageprocessingwithfoundationmodelsatrainingfreesolution AT yanluo streamlinedphotoacousticimageprocessingwithfoundationmodelsatrainingfreesolution AT haifeng streamlinedphotoacousticimageprocessingwithfoundationmodelsatrainingfreesolution AT mingyuanliu streamlinedphotoacousticimageprocessingwithfoundationmodelsatrainingfreesolution AT chengma streamlinedphotoacousticimageprocessingwithfoundationmodelsatrainingfreesolution |