A lightweight transformer with linear self‐attention for defect recognition
Abstract Visual defect recognition techniques based on deep learning models are crucial for modern industrial quality inspection. The backbone, serving as the primary feature extraction component of the defect recognition model, has not been thoroughly exploited. High‐performance vision transformer...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-09-01
|
| Series: | Electronics Letters |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/ell2.13292 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850193007566913536 |
|---|---|
| author | Yuwen Zhai Xinyu Li Liang Gao Yiping Gao |
| author_facet | Yuwen Zhai Xinyu Li Liang Gao Yiping Gao |
| author_sort | Yuwen Zhai |
| collection | DOAJ |
| description | Abstract Visual defect recognition techniques based on deep learning models are crucial for modern industrial quality inspection. The backbone, serving as the primary feature extraction component of the defect recognition model, has not been thoroughly exploited. High‐performance vision transformer (ViT) is less adopted due to high computational complexity and limitations of computational resources and storage hardware in industrial scenarios. This paper presents LSA‐Former, a lightweight transformer architectural backbone that integrates the benefits of convolution and ViT. LSA‐Former proposes a novel self‐attention with linear computational complexity, enabling it to capture local and global semantic features with fewer parameters. LSA‐Former is pre‐trained on ImageNet‐1K and surpasses state‐of‐the‐art methods. LSA‐Former is employed as the backbone for various detectors, evaluated specifically on the PCB defect detection task. The proposed method reduces at least 18M parameters and exceeds the baseline by more than 2.2 mAP. |
| format | Article |
| id | doaj-art-2af147328379430db0eede2338fccd5f |
| institution | OA Journals |
| issn | 0013-5194 1350-911X |
| language | English |
| publishDate | 2024-09-01 |
| publisher | Wiley |
| record_format | Article |
| series | Electronics Letters |
| spelling | doaj-art-2af147328379430db0eede2338fccd5f2025-08-20T02:14:22ZengWileyElectronics Letters0013-51941350-911X2024-09-016017n/an/a10.1049/ell2.13292A lightweight transformer with linear self‐attention for defect recognitionYuwen Zhai0Xinyu Li1Liang Gao2Yiping Gao3School of Mechanical Science and Engineering Huazhong University of Science and Technology Wuhan ChinaSchool of Mechanical Science and Engineering Huazhong University of Science and Technology Wuhan ChinaSchool of Mechanical Science and Engineering Huazhong University of Science and Technology Wuhan ChinaSchool of Mechanical Science and Engineering Huazhong University of Science and Technology Wuhan ChinaAbstract Visual defect recognition techniques based on deep learning models are crucial for modern industrial quality inspection. The backbone, serving as the primary feature extraction component of the defect recognition model, has not been thoroughly exploited. High‐performance vision transformer (ViT) is less adopted due to high computational complexity and limitations of computational resources and storage hardware in industrial scenarios. This paper presents LSA‐Former, a lightweight transformer architectural backbone that integrates the benefits of convolution and ViT. LSA‐Former proposes a novel self‐attention with linear computational complexity, enabling it to capture local and global semantic features with fewer parameters. LSA‐Former is pre‐trained on ImageNet‐1K and surpasses state‐of‐the‐art methods. LSA‐Former is employed as the backbone for various detectors, evaluated specifically on the PCB defect detection task. The proposed method reduces at least 18M parameters and exceeds the baseline by more than 2.2 mAP.https://doi.org/10.1049/ell2.13292automatic optical inspectionconvolutional neural netsimage recognitionobject detection |
| spellingShingle | Yuwen Zhai Xinyu Li Liang Gao Yiping Gao A lightweight transformer with linear self‐attention for defect recognition Electronics Letters automatic optical inspection convolutional neural nets image recognition object detection |
| title | A lightweight transformer with linear self‐attention for defect recognition |
| title_full | A lightweight transformer with linear self‐attention for defect recognition |
| title_fullStr | A lightweight transformer with linear self‐attention for defect recognition |
| title_full_unstemmed | A lightweight transformer with linear self‐attention for defect recognition |
| title_short | A lightweight transformer with linear self‐attention for defect recognition |
| title_sort | lightweight transformer with linear self attention for defect recognition |
| topic | automatic optical inspection convolutional neural nets image recognition object detection |
| url | https://doi.org/10.1049/ell2.13292 |
| work_keys_str_mv | AT yuwenzhai alightweighttransformerwithlinearselfattentionfordefectrecognition AT xinyuli alightweighttransformerwithlinearselfattentionfordefectrecognition AT lianggao alightweighttransformerwithlinearselfattentionfordefectrecognition AT yipinggao alightweighttransformerwithlinearselfattentionfordefectrecognition AT yuwenzhai lightweighttransformerwithlinearselfattentionfordefectrecognition AT xinyuli lightweighttransformerwithlinearselfattentionfordefectrecognition AT lianggao lightweighttransformerwithlinearselfattentionfordefectrecognition AT yipinggao lightweighttransformerwithlinearselfattentionfordefectrecognition |