Benchmarking CNN Architectures for Tool Classification: Evaluating CNN Performance on a Unique Dataset Generated by Novel Image Acquisition System
In this study, we introduce the ToolSurface-144 dataset, which is presented here for the first time. It comprises four subsets – Full R, Full S, Top R, and Top S – each containing 144 tool classes captured under varying illumination conditions and fields of view. A newly develo...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11017576/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850138848515850240 |
|---|---|
| author | Muhenad Bilal Ranadheer Podishetti Daniel Grossmann Markus Bregulla |
| author_facet | Muhenad Bilal Ranadheer Podishetti Daniel Grossmann Markus Bregulla |
| author_sort | Muhenad Bilal |
| collection | DOAJ |
| description | In this study, we introduce the ToolSurface-144 dataset, which is presented here for the first time. It comprises four subsets – Full R, Full S, Top R, and Top S – each containing 144 tool classes captured under varying illumination conditions and fields of view. A newly developed, patented imaging approach was employed to acquire the data. It is compared with conventional diffuse ring illumination to assess its effectiveness in evaluating state-of-the-art convolutional neural networks. This enabled a more targeted investigation of the role of global shape characteristics such as silhouettes versus localized features like the tool face, cutting edges, and delicate geometrical structures under different training strategies. In this study, we evaluate six state-of-the-art convolutional neural networks—AlexNet, DenseNet161, EfficientNet-B0, ResNet152, ResNet50, and VGG16—using three training strategies: fine-tuning, freezing of pre-trained layers, and training from scratch. The results show that EfficientNet-B0 consistently achieved the highest classification accuracy in nearly all experiments and data sets. Especially using the fine-tuning training strategy, the model achieved 99% accuracy in tool classification. ResNet50 benefited greatly from fine-tuning and freezing, achieving a significant increase in performance compared to training from scratch. In contrast, ResNet152, AlexNet, and VGG16 consistently showed poor classification performance, indicating difficulties regarding learning and generalisation. The results show that diffuse illumination and complete tool views provide the best classification conditions, while restricted image sections with homogeneous illumination negatively affect model performance. Among the evaluated training strategies, fine-tuning proved the most efficient training method for developing CNN models for tool classification. |
| format | Article |
| id | doaj-art-e8df76b072c04a25957cf7023e1c632b |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-e8df76b072c04a25957cf7023e1c632b2025-08-20T02:30:30ZengIEEEIEEE Access2169-35362025-01-0113964009642210.1109/ACCESS.2025.357478511017576Benchmarking CNN Architectures for Tool Classification: Evaluating CNN Performance on a Unique Dataset Generated by Novel Image Acquisition SystemMuhenad Bilal0https://orcid.org/0000-0003-4065-8467Ranadheer Podishetti1Daniel Grossmann2https://orcid.org/0000-0002-7388-5757Markus Bregulla3AImotion Bavaria, Application Cluster “Digital Production,” Technische Hochschule Ingolstadt, Ingolstadt, GermanyAImotion Bavaria, Application Cluster “Digital Production,” Technische Hochschule Ingolstadt, Ingolstadt, GermanyAImotion Bavaria, Application Cluster “Digital Production,” Technische Hochschule Ingolstadt, Ingolstadt, GermanyAImotion Bavaria, Application Cluster “Digital Production,” Technische Hochschule Ingolstadt, Ingolstadt, GermanyIn this study, we introduce the ToolSurface-144 dataset, which is presented here for the first time. It comprises four subsets – Full R, Full S, Top R, and Top S – each containing 144 tool classes captured under varying illumination conditions and fields of view. A newly developed, patented imaging approach was employed to acquire the data. It is compared with conventional diffuse ring illumination to assess its effectiveness in evaluating state-of-the-art convolutional neural networks. This enabled a more targeted investigation of the role of global shape characteristics such as silhouettes versus localized features like the tool face, cutting edges, and delicate geometrical structures under different training strategies. In this study, we evaluate six state-of-the-art convolutional neural networks—AlexNet, DenseNet161, EfficientNet-B0, ResNet152, ResNet50, and VGG16—using three training strategies: fine-tuning, freezing of pre-trained layers, and training from scratch. The results show that EfficientNet-B0 consistently achieved the highest classification accuracy in nearly all experiments and data sets. Especially using the fine-tuning training strategy, the model achieved 99% accuracy in tool classification. ResNet50 benefited greatly from fine-tuning and freezing, achieving a significant increase in performance compared to training from scratch. In contrast, ResNet152, AlexNet, and VGG16 consistently showed poor classification performance, indicating difficulties regarding learning and generalisation. The results show that diffuse illumination and complete tool views provide the best classification conditions, while restricted image sections with homogeneous illumination negatively affect model performance. Among the evaluated training strategies, fine-tuning proved the most efficient training method for developing CNN models for tool classification.https://ieeexplore.ieee.org/document/11017576/Tool classificationmachining toolsCNN benchmarkingtransfer learninggeneralization performanceillumination conditions |
| spellingShingle | Muhenad Bilal Ranadheer Podishetti Daniel Grossmann Markus Bregulla Benchmarking CNN Architectures for Tool Classification: Evaluating CNN Performance on a Unique Dataset Generated by Novel Image Acquisition System IEEE Access Tool classification machining tools CNN benchmarking transfer learning generalization performance illumination conditions |
| title | Benchmarking CNN Architectures for Tool Classification: Evaluating CNN Performance on a Unique Dataset Generated by Novel Image Acquisition System |
| title_full | Benchmarking CNN Architectures for Tool Classification: Evaluating CNN Performance on a Unique Dataset Generated by Novel Image Acquisition System |
| title_fullStr | Benchmarking CNN Architectures for Tool Classification: Evaluating CNN Performance on a Unique Dataset Generated by Novel Image Acquisition System |
| title_full_unstemmed | Benchmarking CNN Architectures for Tool Classification: Evaluating CNN Performance on a Unique Dataset Generated by Novel Image Acquisition System |
| title_short | Benchmarking CNN Architectures for Tool Classification: Evaluating CNN Performance on a Unique Dataset Generated by Novel Image Acquisition System |
| title_sort | benchmarking cnn architectures for tool classification evaluating cnn performance on a unique dataset generated by novel image acquisition system |
| topic | Tool classification machining tools CNN benchmarking transfer learning generalization performance illumination conditions |
| url | https://ieeexplore.ieee.org/document/11017576/ |
| work_keys_str_mv | AT muhenadbilal benchmarkingcnnarchitecturesfortoolclassificationevaluatingcnnperformanceonauniquedatasetgeneratedbynovelimageacquisitionsystem AT ranadheerpodishetti benchmarkingcnnarchitecturesfortoolclassificationevaluatingcnnperformanceonauniquedatasetgeneratedbynovelimageacquisitionsystem AT danielgrossmann benchmarkingcnnarchitecturesfortoolclassificationevaluatingcnnperformanceonauniquedatasetgeneratedbynovelimageacquisitionsystem AT markusbregulla benchmarkingcnnarchitecturesfortoolclassificationevaluatingcnnperformanceonauniquedatasetgeneratedbynovelimageacquisitionsystem |