From Simulation to Reality: Transfer Learning for Automating Pseudo‐Labeling of Real and Infrared Imagery
Training a convolutional neural network (CNN) for real‐world applications is challenging due to the requirement of high‐quality labeled imagery. This study employs pseudo‐labeling and transfer learning, built upon a 6D pose estimation framework. A CNN trained on synthetic images predicts bounding bo...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2025-05-01
|
| Series: | Advanced Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/aisy.202400575 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850129523061817344 |
|---|---|
| author | Jeffrey Choate Derek Worth Scott L. Nykl Clark Taylor Brett Borghetti Christine Schubert Kabban Ryan Raettig |
| author_facet | Jeffrey Choate Derek Worth Scott L. Nykl Clark Taylor Brett Borghetti Christine Schubert Kabban Ryan Raettig |
| author_sort | Jeffrey Choate |
| collection | DOAJ |
| description | Training a convolutional neural network (CNN) for real‐world applications is challenging due to the requirement of high‐quality labeled imagery. This study employs pseudo‐labeling and transfer learning, built upon a 6D pose estimation framework. A CNN trained on synthetic images predicts bounding boxes (bbox) for an object's components in a real image. With as few as four bbox predictions, the framework solves for the object's pose relative to the camera and reprojects bboxes for all components onto that image. The pose and reprojections allow filtering of bad predictions, a common issue in pseudo‐labeling. Thereby, enabling automated labeling of large datasets with minimal human intervention. Tested on color and long‐wave infrared imagery captured during December 2023 flight tests, this process demonstrates increased predictions, enhanced performance across situations, reduced reprojection error, and stabilized pose predictions. This technique is significant as it enables labeling of real‐world imagery without expensive truth systems, requiring only a camera. It supports learning and labeling of previously captured imagery without known camera calibrations, facilitating labeled data creation for impractical‐to‐simulate sensors. Ultimately, this transfer learning approach provides a low‐cost and precise method for creating CNNs trained on operationally relevant data, previously unattainable by the everyday user. |
| format | Article |
| id | doaj-art-b2623a7edfd94a1e8dfcdcd9cbfb84db |
| institution | OA Journals |
| issn | 2640-4567 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Wiley |
| record_format | Article |
| series | Advanced Intelligent Systems |
| spelling | doaj-art-b2623a7edfd94a1e8dfcdcd9cbfb84db2025-08-20T02:32:56ZengWileyAdvanced Intelligent Systems2640-45672025-05-0175n/an/a10.1002/aisy.202400575From Simulation to Reality: Transfer Learning for Automating Pseudo‐Labeling of Real and Infrared ImageryJeffrey Choate0Derek Worth1Scott L. Nykl2Clark Taylor3Brett Borghetti4Christine Schubert Kabban5Ryan Raettig6Department of Electrical and Computer Engineering Air Force Institute of Technology Wright‐Patterson AFB OH 45433 USADepartment of Electrical and Computer Engineering Air Force Institute of Technology Wright‐Patterson AFB OH 45433 USADepartment of Electrical and Computer Engineering Air Force Institute of Technology Wright‐Patterson AFB OH 45433 USADepartment of Electrical and Computer Engineering Air Force Institute of Technology Wright‐Patterson AFB OH 45433 USADepartment of Electrical and Computer Engineering Air Force Institute of Technology Wright‐Patterson AFB OH 45433 USADepartment of Mathematics and Statistics Air Force Institute of Technology Wright‐Patterson AFB OH 45433 USADepartment of Electrical and Computer Engineering Air Force Institute of Technology Wright‐Patterson AFB OH 45433 USATraining a convolutional neural network (CNN) for real‐world applications is challenging due to the requirement of high‐quality labeled imagery. This study employs pseudo‐labeling and transfer learning, built upon a 6D pose estimation framework. A CNN trained on synthetic images predicts bounding boxes (bbox) for an object's components in a real image. With as few as four bbox predictions, the framework solves for the object's pose relative to the camera and reprojects bboxes for all components onto that image. The pose and reprojections allow filtering of bad predictions, a common issue in pseudo‐labeling. Thereby, enabling automated labeling of large datasets with minimal human intervention. Tested on color and long‐wave infrared imagery captured during December 2023 flight tests, this process demonstrates increased predictions, enhanced performance across situations, reduced reprojection error, and stabilized pose predictions. This technique is significant as it enables labeling of real‐world imagery without expensive truth systems, requiring only a camera. It supports learning and labeling of previously captured imagery without known camera calibrations, facilitating labeled data creation for impractical‐to‐simulate sensors. Ultimately, this transfer learning approach provides a low‐cost and precise method for creating CNNs trained on operationally relevant data, previously unattainable by the everyday user.https://doi.org/10.1002/aisy.202400575convolutional neural networkPerspective‐N‐Point algorithmpseudo‐labelingreal time pose estimationsynthetic imagery generationtransfer learning |
| spellingShingle | Jeffrey Choate Derek Worth Scott L. Nykl Clark Taylor Brett Borghetti Christine Schubert Kabban Ryan Raettig From Simulation to Reality: Transfer Learning for Automating Pseudo‐Labeling of Real and Infrared Imagery Advanced Intelligent Systems convolutional neural network Perspective‐N‐Point algorithm pseudo‐labeling real time pose estimation synthetic imagery generation transfer learning |
| title | From Simulation to Reality: Transfer Learning for Automating Pseudo‐Labeling of Real and Infrared Imagery |
| title_full | From Simulation to Reality: Transfer Learning for Automating Pseudo‐Labeling of Real and Infrared Imagery |
| title_fullStr | From Simulation to Reality: Transfer Learning for Automating Pseudo‐Labeling of Real and Infrared Imagery |
| title_full_unstemmed | From Simulation to Reality: Transfer Learning for Automating Pseudo‐Labeling of Real and Infrared Imagery |
| title_short | From Simulation to Reality: Transfer Learning for Automating Pseudo‐Labeling of Real and Infrared Imagery |
| title_sort | from simulation to reality transfer learning for automating pseudo labeling of real and infrared imagery |
| topic | convolutional neural network Perspective‐N‐Point algorithm pseudo‐labeling real time pose estimation synthetic imagery generation transfer learning |
| url | https://doi.org/10.1002/aisy.202400575 |
| work_keys_str_mv | AT jeffreychoate fromsimulationtorealitytransferlearningforautomatingpseudolabelingofrealandinfraredimagery AT derekworth fromsimulationtorealitytransferlearningforautomatingpseudolabelingofrealandinfraredimagery AT scottlnykl fromsimulationtorealitytransferlearningforautomatingpseudolabelingofrealandinfraredimagery AT clarktaylor fromsimulationtorealitytransferlearningforautomatingpseudolabelingofrealandinfraredimagery AT brettborghetti fromsimulationtorealitytransferlearningforautomatingpseudolabelingofrealandinfraredimagery AT christineschubertkabban fromsimulationtorealitytransferlearningforautomatingpseudolabelingofrealandinfraredimagery AT ryanraettig fromsimulationtorealitytransferlearningforautomatingpseudolabelingofrealandinfraredimagery |