Density-Aware DETR With Dynamic Query for End-to-End Tiny Object Detection
End-to-end DEtection TRansformer (DETRs) are leading a new trend in various object detection tasks. However, when it comes to the ubiquitous tiny objects in aerial imagery, the potential of DETRs still remains under-explored. In this work, we observe that the expansive field of view of remote sensin...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11007261/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849472105032187904 |
|---|---|
| author | Xianhang Ye Chang Xu Haoran Zhu Fang Xu Haijian Zhang Wen Yang |
| author_facet | Xianhang Ye Chang Xu Haoran Zhu Fang Xu Haijian Zhang Wen Yang |
| author_sort | Xianhang Ye |
| collection | DOAJ |
| description | End-to-end DEtection TRansformer (DETRs) are leading a new trend in various object detection tasks. However, when it comes to the ubiquitous tiny objects in aerial imagery, the potential of DETRs still remains under-explored. In this work, we observe that the expansive field of view of remote sensing images often results in a limited pixel representation of tiny objects coupled with a substantial variance in the number of instances across images. The significantly varied tiny object number per image conflicts with DETRs' fixed set of object queries. A large number of queries are necessary to ensure high recall in dense scenarios, while sparse scenarios benefit from fewer, more distinct queries. To tackle this issue, we propose a Density-aware DETR with Dynamic Query (D3Q). D3Q adaptively determines the optimal number of object queries for each image by explicitly estimating its object density. This dynamic query mechanism enables efficient and accurate tiny object detection under both dense and sparse object distributions. In addition, we introduce a refined box loss designed for tiny object detection that further stabilizes training. Through these strategies, D3Q effectively adapts to both dense and sparse scenarios, overcoming the limitations of fixed query in DETR. Extensive experiments on challenging tiny object detection benchmarks demonstrate the superior performance of D3Q compared to state-of-the-art methods. Particularly, when integrated with DINO, D3Q achieves an impressive 32.1% mAP on the AI-TOD-v2 dataset, setting a new state-of-the-art performance. |
| format | Article |
| id | doaj-art-6f13a2d1e93d40ffac16134d94b2ac4f |
| institution | Kabale University |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-6f13a2d1e93d40ffac16134d94b2ac4f2025-08-20T03:24:37ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118135541356910.1109/JSTARS.2025.357181411007261Density-Aware DETR With Dynamic Query for End-to-End Tiny Object DetectionXianhang Ye0https://orcid.org/0009-0003-3334-4686Chang Xu1https://orcid.org/0000-0002-3078-0496Haoran Zhu2https://orcid.org/0009-0003-0153-1305Fang Xu3https://orcid.org/0000-0003-4260-7911Haijian Zhang4https://orcid.org/0000-0001-8314-6563Wen Yang5https://orcid.org/0000-0002-3263-8768School of Electronic Information, Wuhan University, Wuhan, ChinaSchool of Electronic Information, Wuhan University, Wuhan, ChinaSchool of Electronic Information, Wuhan University, Wuhan, ChinaSchool of Artificial Intelligence, Wuhan University, Wuhan, ChinaSchool of Electronic Information, Wuhan University, Wuhan, ChinaSchool of Electronic Information, Wuhan University, Wuhan, ChinaEnd-to-end DEtection TRansformer (DETRs) are leading a new trend in various object detection tasks. However, when it comes to the ubiquitous tiny objects in aerial imagery, the potential of DETRs still remains under-explored. In this work, we observe that the expansive field of view of remote sensing images often results in a limited pixel representation of tiny objects coupled with a substantial variance in the number of instances across images. The significantly varied tiny object number per image conflicts with DETRs' fixed set of object queries. A large number of queries are necessary to ensure high recall in dense scenarios, while sparse scenarios benefit from fewer, more distinct queries. To tackle this issue, we propose a Density-aware DETR with Dynamic Query (D3Q). D3Q adaptively determines the optimal number of object queries for each image by explicitly estimating its object density. This dynamic query mechanism enables efficient and accurate tiny object detection under both dense and sparse object distributions. In addition, we introduce a refined box loss designed for tiny object detection that further stabilizes training. Through these strategies, D3Q effectively adapts to both dense and sparse scenarios, overcoming the limitations of fixed query in DETR. Extensive experiments on challenging tiny object detection benchmarks demonstrate the superior performance of D3Q compared to state-of-the-art methods. Particularly, when integrated with DINO, D3Q achieves an impressive 32.1% mAP on the AI-TOD-v2 dataset, setting a new state-of-the-art performance.https://ieeexplore.ieee.org/document/11007261/Density estimationdetection transformer (DETR)dynamic query (D3Q)label assignmenttiny object detection |
| spellingShingle | Xianhang Ye Chang Xu Haoran Zhu Fang Xu Haijian Zhang Wen Yang Density-Aware DETR With Dynamic Query for End-to-End Tiny Object Detection IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Density estimation detection transformer (DETR) dynamic query (D3Q) label assignment tiny object detection |
| title | Density-Aware DETR With Dynamic Query for End-to-End Tiny Object Detection |
| title_full | Density-Aware DETR With Dynamic Query for End-to-End Tiny Object Detection |
| title_fullStr | Density-Aware DETR With Dynamic Query for End-to-End Tiny Object Detection |
| title_full_unstemmed | Density-Aware DETR With Dynamic Query for End-to-End Tiny Object Detection |
| title_short | Density-Aware DETR With Dynamic Query for End-to-End Tiny Object Detection |
| title_sort | density aware detr with dynamic query for end to end tiny object detection |
| topic | Density estimation detection transformer (DETR) dynamic query (D3Q) label assignment tiny object detection |
| url | https://ieeexplore.ieee.org/document/11007261/ |
| work_keys_str_mv | AT xianhangye densityawaredetrwithdynamicqueryforendtoendtinyobjectdetection AT changxu densityawaredetrwithdynamicqueryforendtoendtinyobjectdetection AT haoranzhu densityawaredetrwithdynamicqueryforendtoendtinyobjectdetection AT fangxu densityawaredetrwithdynamicqueryforendtoendtinyobjectdetection AT haijianzhang densityawaredetrwithdynamicqueryforendtoendtinyobjectdetection AT wenyang densityawaredetrwithdynamicqueryforendtoendtinyobjectdetection |