Context-Aware Fine-Grained Product Recognition on Grocery Shelves
Product recognition is a fine-grained image retrieval problem because grocery stores can contain several thousand products on their shelves, some of which have minimal visual differences. Given that the products on the store’s shelves change frequently, it would be practical to build a sy...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10849567/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832576787472711680 |
|---|---|
| author | Lovre Antonio Budimir Zoran Kalafatic Marko Subasic Sven Loncaric |
| author_facet | Lovre Antonio Budimir Zoran Kalafatic Marko Subasic Sven Loncaric |
| author_sort | Lovre Antonio Budimir |
| collection | DOAJ |
| description | Product recognition is a fine-grained image retrieval problem because grocery stores can contain several thousand products on their shelves, some of which have minimal visual differences. Given that the products on the store’s shelves change frequently, it would be practical to build a system using a single reference image or a few per product. This task is very challenging in computer vision, whereas humans solve it more effortlessly by relying on contextual information. Our work incorporates semantic and spatial context into a novel product recognition method. We take advantage of stores being organized into different sections and aisles, where similar products are placed nearby. First, we propose the Hierarchical Auxiliary Loss (HAL) for learning an organized feature space in which products from the same category, usually placed in the same store section, are close to each other. Second, we propose the Context-Aware Query Expansion (CAQE) module for the inference phase, in which each feature vector of a product on a shelf is expanded with the feature vectors of neighboring products. The amount of information exchanged between the two products depends on the similarity of their feature vectors and spatial distance on the store shelves. To demonstrate the effectiveness of our contributions, we conducted detailed experiments on publicly available grocery product datasets and showed that our method achieves state-of-the-art results. |
| format | Article |
| id | doaj-art-d1caf5811e464c1686dff5af74f27a71 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-d1caf5811e464c1686dff5af74f27a712025-01-31T00:01:04ZengIEEEIEEE Access2169-35362025-01-0113168241683710.1109/ACCESS.2025.353274510849567Context-Aware Fine-Grained Product Recognition on Grocery ShelvesLovre Antonio Budimir0https://orcid.org/0000-0003-2146-7067Zoran Kalafatic1https://orcid.org/0000-0001-8918-9070Marko Subasic2https://orcid.org/0000-0002-4321-4557Sven Loncaric3https://orcid.org/0000-0002-4857-5351Faculty of Electrical Engineering and Computing, University of Zagreb, Zagreb, CroatiaFaculty of Electrical Engineering and Computing, University of Zagreb, Zagreb, CroatiaFaculty of Electrical Engineering and Computing, University of Zagreb, Zagreb, CroatiaFaculty of Electrical Engineering and Computing, University of Zagreb, Zagreb, CroatiaProduct recognition is a fine-grained image retrieval problem because grocery stores can contain several thousand products on their shelves, some of which have minimal visual differences. Given that the products on the store’s shelves change frequently, it would be practical to build a system using a single reference image or a few per product. This task is very challenging in computer vision, whereas humans solve it more effortlessly by relying on contextual information. Our work incorporates semantic and spatial context into a novel product recognition method. We take advantage of stores being organized into different sections and aisles, where similar products are placed nearby. First, we propose the Hierarchical Auxiliary Loss (HAL) for learning an organized feature space in which products from the same category, usually placed in the same store section, are close to each other. Second, we propose the Context-Aware Query Expansion (CAQE) module for the inference phase, in which each feature vector of a product on a shelf is expanded with the feature vectors of neighboring products. The amount of information exchanged between the two products depends on the similarity of their feature vectors and spatial distance on the store shelves. To demonstrate the effectiveness of our contributions, we conducted detailed experiments on publicly available grocery product datasets and showed that our method achieves state-of-the-art results.https://ieeexplore.ieee.org/document/10849567/Deep learningcontexthierarchical metric learningproduct recognitionquery expansion |
| spellingShingle | Lovre Antonio Budimir Zoran Kalafatic Marko Subasic Sven Loncaric Context-Aware Fine-Grained Product Recognition on Grocery Shelves IEEE Access Deep learning context hierarchical metric learning product recognition query expansion |
| title | Context-Aware Fine-Grained Product Recognition on Grocery Shelves |
| title_full | Context-Aware Fine-Grained Product Recognition on Grocery Shelves |
| title_fullStr | Context-Aware Fine-Grained Product Recognition on Grocery Shelves |
| title_full_unstemmed | Context-Aware Fine-Grained Product Recognition on Grocery Shelves |
| title_short | Context-Aware Fine-Grained Product Recognition on Grocery Shelves |
| title_sort | context aware fine grained product recognition on grocery shelves |
| topic | Deep learning context hierarchical metric learning product recognition query expansion |
| url | https://ieeexplore.ieee.org/document/10849567/ |
| work_keys_str_mv | AT lovreantoniobudimir contextawarefinegrainedproductrecognitionongroceryshelves AT zorankalafatic contextawarefinegrainedproductrecognitionongroceryshelves AT markosubasic contextawarefinegrainedproductrecognitionongroceryshelves AT svenloncaric contextawarefinegrainedproductrecognitionongroceryshelves |