Efficient Stereo Visual Odometry on FPGA Featuring On-Chip Map Management and Pipelined Descriptor-Based Block Matching
Due to its ability to capture natural light without emitting additional signals, a stereo camera is a promising option for low-power visual odometry (VO). However, achieving real-time processing of VO using a stereo camera on embedded CPUs and GPUs is challenging due to the high computational demand...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10689404/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850236480325156864 |
|---|---|
| author | Yuki Ichikawa Kazushi Kawamura Masato Motomura Thiem van Chu |
| author_facet | Yuki Ichikawa Kazushi Kawamura Masato Motomura Thiem van Chu |
| author_sort | Yuki Ichikawa |
| collection | DOAJ |
| description | Due to its ability to capture natural light without emitting additional signals, a stereo camera is a promising option for low-power visual odometry (VO). However, achieving real-time processing of VO using a stereo camera on embedded CPUs and GPUs is challenging due to the high computational demands and irregular computational patterns. These irregular patterns arise from the dynamic and non-uniform nature of feature detection, stereo matching, and map management, which vary significantly with scene complexity and camera movement. Various hardware accelerators have been proposed to address this challenge, but they typically tackle individual aspects of stereo VO, leading to difficulties in combining them efficiently. This paper introduces a comprehensive stereo VO accelerator implemented on an AMD Kria KV260 FPGA board, aimed at improving processing efficiency and reducing energy consumption. Our accelerator features innovative techniques including on-FPGA map management and pipelined stereo matching with the use of descriptors instead of image patches. These advancements result in significant reduction in off-chip data transfer, memory savings, and faster processing times. Additionally, an adaptive bucketing-based feature selection method is employed to enhance system accuracy without significant increases in hardware resource usage. Evaluation using the KITTI dataset shows that our accelerator achieves speedups of up to <inline-formula> <tex-math notation="LaTeX">$3.08\times $ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$2.75\times $ </tex-math></inline-formula> over CPU-only and CPU+GPU implementations of LVT, an optimized algorithm, on an NVIDIA Jetson Nano developer kit B01 4GB, with corresponding energy efficiency improvements of <inline-formula> <tex-math notation="LaTeX">$3.55\times $ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$1.95\times $ </tex-math></inline-formula>. The results demonstrate the accelerator’s effectiveness for real-time applications, highlighting the benefits of FPGA-based solutions in complex visual processing tasks. |
| format | Article |
| id | doaj-art-20fa3052c72f402f8deae676f7e4ea87 |
| institution | OA Journals |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-20fa3052c72f402f8deae676f7e4ea872025-08-20T02:01:57ZengIEEEIEEE Access2169-35362024-01-011217145817147110.1109/ACCESS.2024.346623810689404Efficient Stereo Visual Odometry on FPGA Featuring On-Chip Map Management and Pipelined Descriptor-Based Block MatchingYuki Ichikawa0https://orcid.org/0009-0002-9293-1701Kazushi Kawamura1https://orcid.org/0000-0002-0795-2974Masato Motomura2https://orcid.org/0000-0003-1543-1252Thiem van Chu3https://orcid.org/0000-0002-2003-2574Tokyo Institute of Technology, Yokohama, Kanagawa, JapanTokyo Institute of Technology, Yokohama, Kanagawa, JapanTokyo Institute of Technology, Yokohama, Kanagawa, JapanTokyo Institute of Technology, Yokohama, Kanagawa, JapanDue to its ability to capture natural light without emitting additional signals, a stereo camera is a promising option for low-power visual odometry (VO). However, achieving real-time processing of VO using a stereo camera on embedded CPUs and GPUs is challenging due to the high computational demands and irregular computational patterns. These irregular patterns arise from the dynamic and non-uniform nature of feature detection, stereo matching, and map management, which vary significantly with scene complexity and camera movement. Various hardware accelerators have been proposed to address this challenge, but they typically tackle individual aspects of stereo VO, leading to difficulties in combining them efficiently. This paper introduces a comprehensive stereo VO accelerator implemented on an AMD Kria KV260 FPGA board, aimed at improving processing efficiency and reducing energy consumption. Our accelerator features innovative techniques including on-FPGA map management and pipelined stereo matching with the use of descriptors instead of image patches. These advancements result in significant reduction in off-chip data transfer, memory savings, and faster processing times. Additionally, an adaptive bucketing-based feature selection method is employed to enhance system accuracy without significant increases in hardware resource usage. Evaluation using the KITTI dataset shows that our accelerator achieves speedups of up to <inline-formula> <tex-math notation="LaTeX">$3.08\times $ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$2.75\times $ </tex-math></inline-formula> over CPU-only and CPU+GPU implementations of LVT, an optimized algorithm, on an NVIDIA Jetson Nano developer kit B01 4GB, with corresponding energy efficiency improvements of <inline-formula> <tex-math notation="LaTeX">$3.55\times $ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$1.95\times $ </tex-math></inline-formula>. The results demonstrate the accelerator’s effectiveness for real-time applications, highlighting the benefits of FPGA-based solutions in complex visual processing tasks.https://ieeexplore.ieee.org/document/10689404/Visual odometrystereo matchingORBFPGA |
| spellingShingle | Yuki Ichikawa Kazushi Kawamura Masato Motomura Thiem van Chu Efficient Stereo Visual Odometry on FPGA Featuring On-Chip Map Management and Pipelined Descriptor-Based Block Matching IEEE Access Visual odometry stereo matching ORB FPGA |
| title | Efficient Stereo Visual Odometry on FPGA Featuring On-Chip Map Management and Pipelined Descriptor-Based Block Matching |
| title_full | Efficient Stereo Visual Odometry on FPGA Featuring On-Chip Map Management and Pipelined Descriptor-Based Block Matching |
| title_fullStr | Efficient Stereo Visual Odometry on FPGA Featuring On-Chip Map Management and Pipelined Descriptor-Based Block Matching |
| title_full_unstemmed | Efficient Stereo Visual Odometry on FPGA Featuring On-Chip Map Management and Pipelined Descriptor-Based Block Matching |
| title_short | Efficient Stereo Visual Odometry on FPGA Featuring On-Chip Map Management and Pipelined Descriptor-Based Block Matching |
| title_sort | efficient stereo visual odometry on fpga featuring on chip map management and pipelined descriptor based block matching |
| topic | Visual odometry stereo matching ORB FPGA |
| url | https://ieeexplore.ieee.org/document/10689404/ |
| work_keys_str_mv | AT yukiichikawa efficientstereovisualodometryonfpgafeaturingonchipmapmanagementandpipelineddescriptorbasedblockmatching AT kazushikawamura efficientstereovisualodometryonfpgafeaturingonchipmapmanagementandpipelineddescriptorbasedblockmatching AT masatomotomura efficientstereovisualodometryonfpgafeaturingonchipmapmanagementandpipelineddescriptorbasedblockmatching AT thiemvanchu efficientstereovisualodometryonfpgafeaturingonchipmapmanagementandpipelineddescriptorbasedblockmatching |