A Simulation and Training Platform for Remote-Sighted Assistance
Remote-sighted assistance (RSA) is a technology designed to provide assistance for visually impaired people (VIPs). In this scene, a remote-sighted agent communicates and sends commands to navigate and assist VIPs via real-time video sent back. However, the latency in real-time video and the deviati...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/24/23/7773 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850261079730421760 |
|---|---|
| author | Xuantuo Huang Rong Zhang Yancheng Li Bingao Zhang Jianhua Zhang Jingjing Xu Shengyong Xu |
| author_facet | Xuantuo Huang Rong Zhang Yancheng Li Bingao Zhang Jianhua Zhang Jingjing Xu Shengyong Xu |
| author_sort | Xuantuo Huang |
| collection | DOAJ |
| description | Remote-sighted assistance (RSA) is a technology designed to provide assistance for visually impaired people (VIPs). In this scene, a remote-sighted agent communicates and sends commands to navigate and assist VIPs via real-time video sent back. However, the latency in real-time video and the deviation in the execution of instructions by VIPs are two important factors that affect the performance of agents to guide them. Therefore, how to enable agents to better guide VIPs under conditions of video transmission latency and deviation in instruction execution is an important issue. In this paper, we utilize Unreal Engine to create a virtual training platform for RSA, which simulates VIPs executing instructions in the real world and resembles the environment in RSA systems. We aim to help remote-sighted agents quickly master the set of vibration commands formed after encoding tactile vibrations and enable them to guide VIPs more effectively. Our experiment results show that, compared with untrained novices, when guiding people through the same path, agents trained on this platform reduce their average time by 32.09% and their average number of contacts with the environment by 57.57%. Our work provides agents with a simple and convenient simulation and training platform designed to enhance their performance by guiding VIPs with less travel time and fewer environmental contacts. Through this platform, agents can more effectively assist the visually impaired. |
| format | Article |
| id | doaj-art-72d4852803a64bc987919cd82c44e029 |
| institution | OA Journals |
| issn | 1424-8220 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-72d4852803a64bc987919cd82c44e0292025-08-20T01:55:31ZengMDPI AGSensors1424-82202024-12-012423777310.3390/s24237773A Simulation and Training Platform for Remote-Sighted AssistanceXuantuo Huang0Rong Zhang1Yancheng Li2Bingao Zhang3Jianhua Zhang4Jingjing Xu5Shengyong Xu6Key Laboratory for the Physics and Chemistry of Nanodevices, School of Electronics, Peking University, Beijing 100871, ChinaKey Laboratory for the Physics and Chemistry of Nanodevices, School of Electronics, Peking University, Beijing 100871, ChinaKey Laboratory for the Physics and Chemistry of Nanodevices, School of Electronics, Peking University, Beijing 100871, ChinaKey Laboratory for the Physics and Chemistry of Nanodevices, School of Electronics, Peking University, Beijing 100871, ChinaSchool of Computer Science and Engineering, Tianjin University of Technology, Tianjin 300384, ChinaSchool of Integrated Circuits, Shandong University, Jinan 250100, ChinaKey Laboratory for the Physics and Chemistry of Nanodevices, School of Electronics, Peking University, Beijing 100871, ChinaRemote-sighted assistance (RSA) is a technology designed to provide assistance for visually impaired people (VIPs). In this scene, a remote-sighted agent communicates and sends commands to navigate and assist VIPs via real-time video sent back. However, the latency in real-time video and the deviation in the execution of instructions by VIPs are two important factors that affect the performance of agents to guide them. Therefore, how to enable agents to better guide VIPs under conditions of video transmission latency and deviation in instruction execution is an important issue. In this paper, we utilize Unreal Engine to create a virtual training platform for RSA, which simulates VIPs executing instructions in the real world and resembles the environment in RSA systems. We aim to help remote-sighted agents quickly master the set of vibration commands formed after encoding tactile vibrations and enable them to guide VIPs more effectively. Our experiment results show that, compared with untrained novices, when guiding people through the same path, agents trained on this platform reduce their average time by 32.09% and their average number of contacts with the environment by 57.57%. Our work provides agents with a simple and convenient simulation and training platform designed to enhance their performance by guiding VIPs with less travel time and fewer environmental contacts. Through this platform, agents can more effectively assist the visually impaired.https://www.mdpi.com/1424-8220/24/23/7773assistive technologyhuman–computer interactionremote-sighted assistancesimulation and trainingvisually impaired people |
| spellingShingle | Xuantuo Huang Rong Zhang Yancheng Li Bingao Zhang Jianhua Zhang Jingjing Xu Shengyong Xu A Simulation and Training Platform for Remote-Sighted Assistance Sensors assistive technology human–computer interaction remote-sighted assistance simulation and training visually impaired people |
| title | A Simulation and Training Platform for Remote-Sighted Assistance |
| title_full | A Simulation and Training Platform for Remote-Sighted Assistance |
| title_fullStr | A Simulation and Training Platform for Remote-Sighted Assistance |
| title_full_unstemmed | A Simulation and Training Platform for Remote-Sighted Assistance |
| title_short | A Simulation and Training Platform for Remote-Sighted Assistance |
| title_sort | simulation and training platform for remote sighted assistance |
| topic | assistive technology human–computer interaction remote-sighted assistance simulation and training visually impaired people |
| url | https://www.mdpi.com/1424-8220/24/23/7773 |
| work_keys_str_mv | AT xuantuohuang asimulationandtrainingplatformforremotesightedassistance AT rongzhang asimulationandtrainingplatformforremotesightedassistance AT yanchengli asimulationandtrainingplatformforremotesightedassistance AT bingaozhang asimulationandtrainingplatformforremotesightedassistance AT jianhuazhang asimulationandtrainingplatformforremotesightedassistance AT jingjingxu asimulationandtrainingplatformforremotesightedassistance AT shengyongxu asimulationandtrainingplatformforremotesightedassistance AT xuantuohuang simulationandtrainingplatformforremotesightedassistance AT rongzhang simulationandtrainingplatformforremotesightedassistance AT yanchengli simulationandtrainingplatformforremotesightedassistance AT bingaozhang simulationandtrainingplatformforremotesightedassistance AT jianhuazhang simulationandtrainingplatformforremotesightedassistance AT jingjingxu simulationandtrainingplatformforremotesightedassistance AT shengyongxu simulationandtrainingplatformforremotesightedassistance |