Rebalancing Docked Bicycle Sharing System with Approximate Dynamic Programming and Reinforcement Learning
The bicycle, an active transportation mode, has received increasing attention as an alternative in urban environments worldwide. However, effectively managing the stock levels of rental bicycles at each station is challenging as demand levels vary with time, particularly when users are allowed to re...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-01-01
|
| Series: | Journal of Advanced Transportation |
| Online Access: | http://dx.doi.org/10.1155/2022/2780711 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849395940085989376 |
|---|---|
| author | Young-Hyun Seo Dong-Kyu Kim Seungmo Kang Young-Ji Byon Seung-Young Kho |
| author_facet | Young-Hyun Seo Dong-Kyu Kim Seungmo Kang Young-Ji Byon Seung-Young Kho |
| author_sort | Young-Hyun Seo |
| collection | DOAJ |
| description | The bicycle, an active transportation mode, has received increasing attention as an alternative in urban environments worldwide. However, effectively managing the stock levels of rental bicycles at each station is challenging as demand levels vary with time, particularly when users are allowed to return bicycles at any station. There is a need for system-wide management of bicycle stock levels by transporting available bicycles from one station to another. In this study, a bicycle rebalancing model based on a Markov decision process (MDP) is developed using a real-time dynamic programming method and reinforcement learning considering dynamic system characteristics. The pickup and return demands are stochastic and continuously changing. As a result, the proposed framework suggests the best operation option every 10 min based on the realized system variables and future demands predicted by the random forest method, minimizing the expected unmet demand. Moreover, we adopt custom prioritizing strategies to reduce the number of action candidates for the operator and the computational complexity for practicality in the MDP framework. Numerical experiments demonstrate that the proposed model outperforms existing methods, such as short-term rebalancing and static lookahead policies. Among the suggested prioritizing strategies, focusing on stations with a larger error in demand prediction was found to be the most effective. Additionally, the effects of various safety buffers were examined. |
| format | Article |
| id | doaj-art-b58c49742e4e4aeb977abe96ef436ba9 |
| institution | Kabale University |
| issn | 2042-3195 |
| language | English |
| publishDate | 2022-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Journal of Advanced Transportation |
| spelling | doaj-art-b58c49742e4e4aeb977abe96ef436ba92025-08-20T03:39:28ZengWileyJournal of Advanced Transportation2042-31952022-01-01202210.1155/2022/2780711Rebalancing Docked Bicycle Sharing System with Approximate Dynamic Programming and Reinforcement LearningYoung-Hyun Seo0Dong-Kyu Kim1Seungmo Kang2Young-Ji Byon3Seung-Young Kho4School of CivilDepartment of Civil and Environmental EngineeringSchool of CivilDepartment of Civil Infrastructure and Environmental EngineeringDepartment of Civil and Environmental EngineeringThe bicycle, an active transportation mode, has received increasing attention as an alternative in urban environments worldwide. However, effectively managing the stock levels of rental bicycles at each station is challenging as demand levels vary with time, particularly when users are allowed to return bicycles at any station. There is a need for system-wide management of bicycle stock levels by transporting available bicycles from one station to another. In this study, a bicycle rebalancing model based on a Markov decision process (MDP) is developed using a real-time dynamic programming method and reinforcement learning considering dynamic system characteristics. The pickup and return demands are stochastic and continuously changing. As a result, the proposed framework suggests the best operation option every 10 min based on the realized system variables and future demands predicted by the random forest method, minimizing the expected unmet demand. Moreover, we adopt custom prioritizing strategies to reduce the number of action candidates for the operator and the computational complexity for practicality in the MDP framework. Numerical experiments demonstrate that the proposed model outperforms existing methods, such as short-term rebalancing and static lookahead policies. Among the suggested prioritizing strategies, focusing on stations with a larger error in demand prediction was found to be the most effective. Additionally, the effects of various safety buffers were examined.http://dx.doi.org/10.1155/2022/2780711 |
| spellingShingle | Young-Hyun Seo Dong-Kyu Kim Seungmo Kang Young-Ji Byon Seung-Young Kho Rebalancing Docked Bicycle Sharing System with Approximate Dynamic Programming and Reinforcement Learning Journal of Advanced Transportation |
| title | Rebalancing Docked Bicycle Sharing System with Approximate Dynamic Programming and Reinforcement Learning |
| title_full | Rebalancing Docked Bicycle Sharing System with Approximate Dynamic Programming and Reinforcement Learning |
| title_fullStr | Rebalancing Docked Bicycle Sharing System with Approximate Dynamic Programming and Reinforcement Learning |
| title_full_unstemmed | Rebalancing Docked Bicycle Sharing System with Approximate Dynamic Programming and Reinforcement Learning |
| title_short | Rebalancing Docked Bicycle Sharing System with Approximate Dynamic Programming and Reinforcement Learning |
| title_sort | rebalancing docked bicycle sharing system with approximate dynamic programming and reinforcement learning |
| url | http://dx.doi.org/10.1155/2022/2780711 |
| work_keys_str_mv | AT younghyunseo rebalancingdockedbicyclesharingsystemwithapproximatedynamicprogrammingandreinforcementlearning AT dongkyukim rebalancingdockedbicyclesharingsystemwithapproximatedynamicprogrammingandreinforcementlearning AT seungmokang rebalancingdockedbicyclesharingsystemwithapproximatedynamicprogrammingandreinforcementlearning AT youngjibyon rebalancingdockedbicyclesharingsystemwithapproximatedynamicprogrammingandreinforcementlearning AT seungyoungkho rebalancingdockedbicyclesharingsystemwithapproximatedynamicprogrammingandreinforcementlearning |