Spatio‐temporal dynamic navigation for electric vehicle charging using deep reinforcement learning
Abstract This paper considers the real‐time spatio‐temporal electric vehicle charging navigation problem in a dynamic environment by utilizing a shortest path‐based reinforcement learning approach. In a data sharing system including transportation network, an electric vehicle (EV) and EV charging st...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-12-01
|
| Series: | IET Intelligent Transport Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/itr2.12588 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850117167883747328 |
|---|---|
| author | Ali Can Erüst Fatma Yıldız Taşcıkaraoğlu |
| author_facet | Ali Can Erüst Fatma Yıldız Taşcıkaraoğlu |
| author_sort | Ali Can Erüst |
| collection | DOAJ |
| description | Abstract This paper considers the real‐time spatio‐temporal electric vehicle charging navigation problem in a dynamic environment by utilizing a shortest path‐based reinforcement learning approach. In a data sharing system including transportation network, an electric vehicle (EV) and EV charging stations (EVCSs), it is aimed to determine the most convenient EVCS and the optimal path for reducing the travel, charging and waiting costs. To estimate the waiting times at EVCSs, Gaussian process regression algorithm is integrated using a real‐time dataset comprising of state‐of‐charge and arrival‐departure times of EVs. The optimization problem is modelled as a Markov decision process with unknown transition probability to overcome the uncertainties arising from time‐varying variables. A recently proposed on‐policy actor–critic method, phasic policy gradient (PPG) which extends the proximal policy optimization algorithm with an auxiliary optimization phase to improve training by distilling features from the critic to the actor network, is used to make EVCS decisions on the network where EV travels through the optimal path from origin node to EVCS by considering dynamic traffic conditions, unit value of EV owner and time‐of‐use charging price. Three case studies are carried out for 24 nodes Sioux‐Falls benchmark network. It is shown that phasic policy gradient achieves an average of 9% better reward compared to proximal policy optimization and the total time decreases by 7–10% when EV owner cost is considered. |
| format | Article |
| id | doaj-art-1abdaeba10b34c2a9bc412a09b71aff0 |
| institution | OA Journals |
| issn | 1751-956X 1751-9578 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Intelligent Transport Systems |
| spelling | doaj-art-1abdaeba10b34c2a9bc412a09b71aff02025-08-20T02:36:09ZengWileyIET Intelligent Transport Systems1751-956X1751-95782024-12-0118122520253110.1049/itr2.12588Spatio‐temporal dynamic navigation for electric vehicle charging using deep reinforcement learningAli Can Erüst0Fatma Yıldız Taşcıkaraoğlu1Departmant of Electrical and Electronics Engineering Mugla Sitki Kocman University Mugla TürkiyeDepartmant of Electrical and Electronics Engineering Mugla Sitki Kocman University Mugla TürkiyeAbstract This paper considers the real‐time spatio‐temporal electric vehicle charging navigation problem in a dynamic environment by utilizing a shortest path‐based reinforcement learning approach. In a data sharing system including transportation network, an electric vehicle (EV) and EV charging stations (EVCSs), it is aimed to determine the most convenient EVCS and the optimal path for reducing the travel, charging and waiting costs. To estimate the waiting times at EVCSs, Gaussian process regression algorithm is integrated using a real‐time dataset comprising of state‐of‐charge and arrival‐departure times of EVs. The optimization problem is modelled as a Markov decision process with unknown transition probability to overcome the uncertainties arising from time‐varying variables. A recently proposed on‐policy actor–critic method, phasic policy gradient (PPG) which extends the proximal policy optimization algorithm with an auxiliary optimization phase to improve training by distilling features from the critic to the actor network, is used to make EVCS decisions on the network where EV travels through the optimal path from origin node to EVCS by considering dynamic traffic conditions, unit value of EV owner and time‐of‐use charging price. Three case studies are carried out for 24 nodes Sioux‐Falls benchmark network. It is shown that phasic policy gradient achieves an average of 9% better reward compared to proximal policy optimization and the total time decreases by 7–10% when EV owner cost is considered.https://doi.org/10.1049/itr2.12588electric vehicle chargingintelligent transportation systemslearning (artificial intelligence)intelligent transportation systems |
| spellingShingle | Ali Can Erüst Fatma Yıldız Taşcıkaraoğlu Spatio‐temporal dynamic navigation for electric vehicle charging using deep reinforcement learning IET Intelligent Transport Systems electric vehicle charging intelligent transportation systems learning (artificial intelligence) intelligent transportation systems |
| title | Spatio‐temporal dynamic navigation for electric vehicle charging using deep reinforcement learning |
| title_full | Spatio‐temporal dynamic navigation for electric vehicle charging using deep reinforcement learning |
| title_fullStr | Spatio‐temporal dynamic navigation for electric vehicle charging using deep reinforcement learning |
| title_full_unstemmed | Spatio‐temporal dynamic navigation for electric vehicle charging using deep reinforcement learning |
| title_short | Spatio‐temporal dynamic navigation for electric vehicle charging using deep reinforcement learning |
| title_sort | spatio temporal dynamic navigation for electric vehicle charging using deep reinforcement learning |
| topic | electric vehicle charging intelligent transportation systems learning (artificial intelligence) intelligent transportation systems |
| url | https://doi.org/10.1049/itr2.12588 |
| work_keys_str_mv | AT alicanerust spatiotemporaldynamicnavigationforelectricvehiclechargingusingdeepreinforcementlearning AT fatmayıldıztascıkaraoglu spatiotemporaldynamicnavigationforelectricvehiclechargingusingdeepreinforcementlearning |