A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward Function
Autonomous vehicles (AVs) are increasingly operating in complex traffic environments where safe and efficient decision-making is crucial. Merging into roundabouts is a key interaction scenario. This paper introduces a decision-making approach for roundabout merging that combines human driving behavi...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/13/6/912 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850091422292639744 |
|---|---|
| author | Qingyuan Shen Haobin Jiang Aoxue Li Shidian Ma |
| author_facet | Qingyuan Shen Haobin Jiang Aoxue Li Shidian Ma |
| author_sort | Qingyuan Shen |
| collection | DOAJ |
| description | Autonomous vehicles (AVs) are increasingly operating in complex traffic environments where safe and efficient decision-making is crucial. Merging into roundabouts is a key interaction scenario. This paper introduces a decision-making approach for roundabout merging that combines human driving behavior with advanced reinforcement learning (RL) techniques to enhance both safety and efficiency. The proposed framework models the decision-making process of AVs at roundabouts as a Markov decision process (MDP), optimizing the state, action, and reward spaces to more accurately reflect real-world driving behaviors. It simplifies the state space using relative distance and speed and defines three action profiles based on real traffic data to replicate human-like driving behavior. A force-based reward function, derived from constitutive relations, simulates vehicle-roundabout interactions, offering detailed, physically consistent feedback that enhances learning results. The results showed that this method effectively replicates human-like driving decisions, supporting the integration of AVs into dynamic traffic environments. Future research should address the challenges related to partial observability and further refine the state, action, and reward spaces. This research lays the groundwork for adaptive and interpretable decision-making frameworks for AVs, contributing to safer and more efficient traffic dynamics at roundabouts. |
| format | Article |
| id | doaj-art-ff2f34de29ef40538bd4f534bcad3a10 |
| institution | DOAJ |
| issn | 2227-7390 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj-art-ff2f34de29ef40538bd4f534bcad3a102025-08-20T02:42:22ZengMDPI AGMathematics2227-73902025-03-0113691210.3390/math13060912A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward FunctionQingyuan Shen0Haobin Jiang1Aoxue Li2Shidian Ma3School of Automotive and Traffic Engineering, Jiangsu University, Zhenjiang 212013, ChinaAutomotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, ChinaSchool of Automotive and Traffic Engineering, Jiangsu University, Zhenjiang 212013, ChinaAutomotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, ChinaAutonomous vehicles (AVs) are increasingly operating in complex traffic environments where safe and efficient decision-making is crucial. Merging into roundabouts is a key interaction scenario. This paper introduces a decision-making approach for roundabout merging that combines human driving behavior with advanced reinforcement learning (RL) techniques to enhance both safety and efficiency. The proposed framework models the decision-making process of AVs at roundabouts as a Markov decision process (MDP), optimizing the state, action, and reward spaces to more accurately reflect real-world driving behaviors. It simplifies the state space using relative distance and speed and defines three action profiles based on real traffic data to replicate human-like driving behavior. A force-based reward function, derived from constitutive relations, simulates vehicle-roundabout interactions, offering detailed, physically consistent feedback that enhances learning results. The results showed that this method effectively replicates human-like driving decisions, supporting the integration of AVs into dynamic traffic environments. Future research should address the challenges related to partial observability and further refine the state, action, and reward spaces. This research lays the groundwork for adaptive and interpretable decision-making frameworks for AVs, contributing to safer and more efficient traffic dynamics at roundabouts.https://www.mdpi.com/2227-7390/13/6/912Markov decision processdecision-makingautonomous vehiclesroundaboutmerging model |
| spellingShingle | Qingyuan Shen Haobin Jiang Aoxue Li Shidian Ma A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward Function Mathematics Markov decision process decision-making autonomous vehicles roundabout merging model |
| title | A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward Function |
| title_full | A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward Function |
| title_fullStr | A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward Function |
| title_full_unstemmed | A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward Function |
| title_short | A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward Function |
| title_sort | novel model for optimizing roundabout merging decisions based on markov decision process and force based reward function |
| topic | Markov decision process decision-making autonomous vehicles roundabout merging model |
| url | https://www.mdpi.com/2227-7390/13/6/912 |
| work_keys_str_mv | AT qingyuanshen anovelmodelforoptimizingroundaboutmergingdecisionsbasedonmarkovdecisionprocessandforcebasedrewardfunction AT haobinjiang anovelmodelforoptimizingroundaboutmergingdecisionsbasedonmarkovdecisionprocessandforcebasedrewardfunction AT aoxueli anovelmodelforoptimizingroundaboutmergingdecisionsbasedonmarkovdecisionprocessandforcebasedrewardfunction AT shidianma anovelmodelforoptimizingroundaboutmergingdecisionsbasedonmarkovdecisionprocessandforcebasedrewardfunction AT qingyuanshen novelmodelforoptimizingroundaboutmergingdecisionsbasedonmarkovdecisionprocessandforcebasedrewardfunction AT haobinjiang novelmodelforoptimizingroundaboutmergingdecisionsbasedonmarkovdecisionprocessandforcebasedrewardfunction AT aoxueli novelmodelforoptimizingroundaboutmergingdecisionsbasedonmarkovdecisionprocessandforcebasedrewardfunction AT shidianma novelmodelforoptimizingroundaboutmergingdecisionsbasedonmarkovdecisionprocessandforcebasedrewardfunction |