Exploration-Driven Genetic Algorithms for Hyperparameter Optimisation in Deep Reinforcement Learning
This paper investigates the application of genetic algorithms (GAs) for hyperparameter optimisation in deep reinforcement learning (RL), focusing on the Deep Q-Learning (DQN) algorithm. This study aims to identify approaches that enhance RL model performance through the effective exploration of the...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-02-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/4/2067 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849718864933289984 |
|---|---|
| author | Bartłomiej Brzęk Barbara Probierz Jan Kozak |
| author_facet | Bartłomiej Brzęk Barbara Probierz Jan Kozak |
| author_sort | Bartłomiej Brzęk |
| collection | DOAJ |
| description | This paper investigates the application of genetic algorithms (GAs) for hyperparameter optimisation in deep reinforcement learning (RL), focusing on the Deep Q-Learning (DQN) algorithm. This study aims to identify approaches that enhance RL model performance through the effective exploration of the configuration space. By comparing different GA methods for selection, crossover, and mutation, this study focuses on deep RL models. The results indicate that GA techniques emphasising the exploration of the configuration space yield significant improvements in optimisation efficiency, reducing training time and enhancing convergence. The most effective GA improved the fitness function value from 68.26 (initial best chromosome) to 979.16 after 200 iterations, demonstrating the efficacy of the proposed approach. Furthermore, variations in specific hyperparameters, such as learning rate, gamma, and update frequency, were shown to substantially affect the DQN model’s learning ability. These findings suggest that exploration-driven GA strategies outperform GA approaches with limited exploration, underscoring the critical role of selection and crossover methods in enhancing DQN model efficiency and performance. Moreover, a mini case study on the CartPole environment revealed that even a 5% sensor dropout impaired the performance of a GA-optimised RL agent, while a 20% dropout almost entirely halted improvements. |
| format | Article |
| id | doaj-art-39c0ddea08df4adb9d4b1fa749e5d63b |
| institution | DOAJ |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-39c0ddea08df4adb9d4b1fa749e5d63b2025-08-20T03:12:16ZengMDPI AGApplied Sciences2076-34172025-02-01154206710.3390/app15042067Exploration-Driven Genetic Algorithms for Hyperparameter Optimisation in Deep Reinforcement LearningBartłomiej Brzęk0Barbara Probierz1Jan Kozak2Department of Machine Learning, University of Economics in Katowice, 1 Maja 50, 40-287 Katowice, PolandDepartment of Machine Learning, University of Economics in Katowice, 1 Maja 50, 40-287 Katowice, PolandDepartment of Machine Learning, University of Economics in Katowice, 1 Maja 50, 40-287 Katowice, PolandThis paper investigates the application of genetic algorithms (GAs) for hyperparameter optimisation in deep reinforcement learning (RL), focusing on the Deep Q-Learning (DQN) algorithm. This study aims to identify approaches that enhance RL model performance through the effective exploration of the configuration space. By comparing different GA methods for selection, crossover, and mutation, this study focuses on deep RL models. The results indicate that GA techniques emphasising the exploration of the configuration space yield significant improvements in optimisation efficiency, reducing training time and enhancing convergence. The most effective GA improved the fitness function value from 68.26 (initial best chromosome) to 979.16 after 200 iterations, demonstrating the efficacy of the proposed approach. Furthermore, variations in specific hyperparameters, such as learning rate, gamma, and update frequency, were shown to substantially affect the DQN model’s learning ability. These findings suggest that exploration-driven GA strategies outperform GA approaches with limited exploration, underscoring the critical role of selection and crossover methods in enhancing DQN model efficiency and performance. Moreover, a mini case study on the CartPole environment revealed that even a 5% sensor dropout impaired the performance of a GA-optimised RL agent, while a 20% dropout almost entirely halted improvements.https://www.mdpi.com/2076-3417/15/4/2067genetic algorithmreinforcement learninghyperparameteroptimisation |
| spellingShingle | Bartłomiej Brzęk Barbara Probierz Jan Kozak Exploration-Driven Genetic Algorithms for Hyperparameter Optimisation in Deep Reinforcement Learning Applied Sciences genetic algorithm reinforcement learning hyperparameter optimisation |
| title | Exploration-Driven Genetic Algorithms for Hyperparameter Optimisation in Deep Reinforcement Learning |
| title_full | Exploration-Driven Genetic Algorithms for Hyperparameter Optimisation in Deep Reinforcement Learning |
| title_fullStr | Exploration-Driven Genetic Algorithms for Hyperparameter Optimisation in Deep Reinforcement Learning |
| title_full_unstemmed | Exploration-Driven Genetic Algorithms for Hyperparameter Optimisation in Deep Reinforcement Learning |
| title_short | Exploration-Driven Genetic Algorithms for Hyperparameter Optimisation in Deep Reinforcement Learning |
| title_sort | exploration driven genetic algorithms for hyperparameter optimisation in deep reinforcement learning |
| topic | genetic algorithm reinforcement learning hyperparameter optimisation |
| url | https://www.mdpi.com/2076-3417/15/4/2067 |
| work_keys_str_mv | AT bartłomiejbrzek explorationdrivengeneticalgorithmsforhyperparameteroptimisationindeepreinforcementlearning AT barbaraprobierz explorationdrivengeneticalgorithmsforhyperparameteroptimisationindeepreinforcementlearning AT jankozak explorationdrivengeneticalgorithmsforhyperparameteroptimisationindeepreinforcementlearning |