Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks
The paper presents the GeNNsem (Genetic algorithm ANNs ensemble) software framework for the simultaneous optimization of individual neural networks and building their optimal ensemble. The proposed framework employs a genetic algorithm to search for suitable architectures and hyperparameters of the...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-12-01
|
| Series: | Egyptian Informatics Journal |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1110866524001440 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850119400834727936 |
|---|---|
| author | Lazar Krstic Milos Ivanovic Visnja Simic Boban Stojanovic |
| author_facet | Lazar Krstic Milos Ivanovic Visnja Simic Boban Stojanovic |
| author_sort | Lazar Krstic |
| collection | DOAJ |
| description | The paper presents the GeNNsem (Genetic algorithm ANNs ensemble) software framework for the simultaneous optimization of individual neural networks and building their optimal ensemble. The proposed framework employs a genetic algorithm to search for suitable architectures and hyperparameters of the individual neural networks to maximize the weighted sum of accuracy and diversity in their predictions. The optimal ensemble consists of networks with low errors but diverse predictions, resulting in a more generalized model. The scalability of the proposed framework is ensured by utilizing micro-services and Kubernetes batching orchestration. GeNNsem has been evaluated on two regression benchmark problems and compared with related machine learning techniques. The proposed approach exhibited supremacy over other ensemble approaches and individual neural networks in all common regression modeling metrics. Real-world use-case experiments in the domain of hydro-informatics have further demonstrated the main advantages of GeNNsem: requires the least training sessions for individual models when optimizing an ensemble; networks in an ensemble are generally simple due to the regularization provided by a trivial initial population and custom genetic operators; execution times are reduced by two orders of magnitude as a result of parallelization. |
| format | Article |
| id | doaj-art-4b73c628aa5d4e8a84a1d84a41bf6a07 |
| institution | OA Journals |
| issn | 1110-8665 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Egyptian Informatics Journal |
| spelling | doaj-art-4b73c628aa5d4e8a84a1d84a41bf6a072025-08-20T02:35:39ZengElsevierEgyptian Informatics Journal1110-86652024-12-012810058110.1016/j.eij.2024.100581Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networksLazar Krstic0Milos Ivanovic1Visnja Simic2Boban Stojanovic3Corresponding author.; University of Kragujevac, Faculty of Science, 12 Radoja Domanovica Street, 34000 Kragujevac, SerbiaUniversity of Kragujevac, Faculty of Science, 12 Radoja Domanovica Street, 34000 Kragujevac, SerbiaUniversity of Kragujevac, Faculty of Science, 12 Radoja Domanovica Street, 34000 Kragujevac, SerbiaUniversity of Kragujevac, Faculty of Science, 12 Radoja Domanovica Street, 34000 Kragujevac, SerbiaThe paper presents the GeNNsem (Genetic algorithm ANNs ensemble) software framework for the simultaneous optimization of individual neural networks and building their optimal ensemble. The proposed framework employs a genetic algorithm to search for suitable architectures and hyperparameters of the individual neural networks to maximize the weighted sum of accuracy and diversity in their predictions. The optimal ensemble consists of networks with low errors but diverse predictions, resulting in a more generalized model. The scalability of the proposed framework is ensured by utilizing micro-services and Kubernetes batching orchestration. GeNNsem has been evaluated on two regression benchmark problems and compared with related machine learning techniques. The proposed approach exhibited supremacy over other ensemble approaches and individual neural networks in all common regression modeling metrics. Real-world use-case experiments in the domain of hydro-informatics have further demonstrated the main advantages of GeNNsem: requires the least training sessions for individual models when optimizing an ensemble; networks in an ensemble are generally simple due to the regularization provided by a trivial initial population and custom genetic operators; execution times are reduced by two orders of magnitude as a result of parallelization.http://www.sciencedirect.com/science/article/pii/S1110866524001440Ensemble modelingRegressionANNGenetic algorithmDistributed computing |
| spellingShingle | Lazar Krstic Milos Ivanovic Visnja Simic Boban Stojanovic Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks Egyptian Informatics Journal Ensemble modeling Regression ANN Genetic algorithm Distributed computing |
| title | Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks |
| title_full | Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks |
| title_fullStr | Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks |
| title_full_unstemmed | Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks |
| title_short | Evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks |
| title_sort | evolutionary approach for composing a thoroughly optimized ensemble of regression neural networks |
| topic | Ensemble modeling Regression ANN Genetic algorithm Distributed computing |
| url | http://www.sciencedirect.com/science/article/pii/S1110866524001440 |
| work_keys_str_mv | AT lazarkrstic evolutionaryapproachforcomposingathoroughlyoptimizedensembleofregressionneuralnetworks AT milosivanovic evolutionaryapproachforcomposingathoroughlyoptimizedensembleofregressionneuralnetworks AT visnjasimic evolutionaryapproachforcomposingathoroughlyoptimizedensembleofregressionneuralnetworks AT bobanstojanovic evolutionaryapproachforcomposingathoroughlyoptimizedensembleofregressionneuralnetworks |