AutoML based workflow for design of experiments (DOE) selection and benchmarking data acquisition strategies with simulation models
Abstract Design of experiments (DOE) is an established method to allocate resources for efficient parameter space exploration. Model based active learning (AL) data sampling strategies have shown potential for further optimization. This paper introduces a workflow for conducting DOE comparative stud...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-12-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-024-83581-3 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841559567984492544 |
|---|---|
| author | Xukuan Xu Donghui Li Jinghou Bi Michael Moeckel |
| author_facet | Xukuan Xu Donghui Li Jinghou Bi Michael Moeckel |
| author_sort | Xukuan Xu |
| collection | DOAJ |
| description | Abstract Design of experiments (DOE) is an established method to allocate resources for efficient parameter space exploration. Model based active learning (AL) data sampling strategies have shown potential for further optimization. This paper introduces a workflow for conducting DOE comparative studies using automated machine learning. Based on a practical definition of model complexity in the context of machine learning, the interplay of systematic data generation and model performance is examined considering various sources of uncertainty: this includes uncertainties caused by stochastic sampling strategies, imprecise data, suboptimal modeling, and model evaluation. Results obtained from electrical circuit models with varying complexity show that not all AL sampling strategies outperform conventional DOE strategies, depending on the available data volume, the complexity of the dataset, and data uncertainties. Trade-offs in resource allocation strategies, in particular between identical replication of data points for statistical noise reduction and broad sampling for maximum parameter space exploration, and their impact on subsequent machine learning analysis are systematically investigated. Results indicate that replication oriented strategies should not be dismissed but may prove advantageous for cases with non-negligible noise impact and intermediate resource availability. The provided workflow can be used to simulate practical experimental conditions for DOE testing and DOE selection. |
| format | Article |
| id | doaj-art-6166b51004614799a27c16fe80bb7f52 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-6166b51004614799a27c16fe80bb7f522025-01-05T12:23:33ZengNature PortfolioScientific Reports2045-23222024-12-0114111310.1038/s41598-024-83581-3AutoML based workflow for design of experiments (DOE) selection and benchmarking data acquisition strategies with simulation modelsXukuan Xu0Donghui Li1Jinghou Bi2Michael Moeckel3Aschaffenburg University of Applied Sciences, Faculty of EngineeringAschaffenburg University of Applied Sciences, Faculty of EngineeringDresden University of Technology DE, Faculty of EngineeringAschaffenburg University of Applied Sciences, Faculty of EngineeringAbstract Design of experiments (DOE) is an established method to allocate resources for efficient parameter space exploration. Model based active learning (AL) data sampling strategies have shown potential for further optimization. This paper introduces a workflow for conducting DOE comparative studies using automated machine learning. Based on a practical definition of model complexity in the context of machine learning, the interplay of systematic data generation and model performance is examined considering various sources of uncertainty: this includes uncertainties caused by stochastic sampling strategies, imprecise data, suboptimal modeling, and model evaluation. Results obtained from electrical circuit models with varying complexity show that not all AL sampling strategies outperform conventional DOE strategies, depending on the available data volume, the complexity of the dataset, and data uncertainties. Trade-offs in resource allocation strategies, in particular between identical replication of data points for statistical noise reduction and broad sampling for maximum parameter space exploration, and their impact on subsequent machine learning analysis are systematically investigated. Results indicate that replication oriented strategies should not be dismissed but may prove advantageous for cases with non-negligible noise impact and intermediate resource availability. The provided workflow can be used to simulate practical experimental conditions for DOE testing and DOE selection.https://doi.org/10.1038/s41598-024-83581-3 |
| spellingShingle | Xukuan Xu Donghui Li Jinghou Bi Michael Moeckel AutoML based workflow for design of experiments (DOE) selection and benchmarking data acquisition strategies with simulation models Scientific Reports |
| title | AutoML based workflow for design of experiments (DOE) selection and benchmarking data acquisition strategies with simulation models |
| title_full | AutoML based workflow for design of experiments (DOE) selection and benchmarking data acquisition strategies with simulation models |
| title_fullStr | AutoML based workflow for design of experiments (DOE) selection and benchmarking data acquisition strategies with simulation models |
| title_full_unstemmed | AutoML based workflow for design of experiments (DOE) selection and benchmarking data acquisition strategies with simulation models |
| title_short | AutoML based workflow for design of experiments (DOE) selection and benchmarking data acquisition strategies with simulation models |
| title_sort | automl based workflow for design of experiments doe selection and benchmarking data acquisition strategies with simulation models |
| url | https://doi.org/10.1038/s41598-024-83581-3 |
| work_keys_str_mv | AT xukuanxu automlbasedworkflowfordesignofexperimentsdoeselectionandbenchmarkingdataacquisitionstrategieswithsimulationmodels AT donghuili automlbasedworkflowfordesignofexperimentsdoeselectionandbenchmarkingdataacquisitionstrategieswithsimulationmodels AT jinghoubi automlbasedworkflowfordesignofexperimentsdoeselectionandbenchmarkingdataacquisitionstrategieswithsimulationmodels AT michaelmoeckel automlbasedworkflowfordesignofexperimentsdoeselectionandbenchmarkingdataacquisitionstrategieswithsimulationmodels |