Accelerated Bayesian optimization for CNN+LSTM learning rate tuning via precomputed Gaussian process subspaces in soil analysis
PurposeWe propose an accelerated Bayesian optimization framework for tuning the learning rate of CNN+LSTM models in soil analysis, addressing the computational inefficiency of traditional Gaussian Process (GP)-based methods. This work bridges the gap between computational efficiency and probabilisti...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-08-01
|
| Series: | Frontiers in Environmental Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fenvs.2025.1633046/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849413424363077632 |
|---|---|
| author | Xiaolong Chen Hongfeng Zhang Cora Un In Wong Zhengchun Song |
| author_facet | Xiaolong Chen Hongfeng Zhang Cora Un In Wong Zhengchun Song |
| author_sort | Xiaolong Chen |
| collection | DOAJ |
| description | PurposeWe propose an accelerated Bayesian optimization framework for tuning the learning rate of CNN+LSTM models in soil analysis, addressing the computational inefficiency of traditional Gaussian Process (GP)-based methods. This work bridges the gap between computational efficiency and probabilistic robustness, with broader implications for automated machine learning in geoscientific applications.MethodThe key innovation lies in a subspace-accelerated GP surrogate model that precomputes low-rank approximations of covariance matrices offline, thereby decoupling the costly hyperparameter tuning from the online acquisition function evaluations. By projecting the hyperparameter search space onto a dominant subspace derived from Nyström approximations, our method reduces the computational complexity from cubic to linear in the number of observations. The proposed system integrates seamlessly with existing CNN+LSTM pipelines, where the offline phase constructs the GP subspace using historical or synthetic data, while the online phase iteratively updates the subspace with rank-1 modifications. Moreover, the method’s adaptability to non-stationary response surfaces, facilitated by a Matérn-5/2 kernel with automatic relevance determination, makes it particularly suitable for soil data exhibiting multi-scale features.ResultsEmpirical validation on soil spectral datasets demonstrates a 3–5× speedup in convergence compared to standard Bayesian optimization, with no loss in model accuracy. Experiments on soil spectral datasets show convergence in 23.4 min (3.8× faster than standard Bayesian optimization) with a test RMSE of 0.142, while maintaining equivalent accuracy across diverse CNN+LSTM architectures.ConclusionThe reformulated approach not only overcomes the scalability limitations of conventional GP-based optimization but also preserves its theoretical guarantees, offering a practical solution for hyperparameter tuning in resource-constrained environments. |
| format | Article |
| id | doaj-art-47e706c7733745c1beb980321cb36ced |
| institution | Kabale University |
| issn | 2296-665X |
| language | English |
| publishDate | 2025-08-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Environmental Science |
| spelling | doaj-art-47e706c7733745c1beb980321cb36ced2025-08-20T03:34:08ZengFrontiers Media S.A.Frontiers in Environmental Science2296-665X2025-08-011310.3389/fenvs.2025.16330461633046Accelerated Bayesian optimization for CNN+LSTM learning rate tuning via precomputed Gaussian process subspaces in soil analysisXiaolong ChenHongfeng ZhangCora Un In WongZhengchun SongPurposeWe propose an accelerated Bayesian optimization framework for tuning the learning rate of CNN+LSTM models in soil analysis, addressing the computational inefficiency of traditional Gaussian Process (GP)-based methods. This work bridges the gap between computational efficiency and probabilistic robustness, with broader implications for automated machine learning in geoscientific applications.MethodThe key innovation lies in a subspace-accelerated GP surrogate model that precomputes low-rank approximations of covariance matrices offline, thereby decoupling the costly hyperparameter tuning from the online acquisition function evaluations. By projecting the hyperparameter search space onto a dominant subspace derived from Nyström approximations, our method reduces the computational complexity from cubic to linear in the number of observations. The proposed system integrates seamlessly with existing CNN+LSTM pipelines, where the offline phase constructs the GP subspace using historical or synthetic data, while the online phase iteratively updates the subspace with rank-1 modifications. Moreover, the method’s adaptability to non-stationary response surfaces, facilitated by a Matérn-5/2 kernel with automatic relevance determination, makes it particularly suitable for soil data exhibiting multi-scale features.ResultsEmpirical validation on soil spectral datasets demonstrates a 3–5× speedup in convergence compared to standard Bayesian optimization, with no loss in model accuracy. Experiments on soil spectral datasets show convergence in 23.4 min (3.8× faster than standard Bayesian optimization) with a test RMSE of 0.142, while maintaining equivalent accuracy across diverse CNN+LSTM architectures.ConclusionThe reformulated approach not only overcomes the scalability limitations of conventional GP-based optimization but also preserves its theoretical guarantees, offering a practical solution for hyperparameter tuning in resource-constrained environments.https://www.frontiersin.org/articles/10.3389/fenvs.2025.1633046/fullBayesian optimizationCNN+LSTMsoil analysisGaussian processcomputational efficiencyhyperparameter tuning |
| spellingShingle | Xiaolong Chen Hongfeng Zhang Cora Un In Wong Zhengchun Song Accelerated Bayesian optimization for CNN+LSTM learning rate tuning via precomputed Gaussian process subspaces in soil analysis Frontiers in Environmental Science Bayesian optimization CNN+LSTM soil analysis Gaussian process computational efficiency hyperparameter tuning |
| title | Accelerated Bayesian optimization for CNN+LSTM learning rate tuning via precomputed Gaussian process subspaces in soil analysis |
| title_full | Accelerated Bayesian optimization for CNN+LSTM learning rate tuning via precomputed Gaussian process subspaces in soil analysis |
| title_fullStr | Accelerated Bayesian optimization for CNN+LSTM learning rate tuning via precomputed Gaussian process subspaces in soil analysis |
| title_full_unstemmed | Accelerated Bayesian optimization for CNN+LSTM learning rate tuning via precomputed Gaussian process subspaces in soil analysis |
| title_short | Accelerated Bayesian optimization for CNN+LSTM learning rate tuning via precomputed Gaussian process subspaces in soil analysis |
| title_sort | accelerated bayesian optimization for cnn lstm learning rate tuning via precomputed gaussian process subspaces in soil analysis |
| topic | Bayesian optimization CNN+LSTM soil analysis Gaussian process computational efficiency hyperparameter tuning |
| url | https://www.frontiersin.org/articles/10.3389/fenvs.2025.1633046/full |
| work_keys_str_mv | AT xiaolongchen acceleratedbayesianoptimizationforcnnlstmlearningratetuningviaprecomputedgaussianprocesssubspacesinsoilanalysis AT hongfengzhang acceleratedbayesianoptimizationforcnnlstmlearningratetuningviaprecomputedgaussianprocesssubspacesinsoilanalysis AT corauninwong acceleratedbayesianoptimizationforcnnlstmlearningratetuningviaprecomputedgaussianprocesssubspacesinsoilanalysis AT zhengchunsong acceleratedbayesianoptimizationforcnnlstmlearningratetuningviaprecomputedgaussianprocesssubspacesinsoilanalysis |