Layer wise Scaled Gaussian Priors for Markov Chain Monte Carlo Sampled deep Bayesian neural networks
Previous work has demonstrated that initialization is very important for both fitting a neural network by gradient descent methods, as well as for Variational inference of Bayesian neural networks. In this work we investigate how Layer wise Scaled Gaussian Priors perform with Markov Chain Monte Carl...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-04-01
|
| Series: | Frontiers in Artificial Intelligence |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/frai.2025.1444891/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850144991822741504 |
|---|---|
| author | Devesh Jawla John Kelleher |
| author_facet | Devesh Jawla John Kelleher |
| author_sort | Devesh Jawla |
| collection | DOAJ |
| description | Previous work has demonstrated that initialization is very important for both fitting a neural network by gradient descent methods, as well as for Variational inference of Bayesian neural networks. In this work we investigate how Layer wise Scaled Gaussian Priors perform with Markov Chain Monte Carlo trained Bayesian neural networks. From our experiments on 8 classifications datasets of various complexity, the results indicate that using Layer wise Scaled Gaussian Priors makes the sampling process more efficient as compared to using an Isotropic Gaussian Prior, an Isotropic Cauchy Prior, or an Isotropic Laplace Prior. We also show that the cold posterior effect does not arise when using a either an Isotropic Gaussian or a layer wise Scaled Prior for small feed forward Bayesian neural networks. Since Bayesian neural networks are becoming popular due to their advantages such as uncertainty estimation, and prevention of over-fitting, this work seeks to provide improvements in the efficiency of Bayesian neural networks learned using Markov Chain Monte Carlo methods. |
| format | Article |
| id | doaj-art-e66be50db0b849c38b10f52ebbc1f319 |
| institution | OA Journals |
| issn | 2624-8212 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Artificial Intelligence |
| spelling | doaj-art-e66be50db0b849c38b10f52ebbc1f3192025-08-20T02:28:11ZengFrontiers Media S.A.Frontiers in Artificial Intelligence2624-82122025-04-01810.3389/frai.2025.14448911444891Layer wise Scaled Gaussian Priors for Markov Chain Monte Carlo Sampled deep Bayesian neural networksDevesh Jawla0John Kelleher1School of Computer Science, Technological University Dublin, Dublin, IrelandADAPT Research Centre, School of Computer Science and Statistics, Trinity College Dublin, Dublin, IrelandPrevious work has demonstrated that initialization is very important for both fitting a neural network by gradient descent methods, as well as for Variational inference of Bayesian neural networks. In this work we investigate how Layer wise Scaled Gaussian Priors perform with Markov Chain Monte Carlo trained Bayesian neural networks. From our experiments on 8 classifications datasets of various complexity, the results indicate that using Layer wise Scaled Gaussian Priors makes the sampling process more efficient as compared to using an Isotropic Gaussian Prior, an Isotropic Cauchy Prior, or an Isotropic Laplace Prior. We also show that the cold posterior effect does not arise when using a either an Isotropic Gaussian or a layer wise Scaled Prior for small feed forward Bayesian neural networks. Since Bayesian neural networks are becoming popular due to their advantages such as uncertainty estimation, and prevention of over-fitting, this work seeks to provide improvements in the efficiency of Bayesian neural networks learned using Markov Chain Monte Carlo methods.https://www.frontiersin.org/articles/10.3389/frai.2025.1444891/fullBayesian neural network (BNN)Markov Chain Monte Carlo (MCMC)deep learning artificial intelligenceneural networkBayesian inference (BI) |
| spellingShingle | Devesh Jawla John Kelleher Layer wise Scaled Gaussian Priors for Markov Chain Monte Carlo Sampled deep Bayesian neural networks Frontiers in Artificial Intelligence Bayesian neural network (BNN) Markov Chain Monte Carlo (MCMC) deep learning artificial intelligence neural network Bayesian inference (BI) |
| title | Layer wise Scaled Gaussian Priors for Markov Chain Monte Carlo Sampled deep Bayesian neural networks |
| title_full | Layer wise Scaled Gaussian Priors for Markov Chain Monte Carlo Sampled deep Bayesian neural networks |
| title_fullStr | Layer wise Scaled Gaussian Priors for Markov Chain Monte Carlo Sampled deep Bayesian neural networks |
| title_full_unstemmed | Layer wise Scaled Gaussian Priors for Markov Chain Monte Carlo Sampled deep Bayesian neural networks |
| title_short | Layer wise Scaled Gaussian Priors for Markov Chain Monte Carlo Sampled deep Bayesian neural networks |
| title_sort | layer wise scaled gaussian priors for markov chain monte carlo sampled deep bayesian neural networks |
| topic | Bayesian neural network (BNN) Markov Chain Monte Carlo (MCMC) deep learning artificial intelligence neural network Bayesian inference (BI) |
| url | https://www.frontiersin.org/articles/10.3389/frai.2025.1444891/full |
| work_keys_str_mv | AT deveshjawla layerwisescaledgaussianpriorsformarkovchainmontecarlosampleddeepbayesianneuralnetworks AT johnkelleher layerwisescaledgaussianpriorsformarkovchainmontecarlosampleddeepbayesianneuralnetworks |