Nonlinear Regression Modeling Using Bayesian Neural Networks and Comparative Analysis of Sampling and Variational Inference

Karimi, Omid; Hosseini, Fatemeh

doi:10.22034/aa.2026.2084110.1027

Nonlinear Regression Modeling Using Bayesian Neural Networks and Comparative Analysis of Sampling and Variational Inference

Document Type : Original Article

Authors

Omid Karimi ¹

Fatemeh Hosseini ²

¹ Department of Statistics, Semnan University, Semnan, Iran

² Department of statistics, Semnan University, Semnan, Iran.

10.22034/aa.2026.2084110.1027

Abstract

Modeling nonlinear data, due to the presence of complex relationships and latent structures, requires approaches that go beyond classical linear models. In this study, the Bayesian neural network framework is employed to evaluate the performance of different Bayesian inference methods for nonlinear modeling. Three widely used approaches in this area, namely Hamiltonian Monte Carlo, the adaptive No-U-Turn Sampler, and Variational Inference, are implemented and compared within a nonlinear regression framework. The analysis is conducted using both simulated and real-world datasets, and the methods are evaluated based on predictive accuracy, quality of posterior approximation, and computational efficiency. The results indicate that sampling-based methods provide more accurate approximations of the posterior distribution, particularly in terms of uncertainty quantification, whereas variational inference, despite a slight reduction in accuracy, offers competitive performance due to its high computational efficiency and rapid convergence.

Keywords

Bayesian Neural Networks

Hamiltonian Monte Carlo

No-U-Turn Sampler

Variational Inference

Subjects