Bayesian statistical methods are based on the idea that one can assert prior
probability distributions for parameters of interest. Although this makes Bayesian analysis seem subjective, there are a number of advantages to Bayesianism. It tends to permit more direct conclusions about parameters than the frequentist approach and, once a prior is established, estimation and testing procedures tend to be straightforward. The major downsides of Bayesianism are the requirement that a prior be chosen and the tendency of Bayesian methods to require a great deal of computational power. However, if you do have a lot of prior information, it can be extremely beneficial to incorporate that information into an estimation procedure. We try to find books that offer the Bayesian perspective for all the statistical topics on this site, but most applied books are not strictly Bayesian. However, the books in this category give the orthodox Bayesian perspective.
Recommended Books
A First Course in Bayesian Statistical Methods
Peter D. Hoff
Key Features
- In-text exercises
- Errata and R code
Key Topics
- Belief
- Binomial Model
- Conjugate Priors
- Exchangeability
- Gibbs Sampling
- Group Comparisons
- Hierarchical Modeling
- Latent Variable Methods
- Linear Regression
- Metropolis-Hastings Algorithms
- Mixed Effects Models
- Monte Carlo Approximation
- Multivariate Gaussian/Normal Model
- Poisson Model
- Probability
- Univariate Gaussian/Normal Model
Description
This is an excellent and concise introduction to Bayesian techniques. It will take you all the way from simple, one-dimensional, conjugate-prior-based methods to probit regression. There is a (necessary) focus on Monte Carlo techniques throughout most of the book. There are a limited number of exercises, but the ones that have been included are effective. We didn't realize it when we first read this book, but it is extremely important to note that Hoff has provided the R code he uses at the website linked above. Reading that code can be very helpful for the application exercises.
Bayesian Data Analysis
Andrew Gelman, John B. Carlin, Hal S. Stern, And Donald B. Rubin
Key Features
- In-text exercises
- Errata etc.
Key Topics
- Accounting for Data Collection
- Assessing MCMC Convergence
- Asymptotics
- Bayesian Computation
- Bayesian Inference
- Bayesian Interpretation of Non-Bayesian Approaches
- Comparing Models
- Computationally Efficient Simulation
- Decision Analysis
- Dirichlet Process Models
- Finite Mixture Models
- Gaussian Process Models
- Generalized Linear Models
- Gibbs Sampling
- Hierarchical Linear Models
- Hierarchical Models
- Importance Sampling
- Informative vs Non-informative Priors
- Metropolis Hastings Algorithms
- Modal and Distributional Approximations
- Model Checking
- Models for Missing Data
- Models for Robust Inference
- Multiparameter Models
- Multiple Imputation
- Nonlinear Models
- Nonparametric Models
- Nuisance Parameters
- Numerical Integration
- Probability and Inference
- Regression Models
- Single Parameter Models
- Variational Inference
Description
If you've decided you want to convert to Bayesianism, you ought to read the gospel according to Gelman. We like this book; reading it is like having a long conversation with someone with a lot of experience. However, we often found ourselves having to turn to outside resources in order to complete the exercises, which we attribute to a lack of code examples and somewhat summary, albeit verbose, exposition. You'll learn a lot about practical Bayesian analysis by reading this book; just don't expect it to be self-contained.
