ℹ️ Select 'Choose Exercise', or randomize 'Next Random Exercise' in selected language.

Choose Exercise:
Timer 00:00
WPM --
Score --
Acc --
Correct chars --

Julia Probabilistic Programming: Bayesian Linear Regression

Julia

Goal -- WPM

Ready
Exercise Algorithm Area
1using Turing
2using Distributions
3using Random
4
5# Define the Bayesian Linear Regression model
6@model function bayesian_linear_regression(x, y, ::Type{Val{:prior}} = nothing)
7# Priors for regression coefficients (beta) and noise standard deviation (sigma)
8# Assuming a single predictor variable for simplicity
9beta0 ~ Normal(0, 10) # Prior for intercept
10beta1 ~ Normal(0, 10) # Prior for slope
11sigma ~ Exponential(1.0) # Prior for noise standard deviation
12
13# Likelihood function
14# The mean of the normal distribution for y is determined by the linear model
15mu = beta0 .+ beta1 .* x
16y ~ MvNormal(mu, sigma)
17
18# Return parameters for posterior analysis if needed (optional for basic inference)
19# return beta0, beta1, sigma
20end
21
22# Function to perform inference and get posterior samples
23function infer_bayesian_linear_regression(x_data::Vector{Float64}, y_data::Vector{Float64}, num_samples::Int = 1000, num_chains::Int = 4)
24if length(x_data) != length(y_data)
25error("Input data vectors x and y must have the same length.")
26end
27if num_samples <= 0 || num_chains <= 0
28error("Number of samples and chains must be positive.")
29end
30
31# Set a seed for reproducibility
32Random.seed!(123)
33
34# Perform inference using Turing.jl's NUTS sampler
35# The `prior` argument is not used here for standard inference, but can be for prior predictive checks
36chain = sample(bayesian_linear_regression(x_data, y_data), NUTS(), MCMCThreads(), num_samples, num_chains)
37
38return chain
39end
40
41# Helper function to summarize the results (e.g., mean, median, credible intervals)
42function summarize_regression_results(chain)
43println("--- Posterior Summary ---")
44# Summarize beta0 (intercept)
45println("Intercept (beta0):")
46println(mean(chain[:beta0]))
47println(median(chain[:beta0]))
48println(quantile(chain[:beta0], [0.05, 0.95])) # 90% Credible Interval
49
50# Summarize beta1 (slope)
51println("\nSlope (beta1):")
52println(mean(chain[:beta1]))
53println(median(chain[:beta1]))
54println(quantile(chain[:beta1], [0.05, 0.95])) # 90% Credible Interval
55
56# Summarize sigma (noise std dev)
57println("\nNoise Std Dev (sigma):")
58println(mean(chain[:sigma]))
59println(median(chain[:sigma]))
60println(quantile(chain[:sigma], [0.05, 0.95])) # 90% Credible Interval
61end
62
63# Example Usage:
64# Generate some synthetic data for demonstration
65# Random.seed!(42)
66# N = 100
67# true_beta0 = 2.0
68# true_beta1 = 3.0
69# true_sigma = 1.5
70# x_obs = rand(N) * 10
71# y_obs = true_beta0 .+ true_beta1 .* x_obs .+ randn(N) * true_sigma
72
73# num_samples_inference = 2000
74# num_chains_inference = 4
75# posterior_chain = infer_bayesian_linear_regression(x_obs, y_obs, num_samples_inference, num_chains_inference)
76
77# summarize_regression_results(posterior_chain)
Algorithm description viewbox

Julia Probabilistic Programming: Bayesian Linear Regression

Algorithm description:

This Julia program implements a Bayesian linear regression model using the Turing.jl probabilistic programming library. It defines a probabilistic model with prior distributions for the regression coefficients and noise, and a likelihood function relating the predictors to the response variable. Markov Chain Monte Carlo (MCMC) methods are then used to infer the posterior distributions of the model parameters. This approach provides a richer understanding of parameter uncertainty compared to frequentist methods and is widely used in statistical modeling, econometrics, and machine learning.

Algorithm explanation:

The `bayesian_linear_regression` model defines the probabilistic structure: priors for `beta0` (intercept), `beta1` (slope), and `sigma` (noise standard deviation) are set using `Normal` and `Exponential` distributions. The likelihood `y ~ MvNormal(mu, sigma)` specifies that the observed `y` values are normally distributed around the linear predictor `mu = beta0 .+ beta1 .* x` with standard deviation `sigma`. The `infer_bayesian_linear_regression` function uses Turing's NUTS sampler to draw samples from the posterior distribution of the parameters. The time complexity depends heavily on the sampler and the number of samples/chains, but is generally computationally intensive. Space complexity is O(num_samples * num_chains * num_parameters) for storing the MCMC chain. Edge cases include mismatched data lengths and invalid sample/chain counts, which are handled by errors.

Pseudocode:

function bayesian_linear_regression(x, y):
  // Priors
  beta0 ~ Normal(0, 10)
  beta1 ~ Normal(0, 10)
  sigma ~ Exponential(1.0)

  // Linear predictor
  mu = beta0 + beta1 * x

  // Likelihood
  y ~ Normal(mu, sigma)

function infer_bayesian_linear_regression(x_data, y_data, num_samples, num_chains):
  if length(x_data) != length(y_data): error
  if num_samples <= 0 or num_chains <= 0: error

  // Perform MCMC inference (e.g., NUTS sampler)
  chain = sample(bayesian_linear_regression(x_data, y_data), NUTS(), MCMCThreads(), num_samples, num_chains)

  return chain

function summarize_regression_results(chain):
  print "Posterior Summary"
  for each parameter (beta0, beta1, sigma):
    print mean, median, and credible interval (e.g., 5th and 95th percentiles)