Generalized Linear Models (GLM) — And Why Linear Regression Isn’t Enough

Generalized Linear Models solve a problem that trips up almost every statistics student eventually: what do you do when your response variable isn’t continuous and normally distributed? You’ve been taught linear regression. It’s clean, interpretable, and mathematically elegant. But then you face data where the outcome is a yes/no decision, a count of events, or an insurance claim amount — and linear regression starts producing nonsensical predictions: probabilities above 1, negative counts, and wildly biased estimates.

That’s precisely the gap GLMs were designed to fill. Regression analysis doesn’t have to mean linear regression — GLMs are the broader family within which linear regression sits as a special case. Understanding GLMs means understanding the full scope of what statistical modeling can do.

What Is a Generalized Linear Model?

A Generalized Linear Model (GLM) is a unified statistical framework that extends ordinary linear regression by allowing the response variable to follow any distribution from the exponential family — not just the normal distribution. The generalized linear model may be viewed as a special case where the general linear model has an identity link and normally distributed responses. GLMs generalize beyond this constraint.

The framework was formalized by John Nelder and Robert Wedderburn in their 1972 paper published in the Journal of the Royal Statistical Society. Their insight was profound: many seemingly unrelated statistical models — linear regression, logistic regression, Poisson regression, probit models — all shared a common mathematical structure. By identifying that structure, they created a unified framework capable of handling all of them consistently. Logistic regression, for instance, is not a separate statistical technique — it is simply a GLM with a binomial distribution and a logit link function.

Why Linear Regression Falls Short

Standard linear regression makes four assumptions that real-world data routinely violates. It assumes the response variable is continuous (binary outcomes violate this), normally distributed (count data and skewed data violate this), has constant variance (count data and binary data violate this — variance depends on the mean), and follows a linear relationship with predictors on the raw scale (probabilities must be bounded between 0 and 1, which a straight line cannot guarantee).

When these assumptions are violated, linear regression produces biased estimates, incorrect standard errors, invalid hypothesis tests, and predictions that fall outside the meaningful range of the response variable. Understanding regression model assumptions is the first step toward knowing when to move from linear regression to a GLM. Residual analysis is the diagnostic tool that reveals when a linear model is misspecified — and GLMs offer the corrective framework.

A Quick Historical Note: John Nelder, Robert Wedderburn, and the Birth of GLMs

John A. Nelder — a statistician at Rothamsted Research in the UK — and Robert W.M. Wedderburn co-authored the 1972 paper “Generalized Linear Models” in the Journal of the Royal Statistical Society, Series A. Wedderburn’s contribution was particularly significant on the mathematical side: he developed the concept of quasi-likelihood, which extended GLM theory to distributions beyond the exponential family. Wedderburn died tragically young in 1975, but his influence on the development of GLM theory was lasting.

Nelder went on to develop GLIM (Generalized Linear Interactive Modelling) — one of the first statistical software packages implementing GLMs — and remained a major figure in the field for decades. His later work with Yee on Vector Generalized Linear Models (VGLMs) extended the framework even further.

The Three Components of Every Generalized Linear Model

Every Generalized Linear Model — from the simplest logistic regression to a gamma model for insurance claims — is built from the same three structural components. Understanding these three components is the entire foundation of GLM theory. Once you understand them, reading any GLM specification becomes straightforward.

Component 1: The Random Component — Choosing Your Distribution

The random component specifies the probability distribution of the response variable Y. In a GLM, this distribution must come from the exponential family — a class of distributions that share a common mathematical form allowing unified estimation via maximum likelihood. Probability distributions from the exponential family include the normal, binomial, Poisson, gamma, inverse Gaussian, and negative binomial (as a quasi-GLM extension).

The exponential family can be written in a canonical form as f(y; θ, φ) = exp{[yθ − b(θ)] / a(φ) + c(y, φ)}, where θ is the natural (canonical) parameter, φ is a dispersion parameter, and a(·), b(·), c(·) are specific functions. This canonical form is what unifies these seemingly different distributions and enables the IRLS estimation algorithm to work identically across all of them.

Component 2: The Systematic Component — The Linear Predictor

The systematic component is the linear predictor η (eta). It is simply the linear combination of the explanatory variables: η = β₀ + β₁X₁ + β₂X₂ + … + βₚXₚ. This is exactly the same as the right-hand side of any linear regression equation. Nothing changes here — GLMs retain the linear structure of predictors. What changes is how this linear predictor relates to the mean of the response.

The systematic component keeps GLMs interpretable. Because the predictors enter linearly, each coefficient β represents the effect of a one-unit change in the predictor — on the scale of the link function. This is one of GLMs’ key advantages over more flexible but less interpretable machine learning methods: you can directly quantify and test the effect of each predictor.

Component 3: The Link Function — Connecting Mean to Linear Predictor

The link function g(·) connects the expected value of the response (μ = E[Y]) to the linear predictor (η): g(μ) = η. It transforms the mean response from its natural scale to a scale where the linear predictor is appropriate. The canonical link function is the natural choice for each distribution — it arises directly from the exponential family form and has desirable statistical properties including the simplest sufficient statistics.

Types of Generalized Linear Models: From Logistic to Gamma Regression

Each type of Generalized Linear Model is defined by its distribution and link function combination. Choosing the wrong type means misspecifying the data-generating process — producing biased estimates and invalid inference. The following covers every major GLM type you’ll encounter in statistics courses, research papers, and applied data analysis.

Logistic Regression — The Most Common GLM

Logistic regression is a GLM with a binomial distribution and a logit (log-odds) link function. It models binary outcomes — events that either happen or don’t. Spam vs. not spam. Disease vs. no disease. Customer churn vs. retention. Loan default vs. repayment. The logit link ensures that predicted probabilities always fall between 0 and 1 — a constraint that linear regression cannot satisfy.

The logit link is g(μ) = log(μ/(1-μ)) = log(odds). The coefficient β represents the change in log-odds for a one-unit increase in the predictor. Exponentiating a coefficient gives the odds ratio — the most commonly reported effect size in logistic regression. An odds ratio of 2.5 means the event is 2.5 times as likely for each unit increase in that predictor. Logistic regression is covered in exhaustive detail in that dedicated guide — including multinomial logistic regression for outcomes with more than two categories.

# Fit a logistic regression GLM in R
model <- glm(outcome ~ age + bmi + smoking,
              data = patient_data,
              family = binomial(link = "logit"))

summary(model)

# Odds ratios with 95% CI
exp(cbind(OR = coef(model),
           confint(model)))

Poisson Regression — Modeling Count Data

Poisson regression is a GLM for count data — the number of times an event occurs in a fixed time period or space. Hospital admissions per month. Road accidents per district. Website visits per day. The Poisson distribution assumes variance equals the mean (equidispersion). The log link ensures predicted counts are always positive. Coefficients represent log rate ratios — exponentiated, they give the multiplicative change in expected count per unit increase in the predictor.

When counts are rates (events per unit of time or exposure), an offset term is added to account for varying exposure: log(μ/t) = η, equivalently log(μ) = log(t) + η. The offset log(t) has a fixed coefficient of 1. This allows fair comparison of counts from units with different exposures. Understanding the Poisson distribution is the necessary background before fitting Poisson regression.

import statsmodels.api as sm
import statsmodels.formula.api as smf

# Poisson GLM in Python (statsmodels)
model = smf.glm(formula='count ~ age + group',
                data=df,
                family=sm.families.Poisson(
                    link=sm.families.links.Log()
                )).fit()

print(model.summary())

# Exponentiated coefficients = rate ratios
import numpy as np
print(np.exp(model.params))

Gamma Regression — Handling Positive Skewed Continuous Data

Gamma regression is a GLM for positive, continuous, right-skewed data where variance increases with the mean. Insurance claim amounts, rainfall totals, waiting times, and income distributions often follow this pattern. The gamma distribution is the right choice when your response variable is strictly positive and displays multiplicative rather than additive variation.

The canonical inverse link (g(μ) = 1/μ) is mathematically elegant but not always practical — it can produce negative predictions near zero. The log link is often preferred: it ensures positive predictions and gives coefficients interpretable as proportional effects on the mean (similar to Poisson). Understanding the gamma distribution helps build intuition for when gamma regression is appropriate.

Negative Binomial Regression — Overdispersed Counts

When count data shows overdispersion — observed variance greater than the mean, violating the Poisson assumption — negative binomial regression is the appropriate GLM extension. The negative binomial distribution adds a dispersion parameter k that allows variance to exceed the mean: Var(Y) = μ + μ²/k. As k → ∞, the negative binomial converges to the Poisson.

Overdispersion is extremely common in real count data. It arises from unmeasured heterogeneity between subjects, zero-inflation, or clustering. Fitting a Poisson model to overdispersed data produces artificially small standard errors, inflated test statistics, and false positive findings. Detecting overdispersion is straightforward: compare the residual deviance to its degrees of freedom — a ratio substantially greater than 1 indicates overdispersion.

Probit Regression — An Alternative to Logistic Regression

Probit regression uses the same binomial distribution as logistic regression but substitutes the logit link for the probit link — the inverse of the cumulative standard normal distribution function: g(μ) = Φ⁻¹(μ). The choice between logit and probit is often inconsequential in practice — the two produce very similar predictions. Probit regression is more common in econometrics (where it connects naturally to latent variable models) while logistic regression dominates in biostatistics and epidemiology.

Quasi-GLMs — When You Know the Mean-Variance Relationship

Quasi-likelihood models (quasi-Poisson, quasi-binomial) are not true GLMs in the strict sense — they don’t specify a full probability distribution. Instead, they specify only the mean and variance functions, using Robert Wedderburn’s quasi-likelihood approach. This makes them highly flexible: you can fit a Poisson-like model without committing to exact Poisson variance, estimating a dispersion parameter φ that scales the standard errors appropriately.

Quasi-Poisson and quasi-binomial are the simplest solutions to overdispersion and are widely used in practice. The coefficient estimates are identical to the standard Poisson or binomial GLM — only the standard errors differ (they are multiplied by √φ).

How Generalized Linear Models Are Fitted: MLE and IRLS

Fitting a Generalized Linear Model means estimating the coefficient vector β that maximizes the likelihood of observing the data under the specified distribution and link function. This is maximum likelihood estimation (MLE) — the standard approach for statistical inference in GLMs.

Maximum Likelihood Estimation in GLMs

The log-likelihood function for a GLM from the exponential family is l(β) = Σ [yᵢθᵢ − b(θᵢ)] / a(φ) + c(yᵢ, φ), where the sum is over all observations. Maximizing this with respect to β means solving the score equations ∂l/∂β = 0. These are typically nonlinear equations — no closed-form solution exists except for the normal distribution with identity link (ordinary linear regression). Hypothesis testing in statistics — including Wald tests, likelihood ratio tests, and score tests — applies directly to GLM inference on these estimated coefficients.

Iteratively Reweighted Least Squares (IRLS)

Iteratively Reweighted Least Squares (IRLS) is the numerical algorithm used to solve the GLM score equations. At each iteration, the algorithm constructs a working response z and a weight matrix W, then solves a weighted least squares problem: β̂ = (X’WX)⁻¹ X’Wz. These working quantities are updated at each step using the current parameter estimates, and the process repeats until convergence.

IRLS is elegant because it reduces the nonlinear GLM estimation problem to a sequence of weighted linear regressions — and standard least squares solvers can be applied at each step. The algorithm converges rapidly for well-specified models with sufficient data.

The Dispersion Parameter φ

The dispersion parameter φ controls the variance of the response relative to the mean. For the normal distribution, φ = σ² (the residual variance, estimated from the data). For the Poisson and binomial distributions, φ is fixed at 1 — meaning the model assumes no extra dispersion. For the gamma distribution, φ is estimated from the data.

When φ is fixed at 1 but the actual data shows overdispersion (residual deviance >> degrees of freedom), standard errors are underestimated and test statistics are inflated. This is one of the most common sources of false positives in GLM analyses. Quasi-GLMs address this by estimating φ from the data and adjusting all standard errors accordingly.

Complete Separation in Logistic Regression

Complete separation — also called perfect separation — occurs in logistic regression when a predictor (or combination of predictors) perfectly predicts the binary outcome. The maximum likelihood estimate of the corresponding coefficient diverges to ±∞, and the IRLS algorithm fails to converge. This is a real and common problem in small datasets with many predictors, rare events, or highly correlated variables.

Solutions include Firth’s penalized likelihood approach (which adds a Jeffreys prior to the likelihood, pulling estimates away from infinity), ridge regularization (penalizing large coefficients), or reconsidering the predictor set. Ridge and Lasso regularization as statistical techniques apply directly to logistic regression and other GLMs when separation or collinearity are present.

GLM Diagnostics: Deviance, Residuals, and Goodness of Fit

Fitting a Generalized Linear Model is only the beginning. Model diagnostics — assessing whether the model fits the data adequately and whether assumptions are satisfied — are where most students drop the ball on statistics assignments. A model that fits poorly or violates assumptions produces invalid inference regardless of how elegant the specification looks on paper.

Deviance: The GLM Equivalent of Residual Sum of Squares

Deviance is the primary goodness-of-fit measure for GLMs, defined as D = 2[l(saturated model) − l(fitted model)], where l denotes the log-likelihood. The null deviance measures how well an intercept-only model fits the data — the baseline. The residual deviance measures how well your fitted model (with predictors) fits. The difference in deviance between nested models approximately follows a chi-square distribution with degrees of freedom equal to the difference in number of parameters — this is the likelihood ratio test.

For models with fixed dispersion (Poisson, binomial), residual deviance divided by its degrees of freedom should be close to 1. Values substantially greater than 1 indicate overdispersion or model misspecification. Values substantially less than 1 suggest underdispersion or an over-parameterized model.

Types of Residuals in GLMs

Ordinary residuals (yᵢ – μ̂ᵢ) are not appropriate for diagnosing GLMs because their distribution and variance are not constant across observations. Instead, GLMs use several specialized residual types. Pearson residuals standardize by dividing by the square root of the variance function: eᵢ = (yᵢ – μ̂ᵢ) / √V(μ̂ᵢ). Deviance residuals are signed square roots of each observation’s contribution to the total deviance — the most commonly used for diagnostic plots. Anscombe residuals are transformations designed to be more approximately normal for specific distributions.

Standard diagnostic plots for GLMs include residuals vs. fitted values (checking for patterns suggesting model misspecification), a normal Q-Q plot of deviance residuals (assessing distributional assumptions), Cook’s distance (identifying influential observations), and scale-location plots (assessing variance homogeneity). Residual analysis for statistical modeling covers these diagnostic tools in detail.

Information Criteria: AIC and BIC for GLM Model Selection

AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) are used to compare non-nested GLMs or GLMs with different distributions — situations where the likelihood ratio test cannot be used. AIC = −2l(β̂) + 2p, where p is the number of parameters. BIC = −2l(β̂) + p·log(n). Lower values of both criteria indicate better-fitting models with appropriate penalty for complexity.

AIC tends to select more complex models, making it preferable when prediction accuracy is the goal. BIC penalizes complexity more heavily, favoring simpler models and performing better when identifying the true model is the goal. Model selection using AIC and BIC is foundational to every GLM analysis that compares competing specifications.

Detecting and Handling Overdispersion

Overdispersion is the single most common misspecification in applied GLM analysis. It occurs when the observed variance in the data exceeds what the assumed distribution predicts. For Poisson regression: when Var(Y) > μ. For binomial regression: when Var(Y) > nπ(1−π).

Interpreting GLM Coefficients, Predictions, and Effect Sizes

Fitting a Generalized Linear Model produces coefficient estimates — but interpreting those estimates correctly requires understanding the link function. The scale on which coefficients are estimated is not the scale of the response variable. Getting interpretation right is what separates a competent GLM analyst from someone who can run the code but can’t communicate the results.

Coefficients on the Link Scale vs. the Response Scale

GLM coefficients are always on the link scale — the transformed scale where the linear predictor operates. To convert to the response scale, apply the inverse link function. For logistic regression: coefficients are log-odds, which exponentiated give odds ratios. For Poisson regression: coefficients are log rate ratios, which exponentiated give rate ratios (or incidence rate ratios). For linear regression (identity link): no transformation needed — coefficients are directly on the response scale.

Odds Ratios in Logistic Regression

In logistic regression, the odds ratio (OR) is the primary effect measure. OR = exp(β). An OR of 1 means no association. OR > 1 means the event is more likely as the predictor increases. OR < 1 means the event is less likely. The OR is a multiplicative measure — it multiplies the baseline odds by exp(β) for each unit increase in the predictor.

A critical caution: odds ratios are often misinterpreted as relative risks. The odds ratio approximates the relative risk only when the outcome is rare (prevalence < 10%). For common outcomes, odds ratios overstate the relative risk. For common outcomes, consider using log-binomial regression (which estimates relative risk directly) instead of logistic regression.

Rate Ratios in Poisson Regression

In Poisson regression, exp(β) is the rate ratio (also called the incidence rate ratio, IRR). It represents the multiplicative change in the expected count per unit increase in the predictor, holding all other predictors constant. A rate ratio of 1.8 means the expected count is 80% higher for each unit increase in that predictor.

Predicted Probabilities and Marginal Effects

For any GLM, the most communicable output is the predicted probability (or predicted mean count, or predicted value) at specific values of the predictors. These give audiences something concrete: “The model predicts a 23% probability of loan default for a 35-year-old with an income of $50,000 and no prior default history.” This is far more informative than reporting log-odds coefficients to a non-statistical audience.

Generalized Linear Models in Practice: Applications Across Fields

Generalized Linear Models are not just academic exercises — they are the workhorses of quantitative analysis across virtually every empirical discipline. Understanding where and how GLMs are deployed in practice gives context that makes the theory stick and makes assignments far richer.

Epidemiology and Public Health

In epidemiology, GLMs are fundamental. Logistic regression is the standard method for case-control studies — estimating odds ratios for exposure-disease associations while controlling for confounders. Poisson regression is used in cohort studies to estimate incidence rate ratios when follow-up time varies. Log-binomial regression estimates relative risks directly when outcome prevalence is high.

The Centers for Disease Control and Prevention (CDC) in the US and Public Health England in the UK routinely use GLMs in disease surveillance, risk factor analysis, and policy evaluation. For university students studying epidemiology or public health at institutions like Johns Hopkins Bloomberg School of Public Health or the London School of Hygiene and Tropical Medicine, GLMs are among the first advanced statistical methods taught in core quantitative courses.

Actuarial Science and Insurance

The actuarial profession — through bodies like the Casualty Actuarial Society (CAS) in the United States and the Institute and Faculty of Actuaries (IFoA) in the UK — relies heavily on GLMs for insurance pricing, reserving, and risk classification. Gamma regression models claim severity (the cost of a claim, conditional on a claim occurring). Tweedie regression — a special case bridging the Poisson and gamma — models pure premium (claim cost × frequency combined). Negative binomial regression models claim frequency when Poisson assumptions are violated.

GLMs have largely replaced the minimum bias procedures historically used in US insurance ratemaking. The CAS exam syllabus explicitly tests GLM theory and application.

Ecology and Environmental Science

Ecologists and environmental scientists routinely model count data (species abundances, population sizes, event frequencies) and presence-absence data (species occurrence) using GLMs. Poisson regression and its extensions model species abundance counts. Logistic regression models species distribution (presence vs. absence). Negative binomial regression handles the overdispersion ubiquitous in ecological count data.

Social Sciences and Political Science

In political science, sociology, and economics, GLMs model voting behavior, survey responses, crime rates, and labor market outcomes. Logistic regression models binary outcomes like voting (voted vs. didn’t vote) or survey responses (yes/no). Ordered logistic (proportional odds) regression handles ordinal outcomes like Likert scale responses. Poisson regression models count outcomes like the number of bills passed or arrests per district.

Universities including Harvard, MIT, University of Oxford, and University of Chicago teach GLMs as core quantitative methods in social science doctoral programs.

Data Science and Machine Learning

In data science, GLMs sit at the intersection of classical statistics and machine learning. Logistic regression remains one of the most widely used classification algorithms — not despite its simplicity but because of it. It’s interpretable, fast, regularizable, and well-understood. Major technology companies including Google, Meta, Amazon, and Microsoft use logistic regression for click-through prediction, fraud detection, and recommendation systems, often at massive scale with regularization and distributed computing.

Elastic net regularization applied to logistic regression combines L1 (Lasso) and L2 (Ridge) penalties for variable selection and coefficient shrinkage simultaneously. Ridge and Lasso regularization in the context of GLMs is an increasingly important topic in data science curricula.

Implementing Generalized Linear Models in R and Python

Generalized Linear Models are straightforward to implement in both R and Python — the tricky part is choosing the right specification and interpreting the output correctly. This section covers the essential syntax and key output elements for both languages.

GLMs in R: The glm() Function

R’s built-in glm() function handles all major GLM types through the family argument. The function returns a model object whose summary() method provides coefficient estimates, standard errors, z-statistics (or t-statistics for Gaussian GLMs), p-values, null deviance, residual deviance, and AIC.

# ── Logistic Regression ──────────────────────────────
log_mod <- glm(defaulted ~ income + age + debt_ratio,
               data = loans,
               family = binomial(link = "logit"))
summary(log_mod)
exp(coef(log_mod))  # Odds Ratios

# ── Poisson Regression ───────────────────────────────
pois_mod <- glm(count ~ exposure + treatment,
                data = events,
                family = poisson(link = "log"),
                offset = log(population))
summary(pois_mod)
exp(coef(pois_mod))  # Rate Ratios (IRR)

# ── Gamma Regression ─────────────────────────────────
gamma_mod <- glm(claim_cost ~ vehicle_age + region,
                 data = insurance,
                 family = Gamma(link = "log"))
summary(gamma_mod)

# ── Negative Binomial ─────────────────────────────────
library(MASS)
nb_mod <- glm.nb(count ~ predictors, data = df)
summary(nb_mod)

# ── Model Diagnostics ─────────────────────────────────
par(mfrow = c(2, 2))
plot(log_mod)  # Standard residual plots

# Overdispersion check
summary(pois_mod)$deviance / summary(pois_mod)$df.residual

GLMs in Python: statsmodels and scikit-learn

Python has two main routes to GLMs. statsmodels provides full statistical output including coefficients, standard errors, z-statistics, p-values, deviance, and AIC — essential for statistical inference and reporting. scikit-learn provides GLMs optimized for prediction with regularization — useful for machine learning workflows but lacking the inferential detail of statsmodels. For statistics assignments requiring hypothesis testing and model diagnostics, statsmodels is the correct choice.

import statsmodels.api as sm
import statsmodels.formula.api as smf
import numpy as np
import pandas as pd

# ── Logistic Regression ──────────────────────────────
log_mod = smf.glm(
    formula='defaulted ~ income + age + debt_ratio',
    data=loans,
    family=sm.families.Binomial(
        link=sm.families.links.Logit()
    )
).fit()
print(log_mod.summary())
print(np.exp(log_mod.params))  # Odds Ratios

# ── Poisson Regression with offset ───────────────────
pois_mod = smf.glm(
    formula='count ~ exposure + treatment',
    data=events,
    family=sm.families.Poisson(),
    offset=np.log(events['population'])
).fit()
print(np.exp(pois_mod.params))  # Rate Ratios

# ── Gamma Regression ─────────────────────────────────
gamma_mod = smf.glm(
    formula='claim_cost ~ vehicle_age + region',
    data=insurance,
    family=sm.families.Gamma(
        link=sm.families.links.Log()
    )
).fit()

# ── Overdispersion check ──────────────────────────────
print(pois_mod.deviance / pois_mod.df_resid)

Extensions of GLMs: GLMM, GAM, GEE, and Bayesian GLMs

The Generalized Linear Model framework is powerful — but it has limitations. It assumes observations are independent, the relationship between predictors and the mean is linear (on the link scale), and the distribution family is correctly specified. When these assumptions fail, extensions of the GLM framework provide solutions while preserving much of the core structure.

Generalized Linear Mixed Models (GLMM)

Generalized Linear Mixed Models (GLMMs) extend GLMs by adding random effects to the linear predictor. This allows the model to handle clustered, hierarchical, or longitudinal data where observations are not independent. Students in a school, patients in a hospital, animals in a cage, repeated measurements from the same individual — these all produce correlated data that violates the GLM independence assumption.

The GLMM extends the linear predictor to η = Xβ + Zu, where β are the fixed effects (population-level parameters), u are the random effects (group-level deviations), X and Z are design matrices. The random effects u are assumed to follow a normal distribution: u ~ N(0, D). Fitting GLMMs requires numerical integration (Gaussian quadrature or Laplace approximation) because the marginal likelihood doesn’t have a closed form. In R, the lme4 package’s glmer() function is the standard tool.

Generalized Estimating Equations (GEE)

Generalized Estimating Equations (GEEs) are an alternative approach to correlated data that focuses on population-average (marginal) effects rather than subject-specific effects. GEEs extend the quasi-likelihood approach to handle correlation between observations by specifying a working correlation structure (independent, exchangeable, AR-1, unstructured). Unlike GLMMs, GEEs don’t require specifying the full joint distribution of correlated responses — they are robust to misspecification of the correlation structure.

GEEs are widely used in clinical trials and longitudinal studies in public health and medicine — particularly when the average population effect is the primary scientific interest rather than individual-level prediction. The choice between GLMM and GEE is conceptual: GLMMs for subject-specific inference and prediction, GEEs for population-average inference.

Generalized Additive Models (GAM)

Generalized Additive Models (GAMs) extend GLMs by replacing the linear predictor with a sum of smooth functions of the predictors: η = β₀ + f₁(X₁) + f₂(X₂) + … + fₚ(Xₚ). Each f() is a smooth, non-parametric function estimated from the data using splines or similar methods. GAMs retain the GLM’s distributional flexibility (any exponential family distribution, any link function) while dropping the linearity assumption on the predictor scale.

GAMs are particularly valuable when you suspect non-linear relationships between predictors and the outcome but don’t know the functional form. In R, the mgcv package by Simon Wood is the gold-standard implementation. In Python, the pyGAM library provides similar functionality.

Bayesian Generalized Linear Models

Bayesian GLMs reframe the estimation problem in terms of Bayes’ theorem: instead of finding the maximum likelihood estimate, they characterize the full posterior distribution of the parameters given the data. P(β | data) ∝ P(data | β) × P(β), where P(β) is the prior distribution on the coefficients. This approach is particularly valuable for small samples (where MLE may be unstable), when informative prior knowledge exists, when full uncertainty quantification is needed, and when models are highly complex with many parameters.

In practice, Bayesian GLMs are implemented using Markov Chain Monte Carlo (MCMC) sampling via tools like Stan (accessed via the rstanarm or brms R packages) or PyMC in Python. Regularization in classical GLMs (Ridge, Lasso) corresponds to specific choices of prior distributions in the Bayesian framework — L2 penalization corresponds to a normal prior on coefficients, L1 to a Laplace prior. Bayesian inference provides the full theoretical foundation for understanding this connection.

Writing GLM Assignments and Exams: What Your Professor Is Looking For

GLM questions in statistics assignments and exams test multiple competencies simultaneously. Getting high marks means demonstrating not just that you can run the model in software but that you understand why you chose it, how it works, and what the output means.

Justify Your GLM Choice Before Fitting

The first thing a well-written GLM assignment does is justify the model choice. Describe the response variable — its type (binary, count, continuous), distribution, and constraints. Explain why ordinary linear regression is inadequate. Specify your chosen distribution and link function, and justify each explicitly. This justification is worth marks on nearly every GLM assignment rubric.

Report Model Output Correctly

When reporting GLM results, always include: coefficient estimates and their standard errors, the appropriate transformed effect size (OR, IRR, etc.), 95% confidence intervals (not just p-values), deviance statistics (null and residual), AIC (for model comparison), and a statement about overdispersion if a Poisson or binomial model was used. Don’t just paste software output — explain each number.

Model Diagnostics Are Not Optional

A GLM assignment that fits the model and reports coefficients without diagnostics is incomplete. Check deviance/df for overdispersion. Examine residual plots. Test for influential observations. Compare alternative models using AIC. Many students lose significant marks precisely here — they treat the fitted model as the endpoint rather than the starting point of evaluation.

Common GLM Assignment Mistakes

• Using linear regression on binary or count data — the most fundamental error; always indicates the student didn’t read the response variable correctly
• Choosing the wrong link function — using an identity link with a Poisson model, for instance, can produce negative predicted counts
• Reporting raw coefficients from logistic regression without exponentiating — log-odds by themselves are uninformative to most readers
• Ignoring overdispersion in Poisson models — produces artificially small standard errors and inflated test statistics
• Treating deviance as R² — it isn’t; pseudo-R² measures for GLMs (McFadden, Nagelkerke) must be calculated separately and interpreted differently
• Confusing odds ratios with relative risks — for common outcomes, these are very different quantities
• Not checking for complete separation in logistic regression — produces infinite estimates with no warning in some software

Frequently Asked Questions: Generalized Linear Models (GLM)