Understanding Probability Distributions: Definitions and Examples

What Is a Probability Distribution?

Probability distributions sit at the center of every serious statistical analysis. A probability distribution is a mathematical function that describes the likelihood of each possible outcome for a random variable. In plain terms: it shows you, for every value a variable can take, how probable that value is. If you have ever asked “what are the chances of exactly three customers arriving in the next minute?” or “how likely is a student to score above 90 on this exam?”, you were already thinking in terms of probability distributions. And understanding them is not optional for anyone studying statistics, data science, economics, psychology, or the sciences. Our statistics assignment help team fields questions on this topic constantly — which tells you exactly how central it is to university-level coursework.

Formally, a probability distribution assigns a probability to each measurable subset of outcomes from an experiment. The collection of all these probabilities is exhaustive — they must sum (for discrete variables) or integrate (for continuous variables) to exactly 1. This is the first rule. If your probabilities do not add up to 1, you do not have a valid probability distribution.

What Is a Random Variable?

You cannot fully understand a probability distribution without first understanding a random variable. A random variable is a variable whose value is determined by the outcome of a random process. The key word is random — you do not know in advance what value it will take. What you do know is the set of possible values and, crucially, how probable each value is. That mapping from values to probabilities is the probability distribution.

Random variables come in two types. A discrete random variable takes on a countable set of values: 0, 1, 2, 3, and so on. The number of customers in a queue, the number of defective products in a batch, or the result of rolling a die are all discrete. A continuous random variable, by contrast, can take any value within a range. Height, temperature, time, and exam scores are all examples. The distinction between discrete and continuous matters because it determines which mathematical tools and which distributions you use to model the variable. Learning to recognize this difference is one of the first skills covered in any solid guide to quantitative and qualitative data.

What Does a Probability Distribution Look Like?

A probability distribution can be represented three ways: as a table, as a function, or as a graph. For a discrete variable, you might list each outcome alongside its probability in a table. For a continuous variable, you represent the distribution as a smooth curve — the probability density function (PDF) — and probability is the area under that curve over a specified interval.

Two important functions describe any probability distribution in formal statistics. For discrete distributions, the Probability Mass Function (PMF) gives P(X = x) for each specific value x. For continuous distributions, the Probability Density Function (PDF) gives f(x) where probability is calculated as the integral of f(x) over an interval. And for both types, the Cumulative Distribution Function (CDF) gives P(X ≤ x) — the probability that the variable takes a value at or below x. Understanding these three tools is the foundation for working with any probability distribution you encounter. You can explore this further in our article on data distributions, kurtosis, and skewness.

What Is the Difference Between a PDF and a PMF?

The PMF applies to discrete random variables. It returns the exact probability that X equals a specific value: P(X = 3) = 0.25, for example. The PDF applies to continuous random variables. Here is the subtle but critical point: for a continuous variable, the probability at any single exact point is zero. Probability only makes sense over an interval. So the PDF gives f(x), a density, and you integrate it between two points to get the probability that X falls in that range. If this sounds abstract, think of it this way: the chance that a person is exactly 170.000000 cm tall is effectively zero. But the chance they are between 169 and 171 cm is very real and calculable.

Discrete Probability Distributions: Types, Formulas, and Examples

Discrete probability distributions model random variables with countable outcomes. The three you will encounter most often in introductory and intermediate statistics courses are the binomial, Poisson, and geometric distributions. Each one was designed to model a specific class of real-world random process. Using the wrong distribution for your data is one of the most common errors students make — and it cascades into incorrect probabilities, wrong conclusions, and lost marks. If you are struggling with choosing and applying the right distribution, our statistics assignment experts can guide you through it step by step.

What Is the Binomial Distribution?

The binomial distribution models the number of successes in a fixed number of independent trials, where each trial has exactly two possible outcomes — success or failure — and the probability of success is constant across all trials. This is the distribution for coin flips, yes/no surveys, pass/fail tests, and any scenario that follows the binary structure. The classic academic example: if you flip a fair coin 10 times, the binomial distribution tells you the probability of getting exactly 0, 1, 2, … or 10 heads. You can read a much deeper treatment in our comprehensive binomial distribution guide.

Binomial Distribution Formula

The binomial distribution has mean μ = np and variance σ² = np(1 – p). Four conditions must hold for binomial to apply: a fixed number of trials (n), only two outcomes per trial, independence between trials, and a constant probability of success (p). When all four hold, the binomial distribution is the right tool. When they do not, you need a different distribution.

Binomial Distribution — Worked Example

What Is the Poisson Distribution?

The Poisson distribution models the number of events that occur in a fixed interval of time or space, given that events occur at a known constant average rate and independently of one another. It is the go-to distribution for count data: the number of calls arriving at a call center per hour, the number of typos per page, the number of accidents at an intersection per month. The Poisson distribution is defined by a single parameter, λ (lambda), which represents both the mean and the variance. That λ = μ = σ² is a distinctive and testable property. Explore more in our dedicated Poisson distribution guide.

Poisson Distribution Formula

Poisson Distribution — Worked Example

What Is the Geometric Distribution?

The geometric distribution models the number of trials needed to get the first success in a sequence of independent Bernoulli trials. Where the binomial asks “how many successes in n trials?”, the geometric asks “how many trials until the first success?” It is useful in reliability engineering (how many parts fail before the first failure?), quality control, and gambling scenarios. The geometric distribution has a remarkable property called memorylessness: past failures do not affect the probability of future success. This makes it the discrete analog of the exponential distribution.

What Is the Multinomial Distribution?

The multinomial distribution generalizes the binomial to situations where each trial can result in more than two outcomes. Where the binomial counts successes and failures across n trials, the multinomial counts how many times each of k possible outcomes occurs. Rolling a six-sided die 20 times and asking “how many 1s, 2s, 3s, 4s, 5s, and 6s will I get?” is a multinomial problem. It is widely used in natural language processing, genetics, and survey analysis where responses fall into more than two categories. Our multinomial distribution guide walks through this in full detail.

Continuous Probability Distributions: Types, Formulas, and Examples

Continuous probability distributions model random variables that can take any value within a range. This is the terrain of height, weight, temperature, income, time, and most measurement-based data. Continuous distributions are represented by smooth curves, not bars. And as established earlier, probability for a continuous variable is always an area under that curve over an interval — never a point value. The most important continuous distributions for statistics students are the normal, uniform, exponential, t-distribution, chi-square, and F-distribution. Mastering these is essential for hypothesis testing and confidence interval work. For a closer look at how this connects to statistical inference, see our article on hypothesis testing.

What Is the Normal Distribution?

The normal distribution — also called the Gaussian distribution — is the most important probability distribution in statistics. It is symmetric, bell-shaped, and defined entirely by two parameters: the mean (μ) and the standard deviation (σ). The mean determines where the distribution is centered. The standard deviation determines how spread out it is. Many naturally occurring phenomena — human heights, IQ scores, measurement errors, standardized test scores — follow an approximately normal distribution. This is not coincidence. It flows from the Central Limit Theorem, one of the most powerful results in all of probability theory.

The 68-95-99.7 Empirical Rule

The empirical rule — also called the 68-95-99.7 rule — is a shortcut for the normal distribution that every statistics student must know. For any normally distributed variable: 68% of values fall within 1 standard deviation of the mean; 95% fall within 2 standard deviations; and 99.7% fall within 3 standard deviations. This rule lets you quickly estimate probabilities for normal data without any formula. It also tells you that values more than 3 standard deviations from the mean are genuinely rare — appearing less than 0.3% of the time.

What Is the Standard Normal Distribution (Z-Distribution)?

The standard normal distribution is a special case of the normal distribution with μ = 0 and σ = 1. It is the reference distribution used in z-score calculations, z-tables, and most statistical software outputs. Any normal distribution can be converted to the standard normal using the z-score formula: z = (x – μ) / σ. Once standardized, you can use the z-score table to find probabilities for any normally distributed variable. The z-score tells you how many standard deviations away from the mean a specific value lies.

What Is the Uniform Distribution?

The uniform distribution (continuous) assigns equal probability to all values within a defined interval [a, b]. If you know that a bus arrives at any point between 0 and 10 minutes from now and all times are equally likely, that waiting time follows a uniform distribution. The PDF is simply a flat horizontal line over [a, b] and zero everywhere else. The height of that line is 1/(b – a) — which ensures the total area equals 1. The uniform distribution is often the starting point in probability modeling and simulation. Our deeper article on understanding the uniform distribution covers this in more detail.

What Is the Exponential Distribution?

The exponential distribution models the time between events in a Poisson process. If calls arrive at a support center at a rate of λ per hour (Poisson), then the waiting time between calls follows an exponential distribution with rate parameter λ. It is right-skewed — most waiting times are short, but occasionally you wait a long time. Like the geometric distribution among discrete distributions, the exponential is memoryless: knowing that you have already waited 5 minutes gives you no information about how much longer you will wait. This property is mathematically unique to the exponential distribution among continuous distributions and is heavily tested in probability courses. The exponential is central to survival analysis and time-to-event modeling.

What Is the t-Distribution?

The Student’s t-distribution looks like a normal distribution but with heavier tails. It was developed by William Sealy Gosset, a statistician working at the Guinness brewery in Dublin, published under the pseudonym “Student” in 1908. The t-distribution applies whenever you are working with small samples and the population variance is unknown — which is most of the time in real research. It is defined by one parameter: degrees of freedom (df). As df increases, the t-distribution approaches the normal. With df above 30, the difference is negligible. The t-distribution underlies the one-sample t-test, two-sample t-test, and paired t-test — the workhorses of inferential statistics. See our t-test guide for worked examples across all three test types, and our t-distribution table for the critical values you need.

What Is the Chi-Square Distribution?

The chi-square distribution arises from the sum of squared standard normal variables. If Z₁, Z₂, …, Zₖ are independent standard normal variables, then χ² = Z₁² + Z₂² + … + Zₖ² follows a chi-square distribution with k degrees of freedom. It is right-skewed — especially for small degrees of freedom — and only takes non-negative values. The chi-square distribution has two primary applications in statistics: the goodness-of-fit test (does observed data follow a specified distribution?) and the test of independence (are two categorical variables independent in a contingency table?). Both applications are comprehensively covered in our chi-square test guide.

What Is the F-Distribution?

The F-distribution is the ratio of two chi-square distributions, each divided by their respective degrees of freedom. It is non-negative and right-skewed. The F-distribution is central to Analysis of Variance (ANOVA) — comparing means across three or more groups — and to regression analysis, where the F-statistic tests whether the model as a whole is statistically significant. Understanding the F-distribution is inseparable from understanding regression modeling. Most statistical software (R, SPSS, Python’s SciPy, Excel) computes F-statistics and associated p-values automatically, but knowing what the F-distribution represents is essential for interpreting the output correctly.

Expected Value, Variance, and Standard Deviation in Probability Distributions

A probability distribution is more than a table or a curve. It is a complete description of a random variable’s behavior. The three most important summary statistics extracted from any probability distribution are the expected value (mean), variance, and standard deviation. These numbers tell you where the distribution is centered and how spread out it is — critical information for making decisions under uncertainty.

What Is Expected Value?

The expected value (also written E(X) or μ) is the long-run average of a random variable over many repetitions of an experiment. It is not the most likely outcome. It is the probability-weighted average of all possible outcomes. For a discrete random variable, you compute it by multiplying each value by its probability and summing the results. For a continuous variable, you integrate x × f(x) over the full range. Expected value is foundational to decision theory, insurance pricing, gambling strategy, and financial modeling. Our dedicated article on expected values and variance goes deep on both the math and the applications.

Expected Value — Quick Example

What Is Variance in a Probability Distribution?

The variance (σ²) measures how spread out the distribution is around its expected value. A high variance means outcomes are widely scattered; a low variance means they cluster tightly around the mean. Variance is computed as the expected value of the squared deviation from the mean: Var(X) = E[(X – μ)²]. For a discrete random variable, this becomes Σ[(x – μ)² × P(X = x)]. For continuous variables, you integrate (x – μ)² × f(x) dx. The standard deviation (σ) is simply the square root of variance — more interpretable because it is in the same units as the original variable.

What Is the Moment Generating Function?

The moment generating function (MGF) is an advanced but elegant tool that encodes all the moments (mean, variance, skewness, kurtosis) of a distribution in a single function: M(t) = E[e^(tX)]. The MGF is not always required in introductory statistics, but it becomes essential in theoretical statistics, actuarial science, and probability theory courses. Its critical property: the r-th derivative of the MGF evaluated at t = 0 gives the r-th moment of the distribution. If you are working at this level, our article on hypothesis testing alongside sampling distributions will connect the theory to its applications.

The Central Limit Theorem and Why It Makes the Normal Distribution Dominant

You cannot work seriously with probability distributions without understanding the Central Limit Theorem (CLT). It is the reason the normal distribution appears everywhere — not just in data that is inherently normally distributed, but in the sampling distributions of statistics computed from almost any kind of data. The CLT is the bridge between a single probability distribution and the entire field of inferential statistics. Without it, much of what we do in statistics simply would not work.

What Does the Central Limit Theorem State?

The Central Limit Theorem states that if you take a large enough random sample from any population distribution with a finite mean (μ) and finite variance (σ²), the distribution of the sample mean will be approximately normal, regardless of the shape of the original population distribution. The approximation improves as sample size n increases. A commonly used threshold is n ≥ 30, though the required sample size depends on how non-normal the original distribution is.

The sampling distribution of the sample mean has mean μ (same as the population mean) and standard deviation σ/√n — the standard error. As sample size increases, the standard error shrinks, meaning larger samples give more precise estimates of the population mean. This fact underpins confidence intervals and most parametric hypothesis tests.

Why Does the CLT Matter for Understanding Probability Distributions?

The CLT explains why probability distributions that look nothing like the normal distribution still produce approximately normal sampling distributions for means. A coin flip follows a Bernoulli distribution (p = 0.5) — completely non-normal. But the proportion of heads in 100 coin flips has a sampling distribution that is approximately normal with mean 0.5 and standard error 0.05. Similarly, Poisson-distributed count data, when averaged across large samples, produces a normally distributed sampling distribution. This is what allows us to use z-tests, t-tests, and other normal-theory-based tools on non-normal data, provided n is large enough. The relationship between the CLT and sampling distributions is something every statistics student needs to understand deeply, not just superficially.

How to Choose the Right Probability Distribution for Your Data

Choosing the wrong probability distribution for a problem is one of the most consequential mistakes in statistics. It leads to incorrect probability estimates, flawed inferences, and conclusions that do not reflect reality. The right choice depends on the nature of the random variable, the structure of the data-generating process, and the questions being asked. Here is a systematic approach to making the right call.

A related but advanced challenge is model selection — choosing between competing models for the same data. When two distributions both seem plausible, information criteria like AIC and BIC help identify which model fits the data best relative to its complexity. This matters most in regression modeling and machine learning feature engineering.

Real-World Applications of Probability Distributions

Probability distributions are not abstract mathematical constructs sitting in textbooks. They are active tools used daily across industries where decisions depend on quantifying uncertainty. Understanding where each probability distribution is applied in practice sharpens your ability to choose the right model, interpret results, and communicate findings. Here are the key application domains where probability distributions drive real decisions.

Medicine and Clinical Trials

Clinical trials rest on probability distributions at every stage. The binomial distribution is used to model the number of patients who respond to a treatment in a fixed-size trial group. Normal distributions underlie the t-tests and z-tests used to compare treatment effects between groups. The log-normal distribution is widely used for drug concentration-time profiles in pharmacokinetics because biological data like blood drug levels is typically right-skewed. The chi-square distribution tests whether the observed proportion of adverse events across treatment arms is independent of the treatment received. Statistical power analysis — essential for determining the required sample size in clinical trials — relies on [power analysis and Cohen’s d](https://ivyleagueassignmenthelp.com/power-analysis-cohens-d-guide/) calculations built on the normal and t-distributions.

The U.S. Food and Drug Administration (FDA) and the National Institutes of Health (NIH) both require rigorous probability-based statistical analysis in any clinical trial submission. Understanding probability distributions is not optional for anyone working in clinical research, medical statistics, or public health. According to research published in statistics in medicine, incorrect application of probability distributions is one of the most frequent statistical errors in published medical literature — an issue with genuine patient safety implications.

Finance and Risk Modeling

Financial risk models depend on probability distributions to price assets, measure risk, and simulate portfolio performance. The normal distribution forms the foundation of Modern Portfolio Theory, developed by Harry Markowitz at the University of Chicago, and is used to model daily stock returns in classical finance. The log-normal distribution is used to model stock prices themselves (since prices cannot go negative), and underpins the famous Black-Scholes option pricing model. For extreme risk — the “fat tail” risks that caused the 2008 financial crisis — practitioners use heavy-tailed distributions like the Pareto or Student’s t-distribution. Value at Risk (VaR) — a regulatory risk metric required by Basel III for banks — is computed directly from probability distributions applied to portfolio returns.

Engineering and Quality Control

Manufacturing quality control uses the binomial and Poisson distributions to model defect rates. The normal distribution is central to Statistical Process Control (SPC) — the system of control charts like the X-bar chart, S chart, and p-chart used to monitor production processes. If a manufacturing process is producing parts where the diameter follows a normal distribution with μ = 50mm and σ = 0.5mm, and the specification limits are 49mm to 51mm, the normal distribution tells you exactly what percentage of parts will be out of spec. The Weibull distribution is ubiquitous in reliability engineering, modeling the time to failure of mechanical and electronic components. The exponential distribution models the time between equipment failures in systems with constant failure rates.

Machine Learning and Data Science

Probability distributions are woven into modern machine learning at every level. Naive Bayes classifiers assume that features follow specific distributions (often Gaussian for continuous features) conditional on the class label. Generative models like Gaussian Mixture Models explicitly model data as samples from a mixture of probability distributions. Bayesian inference — now a major paradigm in statistics and machine learning — specifies prior distributions over model parameters and updates them with data to obtain posterior distributions. Our article on Bayesian inference explains this framework in depth. Regularization methods like Ridge and Lasso regression have probabilistic interpretations as maximum a posteriori (MAP) estimation under specific prior distributions over the parameters. See our regularization guide for the full treatment.

Actuarial Science and Insurance

Actuaries at firms like Aon, Mercer, and Swiss Re rely on probability distributions to price insurance policies, set reserves, and manage financial risk. Life tables and mortality models use distributions fitted to historical death data. Claim frequency models use the Poisson or negative binomial distributions to model how many claims will be filed in a period. Claim severity models use the log-normal, gamma, or Pareto distribution to model the size of individual claims. The compound Poisson distribution — which combines a Poisson-distributed number of events with individual event severities drawn from a separate distribution — is a standard model for total insurance losses.

Social Sciences and Psychology

Psychological measurement relies on the normal distribution as the assumed model for many latent constructs: intelligence, personality traits, and self-reported attitudes. Item Response Theory (IRT), used by testing organizations like ETS (Educational Testing Service) to develop and score exams like the SAT, GRE, and TOEFL, uses logistic distributions to model the probability of a correct response as a function of ability. The t-distribution underpins the t-tests used in experimental psychology. The chi-square distribution tests independence in contingency tables of survey data. The F-distribution structures ANOVA designs comparing treatment effects across multiple groups.

Advanced Probability Distributions: Beta, Gamma, Log-Normal, and More

Beyond the common distributions, statistics courses at the graduate level and in applied fields introduce a set of more specialized probability distributions. These are not merely academic curiosities. Each was developed because the common distributions failed to adequately model a specific class of real-world data. Understanding when and why each applies is the mark of statistical sophistication.

What Is the Beta Distribution?

The beta distribution is defined on the interval [0, 1], making it the natural model for random variables that represent proportions, probabilities, or rates — quantities that are inherently bounded between 0 and 1. It is defined by two shape parameters, α and β, and can take a wide range of shapes: symmetric (when α = β), left-skewed (α > β), right-skewed (α < β), U-shaped (α and β both less than 1), or uniform (α = β = 1). In Bayesian inference, the beta distribution is the conjugate prior for the binomial likelihood — meaning if you use a beta prior on a probability p and observe binomial data, the posterior is also a beta distribution. This computational convenience makes it indispensable in Bayesian analysis of proportions, click-through rates, and clinical response rates.

What Is the Gamma Distribution?

The gamma distribution is a flexible, right-skewed continuous distribution for positive-valued data. It generalizes the exponential distribution: where the exponential models the time to the first event in a Poisson process, the gamma models the time to the k-th event. Two parameterizations are in common use: the shape-rate form (α, β) and the shape-scale form (k, θ). The gamma is widely used to model insurance claim severities, income distributions, rainfall amounts, and survival times. The chi-square distribution is a special case of the gamma, which connects the two distributions mathematically.

What Is the Log-Normal Distribution?

A random variable X follows a log-normal distribution if log(X) follows a normal distribution. Log-normal data is always positive and right-skewed. It arises naturally when a random variable is the product of many independent positive random variables — and multiplicative processes are common in nature and economics. Stock prices, incomes, particle sizes, and biological measurement data (enzyme concentrations, blood pressure measurements in specific contexts) often follow log-normal distributions. When you take the logarithm of log-normal data, you get normally distributed data — which means you can apply all the tools of normal-based inference after log-transforming the variable.

What Is the Weibull Distribution?

The Weibull distribution is the most flexible distribution in reliability engineering and survival analysis. It can model increasing failure rates (components that wear out), constant failure rates (exponential — random failures), and decreasing failure rates (infant mortality — early failures predominate). The Weibull has a shape parameter k and a scale parameter λ. When k = 1, it reduces to the exponential distribution. When k = 3 to 4, it closely approximates the normal distribution. This flexibility makes it the standard choice for modeling time-to-failure data across aerospace, manufacturing, and biomedical engineering. It is a core distribution in survival analysis.

What Is the Dirichlet Distribution?

The Dirichlet distribution is the multivariate generalization of the beta distribution. Where the beta models a single probability on [0, 1], the Dirichlet models a vector of probabilities that sum to 1 — a probability simplex. It is the standard prior distribution over the parameters of a multinomial distribution in Bayesian inference, making it essential in natural language processing (topic models like Latent Dirichlet Allocation), genetics (modeling allele frequencies), and any application involving compositional data. Like the beta for binomial problems, the Dirichlet is the conjugate prior for the multinomial likelihood.

For those working on time series data, another important family of distributions enters the picture. ARIMA models for time series are built on assumptions about the distribution of residuals, and understanding how the underlying probability distribution of the error term affects model performance is covered in our time series analysis and ARIMA guide.

Joint, Marginal, and Conditional Probability Distributions

So far, we have focused on distributions of a single random variable. But many real-world problems involve the joint behavior of two or more random variables. How are they related? Does knowing the value of one tell you something about the other? These questions require joint probability distributions — the extension of single-variable distributions to multiple variables.

What Is a Joint Probability Distribution?

A joint probability distribution describes the simultaneous behavior of two or more random variables. For two discrete random variables X and Y, the joint PMF gives P(X = x, Y = y) for all pairs (x, y). For two continuous variables, the joint PDF f(x, y) gives the density at each point in the (x, y) plane. The fundamental requirement: the joint probabilities must sum or integrate to 1 over all possible pairs of values.

What Is a Marginal Distribution?

The marginal distribution of one variable is obtained from the joint distribution by summing (discrete) or integrating (continuous) over all values of the other variable. It gives you the distribution of one variable ignoring the other. The name comes from the practice of computing these sums in the margins of a joint probability table.

What Is a Conditional Distribution?

The conditional distribution of X given Y = y describes the distribution of X when you know that Y has taken a specific value y. It is computed as P(X = x | Y = y) = P(X = x, Y = y) / P(Y = y). Conditional distributions are central to Bayes’ theorem and to Bayesian statistics broadly, where the posterior distribution of a parameter given observed data is a conditional distribution. Our guide to Bayesian inference develops this framework in full.

What Is the Multivariate Normal Distribution?

The multivariate normal distribution generalizes the univariate normal to multiple variables simultaneously. It is characterized by a mean vector μ and a covariance matrix Σ that encodes both the variances of individual variables and the correlations between all pairs. If two variables follow a bivariate normal distribution with correlation ρ, knowing the value of one variable gives you a normally distributed conditional distribution for the other. The multivariate normal is the foundation of principal component analysis (PCA), factor analysis, discriminant analysis, and multivariate regression. Our PCA guide explains how the multivariate normal underlies dimensionality reduction, and our factor analysis article covers the latent variable interpretation.

Understanding how correlation enters the multivariate normal — encoded in the off-diagonal elements of the covariance matrix Σ — is also essential background for MANOVA, the multivariate extension of ANOVA.

How to Fit a Probability Distribution to Real Data

Knowing the theoretical distributions is necessary. Applying them to actual data requires parameter estimation — figuring out the values of μ, σ, λ, p, or whatever parameters define the chosen distribution based on your observed sample. Two major methods dominate statistical practice: Method of Moments (MOM) and Maximum Likelihood Estimation (MLE).

Method of Moments

The method of moments sets the theoretical moments of a distribution (mean, variance, skewness) equal to the sample moments computed from the data, and solves for the unknown parameters. For a normal distribution, this is trivially simple: the moment estimators for μ and σ are just the sample mean x̄ and sample standard deviation s. For a gamma distribution with two parameters, you set the first and second theoretical moments equal to the sample mean and variance and solve the two equations simultaneously. MOM estimators are easy to compute but are not always the most statistically efficient.

Maximum Likelihood Estimation

Maximum Likelihood Estimation (MLE) finds the parameter values that make the observed data most probable under the assumed distribution. You write down the likelihood function — the probability of observing the actual data as a function of the parameters — and then maximize it. For the normal distribution, MLE yields the sample mean and (biased) standard deviation. For the binomial, MLE for p is the sample proportion of successes. For the Poisson, MLE for λ is the sample mean. MLE estimators are generally efficient, consistent, and asymptotically normal — properties that make them theoretically superior to MOM for most applications. MLE is the foundation of logistic regression and most of modern statistical modeling.

Testing Goodness of Fit

After estimating parameters, you need to test whether the data actually fits the chosen distribution — not just assumed. The chi-square goodness-of-fit test divides the data into bins, compares observed frequencies with expected frequencies under the fitted distribution, and computes a chi-square statistic. If the statistic is large relative to the chi-square distribution with appropriate degrees of freedom, you reject the proposed distribution. The Kolmogorov-Smirnov (K-S) test compares the empirical CDF with the theoretical CDF and identifies the maximum deviation between them. The Anderson-Darling test gives more weight to the tails, making it more sensitive for detecting distributional mismatches in the extremes. All three tests are implemented in R, Python, and SPSS. Choosing between them depends on sample size, the importance of tail behavior, and the specific distribution being tested. For a complete treatment of hypothesis testing frameworks, see our hypothesis testing guide and our coverage of Type I and Type II errors.

Probability Distributions: Complete Quick-Reference Summary

Use this table as your go-to reference when identifying and applying probability distributions in coursework, exams, or professional analysis. Each distribution is listed with its type, defining characteristics, key formula, and primary real-world applications.

Frequently Asked Questions About Probability Distributions