Probability Theory

From dice rolls to stochastic differential equations

About This Series

Probability theory is the branch of mathematics that treats uncertain phenomena such as rolling dice and tossing coins. By making "chance" rigorously quantifiable, it provides a scientific foundation for prediction and decision-making.

Probability is used everywhere in the modern world: statistics, machine learning, financial engineering, physics, the life sciences, and beyond. This series builds up probability systematically in four levels, from a high school introduction through graduate-level stochastic processes.

Study by Level

Introduction

High school level

  • What is probability?
  • Permutations and combinations
  • Definition of probability and the addition rule
  • Conditional probability
  • Common pitfalls and misconceptions

Basics

Undergraduate years 1–2

  • Definition of probability spaces
  • Conditional probability and independence
  • Bayes' theorem
  • Random variables and expectation
  • Discrete distributions (binomial, Poisson)

Intermediate

Undergraduate years 3–4

  • Axiomatization of probability
  • σ-algebras and measures
  • Continuous distributions and density functions
  • Lebesgue integration and expectation
  • Modes of convergence and the law of large numbers

Advanced

Graduate level

  • Introduction to stochastic processes
  • Markov processes
  • Brownian motion
  • Stochastic differential equations
  • Central limit theorem and information theory

Learning Path

Introduction High school Basics Undergrad 1–2 Intermediate Undergrad 3–4 Advanced Graduate Intro: probability concept, permutations & combinations, conditional probability Basics: probability spaces, Bayes' theorem, discrete distributions Intermediate: measure-theoretic probability, convergence, law of large numbers Advanced: stochastic processes, Brownian motion, stochastic differential equations

Key Topics

Foundations of Probability

Definition of probability, the addition rule, conditional probability — the basic concepts of the theory.

Probability Distributions

Binomial, Poisson, normal, and other distributions that arise throughout probability and statistics.

Limit Theorems

The law of large numbers, the central limit theorem — the load-bearing theorems of probability theory.

Stochastic Processes

Markov processes, Brownian motion, and stochastic differential equations — randomness evolving in time.

Key Formulas

Kolmogorov's Axioms

$$P(\Omega) = 1, \quad P(A) \geq 0$$

Conditional Probability

$$P(A|B) = \dfrac{P(A \cap B)}{P(B)}$$

Bayes' Theorem

$$P(A|B) = \dfrac{P(B|A)P(A)}{P(B)}$$

Expectation and Variance

$$E[X], \quad V[X] = E[X^2] - (E[X])^2$$