KindiakMath

Category: Exercises

The Gaussian Integral

Problem 1. Evaluate the Gaussian integral $\displaystyle \int_{-\infty}^{\infty} e^{-x^2}\, \mathrm dx$ .

(Click for Solution)

Solution. Since $e^{-x^2}$ is even in $x$ ,

$\displaystyle \int_{-\infty}^{\infty} e^{-x^2}\, \mathrm dx = 2 \cdot \int_{0}^{\infty} e^{-x^2}\, \mathrm dx.$

For the right-hand side,

$\begin{aligned}\left( \int_{0}^{\infty} e^{-x^2}\, \mathrm dx \right)^2 &= \left( \int_{0}^{\infty} e^{-x^2}\, \mathrm dx \right) \cdot \left( \int_{0}^{\infty} e^{-y^2}\, \mathrm dy \right) \\ &= \int_0^\infty e^{-x^2} \cdot \left( \int_0^\infty e^{-y^2}\, \mathrm dy \right)\, \mathrm dx \\ &= \int_0^\infty e^{-x^2} \cdot \left( \int_0^\infty e^{-u^2x^2} \cdot x\, \mathrm du \right)\, \mathrm dx \\ &= \int_0^\infty \int_0^\infty xe^{-x^2(u^2+1)} \, \mathrm du\, \mathrm dx. \end{aligned}$

By Fubini’s theorem,

$\begin{aligned}\left( \int_{0}^{\infty} e^{-x^2}\, \mathrm dx \right)^2 &= \int_0^\infty \int_0^\infty xe^{-x^2(u^2+1)} \, \mathrm du\, \mathrm dx\\ &= \int_0^\infty \int_0^\infty xe^{-x^2(u^2+1)} \, \mathrm dx\, \mathrm du \\ &= \int_0^\infty \left[ -\frac 1{2(u^2+1)} \cdot e^{-x^2(u^2+1)} \right]_0^\infty\, \mathrm du \\ &= \int_0^\infty \frac 1{2(u^2+1)}\, \mathrm du \\ &= \frac 12 \cdot [\tan^{-1}(u)]_0^{\infty} \\ &= \frac 12 \cdot \frac{\pi}2 = \frac{\pi}{4}. \end{aligned}$

Taking square roots,

$\displaystyle \int_{0}^{\infty} e^{-x^2}\, \mathrm dx = \frac{\sqrt{\pi}}{2} \quad \Rightarrow \quad \int_{-\infty}^{\infty} e^{-x^2}\, \mathrm dx = \sqrt{\pi}.$

This calculation was authored by Hirokazu Iwasawa.

—Joel Kindiak, 26 Jul 25, 2156H

January 2, 2026
The Poisson Distribution

Definition 1. The random variable $X : \Omega \to \mathbb N_0$ follows a Poisson distribution with rate parameter $\lambda > 0$ , denoted $X \sim \mathrm{Pois}(\lambda)$ , if

$\displaystyle \mathbb P(X=x) = c_\lambda \cdot \frac{\lambda^x}{x!}$

for some $c_\lambda > 0$ .

Problem 1. Evaluate $c_\lambda$ .

(Click for Solution)

Solution. We require $\sum_{x = 0}^\infty \mathbb P(X=x) = 1$ :

$\begin{aligned}1 = \sum_{x = 0}^\infty \mathbb P(X=x) &= \sum_{x =0}^\infty c_\lambda \cdot \frac{\lambda^x}{x!} = c_\lambda \cdot \sum_{x = 0}^\infty \frac{\lambda^x}{x!} = c_\lambda \cdot e^{\lambda}. \end{aligned}$

Therefore, $c_\lambda = e^{-\lambda}$ .

Problem 2. Evaluate $\mathbb E[X]$ and $\mathrm{Var}(X)$ .

(Click for Solution)

Solution. By definition of the expectation,

$\begin{aligned} \mathbb E[X] &= \sum_{x = 0}^\infty x \cdot \mathbb P(X=x) \\ &= \sum_{x = 1}^\infty x \cdot c_\lambda \cdot \frac{\lambda^x}{x!} \\ &= \lambda \cdot \sum_{x = 1}^\infty c_\lambda \cdot \frac{\lambda^{x-1}}{(x-1)!} \\ &= \lambda \cdot \sum_{x = 0}^\infty c_\lambda \cdot \frac{\lambda^{x}}{x!} = \lambda \cdot 1 = \lambda. \end{aligned}$

For the variance, we compute the term

$\begin{aligned}\mathbb E[X(X-1)] &= \sum_{x = 0}^\infty x(x-1) \cdot \mathbb P(X=x) \\ &= \sum_{x = 2}^\infty x(x-1) \cdot c_\lambda \cdot \frac{\lambda^x}{x!} \\ &= \lambda^2 \cdot \sum_{x = 2}^\infty c_\lambda \cdot \frac{\lambda^{x-2}}{(x-2)!} \\ &= \lambda^2 \cdot 1 = \lambda^2. \end{aligned}$

Therefore,

$\begin{aligned} \mathrm{Var}(X) &= \mathbb E[X^2] - \mathbb E[X]^2 \\ &= \mathbb E[X(X-1)] + \mathbb E[X] - \mathbb E[X]^2 \\ &= \lambda^2 + \lambda - \lambda^2 = \lambda.\end{aligned}$

Problem 3. Given that $X \sim \mathrm{Pois}(\lambda)$ and $Y \sim \mathrm{Pois}(\mu)$ are independent, determine the distribution of $X + Y$ .

(Click for Solution)

Solution. Denoting $W := X+Y$ , we take the discrete convolution of the p.d.f.s of $X,Y$ to obtain

$\begin{aligned}f_W (w) &= \sum_{x = 0}^w f_X(x) \cdot f_Y(w-x) \\ &= \sum_{x=0}^w c_\lambda \cdot \frac{\lambda^x}{x!} \cdot c_\mu \cdot \frac{\lambda^{w-x}}{(w-x)!} \\ &= \sum_{x=0}^w c_\lambda \cdot \frac{\lambda^x}{x!} \cdot c_\mu \cdot \frac{\mu^{w-x}}{(w-x)!} \\ &= c_\lambda \cdot c_\mu \cdot \sum_{x=0}^w \frac{\lambda^x}{x!} \cdot \frac{\mu^{w-x}}{(w-x)!} \\ &= c_\lambda \cdot c_\mu \cdot \frac 1{w!} \cdot \sum_{x=0}^w \frac{w!}{x! \cdot (w-x)!} \cdot \lambda^x \cdot \mu^{w-x} \\ &= c_\lambda \cdot c_\mu \cdot \frac 1{w!} \cdot \sum_{x=0}^w {w \choose x} \cdot \lambda^x \cdot \mu^{w-x} \\ &= c_\lambda \cdot c_\mu \cdot \frac {(\lambda + \mu)^w}{w!}.\end{aligned}$

Furthermore, $c_\lambda \cdot c_\mu = e^{-\lambda} \cdot e^{\mu} = e^{-(\lambda + \mu)} = c_{\lambda + \mu}$ . Hence,

$\displaystyle \mathbb P(X+Y = w) = c_{\lambda + \mu} \cdot \frac{(\lambda + \mu)^w}{w!},$

so that $X +Y \sim \mathrm{Pois}(\lambda + \mu)$ .

Problem 4. Fix $\lambda > 0$ . Suppose $X_n \sim \mathrm{Bin}(n, \lambda/n)$ and $Y \sim \mathrm{Pois}(\lambda)$ . Prove that $f_{X_n} \to f_Y$ .

(Click for Solution)

Solution. Fix $y \in \mathbb N_0$ . For each $n$ ,

$\begin{aligned} f_{X_n}(y) &= {n \choose y} \left(1 - \frac{\lambda}{n}\right)^{n-y} \left( \frac{\lambda}n \right)^y \\ &= \frac{n(n-1) \cdot \dots \cdot (n-y+1)}{y!} \cdot \frac{\lambda^y }{n^y } \cdot \left(1 - \frac{\lambda}{n}\right)^{n-y} \\ &= \frac{n}{n} \cdot \frac{n-1}{n} \cdot \cdots \cdot \frac{n-y+1}n \cdot \left(1 - \frac{\lambda}{n}\right)^{-y} \cdot \left(1 - \frac{\lambda}{n}\right)^{n} \cdot \frac{\lambda^y }{y! } \end{aligned}$

Taking $n \to \infty$ ,

$\displaystyle \lim_{n \to \infty} f_{X_n}(y) = 1 \cdot 1 \cdot e^{-\lambda} \cdot \frac{\lambda^y}{y!} = e^{-\lambda} \cdot \frac{\lambda^y}{y!} = f_Y(y).$

—Joel Kindiak, 1 Aug 25, 1751H

December 30, 2025
Proving Feynman’s Trick
Feynman’s trick in differentiating under the integral sign has been creatively wielded to evaluate otherwise intractable integrals. In this exercise, we prove Feynman’s trick and use it to evaluate the seemingly intractable Dirichlet integral

$\displaystyle \int_{-\infty}^{\infty} \frac{\sin x}{x}\, \mathrm dx.$

Let $(\Omega, \mathcal F, \mathbb R)$ be a measure space and $f : \Omega \times [a, b] \to \mathbb R$ be a function such that for each $t \in [a, b]$ , $f( \cdot, t)$ is measurable.

Problem 1. Suppose the following conditions:
- For any $\omega \in \Omega$ , $f(\omega, \cdot)$ is continuous.
- There exists some non-negative integrable $g : \Omega \to \mathbb R$ such that for any $(\omega, t) \in \Omega \times [a, b]$ , $|f( \omega ,t)| \leq g(\omega)$ .
Prove that the map $F : [a, b] \to \mathbb R$ defined by $\displaystyle F(t) : = \int_{\Omega} f(\cdot , t)\, \mathrm d\mu$ is continuous.

(Click for Solution)

Solution. Fix $t_n \to t$ . For any $\omega \in \Omega$ , since $f(\omega, \cdot)$ is continuous,

$f(\omega, t_n) \to f(\omega, t),$

so that $f(\cdot, t_n) \to f(\cdot, t)$ pointwise. Furthermore,

$\displaystyle \int_{\Omega} |f(\cdot, t)|\, \mathrm d\mu \leq \int_{\Omega} g\, \mathrm d\mu < \infty,$

so that $f(\cdot, t_n)$ and $f(\cdot, t)$ are all integrable.

Since $g$ is integrable, by Lebesgue’s dominated convergence theorem,

$\displaystyle F(t_n) = \lim_{n \to \infty} \int_{\Omega} f(\cdot , t_n)\, \mathrm d\mu= \int_{\Omega} f(\cdot , t)\, \mathrm d\mu = F(t),$

so that $F$ is continuous, as required.

Problem 2. Suppose the following conditions:
- There exists some $t_0 \in [a, b]$ such that $f(\cdot, t_0)$ is integrable.
- For each $\omega \in \Omega$ , $f(\omega, \cdot)$ is differentiable with derivative at $t_0$ denoted by $\frac{\partial f}{\partial t}(\omega,t_0)$ .
- There exists some non-negative integrable $g : \Omega \to \mathbb R$ such that for any $(\omega, t) \in \Omega \times [a, b]$ , $\left|\frac{\partial f}{\partial t} (\omega,t) \right| \leq g( \omega )$ .
Prove that the map $F : [a, b] \to \mathbb R$ defined by $\displaystyle F(t) : = \int_{\Omega} f(\cdot, t)\, \mathrm d\mu$ is differentiable on $[a, b]$ and

$\displaystyle F'(t) = \frac{\mathrm d}{\mathrm dt} \int_{\Omega} f(\omega, t)\, \mathrm d\mu(\omega) = \int_{\Omega} \frac{\partial f}{\partial t}(\omega, t)\, \mathrm d\mu(\omega).$

(Click for Solution)

Solution. We first check that $F$ is well-defined. By hypothesis, $F(t_0)$ is well-defined. Fix $t \in [a, b]$ . By the mean value theorem, there exists $c$ between $t_0$ and latex t$ such that

$\displaystyle \frac{ |f(\omega, t) - f(\omega, t_0)| }{ |t - t_0| } = \left| \frac{ \partial f }{ \partial t} (\omega, c) \right| \leq g(\omega).$

By performing more analysis, $f(\cdot, t)$ is integrable, so that $F(t)$ is well-defined.

Now fix $\omega \in \Omega$ . For any $t_n \to t$ , since each $f(\cdot, t_n) - f(\cdot, t)$ is measurable,

$\displaystyle \frac{\partial f}{\partial t}(\omega, t) := \lim_{n \to \infty} \underbrace{ \frac{f(\omega, t_n) - f(\omega, t)}{ t_n - t} }_{\varphi_n(\omega)}$

is measurable. Furthermore, $\varphi_n \to \frac{\partial f}{\partial t} (\cdot, t)$ pointwise. We claim that $|\varphi_n| \leq g$ , since the mean value theorem gives $c$ between $t_n$ and $t$ such that

$\displaystyle \frac{ | f(\omega, t_n) - f(\omega, t) |}{ | t_n - t | } = \left| \frac{\partial f}{\partial t}(\omega, c) \right| \leq g(\omega).$

By algebruh and the triangle inequality, each $\varphi_n$ is integrable. Hence, by Lebesgue’s dominated convergence theorem,

$\begin{aligned} \lim_{t_n \to t} \int_{\Omega} \frac{f(\omega, t_n) - f(\omega, t)}{t_n - t}\, \mathrm d\mu(\omega) &= \lim_{n \to \infty} \int_{\Omega} \varphi_n\, \mathrm d\mu \\ &= \int_{\Omega} \lim_{n \to \infty} \varphi_n\, \mathrm d\mu = \int_{\Omega} \frac{\partial f}{\partial t} (\cdot, t)\, \mathrm d\mu. \end{aligned}$

On the other hand, by bookkeeping

$\begin{aligned} \frac{ \mathrm d }{ \mathrm dt } F(t) &= \lim_{ t_n \to t } \frac{ F(t_n) - F(t) }{ t_n - t } \\ &= \lim_{t_n \to t} \int_{\Omega} \frac{f(\omega, t_n) - f(\omega, t)}{t_n - t}\, \mathrm d\mu(\omega) \\ &= \int_{\Omega} \frac{\partial f}{\partial t} (\omega, t)\, \mathrm d\mu(\omega). \end{aligned}$

Therefore,

$\displaystyle F'(t) = \frac{\mathrm d}{\mathrm dt} \int f(\omega, t)\, \mathrm d\mu(\omega) = \int \frac{\partial f}{\partial t}(\omega, t)\, \mathrm d\mu(\omega).$

Remark 1. Thanks to Problem 2, our proof that $\mathcal L\{tf'(t)\} = -\mathcal L\{f\}'(s)$ in the study of differential equations becomes a logically correct one.

Problem 3. Use Problem 2 to evaluate $\displaystyle \int_{-\infty}^{\infty} \frac{\sin x}{x}\, \mathrm dx$ .

(Click for Solution)

Solution. Define the function $I$ by

$\displaystyle I(y) := \int_{0}^{\infty} \frac{\sin x}{x} \cdot e^{-xy}\, \mathrm dy$

that satisfies the hypotheses of Problem 2, and our goal is to evaluate $2 \cdot I(0)$ . Applying Problem 2 and integrating by parts,

$\begin{aligned} I'(y) &= \int_{0}^{\infty} \frac{\partial}{\partial y} \left( \frac{\sin x}{x} \cdot e^{-xy} \right) \, \mathrm dx \\ &= \int_{0}^{\infty} \frac{\sin x}{x} \cdot -xe^{-xy} \, \mathrm dx \\ &= -\int_{0}^{\infty} e^{-xy} \cdot \sin x \, \mathrm dx \\ &= -\left[ \frac{e^{-xy}}{(-y)^2 + 1^2} \cdot ((-y) \sin(x) - \cos(x)) \right]_{0}^{\infty} \\ &= -\frac{1}{1+y^2}. \end{aligned}$

Integrating and applying the first fundamental theorem of calculus,

$\displaystyle I(y) = I(0) - \int_0^y \frac{1}{1+y^2}\, \mathrm dy = I(0) - \tan^{-1}(y).$

Since $I$ is continuous,

$\displaystyle \lim_{y \to \infty} I(y) = \int_0^\infty \lim_{y \to \infty} \frac{\sin x}{x} \cdot e^{-xy}\, \mathrm dx = \int_0^\infty 0\, \mathrm dx = 0.$

Taking $y \to \infty$ on all sides, therefore,

$\displaystyle 0 = I(0) - \frac{\pi}{2} \quad \Rightarrow \quad I(0) = \frac{\pi}{2}.$

Therefore, the Dirichlet integral evaluates to

$\displaystyle \int_{-\infty}^{\infty} \frac{\sin x}{x}\, \mathrm dx = 2 \cdot I(0) = \pi.$

—Joel Kindiak, 29 Jul 25, 1319H
December 26, 2025
Binomial Theorem Corollaries

Recall the binomial theorem, which states that for any $n \in \mathbb N$ and $x \in \mathbb R$ ,

$\displaystyle (1 + x)^n = \sum_{k=0}^n {n \choose k} x^k,$

where we define $0^0 := 1$ by convention.

Problem 1. Prove that for any $a, b \in \mathbb R$ ,

$\displaystyle (a + b)^n = \sum_{k=0}^n {n \choose k} a^{n-k} b^k.$

(Click for Solution)

Solution. If $a = 0$ then the result is trivial. If $a > 0$ , then

$\begin{aligned} (a+b)^n &= a^n \cdot \left(1 + \frac ba\right)^n = a^n \cdot \sum_{k=0}^n \left( \frac ba \right)^k \\ &= a^n \cdot \sum_{k=0}^n a^{-k} \cdot b^k = \sum_{k=0}^n a^{n-k} \cdot b^k. \end{aligned}$

Problem 2. Evaluate the sums $\displaystyle \sum_{k=0}^n {n \choose k}$ and $\displaystyle \sum_{k=0}^n (-1)^k {n \choose k}$ .

(Click for Solution)

Solution. By Problem 1,

$\displaystyle (1 + (\pm 1))^n = \sum_{k=0}^n (\pm 1)^k {n \choose k}.$

Therefore, $\displaystyle \sum_{k=0}^n {n \choose k} = (1+1)^n = 2^n$ and $\displaystyle \sum_{k=0}^n (-1)^k {n \choose k} = (1-1)^n = 0$ .

Problem 3. Prove that $\displaystyle \sum_{k \in \mathbb N_0 :\, 2k \leq n} {n \choose 2k} = \sum_{k \in \mathbb N_0 :\, 2k+1 \leq n} {n \choose 2k+1}$ .

(Click for Solution)

Solution. By Problem 2,

$\begin{aligned} 0=\sum_{k=0}^n (-1)^k {n \choose k} &= \sum_{k \in \mathbb N_0 :\, 2k \leq n} (-1)^{2k} {n \choose 2k} + \sum_{k \in \mathbb N_0 :\, 2k+1 \leq n} (-1)^{2k+1} {n \choose 2k+1} \\ &= \sum_{k \in \mathbb N_0 :\, 2k \leq n} {n \choose 2k} - \sum_{k \in \mathbb N_0 :\, 2k+1 \leq n} {n \choose 2k+1}. \end{aligned}$

Therefore,

$\displaystyle \sum_{k \in \mathbb N_0 :\, 2k \leq n} {n \choose 2k} = \sum_{k \in \mathbb N_0 :\, 2k+1 \leq n} {n \choose 2k+1}.$

Problem 4. Evaluate the sums $\displaystyle \sum_{k= 1}^n k {n \choose k}$ and $\displaystyle \sum_{k= 1}^n k^2 {n \choose k}$ .

(Click for Solution)

Solution. Recall the vanilla binomial theorem

$\displaystyle (1 + x)^n = \sum_{k=0}^n {n \choose k} x^k.$

Differentiating on both sides twice,

$\begin{aligned} n(1+x)^{n-1} &= \sum_{k=1}^{n} k {n \choose k} x^{k-1}, \\ n(n-1)(1+x)^{n-2} &= \sum_{k=2}^{n} k(k-1) {n \choose k} x^{k-2}. \end{aligned}$

Setting $x = 1$ in the first identity,

$\displaystyle \sum_{k=1}^{n} k {n \choose k} = n \cdot 2^{n-1}.$

Setting $x = 1$ in the second identity,

$\begin{aligned} n(n-1)\cdot 2^{n-2} &= \sum_{k=2}^{n} k(k-1) {n \choose k} = \sum_{k=2}^{n} k^2 {n \choose k} - \sum_{k=2}^{n} k {n \choose k}. \end{aligned}$

By algebruh, since $\displaystyle 1^r {n \choose 1} = n$ for any $r$ ,

$\displaystyle \sum_{k=1}^{n} k^2 {n \choose k} = n(n-1)\cdot 2^{n-2} + n \cdot 2^{n-1} = n \cdot 2^{n-1} \cdot (2n-1).$

Problem 5. Evaluate $\displaystyle \sum_{k= 0}^r {m \choose k} {n \choose r-k}$ .

(Click for Solution)

Solution. Using the binomial theorem on the product

$\displaystyle (1+x)^{m+n} = (1+x)^m (1+x)^n,$

we have

$\begin{aligned} \sum_{r=0}^{m+n} {m+n \choose r} x^r &= \left(\sum_{k=0}^m {m \choose k} x^k\right) \cdot \left(\sum_{l=0}^n {n \choose l} x^l\right) \\ &= \sum_{k=0}^m \sum_{l=0}^n {m \choose k} {n \choose l} x^{k+l} \\ &= \sum_{r=0}^{m+n} \sum_{k+l=r} {m \choose k} {n \choose l} x^r \\ &= \sum_{r=0}^{m+n} \sum_{k=0}^r {m \choose k} {n \choose r-k} x^r. \end{aligned}$

Comparing the coefficients of $x^r$ ,

$\displaystyle \sum_{k= 0}^r {m \choose k} {n \choose r-k} = {m+n \choose r}.$

This result is known as Vandermonde’s identity.

Problem 6. Evaluate $\displaystyle \sum_{k= 0}^n {n \choose k}^2$ .

(Click for Solution)

Solution. Setting $m = n$ and $r = 2k$ in Problem 5,

$\displaystyle \sum_{k= 0}^n {n \choose k}^2 = \sum_{k= 0}^r {n \choose k} {n \choose 2k-k} = {n + n \choose k} = {2n \choose k}.$

—Joel Kindiak, 1 Aug 25, 1520H

December 23, 2025
Several Probability Puzzles

Problem 1. Let $X_1,\dots, X_n \sim \mathcal U(0, 1)$ be i.i.d.. Let $g : [0,1]^n \to [0,1]^n$ denote the permutation

$g(x_1,\dots, x_n) = (y_1,\dots, y_n)$

such that $y_1 \leq \dots \leq y_n$ . Denoting $(Y_1,\dots, Y_n) = g(X_1, \dots, X_n)$ , evaluate $\mathbb E[Y_i]$ for each $i$ .

(Click for Solution)

Solution. Since $\mathbb P (X_i = X_j) = 0$ whenever $i \neq j$ , we can assume $Y_1 < \dots < Y_n$ .

We will obtain the distribution of $Y_i$ . Fix $x \in [0, 1]$ . Let $V_x \sim \mathrm{Bin}(n-1, x)$ denote the number of sample points that are less than $x$ , which follows a binomial distribution. It follows that $\{ Y_i = x \} = \{ V_x = i-1 \}$ , so that

$\displaystyle f_{Y_i} (x) = \mathbb P(V_x = i-1) = {n-1 \choose i-1} x^{i-1} (1-x)^{n-i}.$

Hence, by recalling the properties of the Beta distribution,

$\begin{aligned} \mathbb E[Y_i] &= \int_0^1 x \cdot f_{Y_i} (x)\, \mathrm dx \\ &= \int_0^1 x \cdot {n-1 \choose i-1} x^{i-1} (1-x)^{n-i}\, \mathrm dx \\ &= \int_0^1 {n-1 \choose i-1} x^i (1-x)^{n-i}\, \mathrm dx \\ &= {n-1 \choose i-1} \int_0^1 x^i (1-x)^{n-i}\, \mathrm dx \\ &= \frac{\Gamma(n)}{\Gamma(i) \cdot \Gamma(n-i+1)} \cdot \frac{\Gamma(i+1) \cdot \Gamma(n-i+1)}{ \Gamma(n+1) } \\ &= \frac{\Gamma(n)}{\Gamma(i) \cdot \Gamma(n-i+1)} \cdot \frac{i \cdot \Gamma(i) \cdot \Gamma(n-i+1)}{ (n+1) \cdot \Gamma(n) } = \frac{i}{n+1}. \end{aligned}$

Problem 2. Calculate the average number of rolls of a fair six-sided die that you need to roll in order for the sum of all rolls to be a multiple of $6$ .

(Click for Solution)

Solution. Let $\xi_i$ denote the $i$ -th roll and $X_n := \sum_{i=1}^n \xi_i$ denote the sum of the first $n$ rolls. Define the stopping time $N$ by

$latex\displaystyle N := \inf_{n \in \mathbb N} \{6 \mid X_n\}.$

We claim that $N \sim \mathrm{Geom}(1/6)$ . For any $n \in \mathbb N$ ,

$\{N = n\} = \{6 \nmid X_1, \dots, 6 \nmid X_{n-1}, 6 \mid X_n\}.$

For each $i$ ,

$\{ 6 \nmid X_{i-1}, 6 \mid X_i\} = \{ \xi_i = X_i - X_{i-1} \},$

which is one of the six possible numbers with equal probability:

$\mathbb P(N = n) = \mathbb P(\{6 \nmid X_1, \dots, 6 \nmid X_{n-1}, 6 \mid X_n\}) = (5/6)^{n-1} \cdot (1/6).$

Therefore, $N \sim \mathrm{Geom}(1/6)$ so that $\mathbb E[N] = 1/(1/6) = 6$ as well.

Problem 3. What is the probability of getting an odd number of heads out of $n$ independent flips of a fair coin?

(Click for Solution)

Solution. Let $X \sim \mathrm{Bin}(n, 1/2)$ denote the number of heads out of $n$ independent flips of a fair coin. Then the required probability is

$\begin{aligned} p_{\text{odd}} &= \sum_{k\, \text{odd}}^{n} \mathbb P(X = k) = \sum_{ k\, \text{odd} }^{ n } {n \choose k} \frac 1{2^n}. \end{aligned}$

Using properties involving the binomial coefficient,

$\begin{aligned} 0 = (1+(-1))^n &= \sum_{k=0}^n {n \choose k} (-1)^k \\ &= \sum_{k\, \text{odd}}^n {n \choose k} (-1)^k + \sum_{k\, \text{even}}^n {n \choose k} (-1)^k \\ &= \sum_{k\, \text{odd}}^n {n \choose k} \cdot (-1) + \sum_{k\, \text{even}}^n {n \choose k} \cdot 1 \\ &= -\sum_{k\, \text{odd}}^n {n \choose k} + \sum_{k\, \text{even}}^n {n \choose k}. \end{aligned}$

Therefore,

$\displaystyle \sum_{k\, \text{odd}}^n {n \choose k} = \sum_{k\, \text{even}}^n {n \choose k}.$

In particular,

$\begin{aligned} p_{\text{even}} &= \sum_{ k\, \text{even} }^{ n } {n \choose k} \frac 1{2^n} = p_{\text{odd}}. \end{aligned}$

Since $p_{\text{odd}} + p_{\text{even}} = 1$ , we must have $p_{\mathrm{odd}} = 1/2$ , as required.

Problem 4. Given $X_1, X_2 \sim \mathcal U(0, 1)$ , calculate $\mathbb E[\min\{X_1,X_2\}]$ .

(Click for Solution)

Solution. Denoting $Y = \min\{X_1, X_2\}$ , we observe that

$\displaystyle \mathbb P(Y > y) = \mathbb P(X_1 > y, X_2 > y) = \mathbb P(X_1 > y) \cdot \mathbb P(X_2 > y) = (1-y)^2.$

Therefore, by the tail integral for expectation,

$\begin{aligned} \mathbb E[Y] &= \int_0^1 \mathbb P(Y > y) \cdot \mathbb I_{[0, 1]}(y) \, \mathrm dy = \int_0^1 (1-y)^2\, \mathrm dy = \int_0^1 y^2\, \mathrm dy = 1/3. \end{aligned}$

Problem 5. You’re the second-best player in a single-elimination tournament with $2^n$ players. Assume the brackets are randomly seeded, and the better player always wins each match. What is the probability you reach the finals?

(Click for Solution)

Solution. Each tournament will have $n$ stages, and at stage $i$ , there will be $2^{n+1-i}$ players. In order to reach the final stage, we need to be in a different “bracket” with the best player. At stage $1$ , there are two “brackets”, and each bracket has $2^{n-1}$ players. Therefore, the required probability is

$\displaystyle \frac{2^{n-1}}{2^n - 1}.$

Problem 6. Consider the sample space $\{0, 1\}$ and the sequence of random variables $X_0, X_1,\dots$ with the property that

$\mathbb P(X_{n+1} = 1 \mid X_n = 1) = p,\quad \mathbb P(X_{n+1} = 0 \mid X_n = 0) = q.$

Assuming that $X_n$ has identical distribution, evaluate $\mathbb P(X_n = 1)$ .

(Click for Solution)

Solution. Denote $\mathbb P(X_n = 1) = s$ . By the law of total probability,

$\begin{aligned} \mathbb P(X_{n+1} = 1) &= \mathbb P(X_{n+1} = 1 \mid X_n = 1) \cdot \mathbb P(X_n = 1) \\ &\phantom{==} + \mathbb P(X_{n+1} = 1 \mid X_n = 0) \cdot \mathbb P(X_n = 0) \\ s &= p \cdot s + (1 - q) \cdot (1-s) \\ (1- p) \cdot s &= (1-q) - (1-q) \cdot s \\ (2 - p - q) \cdot s &= 1 - q \\ s &= \frac{1-q}{2-p-q}. \end{aligned}$

—Joel Kindiak, 17 Oct 25, 1947H

December 19, 2025
Counting Arguments

Recall that the quantity $\displaystyle {n \choose r}$ was defined inductively using Pascal’s identity

$\displaystyle {n \choose r} = {n-1 \choose r} + {n-1 \choose r-1}.$

and denotes the number of $r$ -subsets of a set of size $n$ (i.e. $n$ distinct objects).

Problem 1. Prove that $\displaystyle {n \choose r} = {n \choose n-r}$ .

(Click for Solution)

Solution. Fix a set of $n$ distinct objects. There are $\displaystyle {n \choose r}$ possible $r$ -subsets of items that we can remove from that set. Therefore, every $(n-r)$ -subset of items left behind is obtained by exactly one corresponding $r$ -subset of items removed. Therefore, the number of $(n-r)$ -subsets (left behind) equals the number of $r$ -subsets (removed), yielding

$\displaystyle {n \choose r} = {n \choose n-r}.$

Problem 2. Prove that $\displaystyle {n \choose m} {m \choose r} = {n \choose r} {n-r \choose m-r}$ .

(Click for Solution)

Solution. We can interpret the identity as counting the number of $m$ -committeees out of $n$ persons, and among the $m$ persons, we choose $r$ persons in a “core team”. This is the quantity counted by the left-hand side:

$\displaystyle {n \choose m} {m \choose r}.$

On the right-hand side, we count the same quantity differently: first choose the $r$ “core team” members, then choose the remaining $(m-r)$ members out of the remaining $(n-r)$ persons:

$\displaystyle {n \choose r} {n-r \choose m-r}.$

Since both types of counting give the same total,

$\displaystyle {n \choose m} {m \choose r} = {n \choose r} {n-r \choose m-r}.$

Problem 3. Prove that $\displaystyle {n \choose r} = \frac nr {n-1 \choose r-1}$ .

(Click for Solution)

Solution. Replacing $(n,m,r)$ with $(n,r,1)$ in Problem 2,

$\displaystyle r \cdot {n \choose r} = {n \choose r} {r \choose 1} = {n \choose 1} {n-1 \choose r-1} = n \cdot {n-1 \choose r-1}.$

Problem 4. Prove that $\displaystyle {n \choose r} = \frac {n}{n-r} {n-1 \choose r}$ .

(Click for Solution)

Solution. Using Problems 1 and 3,

$\displaystyle {n \choose r} = {n \choose n-r} = \frac{n}{n-r} \cdot {n-1 \choose n-r-1} = \frac{n}{n-r} \cdot {n-1 \choose r}.$

Problem 5. Prove that $\displaystyle {n \choose r} = \frac {n-r+1}r {n \choose r-1}$ .

(Click for Solution)

Solution. Replacing $(n,m,r)$ with $(n,r,r-1)$ in Problem 2,

$\begin{aligned} r \cdot {n \choose r} ={n \choose r} {r \choose r-1} &= {n \choose r-1} {n-(r-1) \choose r-(r-1)} \\ &= (n-r+1) \cdot {n \choose r-1}. \end{aligned}$

Problem 6. For any $n \in \mathbb N^+, r \in \mathbb N_0$ , count the number of $n$ -tuples $(x_1,\dots,x_n) \in \mathbb N_0^n$ such that

$\displaystyle \sum_{k=1}^n x_k = r.$

(Click for Solution)

Solution. Consider a row of $r$ stars and $(n-1)$ bars. Let $x_1$ denote the number of stars before the 1st bar, $x_2$ denote the number of stars after the 1st bar and before the 2nd bar, and so on and so forth. Each arrangement of the $r$ stars and $(n-1)$ bars then corresponds to each desired $n$ -tuple. Thus, the required number is the number of places to place the bars:

$\displaystyle {r + (n-1) \choose (n-1)} = {n+r-1 \choose n-1} ={n+r-1 \choose r}.$

Problem 7. For any $n \in \mathbb N^+, p \in \mathbb N_0$ , suppose $p$ is prime. Count the number of $(2n)$ -tuples $(x_1,\dots,x_n,y_1,\dots, y_n) \in \mathbb N_0^n$ such that

$\displaystyle \left( \sum_{i=1}^n x_i \right) \cdot \left( \sum_{j=1}^n y_j \right) = p.$

(Click for Solution)

Solution. Since $p$ is prime, we require one of the sums to equal $1$ , and the other to equal $p$ . If the $x_i$ terms sum to $1$ , then there are a total of $n$ possible options of $(x_1,\dots, x_n)$ . By Problem 6, there are a total of

$\displaystyle {n+p-1 \choose p}$

options of $(y_1,\dots, y_n)$ . By symmetry, the required total is

$\displaystyle 2n \cdot {n+p-1 \choose p}.$

—Joel Kindiak, 31 Jul 25, 2114H

December 16, 2025
Multivariate Normal Bookkeeping
For random variables $X_1,\dots, X_n$ , denote $\mathbf X := (X_1,\dots, X_n)^{\mathrm T}$ and define

$\mathbb E[\mathbf X] = (\mathbb E[X_1], \dots, \mathbb E[X_n])^{\mathrm T}.$

Define the covariance matrix $\boldsymbol{ \Sigma }_{\mathbf X} \in \mathcal M_{n \times n}( \mathbb R )$ by

$\displaystyle (\boldsymbol{ \Sigma }_{\mathbf X})_{i,j} = \mathrm{Cov}(X_i, X_j).$

Note that $\boldsymbol{ \Sigma }_{\mathbf X} = \mathbb E[(\mathbf X - \mathbb E[\mathbf X]) (\mathbf X - \mathbb E[\mathbf X])^{\mathrm T}]$ .

Problem 1. For i.i.d. $Z_1,\dots, Z_n \sim \mathcal N(0, 1)$ , define $\mathbf Z := (Z_1,\dots, Z_n)^{\mathrm T}$ . Calculate the p.d.f. $f_{\mathbf Z}$ of $\mathbf Z$ and evaluate $\boldsymbol{ \Sigma }_{\mathbf Z}$ .

(Click for Solution)

Solution. For each $I$ , the p.d.f. $f_{Z_i}$ of $Z_i$ is

$\displaystyle f_{Z_i}(z_i) = \frac{1}{\sqrt{2\pi}} e^{-\frac{z_i^2}{2}}.$

Since $Z_1,\dots, Z_n$ are independent,

$\begin{aligned} f_{\mathbf Z}(\mathbf z) &= \prod_{i=1}^n f_{Z_i}(z_i) = \frac 1{ \sqrt{ (2\pi)^{n} } } e^{-\frac 12 \sum_{i=1}^n z_i^2} = \frac 1{ \sqrt{ (2\pi)^{n} } } e^{-\frac 12 \mathbf z^{\mathrm T} \mathbf z}.\end{aligned}$

Furthermore,

${ ( \boldsymbol{ \Sigma }_{\mathbf Z} )}_{i,j} = \mathrm{Cov}(Z_i, Z_j) = \mathbb I_{\{0\}} (i-j) = \mathbf I_{i,j},$

where $\mathbf I \equiv \mathbf I_n$ denotes the $n \times n$ identity matrix.

Problem 2. For any invertible matrix $\mathbf A \in \mathcal M_{n \times n}(\mathbb R)$ and $\boldsymbol{\mu} \in \mathbb R^n$ , define $\mathbf X = \boldsymbol{\mu} + \mathbf A \mathbf Z$ . Calculate the p.d.f. $f_{\mathbf X}$ of $\mathbf X$ and evaluate $\boldsymbol{ \Sigma }_{\mathbf X}$ .

(Click for Solution)

Solution. We first assume the case $\boldsymbol{ \mu } = \mathbf 0$ . Define the bijective and continuous map. We observe that for $K \in \frak{B}(\mathbb R^n)$ ,

$\mathbb P_{\mathbf X}(K) = \mathbb P(\mathbf X \in K) = \mathbb P(\mathbf Z \in \mathbf A^{-1}(K)) = (\mathbb P_{\mathbf Z} \circ \mathbf A^{-1})(K).$

That is, $\mathbb P_{\mathbf X} = \mathbb P_{\mathbf Z} \circ \mathbf A^{-1}$ is the pushforward measure of $\mathbb P_{\mathbf Z}$ under $g$ . Letting $\lambda$ denote the $n$ -dimensional Lebesgue measure, we leave it as an exercise in linear algebra to verify that

$\displaystyle \frac{ \mathrm d\lambda(\mathbf A^{-1}\mathbf x) }{ \mathrm d\lambda(\mathbf x) } = |\mathbf A^{-1}| = \frac{1}{|\mathbf A|}.$

By a change of variables, for any $K \in \frak{B}(\mathbb R^n)$ ,

$\begin{aligned} \mathbb P_{\mathbf X} (K) &= \int_{\mathbb R^n} \mathbb I_K\, \mathrm d\mathbb P_{\mathbf X} \\ &= \int_{\mathbb R^n} \mathbb I_K\, \mathrm d\mathbb P_{\mathbf X} \\ &= \int_{\mathbb R^n} \mathbb I_K \circ \mathbf A \, \mathrm d\mathbb P_{\mathbf Z} \\ &= \int_{\mathbb R^n} \mathbb I_K( \mathbf A\mathbf z ) \cdot f_{\mathbf Z}(\mathbf z) \, \mathrm d\lambda(\mathbf z) \\ &= \int_{\mathbb R^n} \mathbb I_K(\mathbf x) \cdot f_{\mathbf Z}(\mathbf A^{-1} \mathbf x ) \, \mathrm d\lambda(\mathbf A^{-1} \mathbf x ) \\ &= \int_{\mathbb R^n} \mathbb I_K(\mathbf x) \cdot f_{\mathbf Z}(\mathbf A^{-1} \mathbf x ) \cdot \frac{1}{|\mathbf A|}\, \mathrm d\lambda(\mathbf x) \\ &= \int_{K} f_{\mathbf Z}(\mathbf A^{-1} \mathbf x ) \cdot \frac{1}{|\mathbf A|}\, \mathrm d\lambda(\mathbf x).\end{aligned}$

By the Radon-Nikodým theorem,

$\displaystyle \int_K f_{\mathbf X}(\mathbf x)\, \mathrm d\lambda(\mathbf x) = \mathbb P_{\mathbf X} (K) = \int_{K} f_{\mathbf Z}(\mathbf A^{-1} \mathbf x ) \cdot \frac{1}{|\mathbf A|}\, \mathrm d\lambda(\mathbf x),$

so that

$\begin{aligned} f_{\mathbf X}(\mathbf x) &= \frac{1}{|\mathbf A|} \cdot f_{\mathbf Z}(\mathbf A^{-1}\mathbf x) \\ &= \frac 1{ |\mathbf A| \cdot \sqrt{ (2\pi)^{n} } } \cdot e^{-\frac 12 \mathbf x^{\mathrm T} (\mathbf A\mathbf A^{\mathrm T})^{-1} \mathbf x } \\ &= \frac 1{ \sqrt{ (2\pi)^{n} \cdot |\mathbf A \mathbf A^{\mathrm T}| } } \cdot e^{-\frac 12 \mathbf x^{\mathrm T} (\mathbf A\mathbf A^{\mathrm T})^{-1} \mathbf x }.\end{aligned}$

Furthermore,

$\begin{aligned} \boldsymbol{\Sigma}_{\mathbf X} &= \mathbb E[(\mathbf A \mathbf Z - \mathbb E[\mathbf A \mathbf Z]) (\mathbf A \mathbf Z - \mathbb E[\mathbf A \mathbf Z])^{\mathrm T}] \\ &= \mathbb E[\mathbf A (\mathbf Z - \mathbb E[\mathbf Z]) (\mathbf Z - \mathbb E[\mathbf Z])^{\mathrm T}\mathbf A^{\mathrm T} ] \\ &= \mathbf A \mathbb E[(\mathbf Z - \mathbb E[\mathbf Z]) (\mathbf Z - \mathbb E[\mathbf Z])^{\mathrm T} ]\mathbf A^{\mathrm T} \\ &= \mathbf A \boldsymbol{\Sigma}_{\mathbf Z} \mathbf A^{\mathrm T} = \mathbf A \mathbf A^{\mathrm T} . \end{aligned}$

In the marginally more general case, $\mathbf X - \boldsymbol{\mu} = \mathbf A \mathbf Z$ . We leave it as an exercise to verify that $\mathbb E[\mathbf X] = \boldsymbol{ \mu }$ and $\boldsymbol{ \Sigma }_{\mathbf X} = \boldsymbol{ \Sigma }_{\mathbf A \mathbf Z} = \mathbf A\mathbf A^{\mathrm T}$ , so that finally

$\begin{aligned} f_{\mathbf X}(\mathbf x) = f_{\mathbf A \mathbf Z}(\mathbf x - \boldsymbol{\mu}) &= \frac 1{ \sqrt{ (2\pi)^{n} \cdot |\boldsymbol{ \Sigma }_{\mathbf A \mathbf Z}| } } \cdot e^{-\frac 12 (\mathbf x - \boldsymbol{\mu})^{\mathrm T} \boldsymbol{ \Sigma }_{\mathbf A \mathbf Z}^{-1} (\mathbf x - \boldsymbol{\mu}) }. \end{aligned}$

Definition 1. A random variable $\mathbf X$ is multivariate normal, denoted $\mathbf X \sim \mathcal N(\boldsymbol{ \mu }, \boldsymbol{ \Sigma })$ , if it has a p.d.f. $f_{\mathbf X}$ given by

$\displaystyle f_{\mathbf X}(\mathbf x) = \frac{1}{ \sqrt{ (2\pi)^n \cdot |\boldsymbol{ \Sigma } | } } \cdot e^{-\frac 12 (\mathbf x - \boldsymbol{\mu})^{\mathrm T} \boldsymbol{\Sigma}^{-1} (\mathbf x - \boldsymbol{\mu})}.$

In this case, we say that the mean of $\mathbf X$ is $\mathbb E[ \mathbf X ] = \boldsymbol{\mu}$ , and the covariance of $\mathbf X$ is $\boldsymbol{\Sigma}_{\mathbf{X}} = \boldsymbol{\Sigma}$ . For example, $\mathbf Z \sim \mathcal N(\mathbf 0, \mathbf I_n)$ .

Problem 3. Fix $\mathbf X \sim \mathcal N(\boldsymbol{ \mu }, \boldsymbol{\Sigma} )$ .
- Prove that if $\boldsymbol{\Sigma}$ is diagonal whose diagonal entires are positive $\sigma_i^2 > 0$ , then $X_1,\dots, X_n$ are independent with $X_i \sim \mathcal N( \mu_i, \sigma_i^2 )$ .
- Prove that for any invertible $\mathbf A \in \mathcal M_{n \times n}(\mathbb R)$ and $\boldsymbol{ \nu } \in \mathbb R^n$ , $\mathbf Y := \boldsymbol{\nu} + \mathbf A \mathbf X \sim \mathcal N(\boldsymbol{ \mu }_{\mathbf Y}, \boldsymbol{ \Sigma }_{\mathbf Y})$ for constants $\boldsymbol{ \mu }_{\mathbf Y}, \boldsymbol{ \Sigma }_{\mathbf Y}$ to be calculated.
(Click for Solution)

Solution. For the first point, if $\boldsymbol{\Sigma}$ is diagonal with $\boldsymbol{ \Sigma }_{i,i} = \sigma_i^2 > 0$ , then $\boldsymbol{\Sigma}^{-1}$ is diagonal with $\boldsymbol{ \Sigma }_{i,i} = 1/\sigma_i^2 > 0$ , and

$\displaystyle (\mathbf x - \boldsymbol{\mu})^{\mathrm T} \boldsymbol{\Sigma}^{-1} (\mathbf x - \boldsymbol{\mu}) = \sum_{i=1}^n \frac{ (x_i - \mu_i)^2 }{ \sigma_i^2 }.$

Some algebra then yields

$\begin{aligned} f_{\mathbf X}(\mathbf x) &= \frac{1}{ \sqrt{ (2\pi)^n \cdot |\boldsymbol{ \Sigma } | } } \cdot e^{-\frac 12 (\mathbf x - \boldsymbol{\mu})^{\mathrm T} \boldsymbol{\Sigma}^{-1} (\mathbf x - \boldsymbol{\mu})} \\ &= \prod_{i=1}^n \frac{1}{\sigma_i \sqrt{ 2\pi } } \cdot e^{-\frac 12 \sum_{i=1}^n \frac{ (x_i - \mu_i)^2 }{ \sigma_i^2 } } \\ &= \prod_{i=1}^n \frac{1}{\sigma_i \sqrt{ 2\pi } } \cdot \prod_{i=1}^n e^{-\frac{ (x_i - \mu_i)^2 }{ 2\sigma_i^2 } } \\ &= \prod_{i=1}^n \frac{1}{\sigma_i \sqrt{ 2\pi } } e^{-\frac{ (x_i - \mu_i)^2 }{ 2\sigma_i^2 } } = \prod_{i=1}^n f_{X_i}(x_i). \end{aligned}$

Therefore, $X_1,\dots, X_n$ are independent with $X_i \sim \mathcal N(\mu_i , \sigma_i^2)$ .

For the second point, follow the proof in Problem 2 with needful bookkeeping to obtain

$\begin{aligned} f_{\mathbf Y}(\mathbf y) &= f_{\mathbf A\mathbf X}(\mathbf y - \boldsymbol{\nu}) = \frac 1{|\mathbf A|} \cdot f_{\mathbf X}(\mathbf A^{-1}(\mathbf y - \boldsymbol{\nu})), \end{aligned}$

which simplifies to Definition 1 with $\mathbb E[\mathbf Y] = \boldsymbol{\nu} + \mathbf A \boldsymbol{\mu}$ and $\boldsymbol{\Sigma}_{\mathbf Y} = \mathbf A \boldsymbol{\Sigma}_{\mathbf X} \mathbf A^{\mathrm T}$ .

Problem 4. Let $\mathbf X \sim \mathcal N(\boldsymbol{ \mu } , \boldsymbol{ \Sigma })$ . Prove that there exists an invertible matrix $\mathbf A \in \mathcal M_{n \times n}(\mathbb R)$ such that $\mathbf X = \boldsymbol{ \mu } + \mathbf A \mathbf Z$ . In particular, each $X_i$ is normally distributed.

(Click for Solution)

Solution. By definition,

$\displaystyle f_{\mathbf X}(\mathbf x) = \frac{1}{ \sqrt{ (2\pi)^n \cdot |\boldsymbol{ \Sigma } | } } \cdot e^{-\frac 12 (\mathbf x - \boldsymbol{\mu})^{\mathrm T} \boldsymbol{\Sigma}^{-1} (\mathbf x - \boldsymbol{\mu})}.$

We first assume $\boldsymbol{\mu} = \mathbf 0$ . If we can find some invertible matrix $\mathbf B$ such that $\mathbf B \mathbf X = \mathbf Z$ , then we can set $\mathbf A := \mathbf B^{-1}$ . To that end, we note that $\boldsymbol{\Sigma}$ is symmetric since

$\displaystyle \boldsymbol{\Sigma}_{i,j} = \mathrm{Cov}(X_i, X_j) = \mathrm{Cov}(X_j, X_i) = \boldsymbol{\Sigma}_{j,i} > 0.$

By the spectral theorems, $\boldsymbol{\Sigma}$ is orthogonally diagonalisable. That is, there exists an orthogonal matrix $\mathbf P$ (i.e. $\mathbf P^{-1} = \mathbf P^{\mathrm T}$ ) and a diagonal matrix $\mathbf D$ such that

$\mathbf{P}^{\mathrm T} \boldsymbol{\Sigma} \mathbf{P} = \mathbf{D}\quad \Rightarrow \quad \boldsymbol{\Sigma} = \mathbf{P} \mathbf{D} \mathbf{P}^{\mathrm T}.$

Furthermore, the entries in the diagonal of $\mathbf D$ are all positive, denoted $\sigma_i^2 > 0$ . Define $\boldsymbol{\Delta}$ by

$\displaystyle \boldsymbol{\Delta}_{i,j} := \begin{cases} \sigma_i, & i = j, \\ 0, & i \neq j. \end{cases}$

Then $\boldsymbol{\Delta}\boldsymbol{\Delta}^{\mathrm T} = \mathbf{D}$ , so that

$\boldsymbol{\Sigma} = \mathbf{P} \mathbf{D} \mathbf{P}^{\mathrm T} = \mathbf{P} \boldsymbol{\Delta}\boldsymbol{\Delta}^{\mathrm T} \mathbf{P}^{\mathrm T} = (\mathbf{P} \boldsymbol{\Delta}) (\mathbf{P} \boldsymbol{\Delta})^{\mathrm T}.$

Now define $\mathbf B := ( \mathbf{P} \boldsymbol{\Delta})^{-1}$ . By Problem 3, $\mathbf B \mathbf X$ is multivariate normal. We calculate its expectation via

$\mathbb E[\mathbf B \mathbf X] = \mathbf B \mathbf E[\mathbf X] = \mathbf B \mathbf 0 = \mathbf 0$

and its covariance matrix via

$\begin{aligned}\boldsymbol{\Sigma_{\mathbf B \mathbf X}} &= \mathbb E[(\mathbf B \mathbf X)^{\mathrm T} (\mathbf B \mathbf X)] \\ &= \mathbf B \mathbb E[\mathbf X^{\mathrm T} \mathbf X] \mathbf B^{\mathrm T} \\ &= ( \mathbf{P} \boldsymbol{\Delta})^{-1} (\mathbf{P} \boldsymbol{\Delta}) (\mathbf{P} \boldsymbol{\Delta})^{\mathrm T} (( \mathbf{P} \boldsymbol{\Delta})^{-1})^{\mathrm T} = \mathbf I. \end{aligned}$

Therefore, $\mathbf B \mathbf X \sim \mathcal N(\mathbf 0, \mathbf I)$ . Since $\mathbf Z \sim \mathcal N(\mathbf 0, \mathbf I)$ , we conclude $\mathbf B \mathbf X = \mathbf Z$ , so that $\mathbf X = \mathbf B^{-1}\mathbf Z = \mathbf A \mathbf Z$ , as required.

For the general case, $\mathbf X - \boldsymbol{\mu}$ has expectation $\mathbf 0$ , and so by the first result, there exists some invertible matrix $\mathbf A$ such that

$\mathbf X - \boldsymbol{\mu} = \mathbf A \mathbf Z \quad \Rightarrow \quad \mathbf X = \boldsymbol{\mu} + \mathbf A \mathbf Z,$

as required.

Problem 5. Define $W_1 := \bar X_n$ and for each $i > 1$ , define $W_i := X_i - \bar X_n$ . If $X_1,\dots,X_n \sim \mathcal N(\mu, \sigma^2)$ are i.i.d., prove that $\bar X_n$ and $(W_2,\dots,W_n)$ are independent. Deduce that $\bar X_n$ and $S_n^2$ defined by

$\displaystyle S_n^2 := \frac 1{n-1} \sum_{i=1}^n (X_i - \bar X_n)^2$

are independent.

(Click for Solution)

Solution. For each $i > 1$ ,

$\displaystyle W_i = \sum_{j=1}^n \underbrace{ \left( \mathbb I_{\{ i \}}(j) -\frac{ 1 }{n} \right) }_{\mathbf A_{i,j}} X_j =: \sum_{j=1}^n \mathbf A_{i, j}X_j.$

Furthermore, define $\mathbf A_{1, j} = 1/n$ . Defining $\mathbf W := (W_1,\dots,W_n)^{\mathrm T}$ , $\mathbf W = \mathbf A \mathbf X$ . If $X_1,\dots,X_n$ are i.i.d. normally distributed with variance $\sigma^2$ , then by the converse of the first point in Problem 3 (which holds), $\mathbf X$ is multivariate normal. By the second point in Problem 3, $\mathbf W$ is multivariate normal.

We turn to evaluate its covariance matrix. Firstly, to shorten computations,

$\begin{aligned} \mathrm{Cov}(X_i, \bar X_n) &= \frac 1n \cdot \sum_{k=1}^n \mathrm{Cov}(X_i, X_k) \\ &= \frac 1n \cdot \mathrm{Cov}(X_i, X_i) = \frac{\sigma^2}{n}. \end{aligned}$

Furthermore,

$\begin{aligned} \mathrm{Cov}(\bar X_n, \bar X_n) &= \frac 1n \cdot \sum_{k=1}^n \mathrm{Cov}(X_i, \bar X_n) \\ &= \frac 1n \cdot \sum_{k=1}^n \frac{\sigma^2}{n} = \frac 1n \cdot n \cdot \frac{\sigma^2}n = \frac{\sigma^2}n. \end{aligned}$

Thus, for $j > 1$ ,

$\begin{aligned} (\boldsymbol{\Sigma}_{\mathbf W})_{1,j} &= \mathrm{Cov}(W_1, W_j) \\ &= \mathrm{Cov}(\bar X_n, X_j - \bar X_n) \\ &= \mathrm{Cov}(X_j , \bar X_n) - \mathrm{Cov}(\bar X_n,\bar X_n) \\ &= \frac{\sigma^2}{n} - \frac{\sigma^2}{n} = 0.\end{aligned}$

Therefore, using properties involving the exponential function,

$f_{\mathbf W}(\mathbf w) = f_{W_1}(w_1) \cdot f_{W_2,\dots,W_n}(w_2,\dots,w_n).$

Hence, $W_1$ and $(W_2,\dots,W_n)$ are independent. For the second claim, we first note that

$\displaystyle 0 = \sum_{i=1}^n X_i - n\bar X_n = \sum_{i=1}^n (X_i - \bar X_n) = X_1 - \bar X_n + \sum_{i=2}^n (X_i - \bar X_n),$

so that $X_1 - \bar X_n = -\sum_{i=2}^n (X_i - \bar X_n) = -\sum_{i=2}^n W_i$ . It follows that

$\begin{aligned} S_n^2 &= \frac 1{n-1} \left( (X_1 - \bar X_n)^2 + \sum_{i=2}^n (X_i - \bar X_n)^2 \right) \\ &= \frac 1{n-1} \left( (X_1 - \bar X_n)^2 + \sum_{i=2}^n W_i^2 \right) \\ &= \frac 1{n-1} \left( \left( \sum_{i=2}^n W_i \right)^2 + \sum_{i=2}^n W_i^2 \right). \end{aligned}$

Since $S_n^2$ can be written purely in terms of $(W_2,\dots, W_n)$ , it is independent of $W_1 = \bar X_n$ .

Problem 6. Using the definitions and notation in Problem 5, prove that for any $n \in \mathbb N_{>1}$ ,

$\displaystyle (n-1) \cdot S_n^2 = (n-2) \cdot S_{n-1}^2 + \frac{n-1}{n} \cdot (X_n - \bar X_{n-1})^2.$

Deduce that there exist i.i.d. $Z_1,\dots, Z_{n-1} \sim \mathcal N(0, 1)$ such that

$\displaystyle \frac{(n-1) \cdot S_n^2}{\sigma^2} = \sum_{i=1}^{n-1} Z_i^2.$

(Click for Solution)

Solution. We first observe that

$\begin{aligned} \bar X_n - \bar X_{n-1} &= \frac{1}{n} \cdot \sum_{i=1}^{n-1} X_i + \frac{1}{n} \cdot X_n - \frac 1{n-1} \cdot \sum_{i=1}^{n-1} X_i \\ &= -\frac{1}{n(n-1)} \cdot \sum_{i=1}^{n-1} X_i + \frac{1}{n} \cdot X_n = \frac 1n \cdot ( X_n - \bar X_{n-1} ). \end{aligned}$

Performing algebruh,

$\begin{aligned} &(n-1) \cdot S_n^2 \\ &= \sum_{i=1}^n \left( X_i - \bar X_{n-1} - \frac 1n \cdot ( X_n - \bar X_{n-1})\right)^2 \\ &= \sum_{i=1}^n \left[ (X_i - \bar X_{n-1})^2 - 2 \cdot (X_i - \bar X_{n-1}) \cdot \frac 1n \cdot ( X_n - \bar X_{n-1}) + \frac 1{n^2} \cdot ( X_n - \bar X_{n-1})^2 \right] \\ &= (n-2) \cdot S_{n-1}^2 + (X_n - \bar X_{n-1})^2 - \frac{2}{n} \cdot ( X_n - \bar X_{n-1})^2 + \frac 1{n} \cdot ( X_n - \bar X_{n-1})^2 \\ &= (n-2) \cdot S_{n-1}^2 + \frac{n-1}{n} \cdot (X_n - \bar X_{n-1})^2. \end{aligned}$

We now prove the final equation by induction. In the case $n = 2$ , we have

$\begin{aligned} \frac{S_2^2}{\sigma^2} &= \frac 1{\sigma^2} \cdot \frac 12 \cdot ((X_1 -\bar X_1)^2 + (X_2 -\bar X_1)^2) = \frac{(X_2 - X_1)^2}{2\sigma^2}. \end{aligned}$

Then $Z_1 := (X_2 - X_1)/(\sigma \sqrt 2) \sim \mathcal N(0, 1)$ , so that $S_2^2/\sigma^2 = Z_1^2$ , as required. For the induction step, suppose that the statement holds for $n = k$ . Then

$\displaystyle k \cdot S_{k+1}^2 = (k-1) \cdot S_k^2 + \frac{k}{k+1} \cdot (X_{k+1} - \bar X_k)^2.$

By relabelling the result in Problem 5, we have that $S_k^2$ is independent of $\bar X_k$ and $X_{k+1}$ . By the induction hypothesis, there exist i.i.d. $Z_1,\dots, Z_{k-1} \sim \mathcal N(0, 1)$ such that

$\displaystyle \frac{ (k-1) \cdot S_k^2 }{\sigma^2} := \sum_{i=1}^{k-1} Z_i^2.$

Furthermore, $X_{k+1} - \bar X_k \sim \mathcal N(0, \sigma^2 \cdot (k+1)/k)$ so that

$\displaystyle Z_{k+1} := \frac{\sqrt{k}}{\sqrt{k+1}} \cdot \frac{ X_{k+1} - \bar X_k }{\sigma} = \frac{X_{k+1} - \bar X_k}{ \sigma \cdot \sqrt{ \frac{k+1}{k} } } \sim \mathcal N(0, 1).$

Therefore,

$\displaystyle \displaystyle \frac{ k \cdot S_{k+1}^2 }{ \sigma^2 } = \frac{ (k-1) \cdot S_{k}^2 }{ \sigma^2 } + \frac{k}{k+1} \cdot \frac{ (X_{k+1} - \bar X_{k})^2 }{ \sigma^2 } = \sum_{i=1}^{k-1} Z_i^2 + Z_k^2 = \sum_{i=1}^{k} Z_i^2,$

as required.

—Joel Kindiak, 25 Jul 25, TBCH
December 12, 2025
Formalising the Dirac Delta

The Dirac delta “function” $\delta$ has been a bane in my teaching since I needed to start teaching it in engineering mathematics, since it is, really, not actually a function. Many of its results seem miraculously correct, and yet we don’t have any reasonably accessible introduction to its formal definition.

We will adopt a measure-theoretic interpretation of $\delta$ and regard it as a measure. We will then define the relevant Laplace transform notions on measures that extend the classical setting. Finally, we assert that

$\displaystyle \int_{(-\infty, t]} \mathrm d\delta = U(t) \equiv \mathbb I_{[0,\infty)}(t),$

so that, in some limited sense, $U' = \delta$ .

For any measure $\mu$ on $(\mathbb R, \frak{B}(\mathbb R))$ , if the measurable function $f$ is $\mu$ -integrable, make the definition

$\displaystyle \langle \mu, f \rangle := \int_{\mathbb R} f\, \mathrm d\mu.$

whenever the right-hand side is well-defined. Using the Radon-Nikodým theorem, for suitably-defined functions $f, g$ ,

$\langle \lambda_g, f\rangle = \langle f \cdot g, \lambda \rangle = \langle \lambda_f, g \rangle,$

where the measure $\lambda_f$ is defined by $\lambda_f(S) := \int_S f\, \mathrm d\lambda$ . We remark that by the same theorem, the map $f \to \lambda_f$ is injective $\lambda$ -a.e. in the following sense: if $\lambda_f = 0$ , then $f = 0$ $\lambda$ -a.e.. Furthermore, it is bona-fide injective when restricted to the space of continuous functions (i.e. $f$ is continuous).

Define the map $K : \mathbb R^2 \to \mathbb R$ by $K(s,t) := e^{-st} \cdot U(t)$ .

Definition 1. For any measure $\mu$ on $(\mathbb R, \frak{B}(\mathbb R))$ , if $f$ is $\mu$ -integrable and $K(s, \cdot) \cdot f$ is $\mu$ -integrable, we define the $\mu$ –Laplace transform by

$\displaystyle \mathcal L_{\mu}\{f \} (s) := \langle \mu ,f \cdot K(s,\cdot)\rangle \equiv \int_{\mathbb R} K(s, \cdot) \cdot f\, \mathrm d\mu,$

whenever the right-hand side is well-defined. In particular, define $\mathcal L\{\mu\} := \mathcal L_{\mu}\{1\}$ .

Problem 1. If $K(s,\cdot)\cdot f$ is $\lambda$ -integrable, prove that $\mathcal L_{\lambda}\{f \} = \mathcal L\{f\}$ , where the right-hand side refers to the classical Laplace transform on sub-exponential functions.

(Click for Solution)

Solution. By definition, for any $s$ ,

$\begin{aligned} \mathcal L_{\lambda}\{f \}(s) &= \langle \lambda, f \cdot K(s,\cdot), \rangle \\ &= \int_{\mathbb R} K(s, \cdot) \cdot f\, \mathrm d\lambda \\ &= \int_{-\infty}^{\infty} K(s, t) \cdot f(t)\, \mathrm d \lambda( t ) \\ &= \int_{-\infty}^{\infty} e^{-st} \cdot f(t)\, \mathrm dt = \mathcal L\{f\}(s). \end{aligned}$

Hence, $\mathcal L\{\lambda_f\} \equiv \mathcal L_{\lambda_f}\{1\} \equiv \mathcal L_{\lambda}\{f\} = \mathcal L\{f\}$ .

Problem 2. Prove that the Laplace transform on measures is linear in the measures: for suitably defined measures $\mu,\nu$ , a suitably-defined function $f$ , and $\alpha \in \mathbb R$ ,

$\mathcal L_{\mu + \nu} \{f\} = \mathcal L_{\mu}\{f\} + \mathcal L_{\nu}\{f\},\quad \mathcal L_{\mu}\{\alpha f \} = \alpha \cdot \mathcal L_{\mu}\{f\}.$

Furthermore, if $\mu$ is signed, then $\mathcal L_{\alpha \mu}\{f \} = \alpha \cdot \mathcal L_{\mu}\{f\}$ .

(Click for Solution)

Solution. For additivity, we first prove the additivity of measures: for $S \in \frak B(\mathbb R)$ and suitably defined functions $f$ , we first claim that

$\begin{aligned} \langle \mu + \nu, f\rangle &= \int_{\mathbb R} f\, \mathrm d(\mu + \nu) \\ &= \int_{\mathbb R} f\, \mathrm d\mu + \int_{\mathbb R} f\, \mathrm d\nu \\ &= \langle \mu,f \rangle + \langle \nu, f\rangle, \end{aligned}$

so that for suitably chosen $s$ ,

$\begin{aligned}\mathcal L_{\mu + \nu}\{f\}(s) &= \langle \mu + \nu, K(s, \cdot) \cdot f\rangle \\ &= \langle \mu, K(s, \cdot) \cdot f \rangle + \langle \nu, K(s, \cdot) \cdot f\rangle \\ &= \mathcal L_{\mu }\{f\}(s) + \mathcal L_{ \nu}\{f\}(s) \\ &= (\mathcal L_{\mu }\{f\} + \mathcal L_{\nu }\{f\})(s).\end{aligned}$

To that end, we prove the result for the non-negative simple function $f = \sum_{i=1}^n a_i \cdot \mathbb I_{S_i}$ and leave the extensions via the monotone convergence theorem and decomposition $f = f^+ - f^-$ as an exercise:

$\begin{aligned} \int_{\mathbb R} f\, \mathrm d(\mu + \nu) &= \sum_{i=1}^n a_i (\mu + \nu)(S_i) \\ &= \sum_{i=1}^n a_i \cdot (\mu(S_i) + \nu(S_i)) \\ &= \sum_{i=1}^n a_i \cdot \mu(S_i) + \sum_{i=1}^n a_i \cdot \nu(S_i) \\ &= \int_{\mathbb R} f\, \mathrm d\mu + \int_{\mathbb R} f\, \mathrm d\nu, \end{aligned}$

as required. Scalar multiplication follows similarly.

Problem 3. Prove that for any $a \in \mathbb R$ , the map $\delta_a$ defined by $\delta_a(S) := \mathbb I_{S}(a)$ is a probability measure. We call $\delta := \delta_0$ the Dirac measure and observe that that $\delta_a = \delta ( \cdot - a)$ .

(Click for Solution)

Solution. We verify the three properties of a probability measure. Firstly,

$\delta(\emptyset) = \mathbb I_{\emptyset}(0) = 0.$

Secondly,

$\begin{aligned} \delta \left( \bigsqcup_{i=1}^\infty S_i \right) &= \mathbb I_{\bigsqcup_{i=1}^\infty S_i }(0) = \sum_{i=1}^\infty \mathbb I_{ S_i }(0) = \sum_{i=1}^\infty \delta(S_i).\end{aligned}$

Finally, $\delta(\mathbb R) = \mathbb I_{\mathbb R}(0) = 1$ .

Problem 4. Prove the sifting property:

$\displaystyle \langle \delta_a, f \rangle := \int_{\mathbb R} f\, \mathrm d\delta_a = f(a).$

Deduce that $\mathcal L_{\delta_a}\{f\}= K(\cdot, a) \cdot f$ . In particular, $\mathcal L_{\delta_a}\{1\}(s) = e^{-as}$ .

(Click for Solution)

Solution. We can use the same non-negative simple $\Rightarrow$ non-negative $\Rightarrow$ integrable technique for the sifting property, which we will carry out for completeness.

Let $f = \sum_{i=1}^n a_i \cdot \mathbb I_{S_i}$ be a non-negative simple function. We observe that

$\delta_a(S_i) = 1 \quad \iff \quad a \in S_i \quad \iff \quad f(a) = a_i.$

Then

$\begin{aligned} \int_{\mathbb R} f\, \mathrm d\delta_a &= \sum_{i=1}^n a_i \cdot \delta_a(S_i) = a_i \cdot 1 = f(a).\end{aligned}$

Now suppose $f$ is non-negative. Find a sequence $\varphi_n$ of simple functions such that $\varphi_n \to f$ monotonically. By the monotone convergence theorem,

$\begin{aligned} \int_{\mathbb R} f\, \mathrm d\delta_a &=\lim_{n \to \infty} \int_{\mathbb R} \varphi_n\, \mathrm d\delta_a = \lim_{n \to \infty} \varphi_n(a) = f(a).\end{aligned}$

Finally, suppose $f$ is integrable. Writing $f = f^+ - f^-$ ,

$\begin{aligned}\int_{\mathbb R} f\, \mathrm d\delta_a &= \int_{\mathbb R} f^+\, \mathrm d\delta_a - \int_{\mathbb R} f^-\, \mathrm d\delta_a \\ &= f^+(a) - f^-(a) \\ &= (f^+ - f^-)(a) = f(a).\end{aligned}$

In particular, for any fixed $s$ ,

$\displaystyle \mathcal L_{\delta_a}\{f\}(s) = \langle \delta_a, K(s,\cdot) \cdot f\rangle = \int_{\mathbb R} K(s, \cdot) \cdot f\ \mathrm d\delta_a = K(s, a) \cdot f(a) .$

In particular, $\mathcal L_{\delta_a}\{1\} = K(\cdot, a)$ .

Remark 1. We have shown that $\mathcal L_{\mu}\{f\}$ is a meaningful generalisation of both the usual Laplace transform $\mathcal L\{f\}$ and the Laplace transform of the measure $\mathcal L\{ {\delta_a} \}:=\mathcal L_{\delta_a}\{1\}$ :

$\begin{aligned} \mathcal L \{ \alpha \cdot \lambda_f + \beta \cdot \delta_a \} &= \mathcal L_{ \alpha \cdot \lambda_f + \beta \cdot \delta_a}\{1\} \\ &= \mathcal L_{ \alpha \cdot \lambda_f }\{1\} + \mathcal L_{ \beta \cdot \delta_a}\{1\}\\ &= \alpha \cdot \mathcal L_{\lambda_f}\{1\} + \beta \cdot \mathcal L_{\delta_a}\{1\} \\ &= \alpha \cdot \mathcal L \{\lambda_f\} + \beta \cdot \mathcal L \{\delta_a \} \\ &= \alpha \cdot \mathcal L \{f\} + \beta \cdot \mathcal L \{\delta_a \}. \end{aligned}$

In this manner and usage, we embed $f \to \lambda_f$ as a (signed-)measure, and define $f + \beta \cdot \delta_a := \lambda_f + \beta \cdot \delta_a$ via said embedding, finally permitting engineers to write the following guiltlessly:

$\begin{aligned} \mathcal L\{\alpha \cdot f + \beta \cdot \delta_a\} &\equiv \mathcal L \{ \alpha \cdot \lambda_f + \beta \cdot \delta_a \} \\ &= \alpha \cdot \mathcal L \{f\} + \beta \cdot \mathcal L \{\delta_a \}. \end{aligned}$

Problem 5. Prove that $\displaystyle \int_{(-\infty, t]} \mathrm d\delta_a = U_a(t)$ . Deduce that for $a > 0$ and valid $s$ ,

$\displaystyle \int_{\mathbb R} \mathrm d\delta_a = 1 \quad \text{and} \quad \mathcal L\{\delta_a\}(s) = s \cdot \mathcal L\{U_a\}(s),$

where $U_a = U( \cdot - a)$ .

(Click for Solution)

Solution. For the first claim,

$\begin{aligned} \int_{(-\infty, t]} \mathrm d\delta_a &= \int_{\mathbb R} \mathbb I_{(-\infty, t]}\, \mathrm d\delta_a = \delta_a((-\infty, t]) = \begin{cases} 1, & t \geq a, \\ 0, & t < a. \end{cases}\end{aligned}$

Taking $T \to \infty$ and $T > a$ in particular, by the monotonicity of measures,

$\begin{aligned} \int_{\mathbb R} \mathrm d\delta_a &= \lim_{T \to \infty} \int_{(-\infty, T]} \mathrm d\delta_a = \lim_{T \to \infty} U_a(T) = \lim_{T \to \infty} 1 = 1.\end{aligned}$

For the final result,

$\displaystyle \mathcal L\{U_a\}(s) = \frac{e^{-as}}{s} = \frac{\mathcal L\{\delta_a\}(s)}{s}.$

Remark 2. In classical formulas, if $f$ is differentiable such that $f' \cdot \mathbb I_{(-\infty, t]}$ is $\lambda$ -integrable for any $t \in \mathbb R$ , $\displaystyle \lim_{N \to \infty}f(-N) = 0$ , and has a Laplace transform, we have

$\displaystyle \int_{(-\infty, t]} f'\, \mathrm d\lambda = f(t),\quad \mathcal L\{f'\}(s) = s \cdot \mathcal L\{f\}(s) - f(0).$

Since Problems 3 and 4 agree with classical setups, engineers sloppily write $U' = \delta$ and use these results on their happy merry way, leaving mathematicians infuriated in having to clean up after their…stuff.

This measure-theoretic formalisation only helps us with the integration aspect of the Dirac delta, since measure theory is, in some sense a “completion” of integral calculus. Yet, a full understanding requires us to develop a “completion” of differential calculus—this generalisation builds on measure theory and discusses distribution theory.

We hinted at these objects via the notation $\langle \mu, f \rangle$ . The map

$\langle \mu, \cdot \rangle : \{\text{good functions}\} \to \mathbb R$

is called a distribution. In a sense, then we could define $U := \langle U, \cdot \rangle$ as a distribution (though this requires much more effort), and by defining differentiation of distributions, we get the true equality $U' = \delta$ . But all that work requires a lot more pain and suffering.

As completions, they become indispensable tools in solving many more differential equations than a usual post-secondary or even undergraduate course would present.

—Joel Kindiak, 16 Jul 25, 1150H

November 11, 2025
The Union Bound

Let $(\Omega, \mathcal F, \mathbb P)$ be a probability space.

Problem 1. For $K_1, K_2, \dots\in \mathcal F$ , prove Boole’s inequality:

$\displaystyle \mathbb P \left( \bigcup_{i=1}^\infty K_i \right) \leq \sum_{i=1}^\infty \mathbb P(K_i).$

This result is also known as the union bound.

(Click for Solution)

Solution. Define $L_1 := K_1$ and inductively, $L_{i+1} = K_{i+1} \backslash \bigcup_{j=1}^i K_j$ . We can check that $\{L_i\}$ is pairwise mutually exclusive and $L_i \subseteq K_i$ for each $i$ . Furthermore, it is not hard to prove that

$\displaystyle \bigcup_{i=1}^\infty K_i = \bigsqcup_{i=1}^\infty L_i.$

To see this, we first note that $(\supseteq)$ is obvious. For the direction $(\subseteq)$ , fix $x \in K_i$ for some $i$ . Take $i$ to be the smallest by the well-ordering principle, so that $x \notin \bigcup_{j=1}^{i-1} K_j$ . Therefore, $x \in L_i$ , as required. Therefore,

$\displaystyle \mathbb P\left( \bigcup_{i=1}^\infty K_i\right) = \mathbb P \left( \bigsqcup_{i=1}^\infty L_i \right) = \sum_{i=1}^\infty \mathbb P(L_i) \leq \sum_{i=1}^\infty \mathbb P(K_i) .$

This is known as the disjoint union trick.

Problem 2. For $K_1, K_2, \dots, K_n \in \mathcal F$ , prove Bonferroni’s inequality.

$\displaystyle \mathbb P \left( \bigcap_{i=1}^n K_i \right) \geq \sum_{i=1}^n \mathbb P(K_i) - (n-1).$

(Click for Solution)

Solution. Firstly, setting $K_{n+i} = \emptyset$ for $i \geq 0$ ,

$\displaystyle \mathbb P \left( \bigcup_{i=1}^n K_i \right) = \mathbb P \left( \bigcup_{i=1}^\infty K_i \right) \leq \sum_{i=1}^\infty \mathbb P(K_i) = \sum_{i=1}^n\mathbb P(K_i).$

Defining $L_i := \Omega \backslash K_i$ for $i \geq 1$ , use Problem 1 to deduce

$\begin{aligned} \mathbb P \left( \bigcap_{i=1}^n K_i \right) &= \mathbb P \left( \bigcap_{i=1}^n ( \Omega \backslash L_i ) \right) = \mathbb P \left( \Omega \backslash \bigcup_{i=1}^n L_i \right) \\ &= 1 - \mathbb P \left( \bigcup_{i=1}^n L_i \right) \geq 1 - \sum_{i=1}^n \mathbb P(L_i) \geq 1 - \sum_{i=1}^n \mathbb P(\Omega \backslash K_i) \\ &\geq 1 - \sum_{i=1}^n \mathbb (1 - \mathbb P(K_i)) = \sum_{i=1}^n \mathbb P(K_i) - (n-1). \end{aligned}$

Corollary 1. We have the inequality

$\displaystyle \mathbb P \left( \bigcup_{i=1}^n K_i \right) \leq \sum_{i=1}^n \mathbb P(K_i) \leq \mathbb P \left( \bigcap_{i=1}^n K_i \right) + (n-1).$

—Joel Kindiak, 30 Aug 25, 2048H

October 8, 2025
The Friendship Formula
Friendship is a massively important topic of contemplation in my life, so much to the point I have proposed an equation to quantify the closeness between two friends. Suppose Persons X and Y are friends, and Person X wants to evaluate his closeness with Person Y.

Definition 1. Define the following random variables bounded in $[0, 1]$ :
- $Q$ denotes the space for vulnerability that Person X gives to Person Y (i.e. the extent to which Person Y does not need to feel defensive when interacting with Person X).
- $R$ denotes the space for vulnerability that Person X receives from Person Y.
- $S$ denotes the access that Person X gives to Person Y (i.e. the amount of personal information that Person X discloses to Person Y).
- $T$ denotes the access that Person X receives from Person Y.
Assume the random variables $Q,R,S,T$ are jointly independent continuous random variables.

Note that Person X determines these quantities, and may differ from Person Y’s evaluation (explaining why X can feel close to Y but the feelings are not mutual).

Axiom 1. The mutual closeness $C$ that X feels he shares with Y is defined by the equation

$C := \min\{ Q, R \} \cdot \min\{ S, T \}.$

Problem 1. Prove that $\mathbb E[C]$ is given by the quantity

$\displaystyle \begin{aligned} \left(\mathbb E[Q] + \mathbb E[R] - 1 + \int_{[0, 1]} F_Q \cdot F_R\, \mathrm d\lambda \right) \cdot \left(\mathbb E[S] + \mathbb E[T] - 1 + \int_{[0, 1]} F_S \cdot F_T\, \mathrm d\lambda\right). \end{aligned}$

This quantity evaluates the expected closeness that X feels with Y.

(Click for Solution)

Solution. We notice that all random variables have expectation bounded in $[0, 1]$ . Define

$V := \min\{ Q, R \},\quad A := \min\{ S, T \}.$

We first evaluate $\mathbb E[V]$ . By definition,

$\begin{aligned} \mathbb P(V > v) &= \mathbb P(\min\{Q, R\} > v) \\ &= \mathbb P( Q > v, R > v ) \\ &= \mathbb P( Q > v ) \cdot \mathbb P( R > v ). \end{aligned}$

In particular,

$\begin{aligned}(F_Q \cdot F_R)(v) &= F_Q(v) \cdot F_R(v) \\ &= (1 - \mathbb P(Q > v)) \cdot (1 - \mathbb P(R > v)) \\ &= 1 - \mathbb P(Q > v) - \mathbb P(R > v) + \mathbb P(Q > v) \cdot \mathbb P(R > v) \\ &= 1 - \mathbb P(Q > v) - \mathbb P(R > v) + \mathbb P(V > v). \end{aligned}$

Taking integrals on all sides, using the tail-probability characterisation of the expectation, and performing algebruh,

$\displaystyle \mathbb E[V] = \mathbb E[Q] + \mathbb E[R] - 1 + \int_{[0, 1]} (F_Q \cdot F_R)(v)\, \mathrm dv.$

Similarly,

$\displaystyle \mathbb E[A] = \mathbb E[S] + \mathbb E[T] - 1 + \int_0^1 (F_S \cdot F_T)(a)\, \mathrm da.$

Since $Q,R,S,T$ are jointly independent, $V = \min\{Q, R\}$ and $A = \min\{S, T\}$ are also independent, and

$\mathbb E[C] = \mathbb E[V \cdot A] = \mathbb E[V] \cdot \mathbb E[A],$

giving the desired result, after replacing relevant dummy variables.

—Joel Kindiak, 7 Aug 25, 2324H
August 9, 2025