Density Functions on Steroids

We have seen that a random variable $X$ is said to be (absolutely) continuous if there exists a continuous, non-negative, integrable function $f_X$ such that for any Borel-measurable $K$ ,

$\displaystyle \mathbb P_X(K) = \int_K f_X\, \mathrm d\lambda.$

In this case, $\mathbb P_X \ll \lambda$ in the sense that for any Borel-measurable $L$ ,

$\lambda(L) = 0 \quad \Rightarrow \quad \mathbb P_X(L) = 0.$

It turns out that the reverse is true.

Theorem 1 (Radon-Nikodým Theorem). Let $(\Omega, \mathcal F)$ be a measurable space. If $\mu, \nu$ are $\sigma$ -finite measures on $(\Omega, \mathcal F)$ such that $\nu \ll \mu$ , then there exists a measurable function $f : \Omega \to [0, \infty]$ such that

$\displaystyle \nu(K) = \int_K f\, \mathrm d\mu.$

Furthermore, $f$ is unique $\mu$ -a.e., called the Radon-Nikodým derivative, denoted $\displaystyle \frac{\mathrm d \nu}{\mathrm d\mu}$ .

In particular, for any measure $\mathbb P_X$ on $(\mathbb R^n, \frak{B}(\mathbb R^n), \lambda)$ , where $\lambda$ denotes the $n$ -dimensional Lebesgue measure,

$\displaystyle f_X := \frac{\mathrm d\mathbb P_X}{\mathrm d\lambda} \quad \iff \quad \mathrm d\mathbb P_X \equiv f_X\, \mathrm d\lambda$

is called the probability density function of $X$ , where the right-hand side follows a legitimate abuse of notation.

Our goal in this post is to prove the Radon-Nikodým theorem. To achieve that goal, we will need to introduce the notion of a signed measure, and a lemma known as Hahn’s decomposition theorem.

Let $(\Omega, \mathcal F)$ be a measurable space.

Definition 1. The map $\mu : \mathcal F \to \mathbb R$ (excluding infinities) is called a signed measure if $\mu(\emptyset) = 0$ and $\mu$ is countably additive.

Theorem 2 (Hahn Decomposition Theorem). If $\mu$ is a signed measure on $(\Omega, \mathcal F)$ , then there exist disjoint measurable sets $P, N \in \mathcal F$ such that

$\displaystyle \mu( \cdot \cap P) \in [0,\infty),\quad \mu( \cdot \cap N) \in (-\infty, 0],\quad P \sqcup N = \Omega.$

In this case, we call the set $P$ a positive set, denoted $P \geq 0$ , and $N$ a negative set, denoted $N \leq 0$ . A set $M$ is null if it is positive and negative, i.e. $\mu( \cdot \cap M) \in \{0\}$ .

Proof of Theorem 2. We first check that $\alpha := \sup \{\mu(K) : K \geq 0\} < \infty$ . If not, then for any $n \in \mathbb N^+$ , there exists $K_n \in \mathcal F$ with $K_n \geq 0$ such that $\mu(K_n) \geq n$ . Then $K := \bigcup_{n=1}^\infty K_n$ has measure $\mu(K) \geq n$ for any $n$ , leading to $\mu(K) = \infty$ , a contradiction.

Therefore, for any $n \in \mathbb N^+$ , there exists $P_n \geq 0$ such that $\alpha - 1/n < \mu(P_n) \leq \alpha$ . It follows by bookkeeping that $P := \bigcup_{n=1}^\infty P_n$ is positive. We claim that $N:= \Omega \backslash P$ is negative.

Suppose otherwise. Then there exists $K \in \mathcal F$ such that $\mu(K \cap N) > 0$ . Assume $K \subseteq N$ so that $\mu(K) > 0$ . If $K$ is positive, then so is $P \cup K$ , thus $\mu(P \cup K) = \mu(P) + \mu(K) > \alpha$ , a contradiction. Therefore, there exists $K_1 \subseteq K$ such that $\mu(K_1) = \mu(K_1 \cap K) < 0$ . Furthermore, $\mu(K \backslash K_1) = \mu(K) - \mu(K_1) > 0$ .

Define $n_1 := \min \{k \in \mathbb N : \mu(K_1) < -1/k\}$ , which exists by the Archimedean property of $\mathbb R$ . Repeating the argument inductively on the set $K \backslash \bigcup_{j=1}^{i-1} K_j$ , we obtain $K \supseteq K_1 \supseteq K_2 \supseteq \dots$ such that $K_i \subseteq K \backslash \bigcup_{j=1}^{i-1} K_j$ and $\mu(K_i) < -1/{n_i}$ , where $n_i$ is the smallest possible positive integer. Define $L := K \backslash \bigsqcup_{i=1}^\infty K_i$ , which has positive finite measure. Since $\mu$ is countably additive,

$\displaystyle \mu(K) = \mu(L) + \sum_{i=1}^\infty \mu(K_i).$

Hence, $\mu(K_i) \to 0 \iff 1/n_i \to 0$ as $i \to \infty$ .

If we can show that $L \geq 0$ , then $\mu(L \cup P) > \alpha$ , yielding the desired contradiction. To that end, suppose for a contradiction that there exists $M \subseteq L$ such that $\mu(M) = \mu(M \cap L) < 0$ . Then $\mu(M) < -1/(n_i - 1)$ for some $n_i$ , contradicting the construction of $K_i$ , as required.

Now we prove the Radon-Nikodým theorem.

Proof of Theorem 1. We first assume that $\mu, \nu$ are finite. For any $n, k$ , define the signed measure $\lambda_{n,k} := \nu - \frac{k}{2^n} \cdot \mu : \mathcal F \to \mathbb R$ . By the Hahn decomposition theorem, there exist disjoint positive and negative sets $P_{n,k}, N_{n,k}$ with respect to $\lambda_{n,k}$ such that $P_{n,k} \sqcup N_{n,k} = \Omega$ .

Since $k\cdot 2^{-n} \geq l \cdot 2^{-m}$ implies $P_{n,k} \subseteq P_{m,l}$ , for any $n$ , $( P_{n,k} )$ is decreasing in $k$ . Define the set $P_n := \bigcap_{k=1}^\infty P_{n,k}$ , which is positive with respect to all $\lambda_{n,k}$ . Since this set is positive with respect to $\lambda_{n,k}$ ,

$\displaystyle \lambda_{n,k}(P_{n,k}) \geq 0 \quad \Rightarrow \quad \mu(P_n) \leq \mu(P_{n,k}) \leq \frac{2^n}{k} \cdot \nu(P_{n,k}) \leq \frac{2^n}{k} \cdot \nu(\Omega).$

Taking $k \to \infty$ , we obtain $\mu(P_n) = 0$ for any $n$ . Define the simple functions

$\begin{aligned} f_{n}^- &:= \sum_{k=1}^\infty \frac{k}{2^n} \cdot \mathbb I_{P_{n,k} \backslash P_{n,k+1}} + \infty \cdot \mathbb I_{P_n}, \\ f_{n}^+ &:= \sum_{k=1}^\infty \frac{k+1}{2^n} \cdot \mathbb I_{P_{n,k} \backslash P_{n,k+1}} + \infty \cdot \mathbb I_{P_n}. \end{aligned}$

Defining $M_{n,k} := M \cap P_{n,k} \backslash P_{n,k+1}$ ,

$\displaystyle \lambda_{n,k}(M_{n,k}) \geq 0 \quad \Rightarrow \quad \frac{k}{2^n} \cdot \mu(M_{n,k}) \leq \nu(M_{n,k}).$

On the other hand, $M_{n,k} \subseteq \Omega \backslash P_{n, k+1} = N_{n,k+1}$ , so that

$\displaystyle \lambda_{n, k+1}(M_{n,k}) \leq 0 \quad \Rightarrow \quad \nu(M_{n,k}) \leq \frac{k+1}{2^n} \cdot \mu(M_{n,k}).$

Combining both estimates,

$\displaystyle \frac{k}{2^n} \cdot \mu(M_{n,k}) \leq \nu(M_{n,k}) \leq \frac{k+1}{2^n} \cdot \mu(M_{n,k}).$

Summing over $k$ , since $M = \bigsqcup_{k=1}^\infty M_{n,k} \sqcup ( M \cap P_n)$ ,

$\displaystyle \int_{M} f_n^-\, \mathrm d\mu \leq \nu(M \backslash P_n) \leq \int_{M} f_n^+\, \mathrm d\mu .$

Since $\mu(P_n) = 0$ implies $\nu(P_n) = 0$ ,

$\displaystyle \int_{M} f_n^-\, \mathrm d\mu \leq \nu(M) \leq \int_{M} f_n^+\, \mathrm d\mu .$

Therefore, $( f_n^- )$ is monotonically non-decreasing in $n$ , and so converges to some $f^- := \limsup_{n \to \infty} f_n^-$ by the monotone convergence theorem, and

$\begin{aligned} \int_M f^-\, \mathrm d\mu &= \lim_{n \to \infty} \int_M f_n^-\, \mathrm d\mu \leq \nu(M) \leq \int_M f^+\, \mathrm d\mu = \int_M f^-\, \mathrm d\mu + \frac {\mu(\Omega)}{2^n} . \end{aligned}$

Taking $n \to \infty$ , $\displaystyle \nu(M) = \int_M f^-\, \mathrm d\mu$ , so we set $f := f^-$ , as required.

Now suppose that $\mu, \nu$ are $\sigma$ -finite. Write $\Omega = \bigsqcup_{i=1}^\infty K_i$ with each $\mu(K_i) < \infty$ . For each $i$ , obtain a non-negative measurable function $f_i$ such that

$\displaystyle \nu(M \cap K_i) = \int_{M \cap K_i} f_i\, \mathrm d\mu ,\quad M \in \mathcal F.$

Define the non-negative measurable function $f := \sum_{i=1}^\infty f_i \cdot \mathbb I_{K_i}$ . Then for any $M \in \mathcal F$ ,

$\displaystyle \nu(M) = \sum_{i=1}^\infty \nu (M \cap K_i) = \sum_{i=1}^\infty \int_{M} f_i \cdot \mathbb I_{K_i}\, \mathrm d\mu = \int_{M} \sum_{i=1}^\infty f_i \cdot \mathbb I_{K_i}\, \mathrm d\mu = \int_{M} f\, \mathrm d\mu.$

Finally, $\mu$ -a.e. uniqueness holds since

$\displaystyle \left( \int_{M} f\, \mathrm d\mu = 0,\ M \in \mathcal F \right) \quad \Rightarrow \quad f = 0\ \mu\text{-a.e.}.$

We can now finally turn our attention to adding two continuous random variables $X, Y$ properly, since $\mathbb P_{X,Y} \ll \lambda$ so that by the Radon-Nikodým theorem, there exists a density function $f_{X,Y}$ for the joint distribution $\mathbb P_{X,Y}$ .

—Joel Kindiak, 23 Jul 25, 0113H

KindiakMath

Density Functions on Steroids

Response

Leave a comment Cancel reply

Density Functions on Steroids

Share this:

Response

Leave a comment Cancel reply