KindiakMath

Fourier Orthogonality

Let’s discuss Fourier series using orthogonality—a fundamental approximation tool when studying periodic functions. Let \mathbb K = \mathbb R or \mathbb C.

Definition 1. For any K \subseteq \mathbb R, define the subspace \mathcal C(K, \mathbb K) \subseteq \mathcal F(K, \mathbb K) of functions that are continuous on K.

Remark 1. The relevant continuity properties in the case \mathbb K = \mathbb R are mostly preserved in the case \mathbb K = \mathbb C due to the same topological ideas in real and complex analysis. Products of functions remain continuous, and in particular, Riemann-integrable in both cases, with the latter being Riemann-integrable if and only if the real and imaginary parts \mathrm{Re}(f), \mathrm{Im}(f) are Riemann-integrable, in which can we define

\displaystyle \int_a^b f(t)\, \mathrm dt := \int_a^b \mathrm{Re}(f)(t)\, \mathrm dt + i \int_a^b \mathrm{Im}(f)(t)\, \mathrm dt.

Recall the usual the inner product on \mathbb K^n \cong \mathcal F(\{1,\dots,n\}, \mathbb K) by

\displaystyle \langle f, g \rangle = \sum_{k=1}^n f(k) \overline{g(k)}.

Definition 2. Define the subspace

\mathcal L^2([a, b], \mathbb K) := \{f \in \mathcal F([a, b], \mathbb K : |f|^2\ \text{is Riemann-integrable}\} \subseteq \mathcal F([a, b], \mathbb K)

of functions that are square-integrable on K. For simplicity, we make the identification in \mathcal L^2([a, b], \mathbb K):

\displaystyle f = g\quad \iff \quad \int_a^b |(f-g)(t)|\, \mathrm dt = 0.

This is certainly true if f,g are continuous.

Remark 2. The more technical formulation is that the relation \sim on \mathcal L^2([a, b], \mathbb K) is an equivalence relation defined by

\displaystyle f \sim g\quad \iff \quad \int_a^b |(f-g)(t)|\, \mathrm dt = 0,

and that we will regard without ambiguity \mathcal L^2([a, b], \mathbb K)/{\sim} = \mathcal L^2([a, b], \mathbb K). In any case, we have

\mathcal C([a, b], \mathbb K) \subseteq \mathcal L^2([a, b], \mathbb K) \subseteq \mathcal F([a, b], \mathbb K)

as subspaces.

We can extend the inner product idea to \mathcal L^2([a, b], \mathbb K) \subseteq \mathcal F(K, \mathbb K).

Lemma 1. The map \langle \cdot, \cdot \rangle : \mathcal L^2([a, b], \mathbb K) \times \mathcal L^2([a, b], \mathbb K) \to \mathbb K defined by

\displaystyle \langle f, g \rangle := \int_a^b f(t) \overline{g(t)}\, \mathrm dt

is a well-defined inner product.

Proof. To prove that \langle \cdot, \cdot \rangle is well-defined in general requires some advanced analytic machinery known as Hölder’s inequality, and so we will relegate that discussion elsewhere in advanced real analysis. However, in the case f, g are bounded, we also have f \bar g being bounded, so that the integral is well-defined. This certainly holds when f,g are continuous on [a, b] due to the extreme value theorem.

We now verify the rest of the inner product axioms. For any f,

\displaystyle \langle f, f \rangle = \int_a^b f(t) \overline{f(t)}\, \mathrm dt = \int_a^b |f(t)|^2\, \mathrm dt \geq 0,

where equality holds if and only if f = 0. The remaining two properties follow from the linearity of integration.

Corollary 1. A function \alpha : [a, b] \to \mathbb R is a weight if it is nonnegative, continuous on (a, b), and Riemann-integrable on [a, b]. For any weight \alpha, the map \langle \cdot, \cdot \rangle_\alpha : \mathcal L^2([a, b], \mathbb K) \times \mathcal L^2([a, b], \mathbb K) \to \mathbb K defined by

\displaystyle \langle f, g \rangle_\alpha := \langle f, \alpha g\rangle

is a well-defined inner product, in particular with the weight

\displaystyle \alpha := \frac{1}{\langle 1, 1 \rangle} = \frac 1{b-a}.

Henceforth, we will regard \mathcal L^2([a, b]) = (\mathcal L^2([a, b]), \langle \cdot, \cdot \rangle_{\alpha}) as an inner-product space. Furthermore, we denote \langle \cdot, \cdot \rangle = \langle \cdot, \cdot \rangle_{\alpha_{[a, b]}} when there is no ambiguity.

Finally, we are most interested in periodic functions, such as \sin and \cos, which have a period of 2\pi. More precisely, \sin = \sin(\cdot + 2\pi), and for any t < 2\pi, \sin \neq \sin(\cdot + t). In fact, since e^{it} = \cos(t) + i \sin(t) by Euler’s formula, the function f : \mathbb R \to \mathbb C defined by f(t) = e^{it} also has a period of 2\pi.

Definition 3. For any T > 0, we call a function f \in \mathcal F(\mathbb R, \mathbb K) T–periodic if f = f(\cdot + T) and for t < T, f \neq f(\cdot + t). Define the subspace

\mathcal F_T := \{f \in \mathcal F(\mathbb R, \mathbb K) : f\ \text{is}\ T\text{-periodic}\},

as well as the subspace

\mathcal L_T^2 := \{f \in \mathcal F_T : f|_{[-T/2,T/2]} \in \mathcal L^2([-T/2,T/2])\}.

Finally, define the set of periodic functions by \displaystyle \bigcup_{T \in \mathbb R_{> 0}} \mathcal F_T.

Lemma 2. For any a \in \mathbb R,

\mathcal L_T^2 := \{f \in \mathcal F_T : f|_{[a,a+T]} \in \mathcal L^2([a,a+T])\}.

Proof. Suppose a \geq 0 without loss of generality. The proof follows from an integration by substitution:

\begin{aligned} \int_a^{a+T}f(t)\, \mathrm dt &= \int_a^{T/2}f(t)\, \mathrm dt + \int_{T/2}^{a+T}f(t)\, \mathrm dt \\ &= \int_a^{T/2}f(t)\, \mathrm dt + \int_{T/2}^{a+T}f(t-T)\, \mathrm dt \\ &= \int_a^{T/2}f(t)\, \mathrm dt + \int_{-T/2}^{a}f(t)\, \mathrm dt \\ &= \int_{-T/2}^{a}f(t)\, \mathrm dt + \int_a^{T/2}f(t)\, \mathrm dt = \int_{-T}^{T}f(t)\, \mathrm dt.\end{aligned}

Replacing f with f^2 yields the desired results.

An inner product space isn’t much good unless we have a subspace with an orthonormal basis. For brevity, we shall abuse the notation f \equiv f(t) when context is clear.

Theorem 1. For any n \in \mathbb N, the set \mathcal E_n := \{e^{ikt} : -n \leq k \leq n\} \subseteq \mathcal L_{2\pi}^2 is orthonormal. Define the subspace E_n := \mathrm{span}(\mathcal E_n) \subseteq \mathcal L_{2\pi}^2.

Proof. For any k, m \in [-n, n],

\begin{aligned}\langle e^{i k t}, e^{i m t} \rangle &= \frac 1{2\pi} \int_0^{2\pi} e^{i k t } \overline{e^{i m t}}\, \mathrm dt \\ &= \frac 1{2\pi} \int_0^{2\pi} e^{i k t } e^{-i m t}\, \mathrm dt \\ &= \frac 1{2\pi} \int_0^{2\pi} e^{i (k-m) t } \, \mathrm dt.\end{aligned}

Therefore, \langle e^{i k t}, e^{i k t} \rangle = 1 while for k \neq m,

\begin{aligned}\langle e^{i k t}, e^{i m t} \rangle &= \frac 1{2\pi} \int_0^{2\pi} e^{i (k-m) t } \, \mathrm dt =\frac 1{2\pi} \left[ \frac{e^{i(k-m)t}}{i(k-m)} \right]_0^{2\pi} = 0\end{aligned}

by the (2\pi)/k-periodicity of e^{ikt}.

Corollary 2. For any f \in \mathcal L_{2\pi}^2 and any n \in \mathbb N, the n-th Fourier sum S_n(f) of f is given by

\displaystyle S_n(f):= \mathrm{proj}_{E_n}(f) = \sum_{k=-n}^n \hat f(k) e^{ikt},

where the Fourier coefficients are given by

\displaystyle \hat f(k) := \frac 1{2\pi} \int_{-\pi}^{\pi} f(t) e^{-ikt}\, \mathrm dt.

Proof. By the definition of orthogonal projection,

\displaystyle \mathrm{proj}_{E_n}(f) = \sum_{k=-n}^n \langle f, e^{ikt} \rangle e^{ikt}.

By definition of the inner product (on [ -\pi, \pi ]),

\begin{aligned} \langle f, e^{ikt} \rangle &= \frac 1{\pi-(-\pi)} \int_{-\pi}^{\pi} f(t) \overline{e^{ikt}}\, \mathrm dt \\ &= \frac 1{2\pi} \int_{-\pi}^{\pi} f(t) e^{-ikt}\, \mathrm dt = \hat f(k). \end{aligned}

This formula is, in fact, the motivation for the Fourier transform

\displaystyle \hat f(\xi) = \frac 1{2\pi} \int_{-\infty}^{\infty} f(t) e^{-i\xi t}\, \mathrm dt,

which we may explore in the future. A more pressing issue arises. If f : \mathbb R \to \mathbb R, then how do we know that S_n(f) is real-valued?

Corollary 3. If \mathbb K = \mathbb R, then for any f \in \mathcal L_{2\pi}^2 and any n \in \mathbb N,

\displaystyle (S_n(f))(t) = \frac{a_0}{2} + \sum_{k=1}^n (a_k \cos(kt) + b_k \sin(kt)),

where

\begin{aligned} a_n = \frac 1{\pi} \int_{-\pi}^{\pi} f(t) \cos(kt)\,\mathrm dt, \quad b_n = \frac 1{\pi} \int_{-\pi}^{\pi} f(t) \sin(kt)\,\mathrm dt. \end{aligned}

Proof. By Euler’s formula,

\displaystyle \hat f(k) = \frac 1{2\pi} \int_{-\pi}^{\pi} f(t) \cos(kt)\,\mathrm dt- i\cdot \frac 1{2\pi} \int_{-\pi}^{\pi} f(t) \sin(kt)\,\mathrm dt = \frac{a_k}{2} - i \frac{b_k}{2}.

Furthermore, \overline{\hat f(k) e^{ikt}} = \hat f(-k) e^{-ikt}. Hence,

\displaystyle \begin{aligned} \hat f(-k) e^{-ikt} + \hat f(k) e^{ikt} &= \mathrm{Re}(2 \hat f(k) e^{ikt}) \\ &= \mathrm{Re}( ( a_k - i b_k ) (\cos(kt) + i \sin(kt) ) \\ &= \mathrm{Re}(a_k \cos(kt) + b_k \sin(kt) + i ( {\cdots} )) \\ &= a_k \cos(kt) + b_k \sin(kt).\end{aligned}

Since c_0 = a_0/2, the result follows.

Remark 2. Since the results hold for any n, we usually write

\displaystyle f(t) = \frac{a_0}{2} + \sum_{k=1}^\infty (a_k \cos(kt) + b_k \sin(kt)),

where we assume suitable notions convergence (either pointwise or square-integrable) for sufficiently nicely-defined functions, so that we can evaluate several surprising infinite series.

Corollary 3. If \mathbb K = \mathbb R, then for any T > 0, f \in \mathcal L_{T}^2 and any n \in \mathbb N,

\displaystyle (S_n(f))(t) = \frac{a_0}{2} + \sum_{k=1}^n (a_k \cos(kt) + b_k \sin(kt)),

where

\begin{aligned} a_k &= \frac 2{T} \int_{-T/2}^{T/2} f(t) \cos \left(\frac{2\pi k t}{T}\right)\,\mathrm dt, \\ b_k &= \frac 2{T} \int_{-T/2}^{T/2} f(t) \sin\left(\frac{2\pi k t}{T}\right)\,\mathrm dt. \end{aligned}

Proof. Define the differentiable bijection h : t \mapsto t T/(2\pi) so that h' = T/(2\pi), the function g \in \mathcal L_{2\pi}^2 defined by g(t) = (f \circ h)(t) and the orthonormal set \mathcal E_n' := \{h^{-1}(\phi) : \phi \in \mathcal E_n\}. By Lemma 1 and a change-of-variable,

(S_n(f))(t) = \mathrm{proj}_{\mathrm{span}(\mathcal E_n')}(f) = \mathrm{proj}_{\mathrm{span}(\mathcal E_n)}(f \circ h) = (S_n(g))(t).

Applying Corollary 3,

(S_n(g))(t) = \displaystyle \frac{a_0}{2} + \sum_{k=1}^n (a_k \cos(kt) + b_k \sin(kt)),

where we make the change-of-variable u = h(t) to get

\displaystyle \begin{aligned} a_n &= \frac 1{\pi} \int_{-\pi}^{\pi} g(t) \cos(kt)\, \mathrm dt \\ &= \frac 1{\pi} \int_{-\pi}^{\pi} f(h(t)) \cos(kt)\, \mathrm dt \\ &= \frac 1{\pi} \int_{h(-\pi)}^{h(\pi)} f(u) \cos(kh^{-1}(u)) \cdot \frac{1}{h'(t)}\, \mathrm du \\ &= \frac 1{\pi} \int_{-T/2}^{T/2} f(u) \cos \left( \frac{2\pi k u}{T} \right) \cdot \frac{2\pi}{T}\, \mathrm du,\end{aligned}

and b_n is derived similarly.

Using Remark 2, we can obtain rather surprising values for infinite series.

Theorem 2. \displaystyle \frac 1{1^2} + \frac 1{2^2} + \frac 1{3^2} + \cdots = \frac{\pi^2}{6}.

Proof. Define f : [-\pi,\pi] \to \mathbb R by f(t) = t^2, and extend it to a 2\pi-periodic function via f = f(\cdot + 2\pi). Then

\displaystyle a_0 = \frac{1}{\pi} \int_{-\pi}^{\pi} t^2\, \mathrm dt = \frac 1{\pi} \cdot 2 \cdot \frac{\pi^3}{3} = \frac{2\pi^2}{3}.

Furthermore, for any k \neq 0, two applications of integration by parts yields

\displaystyle \begin{aligned} a_k &= \frac 1{\pi}\int_{-\pi}^{\pi} t^2 \cos(kt)\,\mathrm dt \\ &= \frac 1{\pi} \left[ \frac{t^2 \sin(kt)}{k} + \frac{2t \cos(kt)}{k^2} - \frac{2 \sin(kt)}{k^3}\right]_{-\pi}^{\pi} \\ &= \frac 1{\pi} \cdot 4\left[ \frac{t \cos(kt)}{k^2} \right]_{0}^{\pi} = \frac 1{\pi} \cdot \frac{4\pi \cos(k\pi)}{k^2} = {(-1)}^k \cdot \frac{4}{k^2}. \end{aligned}

We leave it as an exercise to verify that for any k, b_k = 0. By Remark 2,

\displaystyle f(t) = \frac{\pi^2}{3} + \sum_{k=1}^\infty {(-1)}^k \cdot \frac{4}{k^2} \cos(kt).

Setting t = \pi,

\displaystyle \pi^2 = f(\pi) = \frac{\pi^2}{3} + 4 \cdot \sum_{k=1}^\infty {(-1)}^k \cdot \frac{1}{k^2} (-1)^k = \frac{\pi^2}{3} + 4 \cdot \sum_{k=1}^\infty \frac{1}{k^2}.

By algebruh,

\displaystyle \frac 1{1^2} + \frac 1{2^2} + \frac 1{3^2} + \cdots = \sum_{k=1}^\infty \frac{1}{k^2} = \frac 14 \cdot \left( \pi^2 - \frac{\pi^2}{3} \right) = \frac{\pi^2}{6}.

Finally, we will discuss one more crucial application—the singular value decomposition—which shall be our capstone result in applied linear algebra.

—Joel Kindiak, 22 Mar 25, 1300H

,

Published by

joelkindiak

Leave a comment