The Linearity of Calculus

The “algebra” in linear algebra essentially refers to the kinds of transformations one can perform on vectors. Since the algebra is linear, we shall define linear transformations, which is rather familiar to us in various fields of study.

Before giving a formal definition of a linear transformation, let’s motivate the discussion through some elementary school geometry. Given a rectangle with base $x$ and height $2$ , what is the area $A(x)$ of the rectangle? It would be $A(x) = 2x$ . This means

$\begin{gathered} A(x+y) = 2(x+y) = 2x + 2y = A(x) + A(y),\\ \quad A(kx) = 2(kx) = k(2x) = kA(x). \end{gathered}$

These properties have been used again and again, especially in calculus, since

$\displaystyle \frac{\mathrm d}{dx}(f+g) = \frac{\mathrm d}{dx}(f) + \frac{\mathrm d}{dx}(g),\quad \frac{\mathrm d}{dx}(kf) = k \frac{\mathrm d}{dx}(f)$

and

$\displaystyle \int (f+g) = \int f + \int g,\quad \int (kf) = k \int f.$

We can now define this notion of a linear transformation in broad generality. The idea is that any result that is true of linear transformations would be true to each of these examples, be it contrived ones like $A(\cdot)$ , and less contrived ones like the calculus operations.

For the rest of this post, let $K$ be any set, and let $V, W$ be vector spaces over a field $\mathbb K$ .

Definition 1. A function $T : V \to W$ is a linear transformation if for any $\mathbf u, \mathbf v \in V$ and $c \in \mathbb K$ ,

$T(\mathbf u + \mathbf v) = T(\mathbf u) + T(\mathbf v),\quad T(c\mathbf v) = cT(\mathbf v).$

These properties mimic the closure properties of a vector space. In fact, the key idea is that we want $T$ to preserve these closure properties. If $\{\mathbf v_1,\dots, \mathbf v_n\}$ forms a basis for $V$ , then it turns out that $V$ and $\mathbb K^n$ are not too different, allowing us to formally define dimensions. Finally, when we are dealing with the special cases $V = \mathbb K^n$ and $W = \mathbb K^m$ , we want to recover the usual notion of a matrix.

We first state a seemingly obvious yet incredibly useful result for linear transformations.

Lemma 1. A linear transformation $T : V \to W$ is injective if and only if for any $\mathbf v \in V$ , $T(\mathbf v) = \mathbf 0 \Rightarrow \mathbf v = \mathbf 0$ .

Proof. For $(\Rightarrow)$ , if $T$ is injective, then $T(\mathbf v) = \mathbf 0 = T(\mathbf 0)$ implies $\mathbf v = \mathbf 0$ , as required. On the other hand, for $(\Leftarrow)$ , for vectors $\mathbf u,\mathbf v \in V$ , suppose $T(\mathbf u) = T(\mathbf v)$ . Then

$T(\mathbf u - \mathbf v) = T(\mathbf u) - T(\mathbf v) = \mathbf 0 \quad \Rightarrow \quad \mathbf u - \mathbf v = \mathbf 0.$

Hence, $\mathbf u = \mathbf v$ , so that $T$ is injective, as required.

Let’s start with a question that sounds obvious but requires more thought: Is the constant $c$ a function? Well, for any $c \in \mathbb K$ , we can define the corresponding function $c \in \mathcal F(K, \mathbb K)$ by $f_c : K \to \mathbb K$ , where $f_c(x) = c$ for any $x \in K$ . Therefore, we can define a linear injection between the two vector spaces.

Lemma 2. Define the function $\iota : \mathbb K \to \mathcal F(K, \mathbb K)$ by the map $c \mapsto f_c$ , where $f_c(x) = c$ for any $x \in K$ . Then $\iota$ is a linear transformation and is injective.

Proof. For linearity, we observe that

$f_{c+d}(x) = c+d = f_c(x) + f_d(x) = (f_c + f_d)(x).$

Therefore, $\iota(c+d) = f_{c+d} = f_c + f_d = \iota(c) + \iota(d)$ . Similarly, $\iota(kc) = k \iota(c)$ , establishing linearity. For injectivity, we notice that $f_c = T(c) = 0$ implies that for any $x \in K$ , $c = f_c(x) = 0(x) = 0$ , so that $c =0$ , as required.

Recall that given any function $f : K \to L$ , $f(K) := \{f(x) : x \in K\} \subseteq L$ . In particular, if $f$ is injective, then we can regard $K \hookrightarrow f(K) \subseteq L$ as a subset via the imbedding $f$ . Contextualising to Lemma 2, we can regard $\mathbb K \hookrightarrow \iota(\mathbb K) \subseteq \mathcal F(K, \mathbb K)$ as a subspace. Therefore, constants are as good as functions, via the imbedding $\iota$ .

Using techniques in real analysis, we can prove that the collection $o(1) \subseteq \mathcal F(\mathbb R \backslash \{0\}, \mathbb R)$ such that

$f \in o(1) \quad \iff \quad f(x) \to 0\ \text{as}\ x \to 0$

is a subspace, with the property that

$c_1 - c_2 \in o(1) \quad \iff \quad c_1 - c_2 \to 0 \quad \iff \quad c_1 = c_2.$

For elements $f \in o(1)$ , we write $f \to 0$ . We can use this definition to extend the calculus-based subsets of functions.

For any function $f \in \mathcal F(\mathbb R \backslash \{0\}, \mathbb R)$ and $L \in \mathbb R$ , define $f \to L$ to mean

$f \to L \quad \iff \quad f(\cdot) - L \to 0 \quad \iff \quad f(\cdot) - L \in o(1).$

Lemma 3. If $f \to L$ and $f \to M$ , then $L = M$ .

Proof. Since $o(1)$ is a subspace,

$L-M = (f(\cdot) - M) + (-1) (f(\cdot) - L) \in o(1) \quad \Rightarrow \quad L = M.$

Henceforth, we will denote $L_f = L$ as the unique limit such that $f \to L$ .

Theorem 1. Define the subset of functions

$\mathcal G_0 = \{f \in \mathcal F(\mathbb R \backslash \{0\}, \mathbb R) : (\exists L \in \mathbb R: f \to L)\}.$

Then

$o(1) \subseteq \mathcal G_0 \subseteq \mathcal F(\mathbb R \backslash \{0\},\mathbb R)$

as subspaces and the map $\displaystyle \lim_{t \to 0} : \mathcal G_0 \to \mathbb R$ defined by

$\displaystyle f \mapsto L_f =: \lim_{t \to 0}f(t)$

is a well-defined linear transformation.

Proof. We first remark that $f \in \mathcal G_0 \iff f(\cdot) - L \in o(1)$ . Hence $f, g \in \mathcal G_0$ implies that $f-L_f, g - L_g \in o(1)$ , which yields

$(f+g) - (L_f+L_g) = (f-L_f) + (g-L_g) \in o(1)$

so that $f+g \in \mathcal G_0$ . Similarly, $cf \in \mathcal G_0$ , so that $\mathcal G_0 \subseteq \mathcal F(\mathbb R \backslash \{0\}, \mathbb R)$ as a subspace. With $L = 0$ , $o(1) \subseteq \mathcal G_0$ . Finally, the uniqueness of limits yields

$\displaystyle \begin{gathered} \lim_{t \to 0}(f(t) + g(t)) = L_{f+g} = L_f + L_g = \lim_{t \to 0}f(t) + \lim_{t \to 0}g(t), \\ \lim_{t \to 0}(cf(t)) = L_{cf} = cL_f = c \lim_{t \to 0}f(t).\end{gathered}$

Having defined limits at $0$ , we can generalise to limits at any other point $c$ . In fact, we don’t need to do a lot of hard work to even show that the set of functions with limits at $c$ exist; we’ll just transport the subspace property in the following manner:

Theorem 2. Let $T : V \to W$ be a linear transformation. Then the range of $T$ defined by

$T(V) := \{T(\mathbf v) : \mathbf v \in V\}$

is a subspace of $W$ .

Proof. For additivity, if $T(\mathbf u), T(\mathbf v) \in T(V)$ , then $\mathbf u + \mathbf v \in V$ yields

$T(\mathbf u) + T(\mathbf v) = T(\mathbf u + \mathbf v) \in V.$

Similarly, $c T(\mathbf v) = T(c\mathbf v) \in T(V)$ for any $c \in \mathbb K$ , thus $T(V) \subseteq W$ as a subspace.

Lemma 4. Let $T : V \to W$ be a linear transformation.

Let $U$ be a vector space over $\mathbb K$ and $S : U \to V$ and $T : V \to W$ be functions. If $S$ is a linear transformation, then so is $T \circ S$ .
If $T$ is bijective, then $T^{-1}$ is a bijective linear transformation.
If $K \subseteq V$ is a subset and $T$ is injective, then $K$ is a subspace of $V$ .

Proof. Exercise.

Corollary 1. For any $c \in \mathbb R$ , define the subset of functions

$\displaystyle \mathcal G_c = \left\{f \in \mathcal F(\mathbb R, \mathbb R) : \lim_{t \to 0} f(c+t)\ \text{exists}\right\}.$

Then $\mathcal G_c \subseteq \mathcal F(\mathbb R \backslash \{c\},\mathbb R)$ as a subspace and the map $\displaystyle \lim_{x \to c} : \mathcal G_c \to \mathbb R$ defined by

$\displaystyle f \mapsto \lim_{t \to 0}f(c+t) =: \lim_{x \to c}f(x)$

is a well-defined linear transformation.

Proof. Define the linear transformation $T : \mathcal G_c \to \mathcal G_0$ by $T(f) = f(c + \cdot)$ . Then $T$ is clearly bijective so that $\mathcal G_c \subseteq \mathcal F(\mathbb R \backslash \{c\}, \mathbb R)$ as a subspace.

Furthermore

$\displaystyle \lim_{x \to c} = \lim_{t \to 0}\ \circ\ T : \mathcal G_c \to \mathbb R$

is a linear transformation.

In most calculus courses, limits are applied to discuss continuous functions, in the following manner: $f$ is continuous at $c$ if and only if

$\displaystyle \lim_{x \to c}f(x) = f(c).$

In other words, $f \in \mathcal G_c$ is not all that is required; we also require

$\displaystyle \lim_{x \to c} f(x) - f(c) = 0.$

We could define this property then manually prove that the set of functions continuous at $c$ is indeed a subspace of $\mathcal G_c$ . However, we have an even more efficient tool up our linear algebraic sleeves.

Theorem 3. Let $T : V \to W$ be a linear transformation. Then the kernel of $T$ defined by

$\ker(T) := \{\mathbf v \in V : T(\mathbf v) = \mathbf 0\}$

is a subspace of $V$ .

Proof. For additivity, if $\mathbf u,\mathbf v \in \ker(T)$ , then

$T(\mathbf u + \mathbf v) = T(\mathbf u) + T(\mathbf v) = \mathbf 0 + \mathbf 0 = \mathbf 0,$

so that $\mathbf u + \mathbf v \in \ker(T)$ . Similarly, $c\mathbf v \in \ker(T)$ for any $c \in \mathbb K$ , thus $\ker(T) \subseteq V$ as a subspace.

Corollary 2. For any $c \in \mathbb R$ , define the subset of functions continuous at $c$ by

$\displaystyle \mathcal C_c = \left\{f \in \mathcal F(\mathbb R, \mathbb R) : \lim_{x \to c} f(x) = f(c)\right\}.$

Then for any $c \in \mathbb R$ ,

$\displaystyle \mathcal C := \bigcap_{c \in \mathbb R} \mathcal C_c \subseteq \mathcal C_c \subseteq \mathcal G_c \cap \mathcal F(\mathbb R,\mathbb R)$

as subspaces. Here $\mathcal C$ denotes the set of functions that are continuous on $\mathbb R$ .

Proof. Since arbitrary intersections of vector spaces remain as vector spaces, it remains to prove that $\mathcal C_c \subseteq \mathcal G_c \cap \mathcal F(\mathbb R,\mathbb R)$ as a subspace (as a subset, this is obvious). To that end, define the linear transformation $T: \mathcal G_c \to \mathbb R$ by

$\displaystyle T(f) = \lim_{x \to c} f(x) - f(c).$

Then the result $\mathcal C_c = \ker(T) \subseteq \mathcal G_c$ establishes the result.

Corollary 3. For any $c \in \mathbb R$ , define the subset of functions differentiable at $c$ by

$\displaystyle \mathcal D_c = \left\{f \in \mathcal F(\mathbb R, \mathbb R) : \lim_{x \to c} \frac{f(x) - f(c)}{x-c}=:f'(c)\ \text{exists}\right\}.$

Then for any $c \in \mathbb R$ ,

$\displaystyle \mathcal D := \bigcap_{c \in \mathbb R} \mathcal D_c \subseteq \mathcal D_c \subseteq \mathcal C_c$

as subspaces. Here $\mathcal D$ denotes the set of functions that are differentiable on $\mathbb R$ . Furthermore, the functions $\displaystyle D_c : \mathcal D_c \to \mathbb R$ and $\displaystyle \frac{\mathrm d}{\mathrm dx} : \mathcal D \to \mathcal F(\mathbb R, \mathbb R)$ defined by

$\displaystyle D_c(f) := f'(c),\quad \frac{\mathrm d}{\mathrm dx} (f) = f'$

are linear transformations.

Proof. To prove that $\mathcal D_c \subseteq \mathcal C_c$ requires the use of several limit laws in calculus. For the subspace property, define the injective linear transformation $Q : \mathcal D_c \to \mathcal G_c$ by

$\displaystyle f \mapsto Q_f,\quad Q_f(x):= \frac{f(x) - f(c)}{x - c}.$

It is not hard to verify that $Q(\mathcal D_c) \subseteq \mathcal G_c$ as a subspace. Thus, $Q : \mathcal D_c \to T(\mathcal D_c)$ is a bijective linear transformation, and thus $\mathcal D_c \subseteq \mathcal C_c$ as a subspace. Finally,

$\displaystyle D_c = \lim_{x \to c}\ \circ\ Q : \mathcal D_c \to \mathbb R$

as a linear transformation, and

$\displaystyle \frac{\mathrm d}{\mathrm dx} : \mathcal D \to \mathcal F(\mathbb R,\mathbb R),\quad \frac{\mathrm d}{\mathrm dx}(f) = f',\quad f'(c) := D_c(f)$

as a linear transformation.

Of course, these ideas extend even to integral calculus, which includes the generalisation of integral transforms, whose special cases include the Laplace transform and the Fourier transform. However, to establish the existence of these objects requires us to create new ideas in Lebesgue integration (and even in Riemann integration we don’t get a sufficiently big picture). Thus, we relegate the discussions therein if, and when, we get there.

Next up: differentiating polynomials which causes us to touch base, pun intended, with bases for vector spaces once again.

—Joel Kindiak, 26 Feb 25, 2344H

KindiakMath

The Linearity of Calculus

Response

Leave a reply to Mid-Term Rally – KindiakMath Cancel reply

The Linearity of Calculus

Share this:

Response

Leave a reply to Mid-Term Rally – KindiakMath Cancel reply