Maths on Steroids: Tute 7

I have no idea why inner products are so named. But I do know that there's something called an outer product...they're probably related.

In any case, inner products give us a notion of length and angle on a vector space. The former is given by taking the distance between two vectors $\vec{u}$ and $\vec{v}$ to be $||\vec{u}-\vec{v}||:=\sqrt{\langle\vec{u}-\vec{v},\vec{u}-\vec{v}\rangle}$. And the latter is defined by taking $$\begin{align}\frac{\langle\vec{u},\vec{v}\rangle}{||\vec{u}||\cdot||\vec{v}||}\end{align}$$ to be $\cos\theta$, where $\theta$ is the angle between $\vec{u}$ and $\vec{v}$. Of course, in order for this to make sense, we'd need $$\begin{align}|\langle\vec{u},\vec{v}\rangle|\leq ||\vec{u}||\cdot||\vec{v}||,\end{align}$$ which is precisely the Cauchy-Schwarz inequality.

Usually it's not all that hard to prove that something is an inner product: the linearity and symmetry conditions are fairly straight-forward to check, and positivity is often a matter of doing completing the square in the right way. Having said that, consider the following inner product $\langle\cdot,\cdot\rangle$ on the vector space $C^0([0,1],\mathbb{R})$ of continuous real-valued functions on $[0,1]$: $$\begin{align}\langle f, g\rangle:=\int_0^{1} f(x)g(x)\; \mathrm{d}x.\end{align}$$ In this case, the positivity condition is intuitively obvious, if a function $f:[0,1]\rightarrow\mathbb{R}$ is not the constant zero function, its square is a non-negative function and should have positive integral over $[0,1]$. But to make this precise, we're going to do a wee bit of analysis. The maths here is, in some sense, fairly easy - it's all in the language. Then again, this language did take some very smart mathematicians ages to formulate, so it's probably not entirely trivial?

Time to think positive!

We just need to show the second part of positivity, the first part is pretty obvious. So, we need to show that if $f\in C^0([0,1],\mathbb{R})$ is not the constant zero function, then $$\begin{align}\langle f,f\rangle=\int_0^1 f(x)^2\;\mathrm{d}x>0.\end{align}$$ Now, we know that $f$ is not everywhere zero, so for some (interior) point $x_0\in [0,1]$, the function $f$ spits out some number $f(x_0)\neq 0$. By the $\epsilon-\delta$ definition of continuity, for any positive number $\epsilon$ there's a small neighbourhood $(x_0-\delta,x_0+\delta)$ around $x_0$ so that $f$ spits out numbers between $f(x_0)-\epsilon$ and $f(x_0)+\epsilon$ on this neighbourhood. So if we take $\epsilon$ to be small enough, then on $(x_0-\delta,x_0+\delta)$, the magnitude of $f$ will be greater than some constant $c>0$. Therefore: $$\begin{align}\langle f,f\rangle=\int_0^1 f(x)^2\;\mathrm{d}x\geq\int_{x_0-\delta}^{x_0+\delta} f(x)^2\;\mathrm{d}x>\int_{x_0-\delta}^{x_0+\delta} c^2\;\mathrm{d}x=2\delta c^2>0.\end{align}$$ And we're done.

So ugh...that was totes not on the course syllabus, although it used to be...sorta...a few years ago. But if you decide to do any maths next year you're pretty likely to bump into this $\epsilon-\delta$ stuff again. So umm...here's something-else that's not quite on the syllabus: well, okay, it's more of a parting thought than anything-else. In playing around with these inner products, sometimes we encounter something that doesn't quite satisfy the second part of the positivity condition. For example: the following symmetric (bi)linear function on $\mathbb{R}^3$ does not satisfy the positivity condition that says that $||\vec{u}||=0$ if and only if $\vec{u}=\vec{0}$, $$\begin{align}\langle(x_1,x_2,x_3),(y_1,y_2,y_3)\rangle:=&2(x_1+2x_2+x_3)(y_1+2y_2+y_3)\\+&e(x_2-x_3)(y_2-y_3)\\+&\pi(x_1+3x_3)(y_1+3y_3).\end{align}$$ Well, there're two ways of fixing this problem so that we end up with a vector space with a legit inner product. The first is to restrict ourselves to a vector subspace on which our inner product does satisfy the final positivity condition. And there are many many such choices of a vector space in this particular case - such as: $$\begin{align}V:=\{(x,y,z)\in\mathbb{R}^3|-3x+y+z=0\}.\end{align}$$ And the second method is to equate certain vectors in our vector space $\mathbb{R}^3$, and thereby creating a new vector space called a quotient space $\mathbb{R}^3/W$, where $$\begin{align}W:=\mathrm{Span}_{\mathbb{R}}([-3,1,1])=\{\text{vectors whose ``length" is zero}\}.\end{align}$$ Note that althought you might find quotient spaces a tad intimidating the first time that you see them, quotienting just means that we think of two vectors $\vec{u}$ and $\vec{v}$ that differ by some element in $W$ (that is: $\vec{u}-\vec{v}\in W$) as being the same vector. In fact, this is how we do modular arithmetic! E.g.: one way of defining $\mathbb{Z}_2$ is to take all the integers and regard the ones that differ by even numbers as being the "same". So, $1$ is the same as $3$ is the same as $5232320009$ (modulo $2$).

Now, this quotient space idea has the slight advantage that there's really just one way of doing it. Although we do need to show that for some some nearly-inner product space $(V,\langle\cdot,\cdot\rangle)$, the set $$\begin{align}W:=\{\text{vectors whose ``length" is zero}\}\end{align}$$ is always a vector subspace. Soooo...Subspace Theorem time!

• Axiom 0: $W$ is non-empty because the vector $\vec{0}$ is always length zero.
• Axiom 2: Since $\langle\alpha\vec{w},\alpha\vec{w}\rangle=\alpha^2\langle\vec{w},\vec{w}\rangle=0$, $W$ is closed under scalar multiplication.
• Axiom 1: This is slightly trickier, we need to show that for two vectors $\vec{w}_1,\vec{w}_2\in W$, their sum $\vec{w}_1+\vec{w}_2$ is in $W$. Well, we know that $$\begin{align}\langle \vec{w}_1+\vec{w}_2,\vec{w}_1+\vec{w}_2\rangle=\langle\vec{w}_1,\vec{w}_1\rangle+2\langle\vec{w}_1,\vec{w}_2\rangle+\langle\vec{w}_2,\vec{w}_2\rangle=2\langle\vec{w}_1,\vec{w}_2\rangle,\end{align}$$ so all we need to do is show that this last term is also zero - so, we just need some weaker version of the Cauchy-Schwarz inequality that applies to a nearly-inner product. Here's a proof: our result is obviously true if either $\vec{w}_1$ or $\vec{w}_2$ is the zero vector, so let's ignore that case. Then we know from the first part of positivity that: $$\begin{align}0\leq\langle\frac{\vec{w}_1}{||\vec{w}_1||}\pm\frac{\vec{w}_2}{||\vec{w}_2||},\frac{\vec{w}_1}{||\vec{w}_1||}\pm\frac{\vec{w}_2}{||\vec{w}_2||}\rangle=1\pm2\langle\frac{\vec{w}_1}{||\vec{w}_1||},\frac{\vec{w}_2}{||\vec{w}_2||}\rangle+1\\\Rightarrow||\vec{w}_1||\cdot||\vec{w}_2||\geq\pm\langle\vec{w}_1,\vec{w}_2\rangle\Rightarrow||\vec{w}_1||\cdot||\vec{w}_2||\geq|\langle\vec{w}_1,\vec{w}_2\rangle|.\end{align}$$ So yea, $|\langle\vec{w}_1,\vec{w}_2\rangle|$ is less than $0=0\cdot0$, and we've shown that $W$ is closed under vector addition.

You might think that this is all pretty useless, but oddly enough this sort of stuff keeps cropping up in maths, often for perfectly legitimate reasons. Like, that inner product that we defined on the space of continuous functions? Well, if you try to ditch the word continuous and replace it with something like square-integrable (and this is a perfectly pragmatic thing to do - sometimes you want to integrate functions which aren't continuous, like car acceleration...etc.), you'll suddenly find that our integral based inner product now fails the last positivity condition. So peeps actually use this quotient space technology to fix this problem and do some functional analysis and stuff...and stuff.

Okay okay okay, we're done. =)