@Chen Siyi

November 30, 2020

Note9 Fourier Series

Let's look at the series methods from a wider view

Inner Product Space

Vector Space

A vector space is a set V , equipped with a rule for addition of any two vectors and for scalar multiplication of a vector by a scalar.

The addition + must satisfy the following axioms:

1. The set $V$ is closed under addition: For any two vectors v and w of $V$, the sum $v+w$ is also in $V$.
2. Additionis commutative: For all $v,w∈V$, $v+w=w+v$.
3. Additionis associative: For all $v,w,y∈V$, $(v+w)+y=v+(w+y)$.
4. There is an additive identity: There exists $0_V ∈ V$ such that $v+0_V = 0_V +v = v$ for all $v ∈ V$.
5. Every element has an additive inverse: for every $v ∈ V$ , there exists a vector $y ∈ V$ such that $v + y = y + v = 0_V$.

The scalar multiplication must satisfy the following axioms:

1. The set $V$ is closed under scalar multiplication: For any vector v in $V$ and any scalar $λ \in \mathbb{R}$, the scalar multiple $λv$ is also in $V$ .
2. For two scalars $a, b ∈ \mathbb{R} \text{ or } \mathbb{C}$, we have $a(bv) = (ab)v$ for all vectors $v ∈ V$ .
3. For $1 ∈ \mathbb{R}$, we have $1v = v$ for all $v ∈ V$ .

And finally, scalar multiplication distributes over addition:

1. $λ(v+w)=λ(v)+λ(w)$ fo rall v,w∈V and all $λ∈\mathbb{R} \text{ or } \mathbb{C}$.
2. $(a+b)v=av+bv$ for all vectors $v∈V$ and all scalars $a,b∈\mathbb{R} \text{ or } \mathbb{C}$.

Inner Product Space

An inner product on a vector space V is a function

$X$ can be $\mathbb{C}$ or $\mathbb{R}$. It must satisfy the following axioms:

1. Symmetry: $⟨f,g⟩=\overline{⟨g,f⟩}$ for all vectors $f,g∈V$;
2. Linearity in each factor: $⟨(f + g),h⟩ = ⟨f,h⟩ + ⟨g,h⟩$ for all vectors f,g,h ∈ V and $⟨λf, g⟩ = \overline{λ}⟨f, g⟩$ for all $λ \in \mathbb{C}$ and all $f, g ∈ V$.
3. Positive Definiteness: $⟨f,f⟩ ≥ 0$ for all $f ∈ V$ with $⟨f,f⟩ = 0$ if only if $f = 0_V$.

A vector space V together with a choice of an inner product $(V, <\cdot , \cdot>)$ is called an inner product space.

As the most familiar example, $V = \mathbb{R}^n$, $<\cdot , \cdot> = \cdot$

Norm and Orthogonality

As soon as you define an inner product space, you can define norm and orthogonality

Norm of a vector $\textbf{v}$ is defined as

Which can be interpreted as the length defined under that inner product.

If the norm of $\textbf{v} = 1$, it is said to be normed or normalized.

And there are some fundamental theorems:

1. Pythagoras’s Theorem:

   Let $(V , ⟨ · , · ⟩)$ be an inner product space and $z = x + y ∈ V$ , where $⟨x , y ⟩ = 0$. Then

   $$\|z\|^{2}=\|x\|^{2}+\|y\|^{2}$$

2. Bessel’s Inequality:

   Let $(V , ⟨ · , · ⟩)$ be an inner product space and $\{e_k\}$, $k∈I ⊂ V, I ⊂ N,$ be an orthonormal system in $V$. Then, for any $v∈V$,

   $$\sum_{k \in I}\left|\left\langle e_{k}, v\right\rangle\right|^{2} \leq\|v\|^{2}$$

Two vectors $\textbf{v}$ and $\textbf{w}$ are orthogonal if

A system of vectors is an orthonormal system if all the vectors are normed, and are orthogonal to each other.

Let's see a really straight-forward way to find orthonormal systems!

As long as initially you have a system, just keep using orthogonal projection...

Gram-Schmidt Orthonormalization

Used to construct an orthonormal system from an existing system.

A Tricky Question:

By the way, does any system returns an orthonormal system of the same size as before?

Orthonormal Basis and Completeness

A system of vectors $\mathcal{B}$ is an orthonormal basis of the vector space $V$ if it is an orthonormal system, and $\textbf{span}\{\mathcal{B}\} = V$.

A Tricky Question:

Can we always construct an orthonormal basis of $V$ using the Gram-Schmidt Orthonormalization approach from an existing basis of $V$?

A theorem but not the definition: An inner product space $(V, <\cdot , \cdot>)$ is complete iff every maximal orthonormal system is an orthonormal basis.

Intuitively you can interpret as...

Any vector in $V$ can be represented "precisely" (by linear combinitions) using the maximal orthonormal system as long as it is a basis.

Otherwise, even you have a maximal orthonormal system, this system can't be used to represent a vector precisely!

Multiple inner products can be defined for a vector space. For example, let $C([a,b])$ be the vector space:

And a sequence is said to converge uniformly if $∥f_n∥∞\rightarrow 0$, converge in the mean if $∥f_n∥_1 \rightarrow 0$ ,and converge in the mean square if $∥f_n∥_2 \rightarrow 0$.

But not all inner products make sure the inner product space is complete.

For example, $(C([a,b]), ∥\cdot∥_2)$ is incomplete. $(L^2([a,b]), ∥\cdot∥_2)$ is complete. Where,

Approximation of "Vectors"

We now "say" the least square estimation is our best approximation.

Let $(V,⟨·, ·⟩)$ be an inner product space, $v ∈ V$ and $B = {e_k}$ an orthonormal system in $V$ . We seek to approximate a vector $v$ using a linear combination of the first $N ∈ \mathbb{N}$ elements (or as many as you can...) of the orthonormal system:

Our approximation is the best when the following "error" is minimized:

Fourier analysis is acting exactly in the same way to find an estimation...

And we choose the inner product space to be $(L^2([a,b]), ∥\cdot∥_2)$. So you will notice, it's generally helpful to represent periodic functions.

Which region $[a,b]$ do we choose?

Which orthonormal system do we choose? Four different choices. Keep in mind it is affected by the region you choose.

A Tricky Question:

Recall The Method of Frobenius, do you find any similarities in the methods?

Fourier Analysis

Basic Definitions about "Measure"

• A set $Ω ⊂ \mathbb{R}$ is said to have

  • measure less than $ε$ if...
  • measure zero if...

• A property is said to hold almost everywhere (abbreviated by a.e.) on a subset $D ⊂ \mathbb{R}$ if...

• The set $Ω = \mathbb{Q}$ of rational numbers has measure zero.

(One of) Our next goal:

Find some (Fourier)series $f'$ equal to a function $f$ almost everywhere within certain region.

The Fourier-Euler Basis

If you choose $[a,b] = [-\pi,\pi]$, then since the below trigonometric polynomials forms an orthonormal basis of $L^2([-\pi,\pi])$, we can choose it as the Fourier-Euler Basis.

1. You can just use the first several terms to form a Fourier-Euler expansion of any $f \in L^2([-\pi,\pi])$ by projecting onto the first sevral vectors in this basis.
2. You can also form a Fourier Series $f'$ which is a linear combination of "every" vectors in the basis, and equals to $f\in L^2([-\pi,\pi])$ almost every where. $f'$ is gained by projecting onto each vector in this basis.

If you choose $[a,b] = [0,L]$, then the Fourier-Euler Basis can be

Exercise:

If possible, uniquely expand $f(x) = x^2$, $0 < x < 2\pi$ in a Fourier series if (a) the period is $2\pi$, (b) the period is not specified.

Exercise:

Using the results of the last exercise to prove

$$\frac{1}{1^{2}}+\frac{1}{2^{2}}+\frac{1}{3^{2}}+\cdots=\frac{\pi^{2}}{6}$$

The Gibbs Phenomenon

A Fourier series may not and does not need to converge uniformly. The occurrence of these peaks of $f'$ at the discontinuities of $f$ is known as the Gibbs phenomenon.

Recall "uniform convergence".

Convergence of Fourier Series(Dirichlet’s rule)

1. On any sub-interval $[a',b']⊂[a,b]$ with $a' >a$ , $b' <b$ on which $f$ is continuous, the Fourier series converges uniformly towards $f$ .

2. At any point $x ∈ [a, b]$, we have the pointwise limit: when $N \rightarrow \infty$

   $$S_{N}(x) \longrightarrow \frac{\lim _{y \nearrow x} f(y)+\lim _{y \backslash x} f(y)}{2}$$

Actually the second point needs certain conditions () to hold, but we don't discuss them in VV286.

The Fourier-Cosine Basis

Cosine functions are even.

• For even functions, we can just use Cosine functions to expand.
• For functions which is not even, we can just use Cosine functions to find its even part.

If you choose $[a,b] = [0,L]$, then the Fourier-Cosine Basis can be

The Fourier-Sine Basis

Since functions are odd.

• For odd functions, we can just use Sine functions to expand.
• For functions which is not even, we can just use Cosine functions to find its odd part.

If you choose $[a,b] = [0,L]$, then the Fourier-Sine Basis can be

A Tricky Question:

For an arbitrary $f \in L^2([a,b])$, if you expand it using the the Fourier-Cosine Basis and the Fourier-Sine Basis separately with $\infin$ terms, and obtain series $f_1$ and $f_2$, would $f_1 + f_2$ and $f$ becomes equal almost every where?

It's interesting to see knowledge links to each other, does this remind you of any fundamental theorems you have seen before?

Exercise:

Explain why any function $F(x)$ is a sum of an even function and an odd function in just one way.

Explain why any odd function $f(x)$ would has its Fourier cosine series be 0.

A Tricky Question:

Can the following question be solved?

Expand $f(x) = sin x$, $0 < x < \pi$, in a Fourier cosine series.

Exercise:

Expand $f(x) = x$, $0 < x < 2$, in a half range (a) sine series, (b) cosine series.

Hint: Can you first "extand" f a little bit?

Just a reminder before the last part, $L^{2}([a, b]):=\left\{f:[a, b] \rightarrow \mathbb{C}: \int_{a}^{b}|f(x)|^{2} d x<\infty\right\}$ is defined to containning complex valued functions of a real variable, not the same as complex functions.

Complex Fourier-Euler Basis

If you choose $[a,b] = [-\pi,\pi]$, then the following would also be a basis of $L^{2}([a, b])$, simply because you can change basis with the Fourier-Euler Basis which we discussed above.

So you can also do orthogonal projection onto each vector of this basis to find an approximation. No matter whether $f$ is real valued or not, since just the $\lambda_i$ might be complex numbers.

Additional Practice

Exercise:

Find a Fourier series for $f(x)= cos \alpha x$, $-\pi < x < \pi$, where $\alpha \neq 0, \pm 1,\pm 2,\pm 3, \ldots$ ... .

Hint: Can you first "extand" f a little bit?

Exercise:

Prove that

$$\sin x=x\left(1-\frac{x^{2}}{\pi^{2}}\right)\left(1-\frac{x^{2}}{(2 \pi)^{2}}\right)\left(1-\frac{x^{2}}{(3 \pi)^{2}}\right) \cdots$$

Hint:

Use the results from the previous exercise.