2.3 Inner products and orthogonality

In discrete-time signal processing, understanding signals as vectors within a vector space allows us to use tools of analysis and linear algebra to examine signal properties. One of the properties we may want to consider is the similarity of (or difference between) two vectors. A mathematical tool that provides insight into this is the inner product .

Transposing vectors

Now, when it comes to complex valued vectors, we can take a transpose in the same way, but for the purposes of finding an inner product we actually need to take the conjugate, or Hermitian, transpose, which involves taking the transpose and then the complex conjugate: $\begin{bmatrix}x[0]\\ x[1] \\ \vdots \\ x[N-1]\end{bmatrix}^H =\begin{bmatrix}x[0]^*&x[1]^*&\cdots&x[N-1]^* \end{bmatrix}$Of course, for real valued vectors, the regular transpose and Hermitian transpose are identical.

The inner product

For two (or three) dimensional vectors, this angle is exactly what you would expect it to be. Let $x=\begin{bmatrix}1 \\ 2 \end{bmatrix}$, ~ $y=\begin{bmatrix}3 \\ 2 \end{bmatrix}$We have $\|x\|_2^2 = 1^2 + 2^2 = 5$, $\|y\|_2^2 = 3^2 + 2^2 = 13$ ,and $\langle x, y \rangle=(1)(3)+(2)(2)=7$.The angle between them is $\arccos\left(\frac{7}{\sqrt{(5)(13)}}\right) \approx 0.519~{\rm rad} \approx 29.7^\circ$. If you plot the vectors out in the Cartesian plane, you will indeed see an angle between them of about 30 degrees.

For higher dimensional signals the result of the inner product--how it relates to the angle between signals--may not seem as intuitive, but the information it provides is still just as useful, and of course it is computed in the same was as with shorter vectors. Consider the signals below: