The impulse response of infinite-length lti systems
Systems are mathematical transformations that take input signals and map them to output signals: Recall that linear systems can be expressed as matrix multiplications, the output $y$ being the product of the infinitely long matrix $H$ and the input vector $x$: $x=Hx$. Recall also that if a linear system is also time-invariant, the matrix $H$ has a special "Toeplitz" structure: each column/row is simply a shifted version of any other column/row: The 0th column of the $H$ matrix for an LTI system has a special interpretation. If we multiply the $H$ matrix by a delta function, then the result will simply be that 0th column: We call the result of this multiplication, i.e. the multiplication of the $H$ matrix by delta (impulse) function, the impulse response of the system. It is usually denoted $h[n]$, or if referring to it simple as a vector, as $h$. The impulse response $h[n]$ is simply the output of a system when the input is $\delta[n]$. Using the matrix notation from above, $h$ is the 0th column of the system matrix $H$.Because the impulse response is simply a column (specifically, the 0th column) of the system matrix $H$, then if $H$ is an LTI system, the impulse response completely characterizes the system. It tells us absolutely everything we need to know about the system! Why is this? Because we can construct the entire system matrix $H$ by simply shifting the impulse response accordingly. The value of the matrix at row $n$ and column $m$ is simply $h[n-m]$.
As a result, we can also express the system input-output matrix multiplication in terms of the impulse response. If we have that $[{\bf H}]_{n,m}$ is the value of the $H$ matrix at row $n$ and column $m$, then the matrix multiplication $y[n]=Hx=\sum_\limits_{m=-\infty}^{\infty}[{\bf H}]_{n,m}x[m]$. From above we have the matrix values expressed in terms of the impulse response; subbing this in we see the output is a sum involving the impulse response and the input: $y[n]=Hx=\sum_\limits_{m=-\infty}^{\infty}h[n-m]x[m]$.
Impulse response of the scaling system
Let's find the impulse response of a simple scaling system: $y[n]=H\{x[n]\})=2x[n]$ To find the impulse response $h[n]$, simply calculate the output when the input is a delta function: $h[n]=H\{\delta[n]\})=2\delta[n]$ If we would like to know what the system matrix $H$ is, we use the formula $[{\bf H}]_{n,m} = h[n-m]= 2\delta[n-m]$. Below is a pictorial representation of that matrix:
Impulse response of the shift system
Now let's try finding the impulse response of a simple time-shifting system: $y[n]=H\{x[n]\}=x[n-2]$ Once again, find the impulse response by giving a delta function as the input:$h[n]=H\{\delta[n]\}=\delta[n-2]$ The system matrix $H$ can be found, as before, as $[{\bf H}]_{n,m} = h[n-m]= \delta[n-m-2]$
Impulse response of a moving average system
Consider now a simple moving average system: $y[n]~=~ {\cal H}\{ x[n] \} ~=~ \frac{1}{2}\left( x[n]+x[n-1]\right)$For this system, we find that the impulse response is: $h[n]~=~ {\cal H}\{ \delta[n] \} ~=~ \frac{1}{2}\left( \delta[n]+\delta[n-1]\right)$ The system matrix is found to be $[{\bf H}]_{n,m} = h[n-m]= \frac{1}{2}\left( \delta[n-m]+\delta[n-m-1]\right)$
Impulse response of the recursive average system
A recursive moving average system has the following input-output relationship: $y[n] ~=~ {\cal H}\{ x[n]\} ~=~ x[n] + \alpha\, y[n-1]$. To find the impulse response of this system, we first let $x[n]=\delta[n]$ and find the output: $h[n]={\cal H}\{ x[n] \}+\alpha\, h[n-1]$ A bit more work is required if we would like a non-recursively defined impulse response. Assuming that the system $H$ is initially at rest (it has no output in the absence of an input), then we can find the output by computing several values of $h[n]$: $h[-1]=0$ $h[0]=\delta[0]+\alpha h[-1]=1+\alpha \cdot 0=1$ $h[1]=\delta[1]+\alpha h[0]=0+\alpha\cdot 1=\alpha$ $h[2]=\delta[2]+\alpha h[1]=0+\alpha\cdot \alpha=\alpha^2$ Having discerned a pattern we have that $h[n]=\alpha^n u[n]$ (this may be more rigorously proven by induction, if desired):
The impulse response of finite-length lti systems
For a finite-length LTI system, just as with infinite length systems, each rows/columns of its system matrices is a shifted version of another row/column. Thus, the system matrices for finite length systems are also of a Toeplitz structure. However, for finite-length systems not only are the rows/columns shifted, they are circularly shifted; the values in the rows/columns "wrap around" when shifted across the boundary of the matrix.As a result, the entire system matrix can be expressed in terms of the 0th column of the matrix, which (as with infinite length systems) is given the mathematical expression $h[n]$ and is named the impulse response. For a length $N$ LTI system, the formula to determine the value of the matrix at a particular row $n$ and column $m$ is $[{\bf H}]_{n,m} ~=~ h[(n-m)_N]$. Note how this is nearly identical to the infinite length formula, except for the modulo-$N$ operator. As with infinite-length systems, being able to express the entire system matrix in terms of the impulse response means that the system's input-output matrix multiplication relationship can be expressed as a sum involving the input and impulse response: $y[n]=Hx=\sum_{m=0}^{N-1} \: h[(n-m)_N]\, x[m]$