Eigendecomposition

The crux of the idea is that when a matrix is multiplied to a vector, the matrix transforms the vector into a new space with different basis vectors.¹

$\mathbf{u}=A\mathbf{v}$

So, are there vectors that only get scaled and not transformed completely when multiplied with matrix A?

Eigenvectors are vectors $\mathbf{v}$ (with a particular basis) that only get scaled by factors, eigenvalues, when transformed by A.² So for eigenvectors the transformation is essentially just a scale multiplication.

${\mathbf{u}}_{\mathbf{i}}={\lambda }_i{\mathbf{v}}_{\mathbf{i}}$

Eigendecomposition ‘decomposes’ the transformation matrix to eigenvectors and eigenvalues. The above equation can be written as:

$A\mathbf{v}=\lambda \mathbf{v}$

This can be generalized for all eigenvectors, represented by matrix Q (each vector is a column vectors):

$AQ=Q\Lambda$
where, $\Lambda$ is a diagonal matrix with eigenvalues as the diagonal elements.

If A has n linearly independent eigenvectors, the eigendecomposition can be written as:

$A=Q\Lambda Q^{-1}$

If the matrix $Q$ is linearly independent, then Q forms a basis for the space. If A is a normal matrix, then Q forms the orthonormal basis for the space. In both these cases, one than think of A matrix multiplication as changing of basis, a scaling and reverting back to the original basis.

$A\mathbf{v}=Q\Lambda Q^{-1}\mathbf{v}$

Footnotes

1. https://www.3blue1brown.com/lessons/linear-transformations ↩

2. https://www.3blue1brown.com/lessons/eigenvalues ↩