Linear transformations and matrices

[[Reading Status Button]]

Linear Transformation:

• Transformation - Function.
• Linear Transformation - takes a vector and transforms(moves) that vector into another vector
• Linear Transformation - 2 properites:
  • All lines remain lines, without getting curved
  • The origin is fixed
  • The resultant grid lines are parallel and evenly spaced
• If you apply a linear transformation on a set of vectors, if you know where the vectors $\hat{i}$ and $\hat{j}$ are, you can find any other vector. It will be the same linear combination $\hat{i}$ and $\hat{j}$, that was before the transformation

$\vec{v}=a\hat{i}+b\hat{j}$ will always be the same.

Matrix mulitplication is just a transformation

$$\left[\begin{array}{c}x\\ y\end{array}\right]\to x\times \hat{i}+y\times \hat{j}$$

This will always be true in a linear transformation. So when you apply linear transformation, just with the co-ordinates of $\hat{i}$ and $\hat{j}$ you can get any other vector, if you know the original co-ordinates of the vector before the transformation.

Example:

you have a vector $\left[\begin{array}{c}-1\\ 2\end{array}\right]$ , which is -1 $-1\times \hat{i}+2\times \hat{j}$ and, suppose after some transformation the $\hat{i}$ lands at $\left[\begin{array}{c}1\\ -2\end{array}\right]$ and $\hat{j}$ lands at $\left[\begin{array}{c}3\\ 0\end{array}\right]$ then the $\vec{v}=-1\times \hat{\mathbf{i}}+2\times \hat{j}$

So the original $\vec{v}$ has moved from co-ordinates (-1,2) to (5,-2) after the transformation.

$$\begin{array}{rl}\vec{v}& =-1\times \left[\begin{array}{c}1\\ -2\end{array}\right]+2\times \left[\begin{array}{c}3\\ 0\end{array}\right]\\ & =\left[\begin{array}{c}-1\times 1\\ -1\times -2\end{array}\right]+\left[\begin{array}{c}2\times 3\\ 2\times 0\end{array}\right]\\ & =\left[\begin{array}{c}-1\\ 2\end{array}\right]+\left[\begin{array}{c}6\\ 0\end{array}\right]\\ & =\left[\begin{array}{c}-1+6\\ 2+0\end{array}\right]\\ & =\left[\begin{array}{c}5\\ 2\end{array}\right]\end{array}$$

A 2 dimensional linear transformation always relies on just 4 numbers, the the 2 co-ordinates of $\hat{i}$ and $\hat{j}$.

These 4 numbers are packed into a 2 x 2 matrix.

The columns denote where the vectors $\hat{i}$ and $\hat{j}$ land.

If the lines where $\hat{i}$ and $\hat{j}$ are linearly dependent, then entire 2D space into a single line, where the 2 vectors sit.

matrices is way to describe transformation, and matrix multiplication is the way to find out what the transformation does to a given vector.

How does matrix multiplication relate to Neural Networks:

• so when performing matrix multiplication on the token’s embeddings, we ensure that the reflected context is transformed onto the other tokens.
• And when a 2 linearly dependent vectors are multiplied, the resulting vector has only one dimension.
• say for example take 2 tokens, first name, and second name, those 2 are nouns, so will occupy the same dimension. and might have linearly dependent vectors. So when you multiply them the result will a noun.