Dot products and duality

[[Reading Status Button]]

Dot Product

• Dot Product means pairing up all of the co-ordinates, multiplying those pair and adding them together
• Example: $\left[\begin{array}{c}1\\ 2\end{array}\right].\left[\begin{array}{c}3\\ 4\end{array}\right]=1.3+2.4$
• A dot product is a score used to find if 2 vectors point in the same direction or not.
• The dot product of 2 vectors before and after the transformation will be different, ref

Important

If 2 vector point in the same direction they have a positive dot product, if perpendicular $\perp$ then 0, if in opposite direction then negative

A dot product is a measure of how much one vector contributes to the direction of the other.

A dot product provides the same result as matrix multiplication. performing a transformation on a vector, when the said transformation converts the space to a single dimension.

• Matrix multiplication: $\left[\begin{array}{c}12\end{array}\right]\left[\begin{array}{c}3\\ 4\end{array}\right]=1.3+2.4$

What is dot product?

so a dot product is a measure of how the vectors would affect each other if they were projected onto a single dimension. will one enhance the other or hinder it or will not affect at all.

Dot product with unit vectors

A dot product of any vector with a unit vector gives the co-ordinate of the vector in the corresponding direction, because in a unit vector, only the co-ordinate of that direction is 1, others are 0.

Dual vector:

The vector formed by the co-ordinates of each vector after they are transformed into a number line.

How dot product relate to neural network:

This gives us a score to decide whether we should combine the attention or context between 2 vectors.

Question

A matrix multiplication is a transformation performed on a vector, but a dot product is a number. How are they same?

Because in a number line, there is only one axis, the direction is determined by whether the value is positive or negative.