Shub Tiwari

MLU Math

MLU Math

https://explained.ai/regularization/index.html

Understanding vector length and angle:

Vector length is defined by standard deviation

  • Standard deviation is defined by a standardized difference from the mean (add up squared distances of each element from mean > average out the squared sum > take an underfoot to get the side of the averaged square for comparison). Sd is used to measure the spread of data (mean can be deceiving as its forcing the data to be a single point). High SD = very spread out data
  • Vectors are defined by taking in all the observations about a feature, as the count of observations increase the dimensionality of the vector increases
  • Imagine a group of 3 people with 3 height values from a village, all the heights are very close to each other and the SD of this dataset is low (since all values are close to the mean). When all these 3 points are plotted on a 3 dimensional plane (imagine a room) and imagine heights to be 5’6, 5’5, 5’4 and we can join the origin to the final point obtained after traversing the x,y,z, planes.
    • mean = 5.5, SD = 0.082, L2 norm = 9.53
  • Imagine a random sample of 3 people from London where we get heights 3’5, 5’5, 7’5, here there is greater variability in the data
    • mean = 5.5, SD = 1.633, L2 norm = 9.94

To further imagine it, we can imagine higher dispersal as more lengths to travel in different dimensions hence the vector is elongated and hence higher SD = higher length, or two vectors which have similar dispersion in their elements have similar lengths

Angle between vectors = similarity between vectors

Perfect similarity = 0 angle (or 180 angle for negative but perfect similarity)

No similarity = 90 degree angle (also called orthogonal)

To imagine this simply - consider a 2D place.

Plot two points (1,0) and (0,1). The first point has no magnitude in the y direction, its simply a measure on the x axis and vice versa for the second point. These vectors are orthogonal or they have no component in common, their projection upon each other is 0 Imagine another vector (1,1) - this has some component on the x axis or the vector (1,0). And hence these vectors are said to be similar, the degree is defined by cos (theta)

This experiment can also be done in higher dimension where vectors can be drawn and can be orthogonal of none of the components have anything in common).

Practical applications - if we are applying force in the vertical direction (pressing an object), we can never move it forward (no movement in horizontal direction) , we can change the angle by a degree and now we are still pressing it an angle of 89 degrees but also with a 1 degree tilt. This 1 degree can be projected on to the horizontal axis to find out the force in the horizontal direction, there is now some similarity between the vectors. The degree of similarity can be found by calculating cos theta. Perfect similarity is when we are pressing the box on the side to slide it.

When comparing two vectors

Vector length = related to the spread in the elements defining the vectors (standard deviation)

Vector similarity = related to angle between them - (cos theta)

Vector dot product is a calculation of a quantity, like the strength of multiplication of two vectors, it does not have a direction, its a scalar quantity

Day 2 (07/14/2021)