Vectors & Operations

A vector in $\mathbb{R}^n$ is an ordered $n$-tuple of real numbers: $\mathbf{v} = \langle v_1, v_2, \ldots, v_n \rangle$. In $\mathbb{R}^2$, a vector is an arrow in the plane; in $\mathbb{R}^3$, an arrow in space. In $\mathbb{R}^n$ for $n > 3$, there is no geometric picture, but the algebra is identical.

What defines a vector is its displacement, not its starting point. Two arrows with the same length and direction, drawn anywhere on the page, are the same vector. This is why vectors model velocities, forces, and displacements — all of these are directional quantities independent of position.

Two vectors are equal if and only if every corresponding component is equal: $\langle u_1, u_2 \rangle = \langle v_1, v_2 \rangle$ iff $u_1 = v_1$ and $u_2 = v_2$. A vector in $\mathbb{R}^2$ and a vector in $\mathbb{R}^3$ can never be equal — they live in different spaces.

The zero vector $\mathbf{0} = \langle 0, 0, \ldots, 0 \rangle$ has zero magnitude and no defined direction. It is the additive identity: $\mathbf{v} + \mathbf{0} = \mathbf{v}$ for any $\mathbf{v}$.

The magnitude (length, norm) is $\|\mathbf{v}\| = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}$, a direct generalisation of the Pythagorean theorem. For $\mathbf{v} = \langle 3, 4 \rangle$: $\|\mathbf{v}\| = \sqrt{9+16} = 5$. Magnitude is always nonneg.

Addition is component-wise: $\mathbf{u} + \mathbf{v} = \langle u_1+v_1, u_2+v_2, \ldots, u_n+v_n \rangle$. Geometrically, place the tail of $\mathbf{v}$ at the head of $\mathbf{u}$; the sum is the arrow from the tail of $\mathbf{u}$ to the head of $\mathbf{v}$. This is called the parallelogram law.

Scalar multiplication scales every component: $c\mathbf{v} = \langle cv_1, cv_2, \ldots, cv_n \rangle$. If $c > 0$, direction is preserved and magnitude is multiplied by $c$. If $c < 0$, direction reverses. If $c = 0$, the result is $\mathbf{0}$.

The magnitude satisfies $\|c\mathbf{v}\| = |c|\,\|\mathbf{v}\|$ — scaling a vector scales its length by the absolute value of the scalar.

A unit vector has magnitude exactly $1$. To normalise any nonzero vector: $\hat{\mathbf{v}} = \mathbf{v}/\|\mathbf{v}\|$. The result $\hat{\mathbf{v}}$ points in the same direction as $\mathbf{v}$ but has length $1$.

The standard unit vectors in $\mathbb{R}^3$ are $\mathbf{e}_1 = \langle 1,0,0 \rangle$, $\mathbf{e}_2 = \langle 0,1,0 \rangle$, $\mathbf{e}_3 = \langle 0,0,1 \rangle$ (also written $\mathbf{i}, \mathbf{j}, \mathbf{k}$). Every vector in $\mathbb{R}^3$ is a linear combination of these: $\langle a,b,c \rangle = a\mathbf{e}_1 + b\mathbf{e}_2 + c\mathbf{e}_3$.

These two operations — addition and scalar multiplication — satisfy eight axioms (commutativity, associativity, distributivity, and so on) that define what it means to be a vector space. Any collection of objects obeying these axioms, whether arrows, polynomials, or signals, is a vector space and the same theory applies.

The dot product (inner product) of $\mathbf{u}, \mathbf{v} \in \mathbb{R}^n$ is $\mathbf{u} \cdot \mathbf{v} = u_1v_1 + u_2v_2 + \cdots + u_nv_n$. It takes two vectors and returns a single scalar — not a vector.

The geometric interpretation is fundamental: $\mathbf{u}\cdot\mathbf{v} = \|\mathbf{u}\|\,\|\mathbf{v}\|\cos\theta$, where $\theta \in [0,\pi]$ is the angle between the vectors. This ties algebraic computation directly to geometry.

Three key cases from the geometric formula: $\mathbf{u}\cdot\mathbf{v} > 0$ means the angle is acute (vectors point broadly the same way); $\mathbf{u}\cdot\mathbf{v} < 0$ means obtuse (they point away from each other); $\mathbf{u}\cdot\mathbf{v} = 0$ means $\theta = 90°$ — the vectors are orthogonal (perpendicular).

Self-dot product gives magnitude squared: $\mathbf{v}\cdot\mathbf{v} = \|\mathbf{v}\|^2$. So $\|\mathbf{v}\| = \sqrt{\mathbf{v}\cdot\mathbf{v}}$.

The Cauchy-Schwarz inequality states $|\mathbf{u}\cdot\mathbf{v}| \leq \|\mathbf{u}\|\,\|\mathbf{v}\|$, with equality iff the vectors are parallel. Rearranging gives the formula for $\cos\theta$; since $|\cos\theta| \leq 1$, the inequality is automatically satisfied.

Algebraic properties: $\mathbf{u}\cdot\mathbf{v} = \mathbf{v}\cdot\mathbf{u}$ (commutative), $\mathbf{u}\cdot(\mathbf{v}+\mathbf{w}) = \mathbf{u}\cdot\mathbf{v}+\mathbf{u}\cdot\mathbf{w}$ (distributive), $(c\mathbf{u})\cdot\mathbf{v} = c(\mathbf{u}\cdot\mathbf{v})$ (scalar associativity). These hold in any dimension.

The projection of $\mathbf{u}$ onto $\mathbf{v}$ is the component of $\mathbf{u}$ that lies exactly in the direction of $\mathbf{v}$ — the 'shadow' of $\mathbf{u}$ on the line through the origin in the direction of $\mathbf{v}$.

Vector projection: $\text{proj}_{\mathbf{v}}\mathbf{u} = \dfrac{\mathbf{u}\cdot\mathbf{v}}{\mathbf{v}\cdot\mathbf{v}}\,\mathbf{v} = \dfrac{\mathbf{u}\cdot\mathbf{v}}{\|\mathbf{v}\|^2}\,\mathbf{v}$. The result is a vector parallel to $\mathbf{v}$.

Scalar projection (signed length of the shadow): $\text{comp}_{\mathbf{v}}\mathbf{u} = \mathbf{u}\cdot\hat{\mathbf{v}} = \dfrac{\mathbf{u}\cdot\mathbf{v}}{\|\mathbf{v}\|}$. Positive if they lean the same way, negative if opposite.

Orthogonal decomposition: every vector splits as $\mathbf{u} = \underbrace{\text{proj}_{\mathbf{v}}\mathbf{u}}_{\parallel\mathbf{v}} + \underbrace{(\mathbf{u}-\text{proj}_{\mathbf{v}}\mathbf{u})}_{\perp\mathbf{v}}$. The error $\mathbf{u} - \text{proj}_{\mathbf{v}}\mathbf{u}$ is perpendicular to $\mathbf{v}$, as you can verify by dotting with $\mathbf{v}$.

Projections are the building block of the Gram-Schmidt process (Chapter 7), least-squares regression, and Fourier series. The single idea of 'drop a perpendicular' reappears throughout mathematics, statistics, and signal processing.

The cross product $\mathbf{u}\times\mathbf{v}$ is defined only in $\mathbb{R}^3$ and produces a vector, not a scalar. The result is perpendicular to both $\mathbf{u}$ and $\mathbf{v}$.

Determinant formula: $\mathbf{u}\times\mathbf{v} = \det\begin{pmatrix}\mathbf{e}_1&\mathbf{e}_2&\mathbf{e}_3\\u_1&u_2&u_3\\v_1&v_2&v_3\end{pmatrix} = \langle u_2v_3-u_3v_2,\;u_3v_1-u_1v_3,\;u_1v_2-u_2v_1\rangle$.

Magnitude: $\|\mathbf{u}\times\mathbf{v}\| = \|\mathbf{u}\|\,\|\mathbf{v}\|\sin\theta$. This equals the area of the parallelogram spanned by $\mathbf{u}$ and $\mathbf{v}$. If the vectors are parallel ($\sin\theta = 0$), the cross product is $\mathbf{0}$.

Anti-commutativity: $\mathbf{u}\times\mathbf{v} = -(\mathbf{v}\times\mathbf{u})$. Reversing the order flips the sign. This is why the cross product is not commutative.

Direction follows the right-hand rule: curl the fingers of your right hand from $\mathbf{u}$ toward $\mathbf{v}$; your thumb points in the direction of $\mathbf{u}\times\mathbf{v}$.

Applications: computing the normal vector to a plane, finding the area of a triangle ($\frac{1}{2}\|\mathbf{u}\times\mathbf{v}\|$), computing torque in physics ($\boldsymbol{\tau} = \mathbf{r}\times\mathbf{F}$).