Vectors

Consider two vectors $(x_1, x_2, \cdots , x_n)$ and $(y_1, y_2, \cdots , y_n)$ in $\mathbb{R}^n$ and $c \in \mathbb{R}$.

Properties of Vectors

Let $v, w$ and $v'$ be vectors in $\mathbb{R}^n$.

$v + w = w + v$
$(v + w) + v' = v + (w + v')$
The $0$ vector satisfies that $v + 0 = 0 + v = v$.
The vector $-v$ satisfies that $v + (-v) = (-v) + v = 0$.
$1 \cdot v = v$.
$(ab)v = a(bv)$.
$a(v + w) = av + aw$.
$(a + b)v = av + bv$.

Vector Spaces

A vector space is a set with two operations (called addition and scalar multiplication with the first and last properties mentioned above).
A vector space $V$ over $\mathbb{R}$ is a set along with two functions

\[+ : V \times V \rightarrow V \text{ and } \cdot : \mathbb{R} \times V \rightarrow V\]

It is standard to suppress the $\cdot$
Addition and scalar multiplication on restricts to the solution set. Hence it is a vector space.

Definition of a Vector Space

$v_1+ v_2$ for all $v_1, v_2 \in V$.
$(v_1 + v_2) + v_3 = v_1 + (v_2 + v_3)$ for all $v_1, v_2, v_3 \in V$.
There exists a vector $0 \in V$ such that $v + 0 = 0 + v = v$ for all $v \in V$.
For each element $v \in V$ there exists an element in $v' \in V$ such that $v + v' = v' + v = 0$.
For each ement in $v \in V$ , $1 \cdot v = v$.
For each pair of elements $a, b \in \mathbb{R}$ and $v \in V$, $(ab)v = a(bv)$.
For each element $a \in \mathbb{R}$ and each pair of ements $v_1$ and $v_2$ a($v_1 + v_2) = av_1 + av_2$.
For each pair of elements $a,b \in \mathbb{R}$ and each element $v \in V$, $(a + b)v = av + bv$.

Example

Matricies

Let $M_{m \times n}(\mathbb{R})$ be the set of all $m \times n$ matrices with entries in $\mathbb{R}$.
Addition and scalar multiplication are defined as usual:
$(A + B)_{ij} = A_{ij} + B_{ij}$
$(cA)_{ij} = cA_{ij}$

Solutions of homogeneous linear equations

Consider the set of solutions $V$ of a homogenous linear equation $Ax = 0$ where $A \in M_{m \times n}(\mathbb{R})$
Nothat that if $v, w \in V$ then , $A(v+w) = Av + Aw = 0 + 0 = 0$
If $c \in \mathbb{R}$ then , $A(c \cdot v) = c(A \cdot v) = c(0) = 0$

Non - Examples

$(x_1, x_2) + (y_1, y_2) = (x_1 + y_1, x_2 + y_2)$ is not a vector space. It is not closed under scalar multiplication.
$c(x_1, x_2) = (cx_1, 0)$ is not a vector space. It is not closed under addition. it fails under the $3^{rd}$, $4^{th}$ and $5^{th}$ properties.

Cancelation Law of Addition

If $v_1, v_2, v_3 \in V$ and $v_1 + v_3 = v_2 + v_3$ then $v_1 = v_2$. This is called the cancelation law of addition.

Linear Dependence

A set of vectors VI, v2, , Vn from a vector space $V$ linearly dependent if there exists scalars $a_1, a_2, \dots , a_n,$ such that is said to be not all zero.

\[a_1v_1 + a_2v_2 + \cdots + a_nv_n = 0\]

The 0 Vector

Let $v_1, v_2, \dots , v_n$ be a set of vectors containing the zero vector $0$.
Suppose $v_i = 0$. Then we can choose $a_i = 1$ and $a_j = 0$ for $j \neq i$.
Then the linear combination $a_1v_1 + a_2v_2 + \cdots + a_nv_n = 0$ is satisfied.
Hence, a set of vectors $v_1, v_2, \dots , v_n$ containing the $0$ vector is always linearly dependent.

Linear Independence

A set of vectors $v_1, v_2, \dots , v_n$ from a vector space $V$ is said to be linearly independent if no scalar $a_1, a_2, \dots , a_n$ can be found such that all are zero.

\[a_1v_1 + a_2v_2 + \cdots + a_nv_n = 0\]

When are two non-zero vectors linearly independent?

Let $v_1, v_2$ be two non-zero vectors in $\mathbb{R}^n$.
Suppose $v_1$ and $v_2$ are linear dependent.
Then $a_1v_1 + a_2v_2 = 0$ for some $a_1, a_2 \in \mathbb{R}$. and atleast one of $a_1, a_2$ is not zero.
Dividing by $a_1$ and putting $c = -\frac{a_2}{a_1}$ we get $v_1 - cv_2 = 0$
we get $v_1 = cv_2$.
Hence $v_1$ is a scalar multiple of $v_2$.
We can reverse the implications above and conclude that if $v_1$ and $v_2$ are multiples of each other then they are linearly dependent.
If $v_1$ and $v_2$ are linearly independent when $v_1$ and $v_2$ are not multiples of each other.

Example

Consider the two vectors $(-1,3)$ and (2,0) in $\mathbb{R}^2$.
Consider the following equation:
$a(-1,3) + b(2,0) = 0$
Hence the following linear equation is satisfied:
$a(-1) + b(2) = 0$
$a(3) + b(0) = 0$
Hence $a = 0$ and $b = 0$ is the only answer.

When are three vectors Linearly Independent

Let $v_1, v_2, v_3$ be a set of vectors in $\mathbb{R}^n$.
Then $a_1v_1 + a_2v_2 + a_3v_3 = 0$ for some $a_1, a_2, a_3 \in \mathbb{R}$ and atleast one of $a_1, a_2, a_3$ is not zero.
If $a_1 = 0$ then $v_1 = b_2v_2 + b_3v_3$, where. $b_2 = -\frac{a_2}{a_1}$ and $b_3 = -\frac{a_3}{a_1}$. Hence $v_1$ is a linear combination of $v_2$ and $v_3$.
Similarly if $a_2 \neq 0$
Since the implication is true in both directions, we conclude that if $v_1, v_2, v_3$ are linearly independent then $v_1$ is not a linear combination of $v_2$ and $v_3$.
Concusion: If $v_1, v_2, v_3$ are linearly independent then non of these vectors is a linear combination of the other two.

Example

Let us consider the vectors $v_1 = (1,1,2)$, $v_2 = (1,2,0)$ and $v_3 = (0,2,1)$ in $\mathbb{R}^3$.

\[a(1,1,2) + b(1,2,0) + c(0,2,1) = 0\]

We get the following equations:
$a + b = 0$
$a + 2b + 2c= 0$
$2a + c = 0$
We get $a = 0$, $b = 0$ and $c = 0$.
Hence $v_1, v_2, v_3$ are linearly dependent.

Linear Independence of n Vectors

Let $v_1, v_2, \dots , v_n$ be a set of vectors in $\mathbb{R}^m$.
In terms of coordinates, let $v_j = (v_{j1}, v_{j2}, \dots , v_{jm})$.
Let us write the linear combination of these vectors with arbitary coefficients $a_1, a_2, \dots , a_n$ as: $a_1v_1 + a_2v_2 + \cdots + a_nv_n = 0$
We get the following equations:
- $a_1v_{11} + a_2v_{12} + \cdots + a_nv_{1n} = 0$
- $a_1v_{21} + a_2v_{22} + \cdots + a_nv_{2n} = 0$
- $\vdots$
- $a_1v_{n1} + a_2v_{n2} + \cdots + a_nv_{nm} = 0$
For linear independence, we need to find a set of coefficients $a_1, a_2, \dots , a_n$ such that all the equations are satisfied.
We need to check if the only choice of $a_i$'s satisfying the above identities is $a_i = 0$ for all $i$.
Concusion: If $v_1, v_2, \dots , v_n \in \mathbb{R}^m$ are linearly independent, we have to check that the homogeneous system of linear eqautions $V_x=0$ has only trivial solution, where $j^{th}$ column of $V$ is $v_j$.

Example

2x2 Matrix

Consider the two vectors $(5,2) \text{ and } (1,3)$ in $\mathbb{R}^2$. Write the linear combination of these vectors with arbitary coefficients $a_1$ and $a_2$ as:

\[a_1(5,2) + a_2(1,3) = 0\]

We get the following equations:
$5x + y = 0$
$2x + 3y = 0$
Since the corresponding matrix is: $\begin{bmatrix} 5 & 1 \\ 2 & 3 \end{bmatrix}$ is invertible, the system of linear equations has only trivial solution.

3x2 Matrix

Consider the two vectors $(1,2,0) \text{ and } (3,3,5)$ in $\mathbb{R}^3$. Write the linear combination of these vectors with arbitary coefficients $a_1$ and $a_2$ as:

\[a_1(1,2,0) + a_2(3,3,5) = 0\]

We get the following equations:
$x + 3y = 0$
$2x + 3y = 0$
$5y = 0$

2x3 Matrix

Consider the three vectors $(1,2), (1,3) \text{ and } (3,4)$. Equate the linear combination of these vectors with arbitary coefficients $a_1, a_2, a_3$ as:

\[a_1(1,2) + a_2(1,3) + a_3(3,4) = 0\]

We get the following equations:
$1x + 1y + 3z = 0$
2x + 3y + 4z = 0$
We can find the values using gaussian elimination.
$\begin{bmatrix} 1 & 1 & 3 \\ 2 & 3 & 4 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}$
$\begin{bmatrix} 1 & 1 & 3 \\ 0 & 1 & -2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}$
$\begin{bmatrix} 1 & 1 & 3 \\ 0 & 1 & -2 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}$
We get infinite solutions. Hence the vectors are linearly dependent.

3x3 Matrix

Consider the three vectors $(1,2,0), (0,2,4) \text{ and } (3,0,0)$. Equate the linear combination of these vectors with arbitary coefficients $x, y, z$ as:

\[x(1,2,0) + y(0,2,4) + z(3,0,0) = 0\]

We get the following equations:
$x + 2y = 0$
$2y + 4z = 0$
$3x = 0$
Since the matrix is $\begin{bmatrix} 1 & 0 & 3 \\ 2 & 2 & 0 \\ 0 & 4 & 0 \end{bmatrix}$ is invertible, the system of linear equations has only trivial solution.

More than 2 vectors in $\mathbb{R}^2$

Suppose we have $n$ vectors in $\mathbb{R}^2$, where $n \geq 3$. To check linear independece, we have to check wheater the corresponding homogeneous linear system $V_x = 0$ has only trivial solution.
Since $n \geq 3 > 2$
We also know that gaussian elimination can yeild infinite solutions.
Hence, any set of $n$ vectors in $\mathbb{R}^2$ with $n \geq 3$ are linearly dependent.

Relation between Linear Independence and Determinant

To Check wheater a set of $n$ vectors in $\mathbb{R}^n$ are linearly independent, we can check wheater the corresponding homogeneous linear system $V_x = 0$ has only trivial solution. Where $V$ is an $n \times n$ matrix obtained by arranging the vectors as columns.
Since $V$ is a square matrix, it has unique solution $x=0$ if and only if $V$ is invertible and it has its $det(V) \neq 0$.
If $A$ is invertible then there exists $A^{-1}$ such that $A^{-1}A = I$. Hence $det(A^{-1}A) = det(I) \neq 0$.

Example

Let us consider the following matrix $(1,4,2), (0,4,3) \text{ and } (1,1,0)$ in $\mathbb{R}^3$.
Let the matrix be: $V = \begin{bmatrix} 1 & 0 & 1 \\ 4 & 4 & 1 \\ 2 & 3 & 0 \end{bmatrix}$
Since the $det(V) = 1$ and $\neq 0$, the system of linear equations has only trivial solution. Hence the vectors are linearly independent.

Vectors

Vector Spaces

Linear Dependence

The 0 Vector

Linear Independence

Linear Independence of n Vectors

More than 2 vectors in \(\mathbb{R}^2\)

Relation between Linear Independence and Determinant