113 Questions

From Mathsoc wiki

This page is far from complete! Can you help by adding anything to it (such as notes or links to useful sites)?

This page gives some general guidelines that may be useful when attempting some Linear Algebra questions.

1 Prerequisites
2 Vectors and Matrices
- 2.1 Linear Independence
- 2.2 Rank
3 Linear and Bilinear Forms
- 3.1 Linear Forms

[edit]

Prerequisites

The following knowledge will be assumed. If any term is not familiar, please look here.

Addition and Multiplication of matrices, including scalar multiplication.
Row Echelon Form and Reduced Row Echelon Form.
Gaussian and Gauss Jordan Elimination of a matrix.
How to compute the transpose, trace and inverse of a matrix.
How to compute the determinant of a matrix (by Cofactor Expansion or otherwise).
A knowledge of the definition of vectors, simple vector arithmetic and $\mathbb{R}^n$ .

[edit]

Vectors and Matrices

[edit]

Linear Independence

A set of vectors $v_{1},v_{2},\dots,v_{k},\in\mathbb{R}^{n}$ is linearly independent if there does not exist constants $c_{1},c_{2},\dots,c_{k}\in\mathbb{R}$ , that are not entirely equal to 0, such that $c_{1}v_{1}+c_{2}v_{2}+\dots+c_{k}v_{k} = 0$ .

If we are given a set of column vectors $v_{1},v_{2},\dots,v_{k},\in\mathbb{R}^{n}$ , the simplest method of showing that the set is linearly independent is:

Form a matrix in which the columns of the matrix are the column vectors of the set, i.e. the coefficient matrix of the system $c_{1}v_{1}+c_{2}v_{2}+\dots+c_{k}v_{k} = 0$ .
Bring this matrix to Reduced Row Echelon Form.
If the system has non-trivial solutions, then the set of vectors is linearly dependent.
If the system only has the trivial solution, then the set of vectors is linearly independent.

Consider the set of vectors in $\mathbb{R}^3$

$\begin{pmatrix} 1 \\ -2 \\ 3 \end{pmatrix} , \begin{pmatrix} 5 \\ 6 \\ -1 \end{pmatrix} , \begin{pmatrix} 3 \\ 2 \\ 1 \end{pmatrix}$

To test if these three vectors are linearly independent, we form the system:

$c_1 \begin{pmatrix} 1 \\ -2 \\ 3 \end{pmatrix} + c_2 \begin{pmatrix} 5 \\ 6 \\ -1 \end{pmatrix} + c_3 \begin{pmatrix} 3 \\ 2 \\ 1 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}$
$\begin{pmatrix} 1 & 5 & 3 \\ -2 & 6 & 2 \\ 3 & -1 & 1 \end{pmatrix} \begin{pmatrix} c_1 \\ c_2 \\ c_3 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}$
$\begin{pmatrix} 1 & 0 & 1/2 \\ 0 & 1 & 1/2 \\ 0 & 0 & 0 \end{pmatrix} \begin{pmatrix} c_1 \\ c_2 \\ c_3 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \end{pmatrix}$

This system has the solution $\left \lbrace -\frac{1}{2}t, -\frac{1}{2}t, t \right \rbrace$ for any $t\in\mathbb{R}$ , so the original set of vectors is linearly dependent, as there exist non-trivial solutions.

Consider the set of vectors in $\mathbb{R}^4$

$\begin{pmatrix} 1 \\ 1 \\ 1 \\ 1 \end{pmatrix} , \begin{pmatrix} 1 \\ 3 \\ 4 \\ 5 \end{pmatrix} , \begin{pmatrix} 1 \\ 9 \\ 16 \\ 25 \end{pmatrix} , \begin{pmatrix} 1 \\ 27 \\ 64 \\ 125 \end{pmatrix}$

To test if these four vectors are linearly independent, we form the system:

$c_1 \begin{pmatrix} 1 \\ 1 \\ 1 \\ 1 \end{pmatrix} + c_2 \begin{pmatrix} 1 \\ 3 \\ 4 \\ 5 \end{pmatrix} + c_3 \begin{pmatrix} 1 \\ 9 \\ 16 \\ 25 \end{pmatrix} +c_4 \begin{pmatrix} 1 \\ 27 \\ 64 \\ 125 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \end{pmatrix}$
$\begin{pmatrix} 1 & 1 & 1 & 1 \\ 1 & 3 & 9 & 27 \\ 1 & 4 & 16 & 64 \\ 1 & 5 & 25 & 125 \end{pmatrix} \begin{pmatrix} c_1 \\ c_2 \\ c_3 \\ c_4 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \end{pmatrix}$
$\begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} c_1 \\ c_2 \\ c_3 \\ c_4 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 0 \end{pmatrix}$

This system only has the trivial solution, so the original set of vectors is linearly independent.

In practice, it suffices to take the coefficient matrix and to show that the set reduces to the identity matrix under Gauss Jordan Elimination to show that the set of vectors is linearly independent.

By theorem, a homogeneous system of linear equations that has more variables than equations has infinitely many solutions. Thus, given a set of $m$ vectors in $\mathbb{R}^n$ , if $m \neq n$ , then this set is linearly dependent.

[edit]

Rank

The rank of a set of vectors in $\mathbb{R}^n$ is defined as the maximal number of linearly independent subsets of the set of vectors.

If we are given a set of column vectors $v_{1},v_{2},\dots,v_{k},\in\mathbb{R}^{n}$ , the way to determine the rank is:

Similar to the test for linear independence, form a matrix in which the columns of the matrix are the column vectors of the set.
Bring this set to Reduced row Echelon Form by Gauss Jordan Elimination.
The number of linearly independent vectors will be the number of leading 1's in the matrix, because if the vectors that correspond to the leading 1's were linearly dependent, then they would have been reduced to 0 during the elimination.

Consider the above example in \mathbb{R}^3 :

$\begin{pmatrix} 1 \\ -2 \\ 3 \end{pmatrix} , \begin{pmatrix} 5 \\ 6 \\ -1 \end{pmatrix} , \begin{pmatrix} 3 \\ 2 \\ 1 \end{pmatrix}$

These vectors form a matrix that reduces (as above) to:

$\begin{pmatrix} 1 & 0 & 1/2 \\ 0 & 1 & 1/2 \\ 0 & 0 & 0 \end{pmatrix}$

Thus, the rank of the above set is 2.

Consider the above example in $\mathbb{R}^4$ :

These vectors form a matrix that reduces (as above) to:

$\begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{pmatrix}$

Thus, the rank of the above set of vectors is 4.

[edit]

Linear and Bilinear Forms

[edit]

Linear Forms

If we have an operator $A : V\rightarrow W$ , where $V$ and $W$ are spaces of some kind ( $\mathbb{R}^n$ , real vector spaces, complex vector spaces, the spaces of polynomials, etc), then we say that $A$ is a linear operator if and only if:

$A(v_1+v_2) = A(v_1)+A(v_2)$ for all $v_1,v_2\in V$ ,
$A(c\cdot v) = c\cdot A(v)$ , for all $c\in\mathbb{R}$ and $v\in V$

We will work in Euclidean n-space, but this also applies to other vector spaces. Let $A : \mathbb{R}^m\rightarrow\mathbb{R}^n$ be a linear operator. We can use matrices in a given basis to represent $A$ instead of possibly long calculations. Let $\left\lbrace e_1, e_2, \dots, e_n\right\rbrace$ be the standard basis of $\mathbb{R}^n$ . Thus, the matrix of $A$ is given by $A = (A(e_1) | A(e_2) | \cdots | A(e_n))$ , where $A(e_i)$ are column vectors. When written in the standard basis, this matrix is called the standard form of $A$ , but any basis can be used.

Consider the operator $T : \mathbb{R}^3\rightarrow\mathbb{R}^3$ given by $T(x, y, z) = (x-y, y-x, x-z)$ . It is easily checked that this operator is linear. We will write the matrix $T$ relative to the bases

$e = \left\lbrace \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix} , \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix} , \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} \right\rbrace$

and

$f = \left\lbrace \begin{pmatrix} 1 \\ 0 \\ 1 \end{pmatrix} , \begin{pmatrix} 0 \\ 1 \\ 1 \end{pmatrix} , \begin{pmatrix} 1 \\ 1 \\ 0 \end{pmatrix} \right\rbrace$

Thus, we get $T(e_1) = (1, -1, 1)$ , $T(e_2) = (-1, 1, 0)$ and $T(e_3) = (0, 0, -1)$ . Thus, T in its standard form is given by $\begin{pmatrix} 1 & -1 & 0 \\ -1 & 1 & 0 \\ 1 & 0 & -1 \end{pmatrix}$

Now, we consider the basis f. $T(f_1) = (1, -1, 0)$ , $T(f_2) = (-1, 1, -1)$ and $T(f_3) = (0, 0, 1)$ . Thus, T relative to the basis f is given by $\begin{pmatrix} 1 & -1 & 0 \\ -1 & 1 & 0 \\ 0 & -1 & 1 \end{pmatrix}$

Sometimes, we are given the matrix of a linear operator relative to one basis, and are then asked to compute the matrix of the linear operator relative to another basis. Let $\mathcal{A} : \mathbb{R}^m\rightarrow\mathbb{R}^n$ be a linear operator, and let $A$ denote the matrix of $\mathcal{A}$ in the basis $e = {e_1, e_2, \dots, e_n}$ . If we wish to represent $\mathcal{A}$ in the basis $f = {f_1, f_2, \dots, f_n}$ , we consider the system of equations:

$A(f_1)$	=	$b_{11}e_1+b_{21}e_2+\dots+b_{n1}e_n$
$A(f_2)$	=	$b_{12}e_1+b_{22}e_2+\dots+b_{n2}e_n$
	$\vdots$
$A(f_n)$	=	$b_{1n}e_1+b_{2n}e_2+\dots+b_{nn}e_n$

Thus, the matrix of $\mathcal{A}$ relative to the basis $f$ is given by $\begin{pmatrix} b_{11} & b_{12} & \dots & b_{1n} \\ b_{21} & b_{22} & \dots & b_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ b_{n1} & b_{n2} & \dots & b_{nn} \end{pmatrix}$

The matrix of a linear operator $\mathcal{A} : \mathbb{R}^3\rightarrow\mathbb{R}^3$ is given (in the standard basis) by

$A = \begin{pmatrix} 1 & 1 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{pmatrix}$ . We'll compute it in the bases $f = {(1, 0, 0),(1, 1, 0),(0, 2, 1)}$ and $g = {(1, -1, 1), (0, 1, 0), (1, 1, 0)}$ . Firstly, we consider f : Af_1 = ***