ODE-Project Linear Algebra in a Nutshell

Section 3.1 Linear Algebra in a Nutshell

Objectives

To understand and be able to apply definitions such as linear dependence and independence, basis, and linear combinations to vectors in $R^{2} .$
To understand and be able to do operations such as matrix arithmetic, computing determinants, finding eigenvalues and eigenvectors, and finding matrix inverses for $2 \times 2$ matrices.

Linear algebra and matrices provide a convenient notation for representing the

2 \times 2

system

🔗

\begin{aligned} \frac{d x}{d t} & = a x + b y, \\ \frac{d y}{d t} & = c x + d y . \end{aligned}

If we let

🔗

A = (\begin{matrix} a & b \\ c & d \end{matrix}) and x (t) = (\begin{matrix} x (t) \\ y (t) \end{matrix}),

then we can rewrite our system as

🔗

(\begin{matrix} x^{'} (t) \\ y^{'} (t) \end{matrix}) = (\begin{matrix} a x (t) + b y (t) \\ c x (t) + d y (t) \end{matrix}) = (\begin{matrix} a & b \\ c & d \end{matrix}) (\begin{matrix} x (t) \\ y (t) \end{matrix}) .

In other words, we can write our system as

🔗

\frac{d x}{d t} = A x,

where

🔗

x^{'} = \frac{d x}{d t} = (\begin{matrix} x^{'} (t) \\ y^{'} (t) \end{matrix}) .

🔗

Subsection 3.1.1 Matrices and Systems of Linear Equations

A short review of linear algebra and

2 \times 2

matrices is useful at this point. Recall that any system of two equations in two variables,

🔗

\begin{aligned} a x + b y & = α, \\ c x + d y & = β, \end{aligned}

can be written as a matrix equation

🔗

\begin{matrix} (3.1.1) & (\begin{matrix} a & b \\ c & d \end{matrix}) (\begin{matrix} x \\ y \end{matrix}) = (\begin{matrix} a x + b y \\ c x + d y \end{matrix}) = (\begin{matrix} α \\ β \end{matrix}) . \end{matrix}

We will denote the

2 \times 2

coefficient matrix by

A .

That is,

🔗

A = (\begin{matrix} a & b \\ c & d \end{matrix}) .

🔗

If a solution for the system (3.1.1) exists, it is easy to find. A unique solution will occur exactly when the matrix

A

is invertible (or nonsingular). The unique solution is given by

🔗

(\begin{matrix} x \\ y \end{matrix}) = {(\begin{matrix} a & b \\ c & d \end{matrix})}^{- 1} (\begin{matrix} α \\ β \end{matrix}),

where

🔗

A^{- 1} = \frac{1}{a d - b c} (\begin{matrix} d & - b \\ - c & a \end{matrix}) .

The matrix

A

is invertible if and only if its determinant is nonzero,

🔗

det (A) = a d - b c \neq 0.

det (A) = 0,

then we either have no solution or infinitely many solutions.

🔗

Let us consider the special case

🔗

A (\begin{matrix} x \\ y \end{matrix}) = (\begin{matrix} 0 \\ 0 \end{matrix}) .

det (A) \neq 0,

we have exactly one solution,

x = 0

and

y = 0 .

On the other hand, if

det (A) = 0,

we have infinitely many solutions. Suppose that

a \neq 0 .

Then

x = - (b / a) y,

and

🔗

- c (\frac{b}{a}) y + d y = 0.

Therefore,

(a d - b c) y = 0 .

Since

det (A) = a d - b c = 0,

the variable

y

can assume any value and

x = - (b / a) y .

Thus, the solutions to our system lie along a line through the origin. In fact, we will always get a line of solutions through the origin as long as at least one entry in our matrix is nonzero.

We will not worry about the

2 \times 2

zero matrix, since it will not play a role in our study of linear equations.

🔗

Subsection 3.1.2 Linear Independence

We say that two vectors

x

and

y

R^{2}

are linearly independent if they do not lie on the same line through the origin. If, on the other hand, they do lie on the same line, then the vectors are linearly dependent. Equivalently, two vectors are linearly dependent if one vector is a multiple of the other. We leave the proof of the following theorem as an exercise.

🔗

Theorem 3.1.1.

Let

x = (x_{1}, x_{2})

and

y = (y_{1}, y_{2}) .

Then

x

and

y

are linearly independent if and only if

🔗

det (\begin{matrix} x_{1} & y_{1} \\ x_{2} & y_{2} \end{matrix}) \neq 0.

🔗

If we have a pair of linearly independent vectors in

R^{2},

then we can write any vector in

R^{2}

as a unique linear combination of the two vectors. That is, given two linearly independent vectors

x = (x_{1}, x_{2})

and

y = (y_{1}, y_{2}),

we can write

z = (z_{1}, z_{2})

🔗

(\begin{matrix} z_{1} \\ z_{2} \end{matrix}) = α (\begin{matrix} x_{1} \\ x_{2} \end{matrix}) + β (\begin{matrix} y_{1} \\ y_{2} \end{matrix}),

where

α

and

β

are unique. To see why this is true, we must solve the equations

🔗

\begin{aligned} z_{1} & = α x_{1} + β y_{1} \\ z_{2} & = α x_{2} + β y_{2} \end{aligned}

for

α

and

β .

However, this system has a unique solution since

🔗

det (\begin{matrix} x_{1} & y_{1} \\ x_{2} & y_{2} \end{matrix}) \neq 0.

Two vectors are said to be a basis for

R^{2}

if we can write any vector in

R^{2}

as a linear combination of these two vectors. By our arguments above, any two linearly independent vectors will form a basis for

R^{2} .

🔗

Example 3.1.2.

The vectors

e_{1} = (1, 0)

and

e_{2} = (0, 1)

form a basis for

R^{2} .

Indeed, if

z = (z_{1}, z_{2}),

then we can write

🔗

z = z_{1} e_{1} + z_{2} e_{2} .

The vectors

e_{1}

and

e_{2}

are called the standard basis for

R^{2} .

🔗

Example 3.1.3.

Let

v_{1} = (2, 1)

and

v_{2} = (3, 2) .

Since

🔗

det (\begin{matrix} 2 & 3 \\ 1 & 2 \end{matrix}) \neq 0,

these vectors form a basis for

R^{2} .

z = (- 5, - 4),

then we can write

🔗

z = 2 v_{1} - 3 v_{2} .

We say that the coordinates of

z

are

(2, - 3)

with respect to the basis

{v_{1}, v_{2}} .

🔗

Example 3.1.4.

The vectors

(1, 1)

and

(- 1, - 1)

do not form a basis for

R^{2}

since these two vectors lie along the same line.

🔗

2 \times 2

matrices and the rest of what we have described above make you nervous, you should work through the exercises at the end of this section.

🔗

Activity 3.1.1. Matrix Operations.

Given the matrices and vectors

🔗

A = (\begin{matrix} 5 & 3 \\ - 6 & 4 \end{matrix}), B = (\begin{matrix} 2 & 3 \\ 1 & 2 \end{matrix}), x = (\begin{matrix} 1 \\ 0 \end{matrix}) y = (\begin{matrix} 2 \\ - 1 \end{matrix})

compute each of the following expressions.

🔗

(a)

A B,

B A

🔗

(b)

A^{- 1},

B^{- 1},

(A B)^{- 1},

B^{- 1} A^{- 1}

🔗

(c)

det (A), det (B), det (A B), det (A^{- 1})

🔗

(d)

A x,

A y,

y^{T} x,

x y^{T},

where

y^{T} = (2, - 1)

🔗

Subsection 3.1.3 Finding Eigenvalues and Eigenvectors

A nonzero vector

v

is an eigenvector of

A

A v = λ v

for some

λ \in R .

The constant

λ

is called an eigenvalue of

A .

Letting

🔗

A = (\begin{matrix} a & b \\ c & d \end{matrix}) and v = (\begin{matrix} x \\ y \end{matrix}) \neq 0,

we have

A x = λ v

A v - λ v = 0 .

In matrix form this is

🔗

\begin{aligned} (\begin{array}{c} a & b \\ c & d \end{array}) (\begin{array}{c} x \\ y \end{array}) - λ (\begin{array}{c} x \\ y \end{array}) & = (\begin{array}{c} a & b \\ c & d \end{array}) (\begin{array}{c} x \\ y \end{array}) - (\begin{array}{c} λ & 0 \\ 0 & λ \end{array}) (\begin{array}{c} x \\ y \end{array}) \\ = (\begin{array}{c} a - λ & b \\ c & d - λ \end{array}) (\begin{array}{c} x \\ y \end{array}) \\ = (\begin{array}{c} 0 \\ 0 \end{array}) . \end{aligned}

This matrix equation is certainly true if

(x, y) = (0, 0) .

However, we seek nonzero solutions to this system. This will occur exactly when the determinant of

🔗

A - λ I = (\begin{matrix} a - λ & b \\ c & d - λ \end{matrix})

is zero. In this case

🔗

det (A - λ I) = det (\begin{matrix} a - λ & b \\ c & d - λ \end{matrix}) = λ^{2} - (a + d) λ + (a d - b c) .

We say that

🔗

det (A - λ I) = λ^{2} - (a + d) λ + (a d - b c)

is the characteristic polynomial of

A .

We summarize the results of this discussion in the following theorem.

🔗

Theorem 3.1.5.

The roots of the characteristic polynomial of

A

are the eigenvalues of

A .

🔗

Example 3.1.6.

Suppose that we wish to find the eigenvalues and associated eigenvectors of

🔗

A = (\begin{matrix} 1 & 2 \\ 4 & 3 \end{matrix}) .

To find the eigenvalues and eigenvectors for

A,

we must solve the equation

🔗

A (\begin{matrix} x \\ y \end{matrix}) = λ (\begin{matrix} x \\ y \end{matrix}) .

If we let

I

denote the

2 \times 2

identity matrix,

🔗

I = (\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}),

we can rewrite this equation in the form

🔗

\begin{matrix} (3.1.2) & (A - λ I) (\begin{matrix} x \\ y \end{matrix}) = (\begin{matrix} 0 \\ 0 \end{matrix}) . \end{matrix}

We know that

A - λ I

is a

2 \times 2

matrix and that this system will only have nonzero solutions if

det (A - λ I) = 0 .

In our example,

🔗

\begin{aligned} det (A - λ I) & = det (\begin{array}{c} 1 - λ & 2 \\ 4 & 3 - λ \end{array}) \\ = (1 - λ) (3 - λ) - 8 \\ = λ^{2} - 4 λ - 5 \\ = (λ - 5) (λ + 1) . \end{aligned}

Thus,

λ = 5

- 1 .

🔗

To see this from a different perspective, we will rewrite equation (3.1.2) as

🔗

\begin{aligned} x + 2 y & = λ x \\ 4 x + 3 y & = λ y . \end{aligned}

This system is equivalent to

🔗

\begin{aligned} (1 - λ) x + 2 y & = 0 \\ 4 x + (3 - λ) y & = 0 \end{aligned}

which can be reduced to

🔗

\begin{aligned} (1 - λ) x + 2 y & = 0 \\ (λ^{2} - 4 λ - 5) y & = 0. \end{aligned}

Therefore, either

λ = 5

λ = - 1

to obtain a nonzero solution.

🔗

If $λ = 5,$ the first equation in the system becomes $- 2 x + y = 0,$ and the eigenvectors corresponding to this eigenvalue are the nonzero solutions of this equation. That is, a vector must be a nonzero multiple of $(1, 2)$ to be an eigenvector of $A$ corresponding to $λ = 5 .$
If $λ = - 1,$ then the corresponding eigenvectors are the nonzero multiples of $(1, - 1) .$

🔗

Activity 3.1.2. Finding Eigenvalues and Eigenvectors.

For each of the following matrices (1) find the characteristic polynomial, (2) find all of the eigenvalues, and (3) find an eigenvector for each eigenvalue.

🔗

(a)

A = (\begin{matrix} 1 & 3 \\ 1 & - 1 \end{matrix}) .

🔗

(b)

A = (\begin{matrix} - 8 & 2 \\ - 15 & 3 \end{matrix}) .

🔗

(c)

A = (\begin{matrix} 4 & 3 \\ - 6 & - 5 \end{matrix}) .

🔗

(d)

A = (\begin{matrix} 7 & 4 \\ - 10 & - 5 \end{matrix}) .

🔗

(e)

A = (\begin{matrix} 3 & 1 \\ - 1 & 1 \end{matrix}) .

🔗

Subsection 3.1.4 Important Lessons

A matrix $A$ is invertible (or nonsingular) if there exists a matrix $A^{- 1}$ such that $A A^{- 1} = A^{- 1} A = I,$ where $I$ is the identity matrix. In the case of $2 \times 2$ matrices,

$I = (\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}) .$
If

$A = (\begin{matrix} a & b \\ c & d \end{matrix}),$

then

$A^{- 1} = \frac{1}{a d - b c} (\begin{matrix} d & - b \\ - c & a \end{matrix}) .$
A matrix $A$ is invertible if and only if its determinant is nonzero,

$det (A) = a d - b c \neq 0.$
We say that two vectors $x$ and $y$ in $R^{2}$ are linearly independent if they do not lie on the same line through the origin. If, on the other hand, they do lie on the same line, then the vectors are linearly dependent. Equivalently, two vectors are linearly dependent if one vector is a multiple of the other.
Let $x = (x_{1}, x_{2})$ and $y = (y_{1}, y_{2}) .$ Then $x$ and $y$ are linearly independent if and only if

$det (\begin{matrix} x_{1} & y_{1} \\ x_{2} & y_{2} \end{matrix}) \neq 0.$
If we have a pair of linearly independent vectors in $R^{2},$ then we can write any vector in $R^{2}$ as a unique linear combination of the two vectors. That is, given two linearly independent vectors $x = (x_{1}, x_{2})$ and $y = (y_{1}, y_{2}),$ we can write $z = (z_{1}, z_{2})$ as

$(\begin{matrix} z_{1} \\ z_{2} \end{matrix}) = α (\begin{matrix} x_{1} \\ x_{2} \end{matrix}) + β (\begin{matrix} y_{1} \\ y_{2} \end{matrix}),$

where $α$ and $β$ are unique.
Two vectors are a basis for $R^{2}$ if we can write any vector in $R^{2}$ as a linear combination of these two vectors. Any two linearly independent vectors will form a basis for $R^{2} .$
The roots of the characteristic polynomial, $det (A - λ I),$ of a matrix $A$ are the eigenvalues of $A .$ Given a specific eigenvalue, $λ,$ for a matrix $A,$ the eigenvectors associated with $A$ are the nonzero solutions of the system of equations

$(A - λ I) (\begin{matrix} x \\ y \end{matrix}) = (\begin{matrix} 0 \\ 0 \end{matrix}) .$
If $v_{1}$ and $v_{2}$ are eigenvectors of two distinct real eigenvalues of a matrix $A,$ then $v_{1}$ and $v_{2}$ are linearly independent.

🔗

Reading Questions 3.1.5 Reading Questions

1.

Explain what it means for two vectors to be linearly independent.

🔗

2.

Explain what it means for a matrix to be nonsingular.

🔗

3.

What is an eigenvalue and an eigenvector?

🔗

Exercises 3.1.6 Exercises

1.

Given a column vector

🔗

x = (\begin{matrix} x_{1} \\ x_{2} \end{matrix}),

we define the transpose of

x

to be

🔗

x^{T} = (\begin{matrix} x_{1} & x_{2} \end{matrix}) .

🔗

A = (\begin{matrix} 3 & - 2 \\ 0 & - 1 \end{matrix}), x = (\begin{matrix} 4 \\ 1 \end{matrix}), and y = (\begin{matrix} - 2 \\ 3 \end{matrix}),

find each of the following.

🔗

$A x$
$A y$
$x^{T} y$
$y^{T} x$

🔗

2.

🔗

A = (\begin{matrix} 1 & - 2 \\ 3 & 1 \end{matrix}) and B = (\begin{matrix} 4 & 1 \\ - 1 & - 2 \end{matrix}),

find each of the following.

🔗

$A + B$
$2 A - 3 B$
$A B$
$B A$
$A^{- 1}$
$B^{- 1}$

🔗

3.

🔗

A = (\begin{matrix} 2 & 1 - i \\ 2 - i & 2 \end{matrix}) and B = (\begin{matrix} 4 i & 1 - i \\ 1 + 3 i & - 2 - i \end{matrix}),

find each of the following.

🔗

$A + B$
$3 A - 2 B$
$A B$
$B A$

🔗

Finding Determinants.

Find the determinant of each of the matrices

A

in Exercise Group 3.1.6.4–13.

🔗

4.

A = (\begin{matrix} 3 & 4 \\ 2 & 1 \end{matrix})

🔗

5.

A = (\begin{matrix} 6 & 3 \\ - 4 & - 1 \end{matrix})

🔗

6.

A = (\begin{matrix} 3 & 1 \\ - 1 & 1 \end{matrix})

🔗

7.

A = (\begin{matrix} - 1 & 6 \\ - 2 & 6 \end{matrix})

🔗

8.

A = (\begin{matrix} 3 & 1 \\ - 2 & 0 \end{matrix})

🔗

9.

A = (\begin{matrix} 1 & - 2 \\ 1 & 3 \end{matrix})

🔗

10.

A = (\begin{matrix} 2 & 0 \\ 0 & - 3 \end{matrix})

🔗

11.

A = (\begin{matrix} 1 & 2 \\ 0 & 3 \end{matrix})

🔗

12.

A = (\begin{matrix} 2 & 1 \\ 1 & 3 \end{matrix})

🔗

13.

A = (\begin{matrix} 1 & - 2 \\ - 3 & 2 \end{matrix})

🔗

Finding Inverses.

Find the inverse (if it exists) of each of the matrices

A

in Exercise Group 3.1.6.14–21. that is, find the matrix

A^{- 1}

such that

A A^{- 1} = A^{- 1} A = I,

where

🔗

I = (\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}) .

🔗

14.

A = (\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix})

🔗

15.

A = (\begin{matrix} 6 & 3 \\ - 4 & - 1 \end{matrix})

🔗

16.

A = (\begin{matrix} 3 & - 7 \\ - 2 & 5 \end{matrix})

🔗

17.

A = (\begin{matrix} 8 & 7 \\ 2 & 2 \end{matrix})

🔗

18.

A = (\begin{matrix} - 3 & 4 \\ - 2 & 5 \end{matrix})

🔗

19.

A = (\begin{matrix} 3 & 2 \\ 4 & 3 \end{matrix})

🔗

20.

A = (\begin{matrix} 4 & 0 \\ 0 & - 3 \end{matrix})

🔗

21.

A = (\begin{matrix} 1 & 2 \\ - 3 & - 6 \end{matrix})

🔗

Finding Eigenvalues and Eigenvectors.

For each of the matrices

A

in Exercise Group 3.1.6.22–31:

🔗

Find the characteristic polynomial of $A .$
Find all of the eigenvalues of $A .$
Find an eigenvector for each eigenvalue of $A .$

🔗

22.

A = (\begin{matrix} 3 & 4 \\ 2 & 1 \end{matrix})

🔗

23.

A = (\begin{matrix} 6 & 3 \\ - 4 & - 1 \end{matrix})

🔗

24.

A = (\begin{matrix} 3 & 1 \\ - 1 & 1 \end{matrix})

🔗

25.

A = (\begin{matrix} - 1 & 6 \\ - 2 & 6 \end{matrix})

🔗

26.

A = (\begin{matrix} 3 & 1 \\ - 2 & 0 \end{matrix})

🔗

27.

A = (\begin{matrix} 1 & - 2 \\ 1 & 3 \end{matrix})

🔗

28.

A = (\begin{matrix} 2 & 0 \\ 0 & - 3 \end{matrix})

🔗

29.

A = (\begin{matrix} 1 & 2 \\ 0 & 3 \end{matrix})

🔗

30.

A = (\begin{matrix} 2 & 1 \\ 1 & 3 \end{matrix})

🔗

31.

A = (\begin{matrix} 1 & - 2 \\ - 3 & 2 \end{matrix})

🔗

32.

For what values of

a

are the vectors

(2, a)

and

(4, - 1)

linearly independent?

🔗

33.

We define the trace of a

2 \times 2

matrix to be the sum of its diagonal entries. That is, the trace of

🔗

A = (\begin{matrix} a & b \\ c & d \end{matrix})

tr (A) = a + d .

Show that

tr (A B) = tr (B A)

for any

2 \times 2

matrices

A

and

B .

🔗

34.

Let

A

and

B

be two

2 \times 2

matrices. Show that

det (A B) = det (A) det (B) .

🔗

35.

Let

A

be a

2 \times 2

matrix. Show that

det (A^{- 1]}) = 1 / .

🔗

36.

Define the

2 \times 2

identity matrix to be

🔗

I = (\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}) .

Show that

A I = I A = A

for any

2 \times 2

matrix.

🔗

37.

An upper triangular matrix

A

is a matrix of the form

🔗

A = (\begin{matrix} α & γ \\ 0 & β \end{matrix}) .

Show that

A

has eigenvalues

α

and

β .

🔗

38.

Let

x = (x_{1}, x_{2})

and

y = (y_{1}, y_{2}) .

Prove that

x

and

y

are linearly independent if and only if

🔗

det (\begin{matrix} x_{1} & y_{1} \\ x_{2} & y_{2} \end{matrix}) \neq 0.

🔗

Subsection 3.1.7 Finding Eigenvalues and Eigenvectors with Sage

Sage can be used to find eigenvalues and eigenvectors for a matrix

A

for now. Consider the matrix

🔗

A = (\begin{matrix} 1 & 3 \\ 1 & - 1 \end{matrix}) .

Using Sage, we would enter the matrix

A

as follows.

🔗

xxxxxxxxxx
 
A = matrix([[1, 3], [1, -1]])
A

We can use the following command to find the eigenvalues of

A .

🔗

xxxxxxxxxx
 
A = matrix([[1, 3], [1, -1]])
A.eigenvalues()

Sage will also allow us to find eigenvectors for each of the eigenvalues of

A .

🔗

xxxxxxxxxx
 
A = matrix([[1, 3], [1, -1]])
A.eigenvectors_right()

Thus, the matrix

A

has two eigenvalues:

λ_{1} = 2

with eigenvector

v_{1} = (1, 1 / 3)

and

λ_{2} = - 2

with eigenvector

v_{2} = (1, - 1) .

🔗

There is a third entry in the Sage output which refers to the multiplicity of the eigenvalue. In the previous example, the multiplicity is 1. In the matrix

🔗

B = (\begin{matrix} 1 & 1 \\ - 1 & 3 \end{matrix})

in the Sage cell below, we obtain a single repeated eigenvalue

λ = 2

and only one eigenvector

v = (1, 1) .

The multiplicity of this eigenvalue is 2. In our previous examples, we obtained two linearly independent eigenvalues allowing us to solve initial value problems given a general solution.

🔗

xxxxxxxxxx
 
B = matrix([[1, 1],[-1, 3]])
B.eigenvectors_right()

We may also have matrices such as

🔗

C = (\begin{matrix} 4 & 1 \\ - 1 & 4 \end{matrix})

has complex eigenvalues,

λ = 4 - i

and

μ = 4 + i .

Eigenvectors for

λ

and

μ

are

u = (1, - i)

and

v = (1, i),

respectively.

🔗

xxxxxxxxxx
 
C = matrix([[4, 1],[-1, 4]])
C.eigenvectors_right()

The complex number

4 - i

is written as 4 - 1*I in Sage.

🔗

Prev Top Next