6 Linear Algebraic Equations PDF

Linear Algebraic Equations
Cheng-Liang Chen
PSE
LABORATORY
Department of Chemical Engineering
National Taiwan University
CL Chen PSE LAB NTU 1
Linear Algebraic Equations and Gauss Elimination
x + y + 2z = 2 (1)
3x y + z = 6 (2)
x + 3y + 4z = 4 (3)
(3)(1)
2y + 2z = 2 (4)
(2)+3(1)
2y + 7z = 12 (5)
(5)2(4)
5z = 10 (6) ( z = 2)
z=2 in (4)
y = 1 (7)
z=2,y=1 in (1)
x = 1 (8)
1. Equation (1) is the pivot equation
2. Multiply it by 1 and add the result to (3)
to obtain 2y +2z = 2 (equivalent to y +z = 1)
x + y + 2z = 2 (1)
3x y + z = 6 (2)
x + 3y + 4z = 4 (3)
3. Next multiply (1) by 3 and add the result to (2)
to obtain 2y + 7z = 12
Thus we have a new set of two equations in two
unknowns (y, z):
y + z = 1 (4)
2y + 7z = 12 (5)
4. Equation (4) is the new pivot equation.
5. Multiply (4) by 2 and add the result to (5)
to obtain 5z = 10 (or z = 2)
6. Substitute z = 2 into (4) to obtain y + 2 = 1 (or y = 1)
7. Then substitute y = 1 and z = 2 into (1)
to obtain x 1 + 4 = 2 (or x = 1)
Test Your Understanding
T6.1-1
Solve the following equations using Gauss elimination:
6x 3y + 4z = 41
12x + 5y 7z = 26
5x + 2y + 6z = 14
(Answer:x = 2, y = 3, z = 5.)
Singular and Ill-Conditioned Problems
3x 4y = 5
6x 10y = 2
_
_
_
unique solution, x = 7, y = 4
3x 4y = 5
6x 8y = 10
_
_
_
singular, innite # solution
3x 4y = 5
6x 8y = 3
_
_
_
singular, no solution
0 1 2 3 4 5 6 7 8 9 10
5
0
5
10
x
y
Unique solution
Line 1
Line 2
0 1 2 3 4 5 6 7 8 9 10
5
0
5
10
x
y
Singular, No unique solution
Line 1
Line 2
0 1 2 3 4 5 6 7 8 9 10
5
0
5
10
x
y
Singular, No solution
Line 1
Line 2
x = [0 : 0.25 : 10];
y_1 = (3*x-5)/4;
y_2 = (6*x-2)/10;
subplot(3,1,1)
plot(x,y_1,x,y_2,o),...
xlabel(x),ylabel(y),...
title(Unique solution),...
legend(Line 1,Line 2)
y_3 = (6*x-10)/8;
subplot(3,1,2)
plot(x,y_1,x,y_3,*),...
title(Singular, No unique soln),...
y_4 = (6*x-3)/8;
subplot(3,1,3)
plot(x,y_1,x,y_4,d),...
title(Singular, No solution),...
0 1 2 3 4 5 6 7 8 9 10
5
0
5
10
x
y
Unique solution
Line 1
Line 2
0 1 2 3 4 5 6 7 8 9 10
5
0
5
10
x
y
Singular, No unique solution
Line 1
Line 2
0 1 2 3 4 5 6 7 8 9 10
5
0
5
10
x
y
Singular, No solution
Line 1
Line 2
Homogeneous Equations (zero RHSs)
6x + ay = 0 (1)
2x + 4y = 0 (2)
(1)3(2)
(a 12)y = 0
Case 1: y = 0 only if a = 12
y=0 in (1)
x = 0
Case 2: 0y = 0 if a = 12
x = 2y innite # of solutions
Ill-Conditioned Equations
An Ill-Conditioned set of equations is a set that is close to being
singular
3x 4y = 5
6x 8.002y = 3
y =
3x 5
4
x = 4668
y =
3x 1.5
4.001
y = 3500
But if we had carried only two signicant gures
We would have rounded the denominator of the latter
expression to 4.0
Two expressions for y have the same slope (parallel)
No solution ill-conditioned status (soln dep.s on accuracy)
T6.1-2
Show that the following set has no solution.
4x + 5y = 10
12x 15y = 8
T6.1-3
For what value of b will the following set have a solution in which
both x and y are nonzero? Find the relation between x and y.
4x by = 0
3x + 6y = 0
(Answer: If b = 8, x = 2. If b = 8, x = y = 0.)
Matrix Methods for Linear Equations
2x
1
+ 9x
2
= 5
3x
1
4x
2
= 7
_
2 9
3 4
_
. .
A
_
x
1
x
2
_
. .
x
=
_
5
7
_
. .
b
Ax = b
a
11
x
1
+ a
12
x
2
+ + a
1n
x
n
= b
1
a
21
x
1
+ a
22
x
2
+ + a
2n
x
n
= b
2

a
m1
x
1
+ a
m2
x
2
+ + a
mn
x
n
= b
m
_
_
a
11
a
12
a
1n
a
21
a
22
a
2n

a
m1
a
m2
a
mn
_
_
. .
A
_
_
x
1
x
2
.
.
.
x
n
_
_
. .
x
=
_
_
b
1
b
2
.
.
.
b
m
_
_
. .
b
Ax = b
Determinant and Singular Problems
A =
3 4 1
6 10 2
9 7 8
A = [3,-4,1; 6,10,2; 9,-7,8];

det(A)
ans =
8
Determinants and Singular Problems
Singular set: 3x 4y = 5
6x 8y = 10
_
_
3 4
6 8
_
_
. .
A
_
_
x
y
_
_
=
_
_
5
10
_
_
. .
b
|A| =
3 4
6 8
= 3(8) (4)(6) = 0
|A| indicates that the equation set is singular
Left-division Method With Three Unknowns
Example: Use the left-division method to solve the following set:
3x + 2y 9z = 65
9x 5y + 2z = 16
6x + 7y + 3z = 5
Solution:
A =
_
_
3 2 9
9 5 2
6 7 3
_
_
We can use MATLAB to check the determinant of A to see
whether the problem is singular.
A = [3, 2, -9; -9, -5, 2; 6, 7, 3];
b = [-65; 16; 5];
det_A = det(A),...
soln = A\b ,... % Ax=b
A*soln
det_A =
288
soln =
2.0000
-4.0000
7.0000
ans =
-65.0000
16.0000
5.0000
Example: An Electrical-resistance Network
The circuit shown in the following Figure has ve resistances and applied voltages.
Assuming that the positive directions of current ow are in the directions shown in
the gure, Kirchhos voltage law applied to each loop in the circuit gives
v
1
+ R
1
i
1
+ R
4
i
4
= 0
R
4
+ R
2
i
2
+ R
5
i
5
= 0
R
5
i
5
+ R
3
i
3
+ v
2
= 0
Conservation of charge applied at each node in the circuit gives
i
1
= i
2
+ i
4
i
2
= i
3
+ i
5
You can use these two equations to eliminate i
4
and i
5
from the rst three
equations. The results is:
(R
1
+ R
4
)i
1
R
4
i
2
= v
1
R
4
i
1
+ (R
2
+ R
4
+ R
5
)i
2
R
5
i
3
= 0
R
5
i
2
(R
3
+ R
5
)i
3
= v
2
Thus we have three equations in three unknowns: i
1
, i
2
, and i
3
.
Write a MATLAB script le that uses given values of the applied voltages v
1
and
v
2
and given values of the ve resistances to solve for the currents i
1
, i
2
, and i
3
.
Use the program to nd the currents for the case R
1
= 5, R
2
= 100, R
3
= 200,
R
4
= 1504, R
5
= 250 k, v
1
= 100, and v
2
= 50 volts. (Note that 1
k = 1000)
Solution:
% File resist.m % Solvers for the currents i_1, i_2, i_3
R = [5, 100, 200, 150, 250]*1000;
v1 = 100; v2 = 50;
A1 = [R(1) + R(4), -R(4), 0];
A2 = [-R(4), R(2) + R(4) + R(5), -R(5)];
A3 = [0, R(5), -(R(3) + R(5))];
A = [A1; A2; A3];
b = [v1; 0; v2];
current = A\b;
disp(The currents are:)
disp(current)
>> resist % run resist.m
The currents are:
1.0e-003*
0.9544
0.3195
0.0664 % i_1,i_2,i_3 = 0.9544, 0.3195, 0.0664 mA
Example: Ethanol Production
Engineers in the food and chemical industries use fermentation in many processes.
The following equation describes Bakers yeast fermentation.
a(C
6
H
12
O
6
) + b(O
2
) + c(NH
3
)
C
6
H
10
NO
3
+ d(H
2
O) + e(CO
2
) + f(C
2
H
6
O)
The variables a, b, . . . , f represent the masses of the products involved in the
equation. In this formula C
6
H
12
O
6
represents glucose, C
6
H
10
NO
3
represents
yeast, and C
2
H
6
O represents ethanol. This reaction produces ethanol, in addition
to water and carbon dioxide. We want to determine the amount of ethanol f
produced. The number of C, O, N, and H atoms on the left must balance those
on the right side of the equation. This gives four equations:
6a = 6 + e + 2f C balance
6a + 2b = 3 + d + 2e + f O balance
c = 1 N balance
12a + 3c = 10 + 2d + 6f H balance
The fermentor is equipped with an oxygen sensor and a carbon dioxide sensor.
These enable us to compute the respiratory quotient R:
R =
CO
2
O
2
=
e
b
Thus the fth equation is Rb e = 0. The yeast yield Y (grams of yeast produced
per gram of glucose consumed) is related to a as follows .
Y =
144
180a
Where 144 is the molecular weight of yeast and 180 is the molecular weight of
glucose. By measuring the yeast yield Y we can compute a as follows:
a = 144/180Y . This is the sixth equation.
Write a user-dened function that computes f, the amount of ethanol produced,
with R and Y as the functions arguments. Test your function for two cases where
Y is measure to be 0.5: (a) R = 1.1 and (b) R = 1.05.
Solution: let x
1
b, x
2
d, x
3
e, x
4
f
x
3
2x
4
= 6 6(144/180Y )
2x
1
x
2
2x
3
x
4
= 3 6(144/180Y )
2x
2
6x
4
= 7 12(144/180Y )
Rx
1
x
3
= 0
In matrix form:
_
_
0 0 1 2
2 1 2 1
0 2 0 6
R 0 1 0
_
_
_
_
x
1
x
2
x
3
x
4
_
_
=
_
_
6 6(144/180Y )
3 6(144/180Y )
7 12(144/180Y )
0
_
_
The function le and the the session:
funtion E = ethanol(R,Y)
% Computes ethanol produced
% from yeast reaction.
A = [0, 0,-1,-2; 2,-1,-2,-1; ...
0,-2, 0,-6; R, 0,-1, 0];
b = [6 - 6*(144./(180*Y)); ...
3 - 6*(144./(180*Y)); ...
7 - 12*(144./(180*Y)); 0];
x = A\b;
E = x(4);
E_1 = ethanol(1.1, 0.5)
E_2 = ethanol(1.05,0.5)
E_1 =
0.0654
E_2 =
-0.0717
Matrix Inverse
Ax = b
A
1
A = AA
1
= I
A
1
Ax = A
1
b
x = A
1
b
Example: Calculation Of Cable Tension
A mass m is suspended by three cables attached at the three points B, C, and D,
as shown in the following gure. Let T
1
, T
2
, and T
3
be the tensions in the three
cables AB, AC, and AD, respectively. If the mass m is stationary, the sum of the
tension components in the x, in the y, and in the z directions must each be zero.
This requirement gives the following three equations:
T
1
35
3T
2
34
+
T
3
42
= 0
3T
1
35
4T
3
42
= 0
5T
1
35
+
5T
2
34
+
5T
3
42
mg = 0
Use MATLAB to nd T
1
, T
2
, and T
3
in terms of an unspecied value of the
weight mg.
Solution: set mg = 1
A =
_
_
1
35

3
34
1
42
3
35
0
4
42
5
35
5
34
5
42
_
_
x =
_
_
T
1
T
2
T
3
_
_
b =
_
_
0
0
1
_
_
% File cable.m
% Computes the tensions in three cables.
A1 = [1/sqrt(35), -3/sqrt(34), 1/sqrt(42)];
A2 = [3/sqrt(35), 0, -4/sqrt(42)];
A3 = [5/sqrt(35), 5/sqrt(34), 5/sqrt(42)];
A = [A1; A2; A3];
b = [ 0; 0; 1];
x = A\b;
disp(The tension T_1 is:)
disp(x(1))
disp(x(2))
disp(x(3))
The tension T_1 is:
0.5071 % 0.5071 mg
The tension T_2 is:
0.2915 % 0.2915 mg
The tension T_3 is:
0.4166 % 0.4166 mg
Example: The Matrix Inverse Method
Solve the following equations using the matrix inverse:
2x + 9y = 5
3x 4y = 7
Solution:
A =
_
2 9
3 4
_
Its determinant is |A| = 2(4) 9(3) = 35, and its inverse is
A
1
=
1
35
_
4 9
3 2
_
=
1
35
_
4 9
3 2
_
The solution is
x = A
1
b =
1
35
_
4 9
3 2
__
5
7
_
=
1
35
_
83
1
_
or x = 83/85 = 2.3714 and y = 1/35 = 0.0286.
The Matrix Inverse in MATLAB
A = [2, 9; 3, -4];
b = [5; 7]
x = inv(A)*b
x =
2.3714
0.0286
T6.2-1
Use the matrix inverse method to solve the following set by hand
by using MATLAB: (Answer:x = 7, y = 4.)
3x 4y = 5
6x 10y = 2
T6.2-2
Use the matrix inverse method to solve the following set by hand
and by using MATLAB:
3x 4y = 5
6x 8y = 2
(Answer: no solution.)
Cramers Method
a
11
x + a
12
y = b
1
a
21
x + a
22
y = b
2
a
22
(a
11
x + a
12
y) = a
22
b
1
a
12
(a
21
x + a
22
y) = a
12
b
2
x =
b
1
a
22
b
2
a
12
a
22
a
11
a
12
a
21
=
b
1
a
12
b
2
a
22
a
11
a
12
a
21
a
22
D
1
D
y =
b
2
a
11
b
1
a
21
a
22
a
11
a
12
a
21
=
a
11
b
1
a
21
b
2
a
11
a
12
a
21
a
22
D
2
D
Note: If D = 0 but D
1
= 0 then x is undened
If D = 0 and D
1
= 0 then x has innitely many soln.s
T6.3-1
Use Cramers method to solve for x and y in terms of the
parameter b. For what value of b is the set singular ?
4x by = 5
3x + 6y = 3
(Answer: x = (10 + b)/(8 b), y = 9/(8 b) unless b = 8.)
T6.2-2
Use Cramers method to solve for y.
Use MATLAB to evaluate the determinants.
2x + y + 2z = 17
3y + z = 6
2x 3y + 4z = 19
(Answer: y = 1.)
Under-determined Systems
An Under-determined
System does not contain
enough information to solve
for all of the unknown
variables (fewer equations
than unknowns)
x = 6 3y
A = [1, 3];
b = 6;
soln = A\b
soln = % infinite # solutions,
0 % giving one of these solutions
2 % (x is set to be zero --> y=2)
Note: An innite number of solutions might exist even if # of equations = #
of unknowns (if |A| = 0)
Matrix inverse method and Cramers method do not work
Left-division generates an error message warning
Use pseudo-inverse method: x = pinv(A)*b
pinv command produces a solution with minimum Euclidean norm
2x 4y + 5z = 4
4x 2y + 3z = 4
2x + 6y 8z = 0
A = [2,-4,5; -4,-2,3; 2,6,-8];
b = [-4, 4, 0];
soln_left_div = A\b
soln_pseudoin = pinv(A)*b
Warning: Matrix is singular
to working precision.
soln_left_div =
Inf
Inf
Inf
soln_pseudoin =
-1.2148
0.2074
-0.1481
Matrix Rank
An mn matrix A has a rank r 1 if and only if |A| contains a
nonzero r r determinant and every square sub-determinant with
r + 1 or more rows is zero
Example: The rank of A =
_
_
3 4 1
6 10 2
9 7 3
_
_
is 2 because |A| = 0
whereas A contains at least one nonzero 2 2 subdeterminant.
eg, |A| =
10 2
7 3
= 44
A = [3,-4,1; 6,10,2; 9,-7,3];
rank(A)
ans =
2
Existence and Uniqueness of Solutions
Existence and Uniqueness of Solutions
The set Ax = b with m equations and n unknowns has solutions
if and only if rank(A) = rank([A b])
Let r = rank(A).
1. If previous condition is satised and if r = n,
then the solution is unique
2. If previous condition is satised and but r < n,
an innite number of solution exists and r unknown variables can
be expressed as linear combination of the other n r unknown
variables, whose values are arbitrary.
Homogeneous Case
1. For the homogeneous set Ax = 0, rank(A) = rank([A b])
always, and thus the set always has the trivial solution x = 0.
2. A nonzero solution (at least one unknown is nonzero) exists if
and only if rank(A) < n
3. If m < n, the homogeneous set always has a nonzero solution
Example: A Set Having A Unique Solution
Determine whether the following set has a unique solution, and if
so,nd it:
3x 2y + 8z = 48
6x + 5y + z = 12
9x + 4y + 2z = 24
Solution:
A =
_
_
3 2 8
6 5 1
9 4 2
_
_
b =
_
_
48
12
24
_
_
x =
_
_
x
y
z
_
_
A = [3, -2, 8; -6, 5, 1; 9, 4, 2];
b = [48; -12; 24];
rank(A)
ans =
3
rank([A b])
ans =
3
x=A\b
x =
2
-1
5
T6.4-1
Use MATLAB to show that the following set has a unique solution
and then nd the solution:
3x + 12y 7z = 5
5x 6y 5z = 8
2x + 7y + 9z = 5
(Answer: The unique solution is
x = 1.0204, y = 0.5940, z = 0.1332.)
The Minimum Euclidean Norm Solution
The pinv command can obtain a solution of an
underdetermined set
v = [x y z]
N =

v
T
v =
_
[x y z]
T
_
_
x
y
z
_
_
=
_
x
2
+ y
2
+ z
2
Example: An Under-determined Set
Show that the following set does not have a unique solution. How many of the
unknowns will be undetermined ? Interpret the results given by the left-division
method.
2x 4y + 5z = 4
4x 2y + 3z = 4
2x + 6y 8z = 0
Solution:
A = [2, -4, 5; -4, -2, 3;...
2, 6, -8];
b = [-4; 4; 0];
r1 = rank(A)
r2 = rank([A b])
r1 =
2
r2 =
2
innite no. of solutions
soln = pinv(A)*b
soln =
-1.2148
0.2074
-0.1481
Ex: A Statically-Indeterminate Problem
Determine the forces in the three equally spaced
supports that hold up a light xture. The supports
are 5 feet apart. The xture weights 400 pounds,
and its mass center is 4 feet from the right end.
(a) Solve the problem by hands. (b) obtain the
solution using the MATLAB left-division method
and the pseudoinverse method.
Solution:
vertical force must cancel; total moments about right endpoint are zero
T
1
+ T
2
+ T
3
400 = 0
400(4) 10T
1
5T
2
= 0
or T
1
+ T
2
+ T
3
= 400
10T
1
+ 5T
2
+ 0T
3
= 1600
_
1 1 1
10 5 0
_
. .
A
_
_
T
1
T
2
T
3
_
_
. .
x
=
_
400
1600
_
. .
b
[A b] =
_
1 1 1 400
10 5 0 1600
_
T
2
=
1600 10T
1
5
= 320 2T
1
T
1
= T
3
80
T
2
= 320 2T
1
= 320 2(T
3
80) = 480 2T
3
A = [1, 1, 1; 10, 5, 0];
b = [400; 1600];
rank(A)
ans =
2
rank([A b])
ans =
2
soln_left = A\b
soln_pseu = pinv(A)*b
soln_left =
160.0000
0
240.0000
soln_pseu =
93.3333 % min norm soln
133.3333
173.3333
norm_left = sqrt(sum(soln_left.^2))
norm_pseu = sqrt(sum(soln_pseu.^2))
norm_left =
288.4441
norm_pseu =
237.7674
T6.4-2
Use MATLAB to nd two solutions to the following set:
x + 3y + 2z = 2
x + y + z = 4
(Answer:
Minimum-norm solution: x = 4.33,y = 1.67,z = 1.34.
Left-division solution: x = 5,y = 1,z = 0.)
The Reduced Row Echelon Form
T
1
= T
3
80
T
2
= 480 2T
3
T
1
T
3
= 80
T
2
+ 2T
3
= 480
_
1 0 1
0 1 2
_
_
_
T
1
T
2
T
3
_
_
=
_
80
480
_
_
1 0 1 80
0 1 2 480
_
rref([A b]) command provides a procedure to reduce an underdetermined set to
such a reduced row echelon form
Its output is the augmented matrix [C d] that corresponds to the equation set
Cx = d
Example: A Singular Set
The following under-determined equation set was analyzed in
previous Example. There it was shown that an innite number of
solutions exists. Use the pinv and the rref commands to obtain
solutions.
2x 4y + 5z = 4
4x 2y + 3z = 4
2x + 6y 8z = 0
Solution:
A = [2, -4, 5; -4, -2, 3;...
2, 6, -8];
b = [-4; 4; 0];
x = pinv(A)*b
x =
-1.2148
0.2074
-0.1481
rref([A b])
ans =
1 0 -0.1 -1.2000
0 1 -1.3 0.4000
0 0 0 0
The answer corresponds to the augmented matrix [C d],
[C d] =
_
_
1 0 0.1 1.2
0 1 1.3 0.4
0 0 0 0
_
_
The matrix corresponds to the matrix equation Cx = d, or
x + 0y 0.1z = 1.2
0x + y 1.3z = 0.4
0x + 0y + 0z = 0.0
x = 0.1z 1.2
y = 1.3z + 0.4
z = arbitrary value
Example: Production Planning
The following table shows how many hours reactors A and B need to produce 1
ton each of the chemical products 1, 2, and 3. The two reactors are available for
40 hours and 30 hours per week, respectively. Determine how many tons of each
product can be produced each week.
Hours Product 1 Product 2 Product 3
Reactor A 5 3 3
Reactor B 3 3 4
Solution:
5x + 3y + 3z = 40
3x + 3y + 4z = 30
A =
_
5 3 3
3 3 4
_
b =
_
40
30
_
x =
_
_
x
y
z
_
_
Note: rank(A) = rank([A b]) = 2, which is less than the number of unknowns
(3). Thus an innite number of solution exists, and we can determine two of the
variables in terms of the third.
Using rref([A b]), where A=[5,3,3; 3,3,4] and b=[40;30], we obtain the following
reduced echelon augmented matrix,
_
1 0 0.5 5
0 1 1.8333 5
_
x 0.5z = 5
y + 1.8333z = 5
x = 5 + 0.5z
y = 5 1.8333z
Suppose we make a prot of $400, $600, $100 per ton for products 1, 2 and 3,
respectively.
P = 400x + 600y + 100z
= 400(5 + 0.5z) + 600(5 1.8333z) + 100z
= 5000 800z
To maximize prot P, choose z = 0 x = y = 5 tons.
Example: Trac Engineering
A trac engineer wants to know whether measurements of trac ow entering
and leaving a road network are sucient to predict the trac ow on each street
in the network. For example, consider the network of one-way streets shown in the
following Figure. The numbers in the gure give the measured trac ows in
vehicles per hour. Assume that no vehicles park anywhere within the network. If
possible, calculate the trac f1, f2, f3, and f4. If this is not possible, suggest
how to obtain the necessary information.
Solution:
100 + 200 = f
1
+ f
4
f
1
+ f
2
= 300 + 200
600 + 400 = f
2
+ f
3
f
3
+ f
4
= 300 + 500
A =
_
_
1 0 0 1
1 1 0 0
0 1 1 0
0 0 1 1
_
_
b =
_
_
300
500
1000
800
_
_
x =
_
_
f
1
f
2
f
3
f
4
_
_
rref([A b])
_
_
1 0 0 1 300
0 1 0 1 200
0 0 1 1 800
0 0 0 0 0
_
f
1
= 300 f
4
f
2
= 200 + f
4
f
3
= 800 f
4
T6.4-3
Use the rref and pinv commands and the left-division method to solve the
following set:
3x + 5y + 6z = 6
8x y + 2z = 1
5x 6y 4z = 5
(Answer: The set has an innite number of solutions. The result obtained
with the rref commands is x = 0.2558 0.3721z, y = 1.0465 0.9767z, z is
arbitrary. The pinv commands gives x = 0.0571, y = 0.5249, z = 0.5340. The
left-division method generates an error message.)
T6.4-4
Use the rref and pinv commands and the left-division method to solve the
following set:
3x + 5y + 6z = 4
x 2y 3z = 10
(Answer: The set has an innite number of solutions. The result obtained with
the rref commands is x = 0.2727z +5.2727 0.3721z, y = 1.3636z 2.3636,
z is arbitrary. The pinv commands gives x = 4.8000, y = 0, z = 1.7333. The
pseudoinverse method gives x = 4.8396, y = 0.1972, z = 1.5887.)
Overdetermined Systems: The Least
Squares Method
Suppose we have the following three data points, and we want to nd the straight
line y = mx + b that best ts the data in some sense.
x y
0 2
5 6
10 11
(a) Find the coecients m and b by using the least squares criterion. (b) Find the
coecients by using MATLAB to solve the three equations (one for each data
point) for the two unknowns m and b. Compare the answers from (a) and (b).
Solution:
J =
i=3
i=1
(mx
i
+ b y
i
)
2
J = (0m + b 2)
2
+ (5m + b 6)
2
+ (10m + b 11)
2
J
m
= 2(5m + b 6)(5) + 2(10m + b 11)(10)
= 250m + 30b 280 = 0
J
b
= 2(b 2) + 2(5m + b 6) + 2(10m + b 11)
= 30m + 6b 38 = 0
250m + 30b = 280
30m + 6b = 38
m = 0.9, b =
11
6
(y = 0.9x + 11/6)
Evaluate y = mx + b at each data point:
0m + b = 2
5m + b = 6
10m + b = 11
_
_
0 1
5 1
10 1
_
_
_
m
b
_
=
_
_
2
6
11
_
_
A =
_
_
0 1
5 1
10 1
_
_
x =
_
m
b
_
b =
_
_
2
6
11
_
_
A = [0, 1; 5, 1; 10, 1];
b = [2; 6; 11];
rank(A)
ans =
2
rank([A b])
ans =
3
A\b
ans =
0.9000
1.8333
A*ans
ans =
1.833
6.333
10.8333
Example: An Over-determined Set
(a) Solve the following equations by hand and (b) solve them using MATLAB.
Discuss the solution for two cases: c = 9 and c = 10.
x + y = 1
x + 2y = 3
x + 5y = c
Solution:
A =
_
_
1 1
1 2
1 5
_
_
[A b] =
_
_
1 1 1
1 2 3
1 5 c
_
_
c = 9 rank(A) = rank([A b]) = 2
A\b gives unique solution x = 1, y = 2
c = 10 rank(A) = 2, rank([A b]) = 3
A\b gives least squares solution x = 1.3846, y = 2.2692
J = (x + y 1)
2
+ (x + 2y 3)
2
+ (x + 5y 10)
2
Fitting Models by Least Squares
inputs = x
i1
, x
i2
, , x
ip
output = y
i
, i = 1, . . . , n
(y
1
; x
11
, x
12
, , x
1p
) (1st observation data)
(y
2
; x
21
, x
22
, , x
2p
) (2nd observation data)
.
.
.
.
.
.
(y
n
; x
n1
, x
n2
, , x
np
) (nth observation data)
y =
1
x
1
+
2
x
2
+ +
p
x
p
(linear model)
y
1
=
1
x
11
+
2
x
12
+ +
p
x
1p
+
1
y
2
=
1
x
21
+
2
x
22
+ +
p
x
2p
+
2
.
.
.
.
.
.
y
n
..
obs
=
1
x
n1
+
2
x
n2
+ +
p
x
np
. .
model output
+
n
..
error
f =
n
i=1
i
2
=
n
i=1
_
_
y
i
j=1
j
x
ij
_
_
2
(SSE)
Q: nd
1
, ,
p
to minimize SSE ?
f
= 0
f
= 0
f
= 0
n
i=1
y
i
x
i1
=

1
n
i=1
x
i1
x
i1
+

2
n
i=1
x
i1
x
i2
+ +

p
n
i=1
x
i1
x
ip
n
i=1
y
i
x
i2
=

1
n
i=1
x
i2
x
i1
+

2
n
i=1
x
i2
x
i2
+ +

p
n
i=1
x
i2
x
ip
.
.
.
.
.
.
.
.
.
n
i=1
y
i
x
ip
=

1
n
i=1
x
ip
x
i1
+

2
n
i=1
x
ip
x
i2
+ +

p
n
i=1
x
ip
x
ip
Q: how to nd

1
, ,

p
?
_
_
y
1
y
2
.
.
.
y
n
_
_
. .
obs.s
=
_
_
x
11
x
12
x
1p
x
21
x
22
x
2p
.
.
.
.
.
.
.
.
.
.
.
.
x
n1
x
n2
x
np
_
_
_
2
.
.
.
p
_
_
. .
model outputs
+
_
2
.
.
.
p
_
_
Y = X
+
f =
T
= (Y X)
T
(Y X) (SSE)
f
= x
T
Y X
T
X
= 0
= (X
T
X)
1
X
T
Y
T6.5-1
Use MATLAB to solve the following set:
x 3y = 2
3x + 5y = 7
70x 28y = 153
(Answer: The unique solution, x = 2.2143, y = 0.0714, is given by the
left-division method.)
T6.5-2
Use MATLAB to solve the following set:
x 3y = 2
3x + 5y = 7
5x 2y = 4
(Answer: no exact solution.)
A MATLAB Program to Solve Linear Equations
% Script file lineq.m
% Solve the set Ax=b, given A and b
% Check the ranks of A and [A b]
if rank(A) == rank([A b]) % A, [A b]: equal ranks
size_A = size(A);
if rank(A) == size_A(2) % rank of A = no unknowns
disp(There is a unique solution: )
x = A\b % solve using left division
else % rank of A # no unknowns
disp(There is an infinite no of solutions.)
disp(The augmented matrix of reduced system: )
rref([A b]) % compute augmented matrix
end
else % A, [A b]: not equal ranks
disp(There are no solutions )
end
A = [1, -1; 1, 1]; b = [3; 5];
lineq
There is a unique solution:
x =
4
1
A = [1, -1; 2, -2]; b = [3; 6];
lineq
There is an infinite no of solutions.
The augmented matrix of reduced system:
ans =
1 -1 3
0 0 0
A = [1, -1; 2, -2]; b = [3; 5];
lineq
There are no solutions

6 Linear Algebraic Equations PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

6 Linear Algebraic Equations PDF

Uploaded by

Copyright:

Available Formats

Linear Algebraic Equations

A = [3,-4,1; 6,10,2; 9,-7,8];

You might also like