Professional Documents
Culture Documents
of Data to a Curve
Gerald Recktenwald
Portland State University
Department of Mechanical Engineering
gerry@me.pdx.edu
Version 0.82
November 6, 2007
page 1
Overview
page 2
(xi, yi), i = 1, . . . , m
Find the coecients and such that
F (x) = x +
is a good t to the data
Questions:
page 3
Plausible Fits
5
y
4
3
2
1
page 4
The Residual
ri = yi F (xi)
= yi (xi + )
4
3
page 5
or
minimize
ri
m
X
[yi (xi + )]
ri2
i=1
page 6
y
d4
d3
d1
x1
d2
x2
x3
x4
page 7
(1)
ri
i=1
=0
=constant
and
=0
=constant
page 8
(2)
Sxx + Sx
Sxy
(1)
Sx + m
Sy
(2)
where
Sxx =
Sxy =
m
X
i=1
m
X
i=1
xixi
xiyi
Sx =
Sy =
m
X
i=1
m
X
xi
yi
i=1
Note: Sxx, Sx, Sxy , and Syy can be directly computed from the given (xi, yi) data.
Thus, Equation (1) and (2) are two equations for the two unknowns, and .
page 9
(3)
1
(SxSy mSxy )
d
1
(SxSxy SxxSy )
d
(3)
(4)
with
2
d = Sx mSxx
(5)
page 10
(1)
Now, lets rederive the equations for the t. This will give us insight into the process or
tting arbitrary linear combinations of functions.
For any two points we can write
x1 + = y1
x2 + = y2
or
x1
x2
1
1
y1
=
y2
page 11
(2)
Writing out the x + = y equation for all of the known points (xi, yi),
i = 1, . . . , m gives the overdetermined system.
2 3
3
2
y1
x1 1
6 y2 7
6 x2 17
7
6 .
7
or
Ac = y
... 5 = 6
4 ... 5
4 ..
xm 1
ym
where
x1
6 x2
A=6
4 ...
xm
3
1
17
... 7
5
1
c=
3
y1
6 y2 7
7
y=6
4 ... 5
ym
Note: We cannot solve Ac = y with Gaussian elimination. Unless the system is consistent (i.e., unless
y lies in the column space of A) it is impossible to nd the c = (, )T that exactly satises
all m equations. The system is consistent only if all the data points lie along a single line.
page 12
= y y (Ac) y y (Ac) + c A Ac
T
= y y 2y Ac + c A Ac.
Minimizing requires
or
T
T
= 2A y + 2A Ac = 0
c
T
(A A)c = A b
This is the matrix formulation of equations (1) and (2).
page 13
linefit.m
The linefit function ts a line to a set of data by solving the normal equations.
function [c,R2] = linefit(x,y)
% linefit
Least-squares fit of data to y = c(1)*x + c(2)
%
% Synopsis:
c
= linefit(x,y)
%
[c,R2] = linefit(x,y)
%
% Input:
x,y = vectors of independent and dependent variables
%
% Output: c = vector of slope, c(1), and intercept, c(2) of least sq. line fit
%
R2 = (optional) coefficient of determination; 0 <= R2 <= 1
%
R2 close to 1 indicates a strong relationship between y and x
if length(y)~= length(x), error(x and y are not compatible); end
x = x(:); y = y(:);
% Make sure that x and y are column vectors
A = [x ones(size(x))]; % m-by-n matrix of overdetermined system
c = (A*A)\(A*y);
% Solve normal equations
if nargout>1
r = y - A*c;
R2 = 1 - (norm(r)/norm(y-mean(y)))^2;
end
page 14
3.5
3
y data and fit function
>> x = [1 2 4 5];
>> y = [1 2 2 3];
>> c = linefit(x,y)
c =
0.4000
0.8000
2.5
2
1.5
1
0.5
0
0
3
x
page 15
R2 Statistic
(1)
R2 is a measure of how well the t function follows the trend in the data. 0 R2 1.
Define:
y
y
Then:
For a line t
yi = c1xi + c2
1 X
y =
yi
m
X
2
)
(
y
i
r22
2
R =X
=1P
2
(yi y)2
(yi y)
page 16
5
y
4
3
2
1
5
y
4
3
2
1
page 17
T (C)
G (GP a)
20 500
1000
1200
1400
1500
203 197
191
188
186
184
GPa
205
Bulk Modulus
200
195
190
185
180
500
1000
1500
T C
page 18
(1)
y = c1ec2x
ln y = x +
y = c1xc2
ln y = ln x +
y = c1xec2x
ln(y/x) = x +
page 19
(2)
Consider
y = c1e
Taking the logarithm of both sides yields
c2 x
(6)
ln y = ln c1 + c2x
Introducing the variables
v = ln y
b = ln c1
a = c2
v = ax + b
page 20
(3)
The preceding steps are equivalent to graphically obtaining c1 and c2 by plotting the data
on semilog paper.
y = c1ec2x
ln y = c2x + ln c1
1
10
4.5
4
3.5
10
2.5
2
1
10
1.5
1
0.5
0
0
0.5
1
x
1.5
10
0.5
1
x
1.5
page 21
(4)
ln y = ln c1 + c2 ln x
(7)
v = ln y
u = ln x
b = ln c1
a = c2
v = au + b
page 22
(5)
The preceding steps are equivalent to graphically obtaining c1 and c2 by plotting the data
on log-log paper.
y = c1xc2
ln y = c2 ln x + ln c1
2
10
14
12
10
8
1
10
0
0
0.5
1
x
1.5
10 2
10
10
10
10
page 23
b = ln c1
v = ax + b
page 24
(6)
The preceding steps are equivalent to graphically obtaining c1 and c2 by plotting the data
on semilog paper.
y = c1xec2x
ln(y/x) = c2x + ln c1
1
0.7
10
0.6
0.5
10
0.4
0.3
1
10
0.2
0.1
0
0
0.5
1
x
1.5
10
0.5
1
x
1.5
page 25
xexpfit.m
xexpfit(x,y)
Least squares fit of data to y = c(1)*x*exp(c(2)*x)
c = xexpfit(x,y)
x,y = vectors of independent and dependent variable values
c = vector of coefficients of
z = log(y./x);
c = linefit(x,z);
c = [exp(c(2)); c(1)];
%
%
%
y = c(1)*x*exp(c(2)*x)
page 26
0.7
Fit y = c1
demoXexp
xec2x
original
noisy
fit
0.6
0.5
c1 = 5.770 c2 = -3.233
0.4
0.3
0.2
0.1
0
0
0.5
1
x
1.5
page 27
Summary of Transformations
x
y
u
v
a
c
=
=
=
=
=
=
...
...
...
...
linefit(u,v)
...
original data
page 28
(1)
f1(x1)
6 f1(x2)
A=6
4 ...
f1(xm)
2
3
x1
f2(x1)
6 x2
f2(x2) 7
6 .
7
=
... 5
4 ..
f2(xm)
xm
3
1
17
... 7
5.
1
page 29
(2)
4. The least-squares principle denes the best t as the values of c1 and c2 that minimize
2
(A A)c = A y,
6. Solving the normal equations gives the slope c1 and intercept c2 of the best t line.
page 30
page 31
(1)
F (x) =
n
X
cj fj (x)
j=1
c1, c2, . . . , cn
are determined by the least squares method.
page 32
(2)
F (x) function can be any combination of functions that are linear in the cj . Thus
2
2/3
1, x, x , x
4x
sin(c1x), e
c3 x
,
c2
are not valid basis functions as long as the cj are the parameters of the t.
For example, the t function for a cubic polynomial is
3
x , x , x, 1.
NMM: Least Squares Curve-Fitting
page 33
(3)
ri = yi F (xi) = yi
n
X
cj fj (xi)
j=1
page 34
(4)
page 35
(5)
Ac = y,
where
f1(x1)
6 f1(x2)
A=6
4 ...
f1(xm)
2 3
c1
c = 4c25 ,
c3
f2(x1)
f2(x2)
...
f2(xm)
3
f3(x1)
f3(x2) 7
... 7
5,
f3(xm)
3
y1
6 y2 7
7
y=6
4 ... 5 .
ym
2
page 36
(6)
If F (x) cannot interpolate the data, then the preceding matrix equation cannot be solved
exactly: b does not lie in the column space of A.
The least-squares method provides the compromise solution that minimizes
r2 = y Ac2.
The c that minimizes r2 satises the normal equations
T
(A A)c = A y.
page 37
(7)
f1(x1)
6 f1(x2)
A=6
4 ...
f1(xm)
2
f2(x1)
f2(x2)
...
f2(xm)
3
c1
6 c2 7
7
c=6
4 ... 5 ,
cn
...
...
...
3
fn(x1)
fn(x2) 7
7,
...
5
fn(xm)
3
y1
6 y2 7
7
y=6
4 ... 5 .
ym
page 38
12
10
1
2
3
4
5
6
10 5.49 0.89 0.14 1.07 0.84
6
y
x
y
(1)
page 39
(2)
y = c1x + c2x + c3
where the ci are the coecients to be determined by the t and the basis functions are
2
f1(x) = x ,
The A matrix is
f2(x) = x,
2
x21
6 x2
2
A=6
4 ...
x2m
x1
x2
...
xm
f3(x) = 1
3
1
17
... 7
5
1
page 40
(3)
0.89
-0.14
-1.07
0.84];
ones(size(x)) ];
441
91
21
91
21
6
page 41
(4)
page 42
(5)
F(x) = c1 x2 + c2 x + c3
12
page 43
(1)
F(x) = c1/x + c2 x
12
c1
F (x) =
+ c2x
x
10
1
,
x
In Matlab:
>>
>>
>>
>>
x
y
A
c
=
=
=
=
(1:6);
[10 5.49 0.89 -0.14 -1.07 0.84];
[1./x x];
(A*A)\(A*y)
page 44
(1)
3
fn(x1)
fn(x2) 7
7
...
5
fn(xm)
page 45
(2)
xfit = linspace(min(x),max(x));
Afit = xinvxfun(xfit);
yfit = Afit*c;
plot(x,y,o,xfit,yfit,--)
page 46
(3)
Advantages:
The basis functions are dened in only one place: in the routine for evaluating the
overdetermined matrix.
Automation of tting and plotting is easier because all that is needed is one routine
for evaluating the basis functions.
End-user of the t (not the person performing the t) can still evaluate the t
function as y = c1f1(x) + c2f2(x) + + cnfn(x).
Disadvantages:
Storage of matrix A for large x vectors consumes memory. This should not be a
problem for small n.
Evaluation of the t function may not be obvious to a reader unfamiliar with linear
algebra.
page 47
...
A = 4f1(x)
...
...
f2(x)
...
...
... 3
fn(x)5
...
The columns of A are the basis functions evaluated at each of the x data points.
As before, the normal equations are
T
A Ac = A y
The user supplies a (usually small) m-le that returns A.
page 48
fitnorm.m
function [c,R2,rout] = fitnorm(x,y,basefun)
% fitnorm
Least-squares fit via solution to the normal equations
%
% Synopsis: c
= fitnorm(x,y,basefun)
%
[c,R2]
= fitnorm(x,y,basefun)
%
[c,R2,r] = fitnorm(x,y,basefun)
%
% Input:
x,y
= vectors of data to be fit
%
basefun = (string) m-file that computes matrix A with columns as
%
values of basis basis functions evaluated at x data points.
%
% Output: c = vector of coefficients obtained from the fit
%
R2 = (optional) adjusted coefficient of determination; 0 <= R2 <= 1
%
r = (optional) residuals of the fit
if length(y)~= length(x); error(x and y are not compatible); end
A = feval(basefun,x(:)); % Coefficient matrix of overdetermined system
c = (A*A)\(A*y(:));
% Solve normal equations, y(:) is always a column
if nargout>1
r = y - A*c;
% Residuals at data points used to obtain the fit
[m,n] = size(A);
R2 = 1 - (m-1)/(m-n-1)*(norm(r)/norm(y-mean(y)))^2;
if nargout>2, rout = r; end
end
NMM: Least Squares Curve-Fitting
page 49
f1(x) = x ,
f2(x) = x,
f3(x) = 1
x(:)
ones(size(x(:)) ]);
page 50
1
,
x
x(:) ]);
page 51
R2 Statistic
(1)
X
2
)
(
y
i
r22
2
R =X
=1P
2
(yi y)2
(yi y)
where yi is the value of the t function evaluated at xi, and y is the average of the
(known) y values.
For a linear combination of basis functions
yi =
n
X
cj fj (xi)
j=1
page 52
R2 Statistic
(2)
To account for the reduction in degrees of freedom in the data when the t is performed,
it is technically appropriate to consider the adjusted coecient of determination
Radjusted = 1
P
(yi y)2
m1
,
P
(yi y)2
mn1
2
fitnorm provides the option of computing Radjusted
page 53
(1)
polyval
page 54
(2)
Syntax:
c = polyfit(x,y,n)
[c,S] = polyfit(x,y,n)
n1
+ + cnx + cn+1
page 55
(3)
page 56
(1)
1
10
2
5.49
3
0.89
4
0.14
5
1.07
6
0.84
In Matlab:
>>
>>
>>
>>
>>
>>
x = (1:6);
y = [10 5.49 0.89 -0.14 -1.07
c = polyfit(x,y,3);
xfit = linspace(min(x),max(x));
yfit = polyval(c,xfit);
plot(x,y,o,xfit,yfit,--)
0.84];
page 57
12
10
2
1
page 58
(1)
35
Conductivity (W/m/C)
30
25
20
15
10
5
0
0
10
20
30
40
Temperature (K)
50
60
page 59
(2)
k(T ) =
1
c1
2
+ c2T
T
(T ) =
which has the basis functions
1
c1
2
=
+ c2T
k(T )
T
1
,
T
page 60
(3)
function y = cuconBasis1(x)
% cuconBasis1 Basis fcns for conductivity model:
y = [1./x x.^2];
An m-le that uses fitnorm to t the conductivity data with the cuconBasis1 function
is listed on the next page.
page 61
(4)
function conductFit(fname)
% conductFit LS fit of conductivity data for Copper at low temperatures
%
% Synopsis: conductFit(fname)
%
% Input: fname
= (optional, string) name of data file;
%
Default: fname = conduct1.dat
%
% Output: Print out of curve fit coefficients and a plot comparing data
%
with the curve fit for two sets of basis functions.
if nargin<1,
fname = cucon1.dat;
end
%
%
%
page 62
page 63