Function ApproximationFunction Approximation Function approximation (Chapters 13 & 14) -- method of least squares -- minimize the residuals -- given data of points have noises -- the purpose is to find the trend represented by data.

Function interpolation (Chapters 15 & 16) -- approximating function match the given data exactly -- given data of points are precise -- the purpose is to find data between these points

Interpolation Interpolation and and

RegressionRegression

Chapter 13Chapter 13

Curve Fitting:Curve Fitting: Fitting a Straight LineFitting a Straight Line

Least Square RegressionLeast Square Regression Curve FittingCurve Fitting Statistics ReviewStatistics Review Linear Least Square RegressionLinear Least Square Regression Linearization of Nonlinear Linearization of Nonlinear

RelationshipsRelationships MATLAB FunctionsMATLAB Functions

Wind Tunnel ExperimentWind Tunnel Experiment

Measure air resistance as a function of velocity

Curve Fitting

(a) Least-squares regression

(b) Linear interpolation

(c) Curvilinear interpolation

Regression and InterpolationRegression and Interpolation

Curve fitting

Least-squares fit of a straight lineLeast-squares fit of a straight line

145083012206105503807025

8070605040302010v, m/s

F, N

Simple StatisticsSimple StatisticsMeasurement of the coefficient of thermal expansion of structural steel [106 in/(inF)]

535666766706592663366276

598649966216445645164036

703654266596624673366626

721639665646435662565556

655647866216543639965526

667632564956775655464856

......

......

......

......

......

......

Mean, standard deviation, variance, etc.

Statistics ReviewStatistics Review Arithmetic mean

Standard deviation about the mean

Variance (spread)

Coefficient of variation (c.v.)

n

yy i

2itt

y yyS 1n

Ss

;

1n

nyy

1n

yys

2

i2i

2i2

y

/

%.. 100y

svc y

1 6.485 0.007173 42.055 2 6.554 0.000246 42.955 3 6.775 0.042150 45.901 4 6.495 0.005579 42.185 5 6.325 0.059875 40.006 6 6.667 0.009468 44.449 7 6.552 0.000313 42.929 8 6.399 0.029137 40.947 9 6.543 0.000713 42.811 10 6.621 0.002632 43.838 11 6.478 0.008408 41.964 12 6.655 0.007277 44.289 13 6.555 0.000216 42.968 14 6.625 0.003059 43.891 15 6.435 0.018143 41.409 16 6.564 0.000032 43.086 17 6.396 0.030170 40.909 18 6.721 0.022893 45.172 19 6.662 0.008520 44.382 20 6.733 0.026669 45.333 21 6.624 0.002949 43.877 22 6.659 0.007975 44.342 23 6.542 0.000767 42.798 24 6.703 0.017770 44.930 25 6.403 0.027787 40.998 26 6.451 0.014088 41.615 27 6.445 0.015549 41.538 28 6.621 0.002632 43.838 29 6.499 0.004998 42.237 30 6.598 0.000801 43.534 31 6.627 0.003284 43.917 32 6.633 0.004008 43.997 33 6.592 0.000498 43.454 34 6.670 0.010061 44.489 35 6.667 0.009468 44.449 36 6.535 0.001204 42.706 236.509 0.406514 1554.198

2i

2i i y yyy i

Coefficient of Thermal ExpansionCoefficient of Thermal Expansion

%.%.

.%..

..

/).(./

..

.

..

64110056976

107770100

y

svc

0116147035

4065140

35

365092361981554

1n

nyys

107770136

4065140

1n

Ss

4065140yyS

5697636

509236

n

yy

y

22

i2i2

y

ty

2

it

i

Sum of the square of residuals

Standard deviation

Variance

Coefficient of variation

Mean

A histogram used to depict the distribution of data For large data set, the histogram often approaches

the normal distribution (use data in Table 12.2)

HistogramHistogram

Normal Distribution

22

1( ) exp

22

x xp x

Regression and ResidualRegression and Residual

Linear RegressionLinear RegressionFitting a straight line to observations

Small residual errors Large residual errors

Equation for straight line

Difference between observation and line

ei is the residual or error

xaay 10

ii10i exaay

Linear RegressionLinear Regression

Least Squares ApproximationLeast Squares Approximation

Minimizing Residuals (Errors) minimum average error (cancellation) minimum absolute error minimax error (minimizing the maximum

error) least squares (linear, quadratic, ….)

)

n

1ii10i

n

1ii xaa(ye

n

1ii10i

n

1ii xaaye

Minimize the Maximum Error

Minimize Sum of Errors

Minimize Sum of Absolute Errors

Linear Least SquaresLinear Least Squares

Minimize total square-error Straight line approximation

Not likely to pass all points if n > 2

),( , , ),( , ),( , ),( 332211 nn yxyxyxyx

i10ii

10

xaaxfy

xaaxf

)(

)(

Linear Least SquaresLinear Least Squares

Total square-error function: sum of the squares of the residuals

Minimizing square-error Sr(a0 ,a1)

),( , , ),( , ),( , ),( 332211 nn yxyxyxyx

n

1i

2i10i

n

1i

2ir xaayeS )(

0a

S

0a

S

1

r

0

r

Solve for (a0 ,a1)

Linear Least SquaresLinear Least Squares Minimize

Normal equation y = a0 + a1x

n

1i

2i10i10r xaayaaS )(),(

n

1iii1

n

1i

2i0

n

1ii

n

1ii1

n

1ii0

n

1iii10i

1

r

n

1ii10i

0

r

yxaxax

yaxna

xxaay20a

S

xaay20a

S

n

xa

n

yxaya

xxn

yxyxna

i1

i10

2

i2i

iiii1

Advantage of Least SquaresAdvantage of Least Squares

Positive differences do not cancel

negative differences

Differentiation is straightforward

weighted differences

Small differences become smaller and

large differences are magnified

Linear Least SquaresLinear Least Squares Use sum( ) in MATLAB

SnS

SSnSa

SnS

SSSSa

S

S

a

a

SS

Sn

yS yxS

xS xS

let

2xxx

yxxy12

xxx

xxyyxx0

xy

y

1

0

xxx

x

n

1iiyi

n

1iixy

n

1iix

n

1i

2ixx

,

,

,,

Correlation CoefficientCorrelation Coefficient Sum of squares of the residuals with respect to the mean

Sum of squares of the residuals with respect to the regression line

Coefficient of determination

Correlation coefficient

n

ii

n

iit y

ny yyS

1

2

1

1 ;)(

2n

1ii10ir xaayS )(

t

rt

S

SSr

trt SSSr /)(2

Correlation CoefficientCorrelation Coefficient Alternative formulation of

correlation coefficient More convenient for

computer implementation

2i

2i

2i

2i

iiii

yynxxn

yxyxnr

)()(

))((

Standard Error of the EstimateStandard Error of the Estimate If the data spread about the line is normal “Standard deviation” for the regression line

2n

SS r

xy /

Standard error of the estimate

No error if n = 2 (a0 and a1)

Spread of data around the mean

Spread of data around the best-fit line

Linear regression reduce the spread of dataLinear regression reduce the spread of data

Normal distributions

Standard Deviation for Regression LineStandard Deviation for Regression Line

Sy/x

Sy

Sy : Spread around the mean

Sy/x : Spread around the regression line

2n

SS

1n

SS

rx/y

ty

Example: Linear RegressionExample: Linear Regression

5.119)5.5(7)0.6)(6()5.2)(5(

)0.4)(4()0.2)(3()5.2)(2()5.0)(1(yxS

1407654321xS

24/7n /Sy ; 0.245.50.65.30.40.25.25.0yS

428/7/nSx ; 287654321xS

iixy

22222222ixx

yiy

xix

99112S714322S5119S140S024S28S

199302908453849557

797206122603636066

589600051051725535

326503265001616044

3473004082069023

5625086220054522

1687057658501501

xaayyyyxxyx

rtxyxxyx

2i10i

2iii

2iii

....

....

....

....

....

....

....

....

)()(

Example: Linear RegressionExample: Linear Regression

8392857140a 0714285730a

5119

24

a

a

14028

287

S

S

a

a

SS

Sn

10

1

0

xy

y

1

0

xxx

x

.,.

.

945716

714322

1n

SS

714322yyS

ty

2it

..

.)(

Standard deviation about the mean

Standard error of the estimate

773505

99112

2n

SS

99112xaayS

rxy

2i10ir

..

.)(

/

Correlation coefficient 932.0

7143.22

9911.27143.22

S

SSr

t

rt

Linear Least Square Regression Linear Least Square Regression

Modified MATLAB M-FileModified MATLAB M-File

» x=1:7x = 1 2 3 4 5 6 7» y=[0.5 2.5 2.0 4.0 3.5 6.0 5.5]y = 0.5000 2.5000 2.0000 4.0000 3.5000 6.0000 5.5000» s=linear_LS(x,y)a0 = 0.0714a1 = 0.8393 x y (a0+a1*x) (y-a0-a1*x) 1.0000 0.5000 0.9107 -0.4107 2.0000 2.5000 1.7500 0.7500 3.0000 2.0000 2.5893 -0.5893 4.0000 4.0000 3.4286 0.5714 5.0000 3.5000 4.2679 -0.7679 6.0000 6.0000 5.1071 0.8929 7.0000 5.5000 5.9464 -0.4464err = 2.9911Syx = 0.7734r = 0.9318s = 0.0714 0.8393 y =0.0714 + 0.8393 x

Sum of squares of residuals Sr

Standard error of the estimate

Correlation coefficient

» x=0:1:7; y=[0.5 2.5 2 4 3.5 6.0 5.5];

Linear regression

y = 0.0714+0.8393x

Error : Sr = 2.9911

correlation coefficient : r = 0.9318

function [x,y] = example1

x = [ 1 2 3 4 5 6 7 8 9 10];

y = [2.9 0.5 -0.2 -3.8 -5.4 -4.3 -7.8 -13.8 -10.4 -13.9];

» [x,y]=example1;» s=Linear_LS(x,y)a0 = 4.5933a1 = -1.8570 x y (a0+a1*x) (y-a0-a1*x) 1.0000 2.9000 2.7364 0.1636 2.0000 0.5000 0.8794 -0.3794 3.0000 -0.2000 -0.9776 0.7776 4.0000 -3.8000 -2.8345 -0.9655 5.0000 -5.4000 -4.6915 -0.7085 6.0000 -4.3000 -6.5485 2.2485 7.0000 -7.8000 -8.4055 0.6055 8.0000 -13.8000 -10.2624 -3.5376 9.0000 -10.4000 -12.1194 1.7194 10.0000 -13.9000 -13.9764 0.0764err = 23.1082Syx = 1.6996r = 0.9617s = 4.5933 -1.8570 y = 4.5933 1.8570 x

r = 0.9617

Linear Least Square

y = 4.5933 1.8570 x

Error Sr = 23.1082

Correlation Coefficient r = 0.9617

» [x,y]=example2x = Columns 1 through 7 -2.5000 3.0000 1.7000 -4.9000 0.6000 -0.5000 4.0000 Columns 8 through 10 -2.2000 -4.3000 -0.2000y = Columns 1 through 7 -20.1000 -21.8000 -6.0000 -65.4000 0.2000 0.6000 -41.3000 Columns 8 through 10 -15.4000 -56.1000 0.5000

» s=Linear_LS(x,y)a0 = -20.5717a1 = 3.6005 x y (a0+a1*x) (y-a0-a1*x) -2.5000 -20.1000 -29.5730 9.4730 3.0000 -21.8000 -9.7702 -12.0298 1.7000 -6.0000 -14.4509 8.4509 -4.9000 -65.4000 -38.2142 -27.1858 0.6000 0.2000 -18.4114 18.6114 -0.5000 0.6000 -22.3720 22.9720 4.0000 -41.3000 -6.1697 -35.1303 -2.2000 -15.4000 -28.4929 13.0929 -4.3000 -56.1000 -36.0539 -20.0461 -0.2000 0.5000 -21.2918 21.7918err = 4.2013e+003Syx = 22.9165r = 0.4434s = -20.5717 3.6005

Correlation coefficient r = 0.4434

Linear Least Square: y = 20.5717 + 3.6005x

Data in arbitrary order

Large errors !!

Linear regression

y = 20.5717 +3.6005x

Error Sr = 4201.3Correlation r = 0.4434 !!

Linearization of Nonlinear RelationshipsLinearization of Nonlinear Relationships

Untransformed power equation

x vs. y

transformed datalog x vs. log y

Linearization of Linearization of Nonlinear RelationshipsNonlinear Relationships

Exponential equation

Power equation

iiii

11

x1

yx of instead yx use

xy

ey 1

,ln,

lnln

iiii

22

2

yx of instead yx use

x y

xy 2

,log,log

logloglog

log : Base-10

Linearization of Linearization of Nonlinear RelationshipsNonlinear Relationships

Saturation-growth-rate equation

Rational function

iii

i

4444

yx of instead y

1x use

xy

1

x

1y

,,

iiii

3

3

333

yx of instead y

1

x

1 use

x

11

y

1

x

xy

,,

Power equation fit along with the data

x vs. y

Transformed Data

log xi vs. log yi

Example 12.4: Power EquationExample 12.4: Power Equation

y = 2 x 2

12-12

>> x=[10 20 30 40 50 60 70 80];>> y = [25 70 380 550 610 1220 830 1450];

>> [a, r2] = linregr(x,y)a = 19.4702 -234.2857r2 = 0.8805

y = 19.4702x 234.2857

12-13

>> x=[10 20 30 40 50 60 70 80];>> y = [25 70 380 550 610 1220 830 1450];>> linregr(log10(x),log10(y))r2 = 0.9481ans = 1.9842 -0.5620

log x vs. log y

log y = 1.9842 log x – 0.5620

y = (10–0.5620)x1.9842 = 0.2742 x1.9842

Least-square fit of nth-order polynomial p = polyfit(x,y,n)

Evaluate the value of polynomial using

y = polyval(p,x)

MATLAB FunctionsMATLAB Functions

n1n2n

21n

1 pxpxpxpxf )(

CVEN 302-501CVEN 302-501Homework No. 9Homework No. 9

Chapter 13 Prob. 13.1 (20)& 13.2(20) (Hand

Calculations) Prob. 13.5 (30) & 13.7(30) (Hand

Calculation and MATLAB program) You may use spread sheets for your hand You may use spread sheets for your hand

computationcomputation

Due Oct/22, 2008 Wednesday at the Due Oct/22, 2008 Wednesday at the beginning of the periodbeginning of the period