North South University (NSU) Course title: Applied Statistics Chapter: Multiple Regression

Multiple regression A statistical process by which several variables are used to predict another variable is called multiple regression. Sample multiple regression The equation y = 0 + 1 X 1 + 2 X 2 + ... + k X k is the sample multiple regression of y on X 1 , X 2 ,..., X k . Where y is the dependent variable X 1 , X 2 ,..., X k are the independent variable. 0 is the intercept made by the regression line 1 , 2 ,..., k are the regression coefficient represent the rate of change of the dependent variable per unit change in each of the independent variables when the other independent variables are held constant.

Estimating of regression coefficients Let us consider the regression model with only two independent variables. So the regression equation can be written as y = 0 + 1 X 1 + 2 X 2 The coefficient estimators can be solved for the following forms:

1 = 2 =

S y rX1 y rX1 X 2 rX 2 y
2 S X1 1 rX1 X 2

S y rX 2 y rX1 X 2 rX1 y
2 S X 2 1 rX1 X 2

) )

Multiple Regression

Where, rX1 y rX 2 y is the sample correlation between X 2 and y . rX1 X 2 is the sample correlation between X 1 and X 2 . S X1 is the sample standard deviation for X 1 . S X 2 is the sample standard deviation for X 2 . S y is the sample standard deviation for y .

0 = y 1 X 1 2 X 2 . is the sample correlation between X 1 and y .

Example Given the following set of data a) Calculate the multiple regression plane. b) Predict y when X 1 = 3.0 and X 2 = 2.7 . y 25 30 11 22 27 19 X1 3.5 6.7 1.5 0.3 4.6 2.0 X2 5.0 4.2 8.5 1.4 3.6 1.3

Solution The multiple regression plane of y on X 1 and X 2 is y = 0 + 1 X 1 + 2 X 2 (a) We know the least square estimates of the coefficients are

1 = 2 =

S y rX1 y rX1 X 2 rX 2 y S X1 1 r S y rX 2 y rX1 X 2 rX1 y SX2 1 r

2 X1 X 2

) )

2 X1 X 2

0 = y 1 X 1 2 X 2 .
Page 2 of 6 Dr. Mohd. Muzibur Rahman Professor, Department of Statistics, JU

Multiple Regression

From the data we get, X 1 = 3.1 X2 = 4 y = 22.33 So, S X1 = 2.32 S X 2 = 2.66 S y = 6.74 rX1 y = 0.75 rX 2 y = 0.432 rX1 X 2 = 0.12

1 = 2 =

6.74 0.75 0.12 ( 0.432 )

2 2.32 1 ( 0.12 ) 6.74 [ 0.432 0.12 0.75] 2 2.66 1 ( 0.12 )

= 2.57

= 1.342

0 = 22.33 2.257 3.1 ( 1.342 ) 4 = 20.7

So, the estimated multiple regression plane of y on X 1 and X 2 is y = 20.7 + 2.257 X 1 1.342 X 2 (b) When X 1 = 3 and X 2 = 2.7 , the predicted value of y is y = 20.7 + 2.257 3 1.342 2.7 = 23.85 Example: The following information has been gathered from a random sample of apartment rents in a city. We are trying to predict rent (in dollars per month) based on the size of the apartment (number of rooms) and the distance from town (in miles). Rent ($) 360 1000 450 525 350 300 Number of rooms 2 6 3 4 2 1 Distance from town 1 1 2 3 10 4

(a) Calculate the least squares equation that best relates these three variables. (b) If some one is looking for a two bedrooms apartment and 2 miles from the town, what rent should be expect to pay?

Page 3 of 6

Dr. Mohd. Muzibur Rahman Professor, Department of Statistics, JU

Multiple Regression

Solution: Our least square equation is y = 0 + 1 X 1 + 2 X 2 (a) We know the least square estimates of the coefficients are

1 = 2 =

S y rX1 y rX1 X 2 rX 2 y S X1 1 r S y rX 2 y rX1 X 2 rX1 y SX2 1 r

2 X1 X 2

) )

2 X1 X 2

0 = y 1 X 1 2 X 2 .
From the data we get, X1 = 3 X 2 = 3.5 y = 497.5 So, S X1 = 1.79 S X 2 = 3.39 S y = 258.92 rX1 y = 0.956 rX 2 y = 0.436 rX1 X 2 = 0.429

1 = 2 =

258.92 0.956 ( 0.429 ) ( 0.436 )

2 1.79 1 ( 0.429 ) 258.92 0.436 ( 0.429 ) 0.956 2 3.39 1 ( 0.429 )

= 136.28

= 2.42

0 = 479.5 136.28 3 ( 2.42 ) 3.5 = 97.14

So, the least squares equation of rent on number of rooms and distance from town is y = 97.14 + 136.28 X 1 2.42 X 2 (b) If some one is looking for a two bedrooms apartment and 2 miles from town, the expected rent would be y = 97.14 + 136.28 2 2.42 2 = 364.86

Page 4 of 6

Dr. Mohd. Muzibur Rahman Professor, Department of Statistics, JU

Multiple Regression

Example: Mr. X, owner and general manager of the campus stationary store, is concerned about the sales behavior of a CD sold at the store. He realizes that there are many factors that might help explain sales, but believes that advertising and price are major determinants. He has collected the following data: Sales (unit sold) 33 61 70 82 17 24 Advertising (no. of Ads) 3 6 10 13 9 6 Price (Tk.) 125 115 140 130 145 140

(a) Calculate the least squares equation to predict sales from advertising and price. (b) If advertising is 7 and price is Tk. 132, what sales would you predict?

Solution: Our least square equation of sales on advertising and price is y = 0 + 1 X 1 + 2 X 2 (a) We know the least square estimates of the coefficients are

1 = 2 =

S y rX1 y rX1 X 2 rX 2 y S X1 1 r S y rX 2 y rX1 X 2 rX1 y SX2 1 r

2 X1 X 2

) )

2 X1 X 2

0 = y 1 X 1 2 X 2 .
From the data we get, X 1 = 7.83 X 2 = 132.5 y = 47.83 So, S X1 = 3.545 S X 2 = 11.292 S y = 26.724 rX1 y = 0.591 rX 2 y = 0.399 rX1 X 2 = 0.362

Page 5 of 6

Dr. Mohd. Muzibur Rahman Professor, Department of Statistics, JU

Multiple Regression

1 = 1 =

26.724 0.591 0.362 ( 0.399 )

2 3.545 1 ( 0.362 ) 26.724 0.399 0.362 ( 0.591) 2 11.292 1 ( 0.362 )

= 6.38 = 1.67

0 = 47.833 6.38 7.833 ( 1.67 ) 132.5 = 219.15

So, the least squares equation of sales on advertising and price is y = 219.15 + 6.38 X 1 1.67 X 2 (b) If the number of advertising is 7 and price is Tk. 132, then the predicted sales would be y = 219.15 + 6.38 7 1.67 132 = 43.37 ; 43

Page 6 of 6

Dr. Mohd. Muzibur Rahman Professor, Department of Statistics, JU