Multiple Regression

Dr. Rohit Vishal Kumar

Reader, Department of Marketing
Xavier Institute of Social Service
PO Box No 7, Purulia Road
Ranchi – 834001, Jharkhand, India
Email: rohitvishalkumar@gmail.com
 Types of Regression Models

1 Explanatory Regression 2+ Explanatory

Variable Models Variables

Simple
Simple Multiple

Non- Non-
Linear Linear
Linear Linear
 Regression Modeling Steps

1. Specify the model and estimate all

unknown parameters

2. Evaluate Model

3. Use Model for Prediction & Estimation

Model Specification
• Decide on the dependent variable

• List all potential Independent

variables
Linear Multiple Regression Model
1.Relationship between 1 dependent
& 2 or more independent variables
is a linear function

Population Population Random

Y-intercept slopes error

Yi   0   1X 1i   2 X 2i   k X ki   i
Dependent Independent
(response) (explanatory)
variable variables
 Linear Regression Assumptions
• Mean of Distribution of Error Is 0

• Distribution of Error Has Constant

Variance

• Distribution of Error is Normal

• Errors Are Independent l yy

r e mm ee l
x
EEx ttr e t
r ttaa nn t
m
IIm ppoor
 Parameter Estimation
• Step 1:
– Gather Data for all the Independent
and Dependent Variables
• Step 2:
– Estimate the Parameters using the
Least Square Method
 Estimating the Parameter
• Do it manually:
– Requires knowledge of Matrix
Manipulation of Huge Sizes
– B = (X’X)-1X’Y
• Use a Software
– MS Excel Can handle 15 independent
Variables
– No Limit on Statistical Software
Interpretation of Estimated Coefficients

1. Slope (k)
– Estimated average change in Y by k for
1 Unit Increase in Xk Holding All Other
Variables Constant

–Example:
^
• If 1 = 0.13, then Y is expected to
Increase by 0.13 for Each 1 unit
increase in X1 Given X2 X3 X4… Xn are
held constant
Interpretation of Estimated Coefficients

• 2. Constant (B0)
– The value of Y when all other Variables
are = 0

– Also Know As the “Autonomous Value” of

Y
 Evaluating Multiple Regression Models
• Examine Variation Measures

• Test Significance of Overall Model, portions of

overall model and Individual Coefficients

• Other Things that needs to be Checked:

– Check conditions of a multiple linear regression model
using Residuals
– Assess Multi-co linearity among independent variables
 Variation Measures 1
• Coefficient of Multiple Determination

• Proportion of Variation in Y ‘Explained’ by All X

Variables Taken Together

2 Explained variation SS yy  SSE SSE

R    1
Total variation SS yy SS yy
 Variation Measures 2
• Adjusted R2
• R2 Never Decreases When New X Variable Is
Added to Model (Disadvantage When Comparing
Models)

• Solution: Adjusted R2
– Each additional variable reduces adjusted R2, unless SSE
goes up enough to compensate

2  n  1  SSE SSE 2
Ra  1     1 R
 n   k  1  SS yy SSyy
 Testing Overall Significance
1. Tests if there is a Linear Relationship Between All X
Variables Together & Y

2. Hypotheses
– H0: 1 = 2 = ... = k = 0
• No Linear Relationship
– Ha: At Least One Coefficient Is Not 0
• At Least One X Variable linearly Affects Y
3. Uses F test statistic

SSR /(k ) ( SS yy  SSE )/ k R /k 2

F  
SSE /(n  k  1) SSE /  n   k  1   1  R 2  /  n   k  1 
H0
~ Fk ,nk 1