Forecast Error

Suresh K Jakhar, PhD

Indian Institute of Management Lucknow
 Forecast Error (FE)
 Every instance of demand has a random
component.
 A good forecasting method should capture the
systematic component of demand but not the
random component.
 The random component manifests itself in the
form of forecast error.
 Valuable information from FE
 Managers use error analysis to determine
whether the current forecasting method is
predicting the systematic component of demand
accurately.
 Overestimating/Underestimating
 All contingency plans must account for
forecasting error.
 Continue..
 Consider a mail order company with two suppliers.
The first one is in the far east and has a lead time of
two months. The second is local and can fill orders
with one week’s notice.
 Continue..
 The local supplier is more expensive than the Far
East supplier.
 The mail order company want to contract a
certain amount of contingency capacity with the
local supplier to be used if the demand exceeds
the quantity the Far East supplier provides.
 The decision regarding the quantity of local
capacity to contract is closely linked to the size of
the forecast error with a two-month lead time.
 Forecast Error
 As long as observed errors are within
historical error estimates, firms can continue
to use their current forecasting method.
 Finding an error that is well beyond historical
estimates may indicate that the forecasting
method in use is no longer appropriate or
demand has fundamentally changed.

Et = Ft – Dt
 Forecast Error
 It is important that a manager estimate the error
of a forecast made at least as far in advance as
the lead time required to take whatever action
the forecast is to be used for.
 Measures of Forecast Error
1. Mean Squared Error (MSE)
2. Mean Absolute Deviation (MAD)
3. Mean Absolute Percentage Error (MAPE)
4. Bias
5. Tracking Signal
Mean Squared Error

1 n 2
MSEn   Et
n t 1

 Using MSE is appropriate when distribution of

FE is symmetric about zero.
 The MSE can be related to the variance of the
FE. In effect, we estimate that the random
component of demand has a mean of zero and
variance of MSE.
It is good to use MSE to compare
forecasting methods if the cost of large
error is much larger than the gains
from very accurate forecasts.
 Mean Absolute Deviation(MAD):

 The absolute deviation in period t, At to be the

absolute value of the error in period t;
At  Et
 The MAD to be average of absolute deviation over
all periods, as expressed by
1 n
MADn   At
n t 1
 Mean Absolute Deviation (MAD):

 The MAD can be used to estimate the standard

deviation of random component assuming that
the random component is normally distributed.
 MAD is appropriate choice when selecting
forecasting method if the cost of a forecasting
error is proportional to the size of the error.
 Mean Absolute Percentage Error (MAPE)

 MAPE is the average absolute error as a

percentage of demand and is given by:
n
Et

t 1 Dt
100
MAPEn 
n
 The MAPE is a good measure of forecast error
when the underlying forecast has significant
seasonality and demand varies considerably from
one period to next.
 Consider a scenario in which two methods are used to
make quarterly forecasts for a product.
Method Forecast Error
M1 190 200 245 180
M2 100 120 500 100
Demand 1000 1200 4800 1100

 Compare MSE, MAD and MAPE. Which methods

would you recommend.
 Bias:
 When a forecast method stops reflecting the
underlying demand pattern, the forecast errors
are unlikely to be randomly distributed around
0.
 Bias:
 In general. One needs a method to track and
control the forecasting method.
n
bias n   Et
t 1

 The bias will fluctuate around 0 if the error is

truly random. Ideally, if we plot all the errors,
the slope of the best straight line passing
though should be 0.
 bias t
Tracking Signal TSt 
MADt

 Forecast is biased and is either:- Underforecasting (if

TS<-6) or Overforecasting (if TS>+6)
 One instance in which a large negative TS will result
occurs when demand has a growth trend and the
manager is using a forecasting method such as moving
average.
 The TS may get large (positive or negative) when
demand has suddenly dropped (as it did for many
industries in 2008-09) or increased by a significant
amount, making historical data less relevant.
 If demand has suddenly dropped, it make sense to
increase the weight on current data relative to older
data when making forecast.
 McClain (1981) recommends the “declining alpha”
method when using exponential smoothing when the
smoothing constant starts large (to give greater
weights to recent data) but then decrease over time.
 If we are aiming for a long term smoothing
constant of α = 1-ρ, a declining alpha approach
would be to start with α0 = 1 and reset the
smoothing constant as follows:

at –1 1– r
at = =
r + a t –1 1– r t

 In the long term, the smoothing constant will

converge to α = 1-ρ with the forecasting becoming
more stable over time.
 Best Smoothing Constant
 When using exponential smoothing, the value of
smoothing constant chosen has a direct impact on the
sensitivity of the forecast to recent data.
 If a manager has a good sense of underlying demand
pattern, it is best to use a smoothing constant that is no
larger than 0.2.
 In general, it is best to pick smoothing constants that
minimize the error term that a manager is most
comfortable with among MSE, MAD, and MAPE. In
the absence of preference among error terms, it is best
to pick smoothing constants that minimize the MSE.
 Measures of Forecast Error
Et = Ft – Dt
n
Et
MSEn = å Et2
1 n
å Dt
100
n t=1 MAPEn = t=1

n
n
1 n
At = Et MADn = å At biasn = å Et
n t=1
t=1

bias t
s = 1.25MAD TSt 
MADt

at –1 1– r
Declining alpha at = =
r + a t –1 1– r t