International Journal of Forecasting 5 (1989) 307-319 307

North-Holland

Political and organizational influences

on the accuracy of forecasting state
government revenues 1
Stuart I. BRETSCHNEIDER
Syracuse University, Syracuse, NY 13244, USA

Wilpen L. GORR
Carnegie-Mellon University, Pittsburgh, PA 15213, USA

Gloria GRIZZLE and Earle KLAY

Florida State University, Tallahassee, FL 32306, USA

Abstract: This paper tests a general theory of the factors influencing the accuracy of state government
revenue forecasts. Besides the more familiar hypotheses on forecasting techniques and randomness of
dependent variable time series, our theory includes hypotheses on the political environment and organiza-
tional procedures used in forecasting. The primary data are from three surveys of state governments and
include percentage forecasts errors for total and sales tax revenues. The analysis uses two measures of
forecast accuracy, the mean and median absolute percentage errors. These are estimated in a linear model
that uses ordinary least squares and least absolute value regressions. The results confirm most parts of the
theoretical model, subject to the caveats of field data. Forecast accuracy increases when there are
independent forecasts from competing agencies. It increases even more when formal procedures exist to
combine competing forecasts. It decreases when outside expert advisors are used and when there is a
dominant political party or ideology. Finally, it increases when simple regression models and judgmental
methods are used as opposed to univariate time series methods or econometric models.

Keywords: Field study, Government forecasting, Revenue forecasting

1. Introduction The purpose of this paper is to investigate the

factors that influence the ability of state govern-
Revenue forecasts of tax and fee collections ment to forecast revenues accurately. We build
play an important role in government budgeting. upon a general model by Bretschneider and Gorr
They place an upper limit on total planned ex- (1987) survey research by Klay and Zingale (1980).
penditures and thus help to determine which and the Public Policy Institute of New York (1985)
to what extent public programs and services will and Klay and Grizzle (1986). Much of the previ-
be funded. ous work on the determinants of forecast accuracy
has been limited to investigating alternative fore-
casting techniques while controlling for the nature
* The authors wish to thank J. Scott Armstrong, Robert
of the time series being forecasted. Well-known
Fildes, Spyros Makridakis, Daniel Nagin, Steven Schnaars,
and William Shkurti for their comments on this paper. Any
examples include forecast experiments and com-
errors in the paper are solely the responsibility of the petitions with real data but simulated conditions
authors. (see, e.g., Nelson, 1973; Makridakis and Hibon,

0169-2070/89/$3.50 0 1989, Elsevier Science Publishers B.V. (North-Holland)

308 S. I. Bretschneider et al. / Accuracy of forecasting state government revenues

1979; Makridakis et al., 1982; Mahmoud, 1984; Exhibit 1

Fildes, 1985; Armstrong, 1983, 1985). We extend Tally of lead times between preparation of forecast and begin-
ning of fiscal year forecasted (1985).
this research to include the influence of political
and organizational variables on forecast accuracy. Lead time Number of Relative Cumulative
This requires data on forecast errors made in the (months) states frequency ( W) relative
frequency (96)
field under real conditions and across a number of
like organizations to obtain variation in the politi- 2 3 12 12
3 2 8 20
cal and organizational variables. 4 _ _ _
Section 2 reviews the literature on government 5 3 12 32
revenue forecasting, including our general model 6 9 36 68
of factors influencing forecast accuracy in section I 1 4 72
3. Section 4 provides a description of the data 8 1 4 76
9 1 4 80
used in this study. These include signed per-
10 1 4 84
centage forecast errors on sales tax receipts made 11 1 4 88
by 26 states over five budget cycles and absolute 12 1 4 92
percentage forecast errors on total revenue made 13 _ _

by 29 states over one to seven budget cycles. 14 2 8 100

Section 5 describes the analysis of empirical re- Total 25 100

sults and includes estimated models of the mean
absolute percentage error (MAPE) and median
absolute percentage error (MdAPE) criteria of termined by economic conditions and perhaps by
forecast accuracy. Section 6 concludes the paper the magnitude of the tax rate itself. Annual fore-
with a set of recommendations for improving reve- casts are thus made for the tax base and its
nue forecast accuracy in state governments and collection rate (i.e., the percentage of levied taxes
suggestions for future research. actually collected). These are multiplied together
and then multiplied by proposed or enacted tax
rates.
2. State government revenue forecasting
2.1. Forecast horizons and revenue sources
The taxes and fees collected by state govern-
ments are partly controllable. Tax rates can be Government revenue forecasts must be made
changed, tax base definitions can be modified or with a lead time of several months prior to the
exemptions made, and collection activities can be start of the budget period to allow time for the
tightened or loosened. The future value of an budget decision process. Exhibit 1 contains a tabu-
established tax base is uncontrollable, being de- lation of revenue forecast lead times (Klay and

Exhibit 2
State and local government revenue sources in millions of dollars (1985).

Source State Local

$ (millions) Percentage $ (millions) Percentage
Sales and gross receipts tax 105 23.9 21 5.2
Intergovernmental 89 20.3 138 34.3
Personal income tax 64 14.6 6 1.5
Corporate income tax 18 4.1 2 0.5
Licenses 14 3.2 0 0.1
Severance tax I 1.6 _ _
Property tax 4 0.9 100 24.9
Gift and death tax 2 0.5 _ _
Others 136 31 .o 135 33.6

Total 439 100.0 402 100.0

 XI. Bretschneider et al. / Accuracy offorecastingstate government reuenues 309

Grizzle, 1986) showing that forecasts are made 2.3. Political and organizational influences
anywhere from two months to over a year prior to
the start of the budget cycle. Sixty-eight percent of The American form of government gives rise to
all states prepare forecasts no more than six political, organizational, and procedural factors
months ahead, 20% prepare forecasts more than that affect revenue forecast accuracy. Perhaps the
six months but less than a year ahead, while the most distinctive of these factors are the balanced
remaining 12%, on biennial fiscal cycles, include budget laws of state and local governments and
forecasts a year or more before the beginning of a the separation-of-powers design of American
budget year. governments. Both factors apply pressure for ac-
Generally state governments forecast revenues curate revenue forecasts. In contrast, American
for as few as 10 or as many as 100 different business firms are believed to sometimes bias sales
sources individually (including an ‘other’ cate- forecasts high in order to set targets and motivate
gory). Results are then aggregated to produce total sales forces (see, e.g., Wildavsky, 1982; Majani,
revenue forecasts. Exhibit 2 shows national totals 1982).
for eight major categories of state and local State and local governments have balanced
government revenues (U.S. Bureau of the Census, budget requirements, unlike the federal govern-
197551984). State governments tend to rely heavily ment, so they need accurate revenue forecasts to
on sales taxes and personal income taxes, whereas determine the size of the budget. Short-run politi-
local governments rely more heavily on the prop- cal goals may be served by under- or overforecast-
erty tax. Both of these levels of government have ing; however, deficits or surpluses must be rec-
become increasingly dependent upon grants from onciled during the course of the budget cycle by
the federal government. public managers and elected officials.
Each revenue source has different generating Cassidy, Kamlet and Nagin (1989) review
factors that influence the degree to which revenue mechanisms used by state governments to protect
may be forecasted accurately. For example, sales, against revenue shortfalls; for instance, they may
personal income, and corporate income taxes tend spend ‘rainy day’ funds and unencumbered fund
to follow the business cycle, leading to a certain balances. When built-in contingencies are ex-
pattern of forecast errors (see Cassidy, Kamlet hausted or inaccessible, revenue shortfalls can be
and Nagin, 1989). In contrast, intergovernmental managed by expenditure cutbacks to a point, after
grants depend on decisions made by a higher level which temporary tax increases may be needed for
of government and are often unpredictable. large deficits. Small revenue surpluses can always
be put to good use, but large revenue surpluses
leave the residents feeling over-taxed. The conse-
2.2. Forecasting techniques quences of forecast errors can thus be severe for
both over- and underforecasts (see Shkurti and
A wide range of forecasting techniques are used Winefordner, 1989, for a case study of revenue
in state government revenue forecasting. A survey forecast errors and their consequences in the Ohio
by Gambill (1978) found that 45% of responding state government).
states made use of econometric models to do some The separation of powers in state government
or all of their revenue forecasting. Models ranged has traditionally led to rivalry between its execu-
in size from 2 to 400 equations and outside con- tive and legislative branches. While the executive
sultants either developed or maintained over 60% branch has the responsibility for initiating the
of these models. Bretschneider and Gorr (1987) budgeting process and making revenue forecasts,
performed a content analysis of responses pro- legislatures enact budgets. In the process of doing
vided in surveys by Klay and Zingale (1980) and so, legislatures are increasingly producing their
the Public Policy Institute of New York (1985). own independent forecasts. Klay and Zingale
The results indicated that states use a mixture of (1980) found that 82% of the state legislatures
methods, with 88% using simple econometric mod- surveyed made or commissioned their own reve-
els (multiple regression models), 48% using time nue forecasts. The Public Policy Institute of New
series techniques, 40% using judgment, and 30% York (1985) found that 64% of the states surveyed
using simultaneous equation econometric models. had formal legislative agencies responsible for
310 S. I. Breischneider et al. / Accuracy of forecasting state government revenues
making independent revenue forecasts. The de-
velopment of rival forecasts can challenge assump-
tions and expose political positions to debate,
thereby improving forecast accuracy.
Executive and legislative forecasts are often
combined to produce the final forecast. Klay and
Zingdale (1980) found that 30% of the states
surveyed had a formal process for combining ex-
ecutive and legislative forecasts. Klay and Grizzle
(1986) found that 64% had tried such a process.
Generally, the combination of disparate forecasts
increases forecast accuracy (see, e.g., Newbold and
Granger, 1974; Makridakis and Winkler, 1983;
Fullerton, 1989).
Partisan politics may decrease forecast accu-
racy if one party or ideology dominates govern-
ment. Political goals can distort forecasts. For
example, Ellwood (1983), Kamlet and Mowery
(1985), and Kamlet, Mowery and Su (1987) have
found evidence that macroeconomic forecasts used
by both executive and legislative branches of the
federal government are based in part on efforts to
further political goals.
3. General model of revenue forecast accuracy
Exhibit 3 presents a general model, developed
by Bretschneider and Gorr (1987), of factors in-
fluencing government revenue forecast accuracy.
Among these factors are forecasting techniques
organizational design and
used, randomness,
processes, and political climate. Indirect in-
fluences are revenue structure and economic
climate.
Randomness refers to the inherent difficulty of
forecasting a time series. Exhibit 3 shows this
Revenue structure ~~~~
-1
i J,
Forecasting
techniques used
57-j Forecast_i accuracy
Exhibit 3. Theoretical model of government
accuracy.
factor to be influenced by revenue structure and
economic conditions. As mentioned earlier, some
revenue sources are more difficult to forecast than
others, so the particular mix of revenue sources in
a particular state in part determines randomness.
Also, many revenue sources are directly dependent
on economic activity levels, so that regional and
national business cycles and trends affect the ran-
domness of state revenues.
One measure of ‘randomness’ (Makridakis and
Hibon, 1979) used in this paper is an estimation
error summary statistic, the MAPE, derived from
a simple time series model of the dependent vari-
able. We also include measures for unusual events
{tax rate or base changes) and forecast lead time.
Unanticipated events can cause relatively large
changes in the pattern of a time series, thereby
making the time series difficult to forecast (Gorr,
1986a, 1986b). Also, it is generally believed that,
in general, the further ahead the forecast, the
larger the forecast error. The hypothesis here is
thus: the following.
Hypothesis 1. State revenue forecast accuracy de-
creases as randomness increases.
There is a large body of literature on the effect
forecasting techniques have on forecast accuracy
(see, e.g., Nelson, 1973; Makridakis and Hibon,
1979; Makridakis et al., 1982; Mahmoud, 1984;
Fildes, 1985; Armstrong, 1983, 1985). Begun in
the 1970’s with case study comparisons between
econometric and time series methods, the litera-
ture expanded to include more closely controlled
experimental studies including qualitative as well
as quantitative techniques. One finding was that
econometric techniques produce more accurate
forecasts when the independent variables undergo
large changes (i.e., for large forecast horizons). But
for short-term forecasts, which probably corre-
spond to small changes in independent variables,
econometric methods have little advantage in ac-
Political climate curacy over time series methods. Given that state
revenue forecasts have horizons of one to two
years, this suggests the following hypothesis.
Organizational
design and process Hypothesis 2. Time series techniques are more
accurate than econometric techniques for state
revenue forecasting.
One data set in this paper has econometric
revenue forecast
methods categorized as being simple (single equa-
 S.I. Bretschneider et al. / Accuracy of forecasrlng safe government revenues 311

tion models) or complex (simultaneous equation hold down planned spending. At the same time,
models). Simultaneous equation models have not the Democrats may go in the opposite direction,
improved forecast accuracy (see, e.g., Armstrong, overforecasting, to justify larger-planned expendi-
1978). Thus we include the following hypothesis. tures. If there is a political imbalance (i.e., there is
a dominant ideology or political party), revenue
Hypothesis 3. Simple econometric models are as forecasts may be biased and thus inaccurate. This
accurate as, or more accurate than, complex leads to the following hypothesis.
econometric models for state revenue forecasting.
Hypothesis 5. States with a single dominant polit-
Finally, with regard to forecast techniques, since ical party or ideology are less likely to have accu-
this paper in part re-analyzes data from rate forecasts than those that have more balanced
Bretschneider and Gorr (1987), their hypothesis representation.
on qualitative techniques is retained although the
authors rejected it. The balance of power between branches of
government is maintained with checks and bal-
Hypothesis 4. Objective forecast techniques are ances. Executive and legislative roles and processes
more accurate than qualitative techniques for state for revenue forecasting lead to the following hy-
revenue forecasting. pothesis.

Much of the literature on objective versus sub- Hypothesis 6. States that have an independent
jective forecasts shows objective methods to be legislative agency for generating revenue forecasts
superior, but not unequivocally so. Huss (1985) make more accurate forecasts than those with only
found that electric utility companies’ forecasts of an executive forecast agency.
the use of electric power for the first year ahead of
electricity use were more accurate than the time For those states having both executive and
series and econometric forecasts that he made; legislative agency forecasts, there is the need
thereafter, his forecasts tended to be superior. He ultimately to develop a single forecast around
speculated that the success of utility companies which to build the state budget. This requires
with the first-year forecasts was due to the fore- some attempt at consensus-building or cooper-
casters’ knowledge of special events, such as major ation between the competing forecasters. The pro-
customers’ plans for large-scale expansion. Simi- cess of combining forecasts leads to the following
larly, it is possible that state revenue forecasters hypothesis.
have knowledge of future special events that they
use to improve forecast accuracy. Hypothesis 7. States that have formal procedures
The political climate may affect state revenue for combining legislative and executive branch
forecast accuracy. The two-party system in the forecasts have improved forecast accuracy over
United States stereotypically fosters two sets of those that do not.
priorities. In the most simplistic terms, Repub-
licans (conservatives) want to decrease spending Many states use councils of economic advisors
and Democrats (liberals) want to increase it. As and external consultants for obtaining general in-
Cassidy, Kamlet and Nogin (1989) point out, for formation on the economy as well as forecasts for
certain conditions either party may find it desira- key macroeconomic indicators used as indepen-
ble to over- or to underforecast. In addition, we dent variables in econometric revenue forecast
wish to point out that any such biases are counter- models (see, e.g., Shkurti and Winefordner, 1989).
acted by opposing parties. For example, Repub- One point of view suggests that the more indepen-
licans may overforecast to refute the need for a dent the forecasters are of the organization need-
tax increase proposed by Democrats. At the same ing forecasts, the more accurate the forecasts will
time, Democrats may underforecast to justify the be. Reasons for this are the detachment, objectiv-
need to raise additional revenues through a tax ity, and sophistication of outside experts. In con-
increase. In the absence of any proposal to in- trast to this is the empirical finding that experts
crease taxes, Republicans may underforecast to are good at evaluating current and past situations
312 XI. Bretschneider et al. / Accuracy of forecasting state government revenues

but relatively poor at forecasting the future Exhibit 4

(Armstrong, 1985). Another possibility is that Order statstics for dependent variables.
those states with the most difficult forecasting Statistic Sales tax Total revetme
problems (and therefore the lowest accuracy) are Percentage Absolute Absolute
the ones that seek the help of outside advisors. ’ error percentage percentage
Since the first of these three possibilities is at odds error error
with the other two, we will not make a directional Upper extreme 40.6 40.6 24.1
hypothesis on the effect of this factor on forecast Upper quartile 2.8 6.6 6.0
accuracy. Median -0.6 3.3 3.0
Lower quartile -4.2 1.5 1.7
Lower extreme - 31.6 0.0 0.1

4. Data N 136 136 85

Midspread 7.0 5.1 4.3
Range 72.2 40.6 24.6
The primary data used to test the hypotheses of
the previous section were taken from three surveys
of state government revenue forecasting: the Pub-
lic Policy Institute of New York (1985), Klay and normally distributed (the Kolomogorov-Smirnov
Zingale (1980), and Klay and Grizzle (1986). The test is significant at the 0.01 level) and a t-test that
first is concerned with total revenue forecasts and the mean is different from zero fails to reject the
was analyzed in Bretschneider and Gorr (1987); null hypothesis (t = -0.77). The absolute per-
thus it will not be discussed at length. The second centage error (APE) distributions of sales tax and
and third are analyzed for the first time in this total revenue forecasts are skewed to the right,
paper, using the model summarized in exhibit 3. since the forecasts are bounded below by zero,
The survey conducted by Klay and Zingale and large forecast errors occur. Note that the
focused on organizational structures and processes midspread (i.e., the difference between the 75th
involved in forecasting revenues, as well as on the and 25th fractile points) and range of the total
nature of interactions between forecasters and of- revenue distribution is smaller, as expected, than
ficials. A questionnaire was mailed to both legisla- that of the sales tax distribution.
tive and executive branches of each of the states, Exhibits 5 and 6 provide classical descriptive
and 97 responses were received after a single statistics for the variables of the total and sales tax
follow-up for nonrespondents. Klay and Grizzle’s revenue data sets, respectively. These are useful in
survey was mailed solely to state budget officials. interpreting the estimated models shown below. In
Subsequent to follow-ups to nonrespondents, usa- exhibit 5, COEFVARIATION, a measure of random-
ble responses from 28 states included sales tax ness, is the coefficient of variation of six data
forecasts and actual receipts for fiscal years points on total revenue collections for each state
1975/76 through 1984/85. between 1979 and 1984 (U.S. Bureau of the Census,
Since our data are cross-sectional, we do not 1979-1984). The four variables on forecast
use the MSE forecast accuracy criterion because it techniques used- QUALITATIVE (judgmental
would reflect the differences in magnitudes of method), TIMEsERIEs (univariate time series
revenue collections of the various states. Instead, technique), ECONSIMPLE (multiple regression mod-
we use the MAPE and MdAPE to study the els), and ECONCOMPLEX (simultaneous equation
spread of forecast errors. econometric models)-indicate the mixture of
Exhibit 4 shows order statistics for annual sales techniques used over the various component fore-
tax and total revenue forecast errors, where per- casts that are aggregated to the total revenue level.
centage error is calculated as 100 (actual - For example, a respondent who indicated that
forecast)/actual. The percentage error distribu- some revenue components were forecasted using
tion of sales tax forecast errors is approximately multiple regression models while the remaining
were done using judgment results in ECONSIMPLE
and QUALITATIVE equal to 1 and the other two
’ In this ease, the field data are deficient due to lack of variables set to 0. From the means of these varia-
randomization in the assignment of subjects to treatments. bles, we see that 49% of the respondents used
 XI. Bretschneider et al. / Accuracy o/forecasting state government revenues 313

Exhibit 5
Descriptive statistics for total revenue forecast accuracy model (n = 74)

Variable Description Mean Standard

deviation

APE Dependent variable, absolute percentage error of total revenue forecast 4.58 5.04
COEFVARIATION Coefficient of variation of total revenue 0.18 0.04
QUALITATIVE 0 = no qualitative techniques used, 0.38 0.49
1 = qualitative techniques used
TIMESERIES 0 = no time series techniques used, 0.50 0.50
1 = time series techniques used
ECONSIMPLE 0 = no simple econometric models used, 0.93 0.25
1 = simple econometric models used
ECONCOMPLEX 0 = no complex econometric models used, 0.36 0.48
1 = complex econometric models used
BALANCE Agencies producing revenue forecasts: 0.51 0.50
0 = executive branch only,
1 = executive and legislative branches
CONSENSUS Procedures for combining executive and legislative forecasts: -0.18 0.88
- 1 = no consensus process,
0 = informal process,
1= formal process
ECONADVISORS 0 = no council of economic advisors used, 0.27 0.45
1 = council of economic advisors used
IDEOLOGYDOM Scale between 0 and 1 with: 0.03 0.03
0 = balance of conservative and liberal ideologies,
1 = dominance of either conservative or liberal ideology

Exhibit 6
Descriptive statistics for sales tax receipts forecast accuracy model variables (n = 106).

Variable Description Mean Standard

deviation

APE Dependent variable, absolute percentage error of sales tax revenue forecast 6.18 7.85
FITMAPE a MAPE of fitting errors from time regression of sales tax receipts 5.82 4.11
TAXCHANGE 0 = no sales tax change, 0.26 0.44
1= sales tax base or rate change in tax year
FORECASTLEAD Time gap between forecast and start of fiscal year forecasted (months) 6.69 3.43
ECON 0 = no econometric model used, 0.90 0.31
1= econometric model used
QUESTASSUMP Propensity of officials to question assumptions about the national economy 3.50 0.67
(scale 1 to 4)
BALANCE Agencies producing sales forecasts: 0.68 0.47
0 = executive branch only,
1 = executive and legislative branches
CONSENSUS 0 = no consensus process, 0.63 0.49
1 = consensus process used to combine executive and legislative agency forecasts
COOPERATIVE 0 = no cooperative processes, 0.64 0.48
1= cooperative processes used to prepare forecasts
OUTADVISORS 0 = no outside advisors consulted, 0.20 0.40
1 = outside advisors consulted to prepare forecast
IDEOLOGYDOM Scale between 0 and 1 with: 0.03 0.04
0 = balance of conservative and liberal ideologies,
1 = dominance of either conservative or liberal ideology
PARTYDOM Ratio of democrats/republicans or republicans/democrats, whichever is > 1, 4.02 6.66
in the senate
-
a Variable used in this model is FITMAPE/lOO.
314 XI. Bretschneider et al. / Accuracy offorecastingstate gouernment revenues

judgmental methods, 50% used time series meth- same variable as in the previous data set. Finally,
ods, 25% used regression models, and 48% used PARTYDOM is an indicator of dominance by a
simultaneous equation models. party of the legislature of a state. It is the ratio of
BALANCE and CONSENSUS are indicator varia- the number of Democrats to Republicans or vice
bles for organizational design and procedures of versa depending on which ratio is larger than 1.
forecasting. BALANCE is 0 if only the executive This variable has a large mean ratio of 4.02.
branch makes the revenue forecasts and 1 if both
executive and legislative branches make indepen-
dent forecasts. CONSENSUS is 1 if there are formal 5. Analysis and results
procedures for combining executive and legislative
forecasts and is 0 otherwise. ECONADVISORS is 1 if This section presents empirical results obtained
a state uses a council of economic advisors. Fi- for the model of state revenue forecast accuracy
nally, IDEOLOGYDOM is a measure of the domi- depicted in exhibit 3. The dependent variable is
nance of the political environment by a conserva- the absolute percentage error (APE) computed
tive or liberal ideology. Wright and McGuire from a state’s forecasted and observed revenue
(1985) compiled data for this variable from the values in a given year. The independent variables,
results of various political poles run between 1974 used in simple linear model specifications, repre-
and 1982. sent the four components of the theory set forth in
Exhibit 6 contains descriptive statistics for the exhibit 3.
variables used to model sales tax forecast errors Ordinary least squares (OLS) regression esti-
below. FITMAPE, TAXCHANGE, and FORECASTLEAD mates for the total revenue and sales tax data sets
are all measures of randomness for the data set on yield results on the forecast MAPE conditioned by
sales tax forecasts. FITMAPE is the MAPE of esti- values of the independent variables. Least ab-
mation residuals from a linear time trend model solute value (LAV) estimates, obtained by means
applied to ten years (197551984) of each state’s of linear programming (Dielman and Pfaffen-
sales tax revenues (U.S. Bureau of the Census, berger, 1982) yield comparable results on the
1975-1984). TAXCHANGE with a value of 1 indi- forecast MdAPE. Exhibit 4 showed that the APE
cates a tax base or rate change during a data distributions for both the total revenue and sales
point’s year. Finally, FORECASTLEAD is the number tax data sets are skewed to the right so that the
of months between completion of a forecast and MdAPE is generally smaller than the MAPE. Bas-
the start of the fiscal year. Exhibit 1 showed the sett and Koenker (1978) have shown thaat LAV
distribution of this variable. estimates are more efficient than OLS estimates
There is little variation in the techniques used for skewed distributions; therefore, LAV-esti-
to forecast sales tax revenues. As indicated by mated coefficients will tend to be more significant
ECON, which is 1 if a state uses an econometric than those obtained from OLS.
model to forecast sales taxes, 90% of the sampled Exhibit 7 summarizes total revenue and sales
states used some kind of econometric technique. tax results estimated by OLS and LAV regressions
This is not surprising since sales tax collections in terms of individual coefficient significance tests.
are tied directly to a state’s economy. QUESTAS- Exhibit 8 reports OLS and LAV estimates for the
SUMP is an indicator, on a scale of 1 to 4, of how total revenue data set, and exhibit 9 does the same
critical a state was in questioning the applicability for the sales tax data set.
to the state level of assumptions about the na- First note in exhibit 7 that all coefficients for
tional economy. BALANCE and CONSENSUS are sim- the total revenue data set are significant at the 5
ilar to variables in the total revenue data set. Here, or 1% level except ideological dominance
outside-advisors, as represented by OUTADVISORS, (IDEOLOGYDOM) for the MAPE estimates and
are not limited to membership on a council of simultaneous equation econometric methods
economic advisors, but can be consultants and (ECONCOMPLEX) for both sets of estimates. All
other kinds of external experts. COOPERATIVE indi- significant estimates have the expected signs ex-
cates the existence of cooperative efforts such as cept for judgmental forecast methods (QUALITA-
interagency working committees through which TIVE) and time series forecast methods (TIME-
revenue forecasting is done. IDEOLOGYDOM is the SERIES). Since all independent variables are non-
 S.I. Breischneider et al. / Accuracy offorecasting state gouemment revenues 315

Exhibit 7
Summary of empirical results. a

Model component Expected Total revenue Sales tax

variables sign
MAPE MdAPE MAPE MdAPE
Randomness
COEFVARIATION + Y* Y*
FITMAPE + N Y**
TAXCHANGE + Y* Y**
FORECASTLEAD + (NJ Y**

Forecast method
QUALITATIVE +
TIMESERIES -
ECON - N Y**
ECONSIMPLE _ Y* y**
ECONCOMPLEX + N N
QUESTASSUMP _ N Y**

Organizational design and procedures

BALANCE - Y** Y** N Y*
CONSENSUS _ Y** Y** Y Y**
COOPERATE _ N Y*
ECONADVISORS ? Y* Y**
OUTADVISORS ? Y** Y**

Political environment
IDEOLOGYDOM + WI Y** Y** Y**
PARTY DOM + Y* N
a ( ) means: wrong sign, N = not significant, Y = significant at 10% level, Y * = significant at 5% level, and Y * * = significant at 1%
level.

Exhibit 8
Estimates for model on total revenue forecast accuracy (n = 74).

Model Variable Expected Mean absolute percentage Median absolute percentage

component sign error model error mode
Coefficient t-statistic P-value a Coefficient t-statistic p-value a
Intercept ? 2.73 0.78 0.436 3.33 1.96 0.027

Randomness COEFVARIATION + 25.73 2.08 0.021 8.40 1.39 0.043

Forecasting QUALITATIVE + -2.12 - 1.80 0.038 -0.87 - 1.51 0.034

techniques TIMESERIES - 5.04 3.78 0.000 1.89 2.90 0.001
ECONSIMPLE - -4.32 - 1.85 0.034 - 2.94 - 2.57 0.003
ECONCOMPLEX + 1.67 1.26 0.106 0.49 0.76 0.112

Organizational BALANCE - - 4.26 - 2.91 0.002 - 2.39 - 3.35 0.000

design CONSENSUS - - 2.42 - 2.84 0.003 - 1.68 - 4.03 0.000
procedures ECONADVISORS ? 2.56 2.03 0.023 1.84 2.99 0.001

Political IDEOLOGYDOM + -0.24 - 0.01 0.496 43.73 4.24 0.000

Adjusted R-square = 0.316 Kendall’s tau-B = 0.395

Root mean squared error = 4.17 Standard error = 2.04
F-statistic = 4.74,
p-value = 0.0001

a One-sided p-values are reported for all coefficients except the intercept.
316 S.I. Bretschneider et al. / Accuracy of forecasting state government revenues

negative, a negative coefficient signifies an im- cients are significant. All coefficients have the
provement in forecast accuracy. expected signs except for FORECASTLEAD which is
Thus, the results show a reversal in expected not significant. OUTADVISORS, like ECONOADVISORS
effects of time series and judgmental methods: for the total revenue model, shows decreased fore-
states using time series methods have lower fore- cast accuracy associated with the use of outside
cast accuracy than those not doing so, and the experts. Overall, exhibit 7 provides support for all
opposite is true for judgmental methods. One components of our theoretical model: random-
former state revenue forecaster suggested that time ness, forecast method, organizational design and
series methods are used for revenue sources that procedures, and political environment are factors
cannot be otherwise modeled (i.e., these methods that influence revenue forecast accuracy in pre-
are applied to situations difficult to forecast). If dictable ways.
this is the case, then the field data are limited by For the estimated model of total revenue fore-
the lack of randomization in the assignment of cast accuracy in exhibit 8, COEFVARIATION and
subjects to treatments (i.e., self-selection) and we IDEOLOGYDOM are continuous variables, CON-
do not have results that can test the hypotheses on SENSUS is discrete, and the rest are dichotomous.
forecast techniques. In regard to the improvement The contribution of the significant continuous
in accuracy due to the use of judgmental methods, variable, COEFVARIATION, to the model is de-
we suspect that revenue forecasters have special termined by setting this variable to its average
knowledge of their states’ economies for the com- value, 0.18, from exhibit 5, and multiplying by its
ing year, as discussed in section 3. Finally, the net estimated coefficient for the MAPE model, 25.73.
effect of using councils of economic advisors This yields 4.6%. The maximum for COEFVARIA-
(ECONADVISORS) is a decrease in forecast accuracy. TION is 0.37, so the maximum impact, using a
Also note in exhibit 7 that the MdAPE results similar calculation, is to add 9.5%. (All percentage
on individual coefficient significance are equal to changes stated here are absolute changes in the
or stronger than MAPE results in all cases, as MAPE or MdAPE, and in the same percentage
predicted above. Two variables had increased sig- units as these criteria.)
nificance: the use of multiple regression forecast Qualitative techniques reduce the MAPE by
models (ECONSIMPLE) improved from 5 to 1% while 2.1%, time series techniques increase it by 5.0%,
IDEOLOGYDOM went from insignificant to 1% sig- simple econometric techniques reduce it by 4.3%,
nificance. and complex econometric techniques increase it
For the sales tax data set, the MdAPE esti- by 1.7% (which is nearly significant at the 10%
mates are much more significant than those for level). The BALANCE coefficient, representing state
the MAPE, as seen in exhibit 8. This is predictable governments having two independent and com-
from exhibit 3 because the sales tax APE distribu- peting forecast agencies, reduces the forecast
tion is more highly skewed than the total revenue MAPE by 4.3%. Moreover, if formal procedures
APE distribution. All coefficients are significant are used to combine independent forecasts, then
at the 5 or 1% level (and predominantly at the 1% the MAPE decreases another 2.4%. Finally, as to
level) and have the expected signs for the MdAPE organizational factors, if a council of economic
estimates with the exception of political party advisors is used, the MAPE increases by 2.6%. The
dominance (PARTYDOM) which is not significant. results for the MdAPE model are similar, but
In contrast, only five out of eleven coefficients are scaled closer to zero since the MdAPE is smaller
significant for the MAPE estimates. Tax rate or than the MAPE. One difference here is that
base changes (TAXCHANGE) of the three random- IDEOLOGYDOM is significant (at the 1% level) and
ness variables is significant; neither of the two so plays a role. If we set this variable to its mean
forecasting techniques coefficients is significant. value, its effect in the model is to add 1.3% to the
The use of formal procedures to combine execu- forecast MdAPE. The maximum for this variable
tive and legislative forecasts (CONSENSUS) and the is 0.16, so it can add as much as 7.0% for states
existence of outside advisors (OUTADVISORS) coef- that have a high degree of ideological dominance.
ficients of the four organizational coefficients are In exhibit 9 for the significant MAPE estimates
significant and the coefficients of both IDEOLOGY- of the sales tax model, we see first that a tax base
DOM and PARTYDOM political environment coeffi- or rate change increases the forecast MAPE by
 XI. Bretschneider et al. / Accuracy of forecasting state government reuenues 317

Exhibit 9
Estimates for model on sales tax receipts forecast accuracy (n = 106).

Model Variable Expected Mean absolute percentage Median absolute percentage

component Sign error model error model
Coefficient r-statistic p-value a Coefficient l-statistic p-value a

Intercept ? 11.38 1.85 0.067 6.39 3.97 0.000

Randomness FITMAPE + 17.91 1.00 0.160 19.25 4.11 0.000

TAXCHANGE + 3.69 2.17 0.016 1.18 2.65 0.003
FORECASTLEAD + -0.01 - 0.04 0.486 0.19 3.00 0.001

Forecasting ECON - - 2.88 -1.11 0.134 - 1.94 - 2.87 0.002

techniques QUESTASSUMP _ - 1.59 - 1.07 0.144 -1.06 - 2.73 0.002

Organizational BALANCE _ - 1.16 - 0.59 0.277 - 0.61 -1.20 0.058

design and CONSENSUS _ - 2.77 -1.62 0.055 -1.63 - 3.63 0.000
procedures COOPERATIVE - -2.11 -1.06 0.147 -1.06 - 2.04 0.011
OUTADVISORS ? 5.60 2.58 0.006 5.33 9.39 0.000

Political IDEOLOGYDOM + 83.06 4.05 OS@0 51.35 9.59 0.000

climate PARTYDOM + 0.24 1.82 0.036 0.03 0.75 0.114

Adjusted B-square = 0.162 Kendall’s tau-B = 0.357

Root mean squared error = 7.18 Standard error = 1.88
F-statistic = 2.84,
p-value = 0.003

a One-sided p-values are reported for all coefficients except the intercept.

3.7%. (An investigation with signed percentage MdAPE by 5.3%. Finally, the mean effect of ideol-
error data shows that the forecasts errors during ogy and legislature party dominance is to increase
years with such changes do not have a tendency to the MdAPE by 1.5 and O.l%, respectively. Using
be over or under forecasts.) As to organizational maximum values observed for these variables (0.16
variables, a state must have two independent fore- and 38) yields increases of 8.2 and l.l%, respec-
cast agencies as well as formal procedures for tively.
combining forecasts to have a significant effect,
which is to lower the MAPE by 2.8%.
The MdAPE results in exhibit 9 are more inter- 6. Conclusion
esting than those for the MAPE. First, all ran-
domness variables are significant. Setting FITMAPE Some caution needs to be exercised in using the
and FORECASTLEAD to their mean values yields results of this paper due to the limitations of field
increases to the MdAPE of 1.1 and 1.38, respec- data. In particular, experimental subjects cannot
tively; setting them to their maximum values (0.16 be assigned to treatments in a cross-sectional study
and 14) yields increases of 3.2 and 2.7% respec- of state government forecasting, so some results
tively. Both the use of econometric methods and may be due to self-selection biases as opposed to
critical use of national macroeconomic indicators general behavior. For example, councils of eco-
decrease the MdAPE by 1.9 and 1 .l%, respec- nomic advisors may tend to be used by states with
tively. difficult forecasting problems, and time series
The organizational effects are as predicted. If models may be used when econometric models are
there are two independent forecast agencies, pro- not applicable. Both of these means to forecasting
cedures for combining forecasts and cooperative were shown to be positively correlated with a
arrangements such as interagency working com- decrease in revenue forecast accuracy. It is not
mittees, then the MdAPE is reduced by 6.9%. possible to say whether this is due to their inferior-
Almost all of this can be lost, however, by relying ity as a means of government revenue forecasting
on outside experts; this procedure increases the or whether they reflect self-selection bias.
318 XI. Bretschneider et al. / Accuracy of forecasting state government revenues

Given this caveat, several recommendations can examination of bias in revenue forecasts by state govern-
ments”, International Journal of Forecasting, 5, 321-331
tentatively be made. First, forecast accuracy can
(this issue).
be increased by having independent executive and
Dielman, T. and R. Pfaffenberger, 1982, “LAV (least absolute
legislative forecast agencies that have cooperative value) estimation in linear regression: A review”, TIMS
working relationships and, furthermore, are bound Studies m Munagement Science 19, 31-52.
to formal procedures for the combination of inde- Ellwood, J., 1983, “The great expectation: The Congressional
budget process in an age of decentralization”, in: L.C.
pendent forecasts. Second, a mixture of multiple
Dodd and B.J. Oppenheimer, eds., Congress Reconsidered,
regression models and qualitative methods ap-
3rd ed. (Congressional Quarterly Press, Washington, D.C.).
pears to be the most accurate technique. Time Fildes, R., 7985, “Quantitative forecasting. the state of the art:
series techniques and simultaneous equation mod- Econometric models, Journal of the Operational Research
els decrease accuracy. Finally, it appears that re- Society, 36, no. 7, 549-580.
sources should be spent on developing govern- Fullerton, Jr., T.M., 1989, “A composite approach to forecast-
ing state government revenues: Case study for the Idaho
mental expertise in revenue forecasting as opposed
sales tax”, International Journal of Forecasting, 5, 373-380
to using outside experts and councils of economic (this issue).
advisors. Gambill, J., 1978, “State use of econometric models in revenue
Our future research plans include field studies estimating: A report on a survey”, in: Proceedings of NATA
Revenue Estimating Procedures Conference (Federal Tax
on revenue forecast accuracy at the local level of
Revenue Estimating Procedures Conference, Washington,
government. Investigations are under way now
D.C.).
with samples of revenue forecast errors made by Gorr, W.L., 1986, “On the use of special event data in govern-
municipalities and school districts. There are over ment information systems”, Public Administration Reuiew,
25 000 separate municipalities, counties, and school 46 (Specral Issue on Public Management Information Sys-
districts in the U.S., so the potential for collecting tems), 532-539.
Gorr, W.L., 1986, “Special event data in shared databases”,
a great deal of data and observing wide ranges of
MIS Quarterb, 10, 239-250.
behavior is great. We suspect an additional factor Huss, W.R., 1985, “Comparative analysis of company fore-
will need to be added to our general model to casts and advanced time series techniques using annual
accommodate local government forecast behavior; electric utility energy sales data”, Journal of Forecasting, 4,
namely, a government’s level of sophistication in 211-239.
Kamlet, M.S. and D.C. Mowery, 1985, “The first decade of the
financial management and forecasting. Also, in
Congressional budget act: Legislative imitation and adapta-
local economies such unexpected events as plant tion in budgeting”, Policy Sciences, 18, 313-334.
closings can cause large forecast errors and must Kamlet, MS., D.C. Mowery and T. Su, 1987, “Whom do you
be taken into account. trust? An analysis of executive and Congressional economic
forecasts”, Journal of Policy Analysis and Management, 6,
365-384.
Klay, W.E. and G.A. Grizzle, 1986, “Revenue forecasting in
the states: New dimensions of budgetary forecasting”, pre-
References
sented at: National Conference of the American Society for
Public Administration, Anaheim, CA, April 14, 1986.
Armstrong, J.S., 1978, “ Forecasting with econometric meth- Klay, E. and J. Zingale, 1980, “Revenue estimation as seen
ods: Folklore vs. fact”, Journal of Business, 51, 5499600. from the administrative process”, in: Proceedings of NATA
Armstrong, J.S., 1983, “Relative accuracy of judgmental and Revenue Estimating Procedures Conference (Federal Tax
extrapolative methods in forecasting annual earnings”, Administrators, Washington, D.C.).
Journal of Forecasting, 2, 431-447. Mahmoud, E., 1984, “Accuracy in forecasting: A survey”,
Armstrong, J.S., 1985, Long Range Forecastmg, 2nd ed., (Wi- Journal of Forecasting, 3.
ley, New York). Makridakis, S. et al., 1982, “The accuracy of extrapolative
Bassett, Jr., G. and R. Koenker, 1978, “Asymptotic theory of (time series) methods: Results of a forecasting competition”,
least absolute error regression”, Journal of the American Journal of Forecasting, 1, 111-153.
Statistical Association, 73, 618-622. Makridakis, S. and M. Hibon, 1979, “Accuracy of forecasting:
Bozeman, B., 1987, All Organizations Are Public: Bridging An empirical investigation (with discussion)“, Journal of
Public and Priuate Organizational Theories, (Jossey-Bass, the Royal Statistical Society, Series A, 142, part 2, 91-145.
San Francisco, CA). Makridakis, S. and R.L. Winkler, 1983, “Averages of forecasts:
Bretschneider, S.I. and W.L. Gorr, 1987. “State and local Some empirical results”, Management Science, 29, 987-996.
government revenue forecasting”, in: S. Makridakis and S. Majani, B.E., 1982, “Decomposition methods for medium-term
Wheelwright, eds., The Handbook of Forecasting, 2nd ed., planning and budgeting”, in, S. Makridakis and S.
(Wiley, New York). Wheelwright, Eds., The Handbook of Forecasting: A
Cassidy, G.. M.S. Kamlet and D.S. Nagin, 1989, “An empirical Manager’s Guide (Wiley, New York).
 S.I. Bretschneider et al. / Accuracy of forecasting state government revenues 319

Nelson, C., 1973, Applied Time Series Analysis for Managerial Wilpen L. GORR is Professor of Public Policy and Manage-
Forecastrng (Holden-Day, San Francisco, CA). ment Information Systems at the School of Urban and Public
Newbold, P. and C.W.J. Granger, 1984, “Experience with Affairs of Carnegie-Mellon University. His research interests
forecasting and the combination of forecasts”, Journal of include policy evaluation research, design and application of
the Royal Statistreal Society, Series A, 142, part 2, 131-146. adaptive filters, public sector forecasting, and decision support
Public Policy Institute of New York, 1985, An Analysis of State systems. His publications appear in Management Science, Deci-
Revenue Forecasting Systems (The Institute, New York). sion Sciences, Public Admmistration Review, and other journals.
Shkurti, W.J. and D. Winefordner, 1989, “The politics of state
revenue forecasting in Ohio, 198441987: A case study and
research implications”, International Journal of Forecastmg,
5, 361-371 (this issue). Gloria GRIZZLE is Professor of Public Administration and
U.S. Bureau of the Census, 1975-1984, State Government Director of the Ph.D. program in the Department of Public
Finances (The Bureau, Washington, D.C.). Administration at Florida State University. Her research inter-
Wildavsky, A., 1982, “Budgeting as a political process”, in: ests include performance measurement, decision strategies,
F.S. Lane, Ed., Current Issues in Public Administration, 2nd budgeting theory, and policy analysis and administration. Her
ed. (St. Martin’s Press, New York). articles have appeared in Pubhc AdminIstration Review, Public
Wright, G.R., R. Erikson and J. McGuire, 1985, “Measuring Budgeting and Finance, State and Local Government Review,
state partisanship and ideology with survey data”, Journal Administration and Society, Pubhc Administration Quarterly.
of Politrcs, 47, 469-489. and Evaluation Research.

Biographies: Stuart 1. BRETSCHNEIDER is Associate Profes-

sor of Public Administration, and Associate Director of Syra- Earle KLAY is Associate Professor of Public Administration
cuse University’s Technology and Information Policy Program. in Florida State University’s Department of Public Adminis-
His research interests include financial forecasting, public tration. His research interests include policy development and
management information systems, and decision making in futures research. Professor Klay has published articles in Pub-
public organizations. His publications appear in Management lic Administration Review, Futures Research Quarterly, Pubhc
Scrence, Public Administration Revrew, International Journal of Productivity Review, Review of Public Personnel Admrnistration,
Forecastmg, Evaluation Review, and other journals. and the International Journal of Public Administration.