Toward a Sustainable Modern Electricity Grid: The Effects of Smart

Metering and Program Investments on Demand-Side Management

Performance in the US Electricity Sector 2009-2012

Jacqueline Corbett
Université Laval

Kathrine Wardle
Canadian Tire Corp Ltd

Chialin Chen
National Taiwan University
cchen026@ntu.edu.tw

Published: IEEE Transactions on Engineering Management

To cite please consult published version:

Corbett, J., Wardle, K. & Chen, C. (2018). Toward a Sustainable Modern Electricity Grid: The
Effects of Smart Metering and Program Investments on Demand-Side Management Performance
in the US Electricity Sector 2009-2012. IEEE Transactions on Engineering Management, 65(2),
252-263.

Acknowledgement:
This research was funded in part by Natural Sciences and Engineering Research Council
(NSERC) of Canada, Grant RGPGP/328942-2013

1
 Toward a Sustainable Modern Electricity Grid: The Effects of Smart
Metering and Program Investments on Demand-Side Management
Performance in the US Electricity Sector 2009-2012

Abstract
Building a sustainable modern electricity sector is a key part of the United Nations’ 2030
Sustainable Development Goals. Smart meters are expected to contribute to the achievement of
this goal by providing new demand-side management opportunities for utilities. Attempting to
address existing gaps in the extant literature, this paper examines the effects of metering
technologies and demand-side management program investments using panel data from 87
American electricity utilities for the period 2009-2012. Our model provides strong explanatory
power with respect to energy efficiency, and shows that automated meter reading (AMR)
devices, advanced metering infrastructure (AMI) devices, direct program costs and incentive
costs all have a positive influence on energy efficiency effects. In contrast, the results for load
management are largely insignificant, with the exception of incentive costs which has a small,
but significant, negative impact on peak load reductions. We discuss the implications and
potential research directions arising from these findings, in particular the need to consider
interactions between different demand-side programs and the influence of different types of
information technologies that are emerging as part of the smart grid.

Managerial Relevance Statement

Panel data analysis is performed to investigate the impacts of smart metering and program
financial investments on energy demand management by the US electricity sector. Unlike most
existing studies on smart grid implementation which utilize cross-sectional data, the use of panel
data in this research allow for the identification of impacts of the smart grid technology by
observing the performances of the same utilities over time. Decision makers can then input their
specific characteristics, such as numbers of different meters types and their level of investment,
to understand and determine what benefits could be expected given a certain level of technology
implementation or financial investment. Our research also leads to a number of analytical results
which are different from those in the existing literature. In particular, we show that automated
meter reading devices, advanced metering infrastructure devices, direct program costs and
incentive costs all have a positive influence on energy efficiency effects, but not on load
reductions. We also discuss the importance for decision makers to consider the interactions
between different types of demand-side initiatives in conjunction with transformations occurring
on the supply-side as well as the continuing innovation in smart grid technologies in use by
intermediary organizations and consumers.

Key Words: Smart Grid, Green IT, Energy System Management, Demand-Side Management,
Sustainability

2
1. Introduction

Much like clean air to breath and water to drink, energy has become an essential resource of our

modern society, so much so that the United Nations identified “access to affordable, reliable,

sustainable and modern energy for all” as one of the 17 Sustainable Development Goals (SDGs)

for 2030 [1]. Electricity is one of the most important energy types and it continues to be the fast-

growing form of end-use energy consumption. Global demand for electricity in the residential

sector is projected to increase 1.4% annually to 2040 [2]. Beyond simply meeting increasing

demands, the electricity sector has much work to do in order to achieve the SDGs, particularly in

relation to sustainability and climate change. In 2014, 30% of the United States’ greenhouse gas

emissions came from the electricity sector as about 67% of its electricity production was

generated primarily from fossil fuels, such as coal and natural gas [3]. Attaining the goal of

sustainable electricity depends in part on the achievement a modernized electricity sector, or

‘smart’ grid. The smart grid is the next generation electricity grid that “enables bidirectional

flows of energy and uses two-way communication and control capabilities that will lead to an

array of new functionalities and applications” [4]. It is estimated that the electricity industry

spent $18 billion on deploying smart grid technology in the United States from 2010 to 2013 [5],

and the development of the smart grid has raised expectations that the transformation needed to

achieve national energy security goals in conjunction with global sustainability goals is possible.

Smart grid strategies to improve the sustainability of the electricity sector can target both

supply-side (e.g., generation) and demand-side (e.g., consumption) activities, thus allowing the

smart grid to play a significant role in a low-carbon future [6]. In regards to the former, the smart

grid can support the integration of distributed generation and more variable energy sources, such

as wind and solar, while maintaining system reliability. In the case of the latter, information and

3
communications technologies associated with the smart grid can facilitate changes in energy

consumption. Recent developments of the smart grid allow demand-side management (DSM)

techniques to be implemented more effectively and permit the use of new strategies [7]. These

techniques encourage the reduction of the total electricity demand and contribute to smoothing

peak load curves, both of which will contribute to lower carbon emissions [8].

Among the different information and communications technologies involved in bringing

the smart grid to fruition, smart meters have received a significant amount of attention [e.g., 9,

10-12] as they represent one of the first and most tangible signs of the smart grid. There were

58.5 million smart meters in place in the United States in 2014 [13], and globally, it is estimated

that 800 million smart meters will be installed by 2020 [14], making them an important

preoccupation for the industry. The relationship between smart meters and improved DSM is

complex and multifaceted. In general, the goal is to use smart meters to reduce electricity

consumption at an overall level, as well as during critical peak load periods [15], without

significantly reducing the service experienced by customers. To these ends, utilities may use a

range of DSM strategies, from conservation and energy efficiency programs to more active

demand response initiatives. Smart meters are important enabling devices particularly when

integrated into broader energy efficiency or DSM programs and policies [16, 17]. For example,

smart meters can support utilities’ efforts by providing the technological integration, two-way

communications, and analytical and automated decision-making, so-called ‘technology

solutions’[15]. In addition, smart meters can also provide more detailed information to

consumers regarding their consumption of electricity and allow for more informed decisions.

These latter functionalities have been referred to as ‘technology-enabled behavioral solutions’

[15]. With technology-enabled behavioral solutions, the influence of smart meters complements

4
and interacts with other DSM programs put in place by utilities such as education, financial

incentives and variable pricing regimes.

Despite the perceived benefits of smart meters with respect to DSM, research findings to date

are inconclusive. A study based on cross-sectional data from U.S. utilities reveals the presence of

both positive and negative relationships between different metering technologies and energy

efficiency effects [9]. Lang [15] suggests that technology solutions may provide the largest

reductions in peak load and overall consumption because consumers only have one decision to

make (e.g., whether or not to participate), rather than having to engage in a series of decisions

over time; however, behavioral reactions to technology solutions may offset some of the benefits

[15]. Further, Dyson et al. [10] suggest that due to significant customer heterogeneity, demand

response programs should be targeted rather than implemented on mass to maximize cost

effectiveness. Given this uncertainty and the importance of building a sustainable, modern

electricity sector and the equivocality in the extant literature, additional research is warranted.

The goal of this paper is to contribute to the continued development of the smart grid by

examining the impact of smart metering technologies on utilities’ DSM performance, in relation

to both energy efficiency and load management programs. We use panel data econometrics to

perform this analysis over the years 2009 to 2012. As compared to cross-sectional analyses,

panel data allows for the identification of impacts that may arise over time as the technology is

adopted and appropriated by utilities and their customers. In addition, our analyses, which also

take into account utilities’ investments in DSM programs, provide a more comprehensive view

of the relationships between smart grid technologies, DSM program characteristics and energy

savings. For instance, our analytical results will allow a utility to determine the amount of energy

efficiency savings that may be achieved by investing an additional dollar into energy efficiency

5
programs, all other factors being equal. As new technologies become available, it will be

essential for utilities to understand these relationships so that they can effectively design and

tailor their DSM programs to achieve specific performance goals. Our findings also contribute to

academia by highlighting areas in which the benefits of smart grid technologies are present and

where they remain to be developed. In this respect, our work provides direction to engineering

and information systems scholars who will work collaboratively with the electricity sector to

achieve of an affordable, reliable, sustainable and modern energy system for all.

This paper is structured as follows. Section 2 reviews the relevant literature as a

foundation for the study. Section 3 presents the research methodology. Section 4 describes the

results of the study. Section 5 discusses the findings and their implications. To conclude, Section

presents the contributions, limitations and future research directions arising from this work.

2. Literature Review

2.1 Overview of Demand-Side Management in Electricity Sector

Demand-side management is a collection of strategies and techniques aimed at altering

“customer’s electricity consumption patterns to produce the desired changes in the load shapes of

power distribution systems” [18, p. 1245]. Passive DSM involves attempts to permanently

reduce the overall demand for electricity, without considering short-term fluctuations in demand,

while active DSM techniques are more proactive as they seek to alter short-term demand in

anticipation of changes in peak loads. Passive DSM strategies typically focus on conservation

and energy efficiency. These strategies encourage consumers to reduce their total electricity

consumption through education, or incentives such as tax rebates. Consumers may also be

encouraged to switch to more energy efficient appliances (e.g., refrigerators, furnaces) offering

the same services while demanding less electricity. Conservation and energy efficiency programs

6
have been used for decades by utilities and are relatively inexpensive, with reports suggesting

that the cost of energy efficiency programs is around $0.03/kWh [19].

Alternatively, innovations in the smart grid are permitting more active DSM approaches,

referred to as load management, or demand response programs. Demand response implies short-

term changes in the pattern of end-use electricity consumption in response to grid conditions

[10]. Demand response, such as controlling use of air conditioning equipment, can provide

significant flexibility to the grid by reducing demand during peak demand periods and shifting it

to another time [10]. In this type of program, a utility monitors aggregate electricity demand

using real-time data from automated meters and determines when demand is reaching the limits

of their supply. Then, a signal is sent to consumers to request a reduction in consumption, such

as by shutting off equipment (e.g., appliances) or postponing their use to a lower demand (off-

peak) period. Consumers may grant utilities permission to control systems or appliances (e.g.,

hot water tanks, pool pumps) directly, or they may decide on a case-by-case basis whether to

respond to a particular request by their utility. Utilities may encourage their consumers to

participate by implementing time of use (or real-time use) pricing or other financial incentives.

Active DSM strategies, such as load management, rely more strongly on information

technologies because effectively balancing supply and demands requires not only finer grain and

real-time data, but also some form of automation [20].

Although energy efficiency and demand response approaches represent two distinct

demand-side management strategies, research suggests they are complementary [21] and could

offer utilities additional benefits when used in conjunction with each other. For example,

although most demand response programs are not designed to focus on total energy

consumption, many of these programs also deliver at least small energy conservations effects

7
[22]. These effects occur in part due to the feedback mechanisms associated with demand

response programs: with proactive strategies, customers receive additional direct usage

information which increases their energy awareness and leads to permanent changes and

reductions in energy consumption [22]. Passive and active strategies can be mutually reinforcing:

on one hand, energy efficiency improvements help to reduce demand, including at peak times

and provide more flexibility for load shifting strategies [20]; alternatively, load shifting can help

to reduce overall demand [20] because some consumption activities targeted by load

management are not simply shifted, but eliminated entirely. Despite the complementary nature of

energy efficiency and load management DSM strategies, program-level coordination of passive

and active DSM strategies is still not prevalent [23].

One potential reason for the apparent lack of coordination may relate to the different

motivating factors with respect to each type of program. For utilities, the adoption of energy

efficiency is influenced more by regulatory obligations and while the adoption of demand

response is driven more by capacity needs and market conditions [21]. In addition, there exists a

large number of different and innovative options with respect to smart meter enabled services

and measurement, not all of which are applicable for all types of consumers [10, 16]. For

example, despite the potential of more active and automated DSM programs there remains

significant resistance and low adoption from residential consumers with respect to direct load

control practices [17, 20]. However, adoption of automated approaches could be improved when

appropriate complementary information and services are put in place [16, 17] and consumers

perceive that such approaches reflect an appropriate balance between passive and active energy

management, provide environmental benefits to the greater society, and present opportunities for

personal financial advantages [12]. Further, by coupling energy efficiency approaches with

8
demand response, it could be possible to increase adoption of demand response by reducing

customer discomfort and inconvenience, and by increasing the social acceptability of either

manual or automated load-shifting [20].

2.2 Demand-Side Management and Smart Grid Technologies

Investment in smart grid technology has increased significantly over the past decade. In 2009, a

$9 billion public-private investment in smart grid projects was committed under the American

Recovery and Reinvestment Act. This increased investment allowed utilities to initiate a variety

of projects and also stimulated the interest of researchers, seeking to understand the implications

of smart grid technologies on utilities’ DSM activities. Initial results suggest certain utilities have

achieved 25-37% reductions in peak demand from their pilot projects under this program [24].

Further analytical studies have sought to quantify the potential of the smart grid or to determine

models for optimizing energy savings. For example, Logenthiran et al. [18] propose a demand-

side management strategy, which uses a load shifting model formulated as a minimization

problem. A heuristic-based algorithm is used to solve this problem, which allows for the

management of a smart grid with a large number of different types of devices.

Other studies have approached the subject from the customer perspective. Khan et al.

[25] develop a generic model for residential customers to use demand-side management while

reducing their total energy costs and appliance waiting times. Conejo et al. [26] present an

optimization model that adjusts the hourly load of residential customers in response to hourly

electricity prices in order to maximize the utility to the consumer, while Saravanan [27] proposes

an algorithm that considers interactions between the utility and consumers with the objectives of

reducing the consumer costs and the average peak demand. Despite the potential for automated

routines and algorithms, Zheng et al. [28], through an agent-based simulation approach, suggest

9
that to achieve greater energy efficiency and cost savings, customers should become more

experienced smart meter users through behavioral learning, especially in the case when the use

of such technologies are mandated.

While the extant research approaches the question of DSM from different perspectives, a

few common points can be retained. First, the research suggests there is a significant potential to

use smart grid technologies to improve DSM. Second, it appears that there is some variation in

the potential impacts: there is no ‘one size fits all’ solution. Instead, utilities must be somewhat

discriminatory in their deployment of smart meters in order to maximize value. Third, the

proposed models largely focus on one category of DSM - demand response - rather than the full

range of DSM approaches.

In contrast to the range of analytical research from engineering and technology

researchers, we find fewer empirical studies examining the impact of financial and technological

investments on utilities’ DSM performance. Among the studies, Loughran and Kulick’s [29]

report on a panel data study of the effects of DSM expenditures and number of customers on

energy efficiency, using retail sales as a proxy measure for energy efficiency. They find that

from 1989 to 1999, DSM expenditures reduced electricity sales, which suggests that DSM

investments can lead to increased energy efficiency savings [29]. A similar study by Rivers and

Jaccard [30] concludes, not without controversy, that from 1990 to 2005, DSM programs did not

have an impact on overall electricity consumption in Canada. Using a similar time period, Gupta

[31] finds that from 1992 to 2008 there was a positive effect from the introduction of load

management programs, but that the overall amount of peak load reduction was very small, and

that there was substantial regional variability. Finally, Bouhou et al. [32] use a mixed regression

model including many additional factors in order to model the impact of a consumer’s

10
technology on the performance of DSM interventions. The results of this last study suggest that a

household’s relative technological state (e.g., presence of a programmable thermostat) has an

influence on DSM performance [32].

Collectively, the literature suggests that DSM program expenditures and available

technologies choices may contribute to energy efficiency and load management savings.

However, the extent of this impact may be limited or variable depending on other factors.

Further, many of these studies do not directly measure the impact of smart metering

technologies. In one exception, Corbett [9] uses cross-sectional data to examine the relationship

between metering technology and energy efficiency effects of DSM programs. In this study, both

positive and negative effects are found, depending on the type of metering technology in use

[33]. Thus, although some research has been done, there remains a substantial gap in our

understanding of the relationship between smart metering technologies, both energy efficiency

and load management types of DSM programs, and other characteristics of the DSM program,

such as program investments and expenditures.

2.3 Research Questions

Through this research, we seek to address gaps identified in the literature and extend scholarship

in this important area. Specifically, we are interested in empirically assessing the impact of

investment levels and metering technology on utilities’ DSM performance for residential

customers. Our investigation is guided by two questions: 1) what is the impact of metering

technologies on utilities’ DSM performance; and 2) does greater investment in energy efficiency

programs and load management programs lead to greater energy efficiency effects and greater

peak load reductions respectively? We believe that answers to these questions will help in

building an affordable, modern and sustainable electricity system.

11
3. Methodology

A panel study using the data from the same utilities over the four-year period from 2009 to 2012

is used for this research. Panel data, where the same units of observation are examined over

several periods of time, has several advantages over cross-sectional and time series data. By

observing the data from the same utilities, panel analysis allows for the control of unobservables

which are correlated with the regressors and are specific to certain individuals, but which are

time-invariant, or do not change over time. It also allows for the control of unobservables which

are correlated with the regressors and common to all of the individuals. Finally, panel data also

allows for the study of heterogeneous choices, and the dynamics of individual choices, without

making assumptions about aggregation. We chose to use econometric analyses because this

method has the ability to quantify relationships “on the basis of available data and using

statistical techniques, and to interpret, use and exploit the resulting outcomes appropriately” [25].

This ability to determine a quantitative relationship between variables, such as the number of

smart meters installed and the savings from an energy-efficiency program, is necessary in order

to allow utilities make decisions regarding their programs.

An example of a panel data model is shown below:

(1)

In this model, is a vector of independent regressors that explain , the dependent variable.

The subscript i represents the different individuals in the data, while the subscript t represents the

different years of data. In addition, and are independent unobservables with zero mean

which make up the error term of the model. The unobservable is called the individual effect,

or the time-invariant unobservable. This is a factor which is correlated with the regressors and is

specific to certain individuals, but which is time-invariant, or does not change over time. The

12
unobservable is called the transitory shock or the time-variant unobservable. This factor is

correlated with the regressors, but changes over time.

The above model allows us to study the effects of demand-side management on energy

efficiency and load management (as dependent variables) as a result of different forms of

metering technology implementation and financial investments (as independent variables). With

the above model, we can also control for unobservable factors (i.e., data omitted or unreported in

the EIA database used in this study), such as the specific demographic characteristics of

customers or geographic characteristics of regions served by different utilities, by observing

changes in the dependent variable over time. While it is possible to use ordinary regression

techniques on panel data, they may not be optimal. The estimates of coefficients derived from

regression may be subject to omitted variable bias – a problem that arises when there is some

unknown variable or variables that cannot be controlled for that affect the dependent variable

[34]. (Refer to Section 3.3 for the specific models used in this research.)

In our study, the fixed effects model is used to perform panel data analysis. It is noted,

however, that there are other alternative methods, such as simple linear regression and random

effects model, which can be considered for analyzing panel data. The fixed effects model

generally assumes that unobserved heterogeneity will bias the estimates (and thus we need to

control for this), while the random effects model assumes otherwise. The conventional linear

regression with OLS generally omits time/space variables and unobserved heterogeneity [35]. To

gain some insights to the right approach for our study, a simple regression with pooled OLS was

also performed, and it shows that the fixed effects model (using AReg routine) leads to higher

values of adjusted R-square than the simple regression. (Refer to the results from pooled OLS

13
analysis given in Appendix A as an online resource). The choice between the fixed effects model

and random effects model is discussed further in Section 3.2.3 where endogeneity is analyzed.

3.1 Data Description

Consistent with previous studies [e.g., 9], data to create the panel was sourced from the United

States Energy Information Administration (EIA) reports. Specifically, the data comes from the

EIA-861, the Annual Electric Power Industry Report (www.eia.gov/electricity/data/eia861/).

Form EIA-861 data are collected from approximately 2,800 utilities annually and have been

reported since 1990. Although utilities may engage in DSM across their range of customers

(residential, commercial and industrial), we chose to focus on the residential sector. In this way,

our research is consistent with and builds on existing research [e.g., 17, 20, 26, 32, 36]. It has

also been suggested that residential customers have more varied and complex reactions than

industrial or commercial customers [37]. Finally, as a matter of convenience, there is more data

available regarding residential customers than any of the other types of customers.

Data were extracted for the period of 2009 to 20121. In 2012, 789 utilities provided data

regarding DSM performance. Out of these utilities, 87 were identified as having the necessary

amount of complete data over four years regarding their DSM (energy efficiency and load

management) programs: these utilities implemented a DSM program with at least one customer

during the four-year periods. With data for each of these four years, a sample of 348

observations2 was achieved.

1
Although more recent data is available, the structure of the reports was changed substantially in 2013 making it
more difficult to ensure consistency of data across the period of study. Accordingly, we decided to use 2012 as the
terminal year.
2
Similar datasets were extracted from the EIA reports for the years 2007-2012 and 2008-2012 and examined.
Although these datasets had more observations in total, they also had more missing data, creating challenges for the
analysis. Thus the 2009-2012 dataset was selected.

14
 Relevant data points were extracted for these utilities from the EIA-861 annual reports,

including: number of customers by metering technology, the effects of energy efficiency and

load management programs, and the costs for DSM programs. In addition, descriptive data for

each of the utilities was extracted, including: utility location, total number of customers, and

ownership type.

3.1.1 Dependent Variables

For dependent variables, we sought measures that best represent a utility’s performance in

relation to the two main types of DSM programs. The EIA-861 report provides ten different

measures regarding this performance. Some of these measures include only new customers,

while some include existing programs, and others use an incremental time horizon versus an

annual time horizon. For this research we chose measures for annual and actual results, as these

seem to be the most applicable to actual performance: Annual Energy Efficiency Effects

(measured in terms of megawatt hours) for energy efficiency programs, and Annual Load

Management Actual Peak Reduction (measured in terms of megawatts) for load management

programs.

3.1.2 Independent Variables

As the type of metering technology used by utilities is in a state of transition, the EIA-861 report

includes data regarding three types of metering technologies: AMR (automated meter reading

devices), AMI (advanced metering infrastructure devices, or smart meters) and net meters. In

examining the data, we noted that there was a relatively small number of net meters deployed by

utilities as compared to AMR and AMI, and that the correlation between AMI and net meters

was very high (r = .93, p < .001). As a result, we decided to use two metering types – AMR and

AMI as independent variables in this analysis. AMR devices are meters that collect data for

15
billing and information purposes and transmit this data one way, from the customer to the

distribution utility. Technologically, AMI devices, or smart meters, are more advanced than

AMR. AMI meters measure and transmit usage data more frequently (e.g., 15 minute intervals)

and allow for two-way communication between the utility and the device.

In order to assess whether the level of financial investment impacts the performance of

DSM programs, the costs of providing energy efficiency and load management programs are

included as independent variables. The direct cost of both energy efficiency programs and load

management programs are included, which measure the actual cost of implementing these

programs. The costs are reported in thousands of dollars. These costs could involve the cost of

replacing equipment and meters, labour costs, and other costs of implementation. The incentive

costs of energy efficiency and load management programs are also included. These costs involve

the payments made to residential customers in order to create an incentive for them to participate

in these programs. These could be one-time lump sum payments or could involve a calculation of

an incentive per megawatt of electricity saved by the consumer. Finally, we include the

combined indirect cost of these two programs. Indirect costs are costs associated with either type

of DSM programs, but that are not directly relevant to the implementation of the program. These

could involve administration, marketing, or monitoring and evaluation costs. It is assumed that

the direct, incentive and indirect costs noted above represent the utility’s decision regarding the

amount of investment in these two DSM programs.

3.1.3 Control Variables

Three control factors - utility size, ownership type, and location - were included in the analysis as

they could have an influence on DSM performance. For utility size, the total number of

customers was used because it is easily understandable and relates to the independent variables

16
of number of meters. For ownership type, we followed Corbett [9] and regrouped the ten

different ownership type categories from the EIA report into two categories and created a binary

control variable to represent whether the utility is investor-owned (=1) or non-investor-owned

(=0). A final ANOVA showed a significant effect of ownership type on energy efficiency F

(1,348) =26.97, p<0.001 and load management F(1,348) = 19.88, p <.001. The third control

variable was location because different challenges and opportunities may exist for utilities across

regions due to regulatory environment and climatic conditions. A binary control variable was

created for utility location. Using the 2012 State Energy Efficiency Scorecard rankings, the

utilities in the top ten energy efficient states were coded as 1, and the utilities in all the other

states were coded as 0.3 The ANOVA comparing the means between these two groups indicated

a significant difference in the effect on energy efficiency F(1,348) = 11.88, p < 0.001. However,

there does not appear to be a significant difference in the effect on load management F(1,348) =

1.46, p = 0.2276.

3.2 Quality Assessment of Data

As the EIA-861 report data has been used in previous studies [9], we had a reasonably high

degree of confidence in the quality of the data. Still, we conducted appropriate tests to verify the

characteristics and suitability of the data.

3.2.1 Normality and Outliers

Normality was assessed through frequency statistics, histograms, and skew and kurtosis tests for

normality in the statistical package Stata. These tests showed statistically significant (p < 0.001)

3
The American Council for an Energy-Efficiency Economy (ACEEE) has created a ranking program called the
State Energy Efficiency Scorecard, and has been ranking states in terms of energy efficiency since 2006. These
rankings encompass six policy areas, and states are ranked using 30 individual metrics to determine the most energy
efficient states. The top ten states were chosen because the ACEEE splits the states into tiers of ten states each, and
thus selecting only the first tier of ten states appears to be a method of selecting the most highly favorable states for
demand-side management programs.

17
positive skew and kurtosis for all the variables as reported in Table 1. These results are not

surprising given the composition of the U.S. electricity sector: there is a large number of

relatively small utilities and a smaller number of very large utilities. Although log

transformations of data can be useful as remedies for outliers and failures of normality, they are

not universally recommended and may have limited benefits when variables have similar

distributions [38]. In addition, data transformations may result in unintended results [39] and

make the interpretability of results more difficult [40]. In light of these concerns and lacking a

theoretical basis for transforming the data, we retained the untransformed values in the analyses.

Table 1: Descriptive Statistics

s.e. s.d. s.e. s.e.

Variable Min Max Mean Skew Kurtosis
(mean) (mean) (Skew) (Kurtosis)
EFF Effects 0 6304223 219413.00 35761.93 667130.10 7.0555 0.131 57,325 0.261
LM Peak Reduction 0 808 22.97 3.69 68.84 7.0316 0.131 64,485 0.261
AMR Metering 0 1513773 180880.20 17918.09 334257.70 2.1214 0.131 3,595 0.261
AMI Metering 0 4230772 80792.09 21812.36 406904.30 8.0414 0.131 72,484 0.261
Direct Cost- EFF 0 286320 8076.95 1485.69 27715.12 8.1896 0.131 75,289 0.261
Incentive Cost- EFF 0 162995 9864.57 110159 20549.94 3.9424 0.131 20,218 0.261
Direct Cost- LM 0 51849 2411.58 328.97 6136.84 4.8471 0.131 27,920 0.261
Incentive Cost- LM 0 82146 2954.83 48636 9072.97 5.1536 0.131 33,969 0.261
Indirect Costs 0 199954 1711.57 592.03 11044.25 16.7618 0.131 301,479 0.261
Number Customers 5514 4599078 431784.70 37324.15 696272.90 3.4513 0.131 15,573 0.261
n=348

Cook’s distance was used to identify and evaluate the influence of outliers in the data. This test

measures “the aggregate change in the estimated coefficients when each observation is left out of

the estimation” [41, p. 6] which shows whether a large change would occur if the influential

cases were omitted from the regression model. Typically, data with a Cook’s distance greater

than 1.0 are considered potentially influential and worthy of further investigation [38, 42] . For

both Energy Efficiency and Load Management, two observations had a Cook’s distances greater

than 1. Upon further analyses, we decided to retain these observations in the dataset.

18
3.2.2 Collinearity

Collinearity diagnostics for an Energy Efficiency model showed that the VIF (Variance Inflation

Factor) was greater than 10 in the case of three variables, namely AMI Metering, Net Metering

and Direct Cost of Energy Efficiency programs. Variables with VIFs greater than 10 may merit

further investigation, because this suggests the variable could be considered a linear combination

of other independent variables [43]. Thus, there may be a threat of multicollinearity when the

variables AMI Metering, Net Metering, and Direct Cost of Energy Efficiency programs are

included in the model. All models created should be examined for multicollinearity, and these

variables should be considered as candidates for exclusion from Energy Efficiency models. To

reduce the potential risk associated with multicollinarily, we decided to remove the variable Net

Metering (as discussed above). However, we decided to retain the AMI Metering and Direct Cost

of Energy Efficiency Programs variables as these variables are core to answering our research

questions. The correlation matrix for the variables is shown in Table 2.

Table 2: Correlation Matrix

1 2 3 4 5 6 7 8 9 10 11 12
1 EFF Effects 1.00
2 LM Peak Reduction 0.07 1.00
3 AMR Metering 0.15 0.35 1.00
4 AMI Metering 0.87 0.00 -0.10 1.00
5 Direct Cost- EFF 0.91 0.07 0.07 0.91 1.00
6 Incentive Cost- EFF 0.77 0.11 0.27 0.61 0.75 1.00
7 Direct Cost- LM 0.15 0.18 0.38 0.13 0.14 0.26 1.00
8 Incentive Cost- LM 0.17 0.51 0.50 0.03 0.14 0.29 0.42 1.00
9 Indirect Costs 0.36 0.05 0.07 0.25 0.24 0.37 0.36 0.24 1.00
10 Number Customers 0.71 0.25 0.35 0.64 0.77 0.75 0.40 0.36 0.37 1.00
11 Ownership Type 0.27 0.23 0.44 0.17 0.27 0.40 0.37 0.30 0.14 0.52 1.00
12 Location ID 0.18 -0.06 0.03 0.12 0.19 0.21 0.03 0.05 0.08 0.05 -0.09 1.00
Notes: N = 348, EFF = Energy Efficiency, LM = Load Management

19
3.2.3 Endogeneity

Endogeneity is present when a regressor is correlated with the error term and can cause biased

parameter estimates. Endogeneity can be caused by omitted variables, measurement error, and

simultaneity [44]. Methods of controlling for endogeneity, such as the Fixed Effects model and

Random Effects model, can be used with panel data when there is the possibility of omitted

variables in the model. To choose between the two models, we first conducted the standard

Hausman test, but the difference of the two covariance matrices were not positive definite, thus

the statistic may not be accurate. To deal with this issue, we used Stata to conduct Hausman test

with the options Sigmamore/Sigmaless for both Energy Efficiency and Load Management. The

option Sigmamore specifies that the covariance matrices are based on the estimate of disturbance

variance from the efficient estimator, while the Sigmaless option specifies that the matrices are

based on the estimate from the consistent estimator. When both of these options are used, the

differences are no longer positive definite and the results are very similar. The Sigmamore and

Sigmaless test results for Energy Efficiency and Load Management are given in Appendix B (as

an online resource). From these tables, because the p-value is less than 0.05, we reject the null

hypothesis that the difference between the estimators is small and that the Random Effects

estimator is consistent. Thus, the Fixed Effects model should be selected, since the Random

Effects model is inconsistent and the individual effects appear to be correlated with the

regressors. Therefore, it is on this basis that we report our findings in this paper.

3.3 Model Descriptions

In order to test for the effects of different independent variables, hierarchical regressions are

performed. Specifically, four models are created for both Energy Efficiency Savings and Load

Management Peak Reduction as shown in Table 3. The first model includes only the control

20
variables, while in the next two models the number of customers with AMR Metering and AMI

Metering are added. This is done to help isolate the impact of different metering technologies on

DSM program performance. A positive beta indicates that adding more of the certain type of

meter improves the performance of the DSM program, while a negative beta indicates a drop in

performance if more meters are added. The fourth model adds the Direct, Incentive, and Indirect

costs for DSM programs.

Table 3: Regression Models

4. Results

4.1 Energy Efficiency Effects

As shown in Table 4, Model 4 is statistically significant and the variation in this model’s

independent variables explains 98.67% of the variation in energy efficiency savings (F=83.84,

21
p<0.001). All of the other variables are individually statistically significant. With respect to

metering technologies, the beta coefficients for both types are positive and significant.

Accordingly, this model suggests that additional AMR and AMI meters will have a small

positive impact on energy efficiency savings. For example, AMI metering’s beta of 0.649

suggests adding one additional smart metering customer will increase the utility’s energy

efficiency savings by 0.694 MWh. Similarly, the betas for direct cost and incentive costs are

positive, with direct costs having the greatest impact. The results suggest that an additional

thousand dollars of investment directly into energy efficiency programs has the largest impact,

with an increase of 4.52 MWh of savings whereas an additional thousand dollars of incentive

payments leads to an increase of 2.50 MWh of savings.

Table 4: Results of Fixed Effects Regression Analysis for Energy Efficiency

Variables Model 1 Model 2 Model 3 Model 4

Constant 368456.7*** 394532.9*** 208136.8*** 150310.1***
AMR Metering -0.1480476 0.5407673*** 0.478438***
AMI Metering 0.7795841*** 0.6494712***
Direct Cost- EFF 4.522667***
Incentive Cost- EFF 2.500524***
Indirect Costs 1.46878
F 32.92*** 17.82*** 131.61*** 83.84***
R^2 0.9648 0.9652 0.9843 0.9867
Adjusted R^2 0.9531 0.9533 0.9789 0.9819
Notes: *p<0.05, **p<0.01, ***p<0.001

4.2 Load Management Performance

The results for load management are presented in Table 5. With respect to load management

peak reduction, the variation in the independent variables in Model 4 explains 69.47% of the

variation in the dependent variable (F=3.28, p<0.01). Model 4 also shows very small positive,

but statistically insignificant betas, for the variables AMR metering and AMI metering, which

22
suggests that increasing the number of meters has almost no influence on the performance of

load management programs; however, no firm conclusions can be drawn. Unlike the results for

energy efficiency effects, incentive cost is the only investment variable that is statistically

significant in Model 4. The negative beta coefficient of the incentive cost measure suggests that

increasing the amount of incentives given to participants in load management programs will

marginally reduce program performance.

Table 5: Results of Fixed Effects Regression Analysis for Load Management

Variables Model 1 Model 2 Model 3 Model 4

Constant 20.55271* 18.01342 16.58091 19.11243
AMR Metering .0000144 0.0000197 .0000106
AMI Metering 0.00000599 7.61e-07
Direct Cost- LM -.0010206
Incentive Cost- LM -.0027917***
Indirect Costs .0006466
F 0.09 0.16 0.14 3.28**
R^2 0.6712 0.6716 0.6717 0.6947
Adjusted R^2 0.5612 0.5600 0.5584 0.5845
Notes: *p<0.05, **p<0.01, ***p<0.001

5. Discussion and Implications

Our goal in presenting this research is to contribute to the development of a modern and

sustainable ‘smart’ grid by examining the relationship between metering technologies, financial

investments in DSM programs and DSM performance. In this respect, a number of observations

are worthy of further discussion.

5.1 Impact of Metering Technologies

5.1.1 Energy Efficiency

Our first research question asked what the impact of metering technologies was on utilities DSM

performance. In the case of energy efficiency programs, the answer seems clear: the number of

23
meters – whether AMR or AMI – has a positive influence on the energy efficiency effects

associated with DSM programs. Taking a managerial perspective, perhaps this result is simply a

reflection of Peter Drucker’s well-known adage ‘you can’t manage what you can’t measure’. In

the case of energy efficiency, meters provide valuable information to utilities and electricity

consumers regarding consumption levels and the impacts of different energy efficiency

investments, allowing them to make better decisions. For utilities this may allow better decisions

with respect to the design and promotion of energy efficiency programs, whereas for consumers,

the increased information may enable them to become more engaged in utility-sponsored

programs.

That being said, the results suggest there is an additional value associated with more

advanced technologies and builds on the idea that the technological state of the home may

influence consumers’ responses to various DSM initiatives [32]. As compared to AMR, the

influence of AMI on energy efficient savings is stronger. In this respect, our results are

consistent with previous research suggesting positive relationships between smart metering and

energy efficiency [e.g., 9, 16]. To achieve these savings, both consumers and utilities play a part.

The rollout of AMI, with their advanced informational and communication capabilities, has

permitted utilities and third-parties to develop ancillary smart grid enabled services [12], such as

in-home and mobile electricity displays, as well as offer different informational resources, such

as interactive websites, informative billing, and consumption feedback [16]. Through these

services and informational resources, consumers gain access to detailed consumption details and

regular feedback that permit consumers to better understand their electricity use, make more

informed decisions about participation in utility programs, and adopt appropriate behavioral

changes [16, 45]. From the utilities’ perspective, more advanced metering technologies, such as

24
smart meters may provide additional organizational information processing capacities (e.g., data

collection, storage and diffusion) [9], to better measure the performance of various programs to

see which one work better in practice [16].

Finally, influence of AMI metering on DSM performance may have something to do with

the observation that, in response to utilities’ requests to reduce consumption, households may

seek more energy independence by reducing their reliance of the main electricity grid [46]. They

may do so by implementing more energy efficient appliances, consistent with the goals of the

DSM program and also by investing in their own small-scale generation capabilities such as

photovoltaic systems. Given our decision not to test the influence of net metering separate from

AMI metering, we cannot say for certain if this is in fact the case; however, the results point to

the need for further investigation into whether households engaging in distributed generation

(and having net metering devices) are more likely to also engage in energy efficiency programs.

5.1.2 Load Management Effects

While the answer to our first research question is quite clear for energy efficiency effects, the

results are not conclusive with respect to the relationship between metering technologies and

Peak Load reductions associated with load management DSM programs. This result was

unexpected and contrasts with previous research suggesting that the use of smart meters to

enable demand response can result in substantial electricity savings [e.g., 16]. Nevertheless, the

extant literature may also provide some clues to help make sense of these results.

First, low adoption of load management programs by utilities and their consumers may

have made it difficult to find statistical and stable relationships between the dependent variables

and load management performance. As compared to energy efficiency programs, fewer utilities

offer residential load management programs and average investments are lower. In addition,

25
these more active strategies, being relatively new, have experienced a degree of resistance from

consumers [17]. For example, research suggests that while consumers may be generally

receptive to smart grid technologies and demand shifting [17] or automation [12], many barriers

stand in the way of actual adoption. With respect to direct load management by utilities,

consumers perceive threats from smart meters in terms of loss of control and autonomy (by

allowing their utility to control various devices within their home), mistrust the energy providers’

motives behind such programs, and worry about threats to personal privacy [12]. Other factors,

such as market regulations, household practices and usage behaviors, and the informational and

technical aspects of smart meters and load management programs also influence consumers’

decisions regarding participation in load management programs [17].

Even after consumers decide to partake in load management programs, utilities face

substantial challenges in keeping them engaged and deriving benefits from the program. For

example, technology-enabled behavioral solutions (e.g., demand response) that repeatedly ask

consumers to alter their behaviors may be less effective [15] and harder to sustain [46] than other

programs requiring a single decision or intervention. This is in part due to the learning that must

take place for consumers to develop new behaviors [28]. In addition, when faced with the

prospect of automated demand response or the possibility of needing to reduce consumption at

specific periods, consumers may adopt alternative behaviors [15, 46]. In effect, in the face of

load management programs, consumers may choose instead to reduce their energy consumption

by replacing their inefficient appliances with new ones (captured in energy efficiency effects),

rather than participating actively in load shifting. When consumers do not switch their timing of

usage or do not allow utilities to adjust their appliance usage during peak periods, there would be

26
little impact on load management performance even with increased investment in these programs

by utilities.

Beyond considerations with respect to adoption and engagement, an alternative

explanation of our results is that our model fails to capture other important factors such as

customers’ willingness to shift their energy use from peak to off-peak periods. For example, a

more environmentally-conscious or cost-conscious consumer might be more likely to engage in a

load management program and truly change their behaviour by shifting their energy consumption

to off-peak periods. Further, in response to specific DSM programs, consumers may make

strategic decisions which could offset potential benefits [15] or lead them to different behaviors.

Although consumers are generally welcoming of smart grid technologies, they have stronger

preferences for energy savings and energy efficiency [6, 17], which may reduce their likelihood

of actively participating in other programs. As our objective was to examine factors within the

utilities’ control (e.g., metering technology and investments), we did not include factors related

to customers in our models, yet this could be a valuable extension to our work as we describe

further below.

Continuing with the idea that other factors may be at play, a final explanation could be

that smart meters on their own are not be sufficient to produce significant load management

benefits such as peak load reductions. In this regard, smart meters could be considered necessary,

but not sufficient enabling devices for achieving significant peak load reductions. On one hand,

residential consumers may require additional information systems and technologies, such as in-

home, always-on displays [45], home area networks [47] or mobile applications that provide

detailed consumption feedback allowing them to more easily control and shift their consumption

loads. Just as for energy efficiency programs, the degree of technology adoption within a home

27
can influence the households’ response to DSM initiatives [32], but this is more pronounced for

load management programs because it requires that consumers make repeated decisions about

whether to accept the utilities’ request for a change in consumption [15]. In the second case,

utilities may implement, either directly or indirectly, direct load control programs. For these

purer technology solutions, consumers are not required to make consumption decisions once they

enter the program. Instead, the utility can directly send signals to specific appliances (e.g., air

conditioners, water tanks, pool pumps) to shut off or reduce their electricity demand during

critical peak periods. In such an arrangement, referred to as direct-load control, the meter is

effectively by-passed because the communications occurs directly between the utility and the

appliance. Thus, the metering device is less relevant as a factor of success, particularly if the

direct load program is managed by a third party intermediary that combine and manage the load

of multiple consumers [47].

5.2 Influence of Program Investments

Our second research question concerned the impact of utilities’ financial investments on their

DSM performance. With respect to program investment costs, our findings show a positive

relationship between financial investments and DSM program success. Our research goes further

by parsing the different types of costs (i.e., direct, incentive and indirect) incurred by utilities,

showing that investments in direct costs have the greatest impact, followed by inventive costs.

This means that customers’ energy saving habits are not influenced as much by actual dollar

incentives they could receive, but are motivated by such things as advertising or awareness

campaigns. Therefore, if a utility has additional investment funds available, it may be more

effective to direct these dollars towards the direct cost of the energy efficiency program rather

than increasing the amount of incentives given to customers.

28
 With respect to load management performance, the results are less encouraging. Here,

direct and indirect costs are statistically insignificant and incentive costs have a small, but

statistically significant, negative influence on peak load reductions. We propose two potential

explanations for this result. First, there may be spillover effects between different types of DSM

programs. It is likely that residential customers who sign up for load management programs are

likely those who are most concerned about their electricity bills. Therefore, in order to save

money, they also tend to conserve their energy usage overall and partake in their utility’s energy

efficiency programs. In this respective, costs associated with promoting and incentivizing

participation in load management programs may actually have some spillover effects onto the

energy efficiency programs without it being captured as part of peak load reduction.

Second, utilities may have developed an over-capacity with respect to peak load shifting.

Two common incentive plans are sign-up payments (e.g., a stand-by fee) and flat-rate incentive

payments for participation in a program (e.g.; customers receive $5 per month for installing a

control device on a hot water heater). Utilities pay these incentives regardless of whether the

customer is called upon to shift loads during critical peak periods. Most of the incentives are

likely provided to customers with AMR or AMI meters, but according to our results (Table 5),

none of the device types has any significant effect on load management. As a result, demand

fluctuations from year to year could have caused the negative relationship between incentive cost

and load management since the metering technologies intended for load management were

underutilized. In other words, it is possible that some customers may not be called upon to

reduce their load.

In terms of over-capacity, it is also important to consider the interactions between passive

and active DSM strategies. Both energy efficiency programs and demand response can be used

29
by utilities to support capacity adequacy as a cost effective alternative to new electricity

generation [21]. However, as discussed previously, each of these programs responds to different

motivations and provides different benefits to utilities [20]. Being a dispatchable resource with

relatively short-term (non-permanent) benefits, demand response provides utilities great

flexibility, but they cannot ignore the effects of energy efficiency programs in their capacity

planning. More specifically, utilities must be attentive to the permanent reductions in peak load

that result from energy efficiency programs in order to avoid excess capacity procurement [20,

21]. In addition, utilities are advised to adopt measurement and verification procedures that

consider the possibility that consumers may engage in both types of programs [21]. Practically,

the actual peak load savings from energy efficiency programs may relatively small, but under

certain conditions, it could create over-capacity situations and confound the evaluation of load

management program performance.

5.3 Managerial and Policy Implications

A number of important managerial implications derive from this research. First, given the

explanatory power of the energy efficiency model, this model could be used to support decision-

making with respect to DSM programs. For instance, a utility could input their specific

characteristics, such as number of customers, numbers of different meters types, their location,

and their level of investment, and then determine what their energy efficiency savings would be

on average. Such a decision-support tool could be beneficial as a starting point for determining

the level of savings the utility could expect at a certain level of investment, and could help

utilities to determine whether any adjustments might need to be made in order to meet their

desired program savings. In addition, the estimates could provide a benchmark against which the

utility could track and improve its performance.

30
 Second, it will be important for utilities and policy makers to consider the interactions

between different types of DSM initiatives in conjunction with transformations occurring on the

supply-side. As discussed, it is possible that the implementation of demand response programs

may indirectly contribute to energy efficiency effects associated with these types of more passive

DSM strategies as consumers make trade-offs and choices in their behaviors responses. In

practice, this type of coordination is still not widespread [23]. Thus, rather than treating DSM

programs discreetly, utilities and policymakers may be well-advised to consider indirect effects

and common factors influencing the effectiveness and efficiency of these programs. We also

agree with other researchers [10] who have suggested that utilities may want to be more selective

in their targeting of customers for different DSM programs. Finally, it will also be important

going forward for utilities to consider changes in demand-side activities in conjunction with

supply-side activities. Increasingly, there is a link between these two sides of utility operations

[46] and a failure to take into account the influence of changing one on the other may create

problems for utilities in the future.

Third, it will be important for utilities and policy-makers to be aware of the continuing

innovation in smart grid technologies, not just in use by the utilities themselves, but also by

intermediary organizations and consumers. As the electricity sector continues to modernize, new

information technologies will inevitably be introduced by third parties that could mediate the

relationships between utilities and their customers. The advent and adoption of smart appliances

and integrated home management systems will also change the landscape of how utilities

communicate with their customers and their ability to effect desired changes in electricity

consumption behaviors.

31
6. Conclusion

6.1 Contributions

The world has set out an ambitious target of making affordable, reliable and sustainable energy

available to all by 2030. Achieving this goal will require the rapid development of a modern

‘smart’ grid. In presenting our research, we hope to contribute to ongoing advancement in this

area. In particular, our research makes two notable contributions. First, we provide an empirical

examination of the relationship between metering technologies, financial investments and DSM

performance in terms of both energy efficiency and load management. In addition, our research

uses panel data over a four year period that provides greater insight than cross-sectional studies.

To date, empirical studies have been lacking in this area and the previous is fragmented,

considering only parts of these questions. Through our work, we provide a more comprehensive

view into the interactions between these factors and their implications for success. A second

important contribution of this research relates to the managerial and policy implications that

derive from our findings. Specifically, we highlight the fact that demand-side and supply side

activities must be considered and managed more holistically by utilities, taking into account the

impact of one on the other. We also highlight the importance of being aware of the emerging

technologies. While the promise of the smart grid is to permit the development of “new

functionalities and applications” [4], this may also be one of its greatest challenges.

6.2 Limitations and Future Research

The most important limitation of this research concerns the data sample which is restricted to 78

utilities in the United States over the period from 2009 to 2012. The extent to which this data is

representative of all utilities around the world affects the generalizability of the results. Without

a doubt, different jurisdictions have different regulatory regimes, energy concerns, national

32
cultures and sustainability preoccupations that could lead to different outcomes. By considering

relevant research conducted in other countries, we believe there is a reasonably broad

applicability to our findings. That being said, future research that attempts to replicate this study

in other jurisdictions would be welcome and could provide interesting insights into the

development of smart grid on a global scale.

Second, the statistically insignificant results related to load management limit our ability

to draw any firm conclusions with respect to the relationships between technology, program

investments and peak load reductions. The results obtained for load management do not

necessarily mean that such programs are not effective, but rather that the critical explanatory

factors, whether consumers’ attitudes, household responses, or other technologies, may not have

been captured in the EIA database and our model. This is a particularly important area for future

research because the ability to manage demand in a more dynamic and real-time fashion has

been touted as one of the main benefits of smart meters. If nothing else, our research raises

questions about that potential. We encourage research that seeks to identify the technological,

organizational, individual factors that contribute to reductions in peak loads as part of load

management DSM programs. Such research could consider both pure technology solutions that

involve direct control and intelligent systems, as well as technology-enabled behavioral solutions

in which technology and consumers interact in order to achieve desired changes in consumption

patterns. This type of research will be essential for turning the promise into reality and will aide

utilities and policy-makers in the design and development of effective programs.

6.3 A Final Word

The United Nation’s Sustainable Development Goal of ensuring universal access to affordable,

reliable, sustainable and modern energy is an ambitious target that will require the engagement

33
of all stakeholders and collaboration between practice and research. Information technologies

associated with the smart grid will play a key role. Given the urgency of the goals and

increasing constraints on resources it is essential that we seek to minimize waste of all types – of

money, time, information and technologies. We hope that our research has helped to build our

collective knowledge in this area and provide concrete directions for moving forward. The job

now is to ensure that the smart grid and its related technologies and programs are mindfully

developed and implemented so as to provide positive, sustainable outcomes for all.

References

[1] United Nations. (2015, December 8). Sustainable Development Knowledge Platform.

Available: https://sustainabledevelopment.un.org/sdgs

[2] EIA. (2016, June 13). International Energy Outlook 2016. Available:

http://www.eia.gov/forecasts/ieo/exec_summ.cfm

[3] EPA. (2016, June 13). Sources of Greenhouse Gas Emissions. Available:

https://www3.epa.gov/climatechange/ghgemissions/sources.html

[4] NIST. (2016). Smart Grid. Available: www.nist.gov/smartgrid/

[5] United States Department of Energy. (2014). 2014 Smart Grid System Report: Report to

Congress. Available: http://energy.gov/sites/prod/files/2014/08/f18/SmartGrid-

SystemReport2014.pdf

[6] D. Ngar-yin Mah, J. Marius van der Vleuten, J. C. Ip, and P. R. Hills, "Governing the

transition of socio-technical systems: A case study of the development of smart grids in

Korea," Energy Policy, vol. 45, pp. 133-141, 2012.

[7] S. D. Ramchurn, P. Vytelingum, A. Rogers, and N. R. Jennings, "Putting the 'Smarts' into

the Smart Grid," Communications of the ACM, vol. 55, pp. 86-97, 2012.

34
[8] S.-I. Inage. (2010, July 11). Modeling Load shifting Using Electric Vehicles in a Smart

Grid Environment. Available:

https://www.iea.org/publications/freepublications/publication/load_shifting.pdf

[9] J. Corbett, "Using information systems to improve energy efficiency: Do smart meters

make a difference?," Information Systems Frontiers, vol. 15, pp. 747-760, 2013.

[10] M. E. H. Dyson, S. D. Borgeson, M. D. Tabone, and D. S. Callaway, "Using smart meter

data to estimate demand response potential, with application to solar energy integration,"

Energy Policy, vol. 73, pp. 607-619, 2014.

[11] R. Hartway, S. Price, and C. K. Woo, "Smart meter, customer choice and profitable time-

of-use rate option," Energy, vol. 24, pp. 895-903, 1999.

[12] K. Buchanan, N. Banks, I. Preston, and R. Russo, "The British public's perception of the

UK smart metering initiative: Threats and opportunities," Energy Policy, vol. 91, pp. 87-

97, 2016.

[13] EIA. (2016, September 28). How many smart meters are installed in the United States,

and who has them? Available: http://www.eia.gov/tools/faqs/faq.cfm?id=108&t=3

[14] Telefonica. (2014, September 28). m2m: 800 million electric smart meters to be installed

globally by 2020. Available: https://iot.telefonica.com/press/m2m-800-million-electric-

smart-meters-to-be-installed-globally-by-2020

[15] C. Lang and E. Okwelum, "The mitigating effect of strategic behavior on the net benefits

of a direct load control program," Energy Economics, vol. 49, pp. 141-148, 2015.

[16] A. Podgornick, B. Sucic, P. Bevk, and D. Stanicic, "The Impact of Smart Metering on

Energy Efficiency in Low-Income Housing in Mediterranean," in Climate-Smart

Technologies, W. L. e. a. Filho, Ed., ed Berlin: Springer-Verlag, 2013.

35
[17] M. A. R. Lopes, C. H. Antunes, K. B. Janda, P. Peixoto, and N. Martins, "The potential

of energy behaviours in a smart(er) grid: Policy implications from a Portuguese

exploratory study," Energy Policy, vol. 90, pp. 233-245, 2016.

[18] T. Logenthiran, D. Srinivasan, and T. Z. Shun, "Demand Side Management in Smart Grid

Using Heuristic Optimization," IEEE Transactions on Smart Grid, vol. 3, pp. 1244-1252,

2012.

[19] M. Curtis and A. Khare, "Energy conservation in electric utilities: An opportunity for

restorative economics at SaskPower," Technovation, vol. 24, pp. 395-402, 2004.

[20] S. J. Darby and E. McKenna, "Social implications of residential demand response in cool

temperate climates," Energy Policy, vol. 49, pp. 759-769, 2012.

[21] Y. Liu, "Demand response and energy efficiency in the capacity resource procurement:

Case studies of forward capacity markets in ISO New England, PJM and Great Britain,"

Energy Policy, vol. 100, pp. 271-282, 2017.

[22] C. King and D. Delurey, "Energy and Demand Response: Twins, Siblings or Cousins?,"

Public Utilities Fortnightly, vol. March 2005, 2005.

[23] C. Goldman, M. Reid, R. Levy, and A. Slilverstein, Coordination of Energy Efficiency

and Demand Response: Environmental Energy Technologies Division, Ernest Orlando

Lawrence Berkeley National Laboratory, 2010.

[24] DOE. (2012). Demand Reductions from the Application of Advanced Metering

Infrastructure, Pricing Programs, and Customer Based Systems - Initial Results, Smart

Grid Investment Grant Program. Available:

https://www.smartgrid.gov/files/peak_demand_report_final_12-13-2012.pdf

36
[25] M. A. Khan, J. Nadeem, A. Mahmood, Z. A. Khan, and N. Alrajeh, "A Generic

demand‐side Management Model for Smart Grid," International Journal of Energy

Research, vol. 39, pp. 954-964, 2015.

[26] A. J. Conejo, J. M. Morales, and L. Baringo, "Real-time Demand Response Model,"

IEEE Transactions on Smart Grid, vol. 1, pp. 236-242, 2010.

[27] B. Saravanan, "DSM in an area consisting of residential, commercial and industrial load

in smart grid," Frontiers in Energy, vol. 9, pp. 211-216, 2015.

[28] T. Zhang, P.-O. Siebers, and U. Aickelin, "Simulating user learning in authoritative

technology adoption: An agend based model for council-led smart meter deployment

planning in the UK," Technological Forecasting & Social Change, vol. 106, pp. 74-84,

2016.

[29] D. S. Loughran and J. Kulick, "Demand-Side Management and Energy Efficiency in the

United States," The Energy Journal, vol. 25, pp. 19-43, 2004.

[30] N. Rivers and M. Jaccard, "Electric Utility Demand Side Management in Canada," The

Energy Journal, vol. 32, pp. 93-116, 2011.

[31] P. Gupta, "Impact of Demand Side Management Programs on Peak Load Electricity

Demand in North America, 1992 to 2008," International Association for Energy

Economics, vol. 1st Quarter, pp. 34-37, 2013.

[32] N.-E. I. Bouhou, M. F. Blackhurst, and P. Torres, "An empirical analysis of joint

residential electricity efficiency gains within and across end uses: implications for

demand-side management," Ecological Economics, vol. 110, pp. 61-70, 2015.

37
[33] J. Corbett, "Designing and Using Carbon Management Systems to Promote Ecologically

Responsible Behaviors," Journal of the Association for Information Systems, vol. 14, pp.

39-378, 2013.

[34] B. H. Baltagi, Econometric Analysis of Panel Data. Chichester: John Wiley & Sons,

2013.

[35] F. Hayashi, Econometrics. Princeton, NJ: Princeton University Press, 2000.

[36] K. Herter, "Residential implementation of critical-peak pricing of electricity," Energy

Policy, vol. 35, pp. 2121-2130, 2007.

[37] K. Herter, P. McAuliffe, and A. Rosenfeld, "An exploratory analysis of California

residential customer response to critical peak pricing of electricity," Energy, vol. 32, pp.

25-34, 2007.

[38] B. G. Tabachnick and L. S. Fidell, Using Multivariate Statistics, Fifth Edition ed. Boston:

Pearson Education, Inc., 2007.

[39] L. A. Polgreen and J. M. Brooks, "Estimating Incremental Costs with Skew," Applied

Health Economics and Health Policy, vol. 10, pp. 319-329, 2012.

[40] C. Feng, H. Wang, N. Lu, T. Chen, H. He, Y. Lu, and X. M. Tu, "Log-transformation and

its implications for data analysis," Shanghai Archives of Psychiatry, vol. 26, pp. 105-109,

2014.

[41] R. Williams. (2016, July 12). Outliers. Available:

https://www3.nd.edu/~rwilliam/stats2/l24.pdf

[42] A. Field, Discovering Statistics Using SPSS. London: Sage Publications Ltd, 2005.

38
[43] X. Chen, P. B. Ender, M. Mitchell, and C. Wells. (2003). Chapter 2 - Regression

Diagnostics. Available:

http://www.ats.ucla.edu/stat/stata/webbooks/reg/chapter2/statareg2.htm

[44] W. H. Green, Econometric Analysis, Seventh ed. Upper Saddle River: Prentice Hall -

Pearson Education, 2012.

[45] Y. Riche, J. Dodge, and R. A. Metoyer, "Studying Always-On Electricity Feedback in the

Home," in SIGCHI Conference on Human Factors in Computing Systems (CHI'10),

2010, pp. 1995-1998.

[46] T. Nakada, K. Shin, and S. Managi, "The effect of demand response on purchase

intention of distributed generation: Evidence from Japan," Energy Policy, vol. 94, pp.

307-316, 2016.

[47] S. Feuerriegel, P. Bodenbenner, and D. Neumann, "Value and granularity of ICT and

smart meter data in demand response systems," Energy Economics, vol. 54, pp. 1-10,

2016.

39