Soft Grid 2013 2020 Big Data Utility Analytics Smart Grid PDF

A Greentech Media Company
THE SOFT GRID 2013-2020:

Big Data & Utility Analytics for Smart Grid
Research Excerpt
THE SOFT GRID 2013-2020, SAS RESEARCH EXCERPT
GTMRESEARCH APRIL 2013 2 2013, GREENTECH MEDIA INC ALL RIGHTS RESERVED
RESEARCH EXCERPT
Research Excerpt
This is an excerpt from the December 2012 GTM Research report The Soft Grid 2013-2020: Big Data and
Utility Analytics for Smart Grid. Research for this report was conducted over a six-month span and included
primary and secondary research as well as extensive interviews with both industry players and utilities.
GTM Research, a division of Greentech Media, provides critical and timely market analysis in the form of
research reports, data services, advisory services and strategic consulting. GTM Researchs analysis also
underpins Greentech Medias webinars and live events.
THE SOFT GRID 2013-2020, SAS RESEARCH EXCERPT Executive Summary And Key Findings
1. EXECUTIVE SUMMARY AND KEY FINDINGS
Analytics: Key Findings
1. While utilities like to claim that they have analytics, they really dont. Utilities tend to have last-gen business
intelligence (BI) reporting solutions that they call analytics, but that typically amount to not much more than
reporting tools or descriptive analytics (primarily based on older database architectures running SQL), as
opposed to the real-time and predictive analytics using complex event processing to which the term analytics
is now commonly understood to refer.
2. Utilities are now seeking to become more proactive in decision-making, adjusting their strategies based on
reasonable predictive views into the future, thus allowing them to side-step problems and capitalize on the
smart grid technologies that are now being deployed at scale. Predictive analytics, capable of managing
intermittent loads, renewables, rapidly changing weather patterns and other grid conditions, represent the
ultimate goal for smart grid capabilities.
3. In this report, we present a taxonomy that identies the three major domains in which analytics can aid utilities,
all of which are ripe with opportunity.
A. Enterprise analytics
B. Grid operations analytics
C. Consumer analytics
FIGURE 1-1: UTILITIES THREE PRIMARY DOMAINS FOR ANALYTICS
SOURCE: GTM RESEARCH
GENERATION
UTILITY INFRASTRUCTURE CONSUMER
TRANSMISSION
SUBSTATION
DISTRIBUTION
HOME / BUILDING /
DATA CENTERS
DISTRIBUTED GENERATION
AND STORAGE
POWER LAYER INFRASTRUCTURE
COMMUNICATION LAYER END-TO-END COMMS PLATFORM
Moving from Traditional, Historical Analytics
to Real-Time Predictive Analytics
Complete Situational Awareness
Business Intelligence (BI)
Trading with live look at the Grid
Simulation/Visualization
ENTERPRISE ANALYTICS CONSUMER ANALYTICS
Behavioral Analytics
Tiered Pricing - Trading, Selling Megawatts (DR)
Building Energy Management
Power Analytics (Load Flow)
Social Media Data Intergration
DG/EV/Microgrid Analytics
GRID OPERATIONS ANALYTICS
Grid Optimization and Operational Intelligence
Asset Management Analytics
Crisis Management Analytics
DMS Analytics
Outage Management Analytics/Fault
Detection and Correction
Weather/Location data
Mobile Workforce Management
Energy Theft
THE SOFT GRID 2013-2020, SAS RESEARCH EXCERPT Executive Summary And Key Findings
4. It is our prediction that in three years, talking about analytics without mentioning big data will be a bit like
talking about email without mentioning the internet - - the two will become intrinsically linked, with one being an
application (analytics) sitting on top of the other, the foundational layer (big data storage and processing).
5. GTM Research believes that it is high-performance analytics, such as predictive analytics, which will prove to
be the most signicant value-add in the big data age, as new data management technologies prove reliable and
fundamental, and as data storage infrastructure moves to commoditization.
Utilities Limited Experience With Analytics: Key Findings
1. Based on discussions with utility CIOs, the utility industry appears to be weary of the process of selecting and
commissioning custom products from vendors and of the consultant-heavy experience of deploying them; open-
source big data products oer a future with more exibility and lower costs.
2. The four biggest challenges for utilities in terms of having enterprise IT architectures suciently prepared for
smart grid and big data are:
A. Siloed systems that hinder easy data sharing
B. Systems integrations is no small task
C. No existing platform in place for unstructured data
D. No single platform is going to be able to handle all needs
All of these obstacles speak to the central challenge of making disparate, incompatible datasets usable and valuable
across the enterprise.
THE SOFT GRID 2013-2020, SAS RESEARCH EXCERPT The Emergence Of The Soft Grid
2. THE EMERGENCE OF THE SOFT GRID
2.1Utilities Existing and Evolving IT Architecture Challenges
It is necessary to stress that utility IT architectures are in many ways only the jump-o point when it comes to
realizing all of the benets that can be reaped via big data and analytics. In other words, smart IT architecture
has to be viewed as the gateway to a smart grid. An informal survey of utility CIO and CTOs that GTM Research
conducted conrms that very few utilities have an ocial, overarching data strategy in place today.
The deployment of smart hardware, including smart meters and distribution devices such as automatic
voltage regulators, is not only continuing, but will also accelerate as these devices become even more
aordable. However, in order to leverage the capabilities of these new devices and to implement smart grid
capabilities like dynamic pricing, grid optimization, self-healing grids and renewables integration, utilities
desperately need to turn their attention to upgrading their IT systems and architecture. To the industrys
credit, many utilities are currently engaged in this process.
As they do so, many are discovering that they have an out-of-date patchwork of legacy systems with little,
if any, architectural consistency. In the past, ad hoc point-to-point integration between pairs of applications
was sucient to handle basic needs, such as entering outage reports from customer service applications
into an outage management system, or creating an engineering work order for execution by a maintenance
crew using a mobile workforce management application. However, the age of big data could have
devastating results on utility systems if IT architectures are not suciently designed and engineered for the
level of performance and sophistication it will require.
Over the past three to ve years, utility executives have been discovering that the ad hoc and unplanned
nature of their systems threatens to block their forward progress in achieving smart grid business goals
and frankly this realization came before it was evident that big data was on the way! Every utility now
needs a pragmatic roadmap that delivers on the promise of smart grid by leveraging and integrating legacy
systems too complex to replace (over the immediate term), while putting a comprehensive plan in place to
account for big data, as well as overcome the four biggest data challenges utilities are facing (see below).
FIGURE 2-1: FOUR MAJOR IT ARCHITECTURE CHALLENGES FOR UTILITIES
1. Siloed systems that prohibit easy data sharing. Many smart grid applications are composite applications
that draw on data and functions from multiple systems.
2. Systems integration is no small task. Related to the challenge of siloed systems is the challenge of creating
the underlying architectures that allow easy data access, sharing, and collaboration between systems. It
is particularly dicult to upgrade architectures that serve as the foundation for electric grids on which
millions of customers depend.
3. No existing platform in place for unstructured data. An estimated 75% to 90% of all new data being
generated is unstructured. Utilities as a group are ill prepared for this shift, and most have not explored or
tested big data platforms in a meaningful way.
4. No single platform is going to be able to handle all needs. Companies like Facebook and Twitter have had
to constantly rebuild and update their architectures in order to meet their ever-evolving, rapidly expanding
needs. This experience likely will be applicable to the utility space, as well. Utilities must look to hybrid
architectures to integrate the totality of their smart grid systems, as well as their emerging big data needs.
Further, massive data warehouses are dicult to support over the long term; often the best data architecture
designs are those that keep master data close to the processing engine/analytics, or vice versa.
The previously published GTM Research report The Smart Utility Enterprise concluded that only a hybrid
architecture that achieves the following conditions will truly be equipped to implement a smart grid.
Separates the data management; application logic and presentation into separate layers (i.e., is multitier)
Has helper applications that surround legacy systems with new functionality
Embraces service-oriented architecture (SOA) that encapsulates application functions into modular components for reuse
Utilizes agent-based architecture for distributing intelligence to nodes like IED
Supports big and unruly (i.e., unstructured) data
Finally, the following list of suggested best practices for utilities moving into smart grid was generated by
systems integrator Accenture.
FIGURE 2-2: KEY BEST PRACTICES FOR DEVELOPING AND IMPLEMENTING SMART GRID SOLUTIONS
1. Recognize smart grid data classes and their characteristics to develop comprehensive smart grid data
management and governance capabilities.
2. Consider how data sources can support multiple outcomes via analytics and visualization to realize the
maximum value from the sensing infrastructure.
3. Consider distributed data, event processing and analytics architectures to help resolve latency, scale and
robustness challenges.
4. Consider the whole smart grid challenge when planning data management, analytics and visualization
capabilitiesnot just advanced metering infrastructureto avoid stranded investments or capability
impediment.
5. Design data architectures that leverage quality master data to match data classes and analytics/
application characteristics. A giant data warehouse is rarely maintainable.
6. Look to new tools such as complex event processing to handle challenges around processing new data
classes. Managing the new smart grid data deluge via historical transaction processing approaches is likely
not scalable.
7. Develop business process transformation plans at the same time asand in alignment withsmart grid designs.
SOURCE: ACCENTURE
THE SOFT GRID 2013-2020, SAS RESEARCH EXCERPT Big Data And Analytics
3. BIG DATA AND ANALYTICS
3.1Top 10 Smart Grid Drivers Of Big Data and Analytics
The following list identies ten drivers that will likely increase the speed at which big-data and analytics
technologies will be adopted in the utility industry.
FIGURE 3-1: TEN DRIVERS WHICH WILL MOVE UTILITES TO BIG DATA AND ANALYTICS
1. Utilities seeking ROI for advanced metering investments to justify the billions spent on AMI infrastructure.
2. The new technologies will improve the usefulness and granularity of demand-side management and
demand response programs in terms of better customer segmentation and other benets.
3. The new technologies will improve asset management in an asset-intensive industry.
4. More data and analytics will lead to better grid operations management in extreme weather, including
reduced outage times, cost savings from better SAIFA and SAIDI indexes, and fewer dissatised customers.
5. The new technologies will lead to reduced energy theft and other non-technical losses.
6. The new technologies will smooth the integration of renewables and EVs.
7. The new technologies will facilitate the use of geospatial intelligence to visualize grid operations.
8. The new technologies will ease the strain being placed on traditional business intelligence (BI) and analytic
solutions from the exponential growth of data.
9. The speed of adoption will likely increase when key stakeholders in the utility industry acknowledge that
todays utility enterprise IT architectures are not sucient to meet future needs, specically in terms of
their lack of cross-departmental data sharing capabilities.
10. New vendor technologies are driving shifts in terms of both what is aordable and what is possible.
THE SOFT GRID 2013-2020, SAS RESEARCH EXCERPT Vendor Proles And Comparative Analysis
4. VENDOR PROFILES AND COMPARATIVE ANALYSIS
4.1Introduction
In this section, we delve more deeply into the leading technology vendors oering solutions across the
various subsectors and submarkets of the soft grid space, including data storage, data infrastructure, data
management, and the growing application layer of smart grid.
2012 has presented utilities with a growing number of new oerings, including those with features such as geospatial
visualization, cloud-based solutions, cluster analysis tools, intelligent alarm ltering capabilities, and others. These
advances have generated a great deal of enthusiasm, as well as a considerable amount of confusion.
This seems like a tting analogy for where the market stands today: it has thus far been successful in
terms of creating a lot of excitement around big data and analytics, but hasnt yet been equally successful
in demonstrating either the capabilities or the business case for these new technologies. To some utility
executives, the need for analytics is clearly obvious, but it appears that the majority of industry insiders
in this traditionally change-averse industry still need a fair amount of education in order to be apprised of
current and emerging technologies. Adding to the confusion is the fact that seemingly every company in
the market even those with little applicable experience in the eld is suddenly developing or oering
analytics products. Ultimately, however, both educational eorts and ROI prove-out will need to take place
in order to spur investment and market proliferation.
However, after having spoken to dozens of utility executives on this topic, as well as conducting an
extensive survey of more than 70 North American utility executives, it is clear that interest in these
emerging technologies is now beginning to mount. Soon, it is likely that utilities will begin to adopt analytics
technologies that will allow them to become more proactive in decision-making and to adjust their strategy
based on the predictive views into the future that the technologies will facilitate. This will allow utilities
to capitalize on the smart grid technologies that are now being deployed at scale; side-step potential
problems; and better handle the steep challenges facing an industry in transition.
4.2Vendor Taxomony and Vendor Rankings
FIGURE 4-1: LEADING VENDORS IN SOFT GRID
4.2.1 Data Management and Movement Layer
The data management layer has been a focal point of this report, and along with the enterprise IT
architecture that supports it, it represents both the biggest challenge and the biggest opportunity for
todays utilities. It is abundantly clear that we are now in the big data age, but how utilities will manage
this paradigm shift remains to be seen. Across the industry, a gauntlet has been thrown down, and
upstarts springing out of the distributed processing world of Hadoop see a multi-billion-dollar market up for
grabs, as the need for real-time analytics in a world of massive, unstructured and complex data demands
performance requirements above and beyond the capabilities of the legacy relational database management
systems of yesteryear. Todays data no longer ts neatly into columns and rows, and is likely to be
generated on the terabyte- or petabyte-scale. As such, old and antiquated architectures are destined to fall.
Predictably, there are many vendors (and utilities) taking an If it aint broke, dont x it stance on the issue
of data management. Year after year, companies like Oracle continue their incremental gains in the speed
and performance of their relational database management systems. However, the emerging technologies
are not ready to fully replace their predecessors. Principal Hadoop founder Doug Cutting describes his
companys platform as augmenting and not replacing regular databases. As should be expected, Oracle,
Microsoft and others are experimenting with big-data products and platforms, but every database expert
consulted for this report cautioned that at the moment, those oerings remain immature and experimental.
The implications of data management for smart grid are vast. Having said that, however, we dont expect
utilities to begin making large bets on technologies like Hadoop in the near term, for several reasons. First,
the solution oerings are relatively young and utilities historically arent big risk-takers on new technologies.
Second, and more importantly, utilities havent yet fully grasped the true value and potential of distributed
data processing. This may be due to the fact that utilities rst foray into dealing with big data namely,
smart meter data -- has relied upon meter data management systems that are based on older relational
database management systems.
Over the past ve years, utilities chief data concern has been ensuring that smart meter data owed
reliably into their CIS/billing systems, so that the utility could ensure payement. There have been some
gestures and claims made in the industry about integrating siloed departments and building intelligent IT
enterprises, but in truth, maintaining accurate and ecient billing standards has been the leading concern.
As 2013 approaches, meter data management systems have now been proven to be reliable, and many other
concerns, including the question of the reliability of AMI networks, have largely been worked out. As a result,
utility CIOs and data experts have been freed up to focus on extending smart grid into other applications. In
undertaking this process, they will begin to re-examine how their data is architected and managed.
The immense wave of unstructured data that is coming to the grid in the near future is the real big data
challenge. Up until now, the data that utilities have had to manage has been predictable. For example,
utilities know when data from an average meter read will be sent, and roughly how big the resulting data
will be. On the other hand, it is very dicult to anticipate the deluge of data that an extreme weather event
will initiate, including inputs from DMS, OMS, integrated weather systems and other systems and sensors
related to other grid assets that may be experiencing unusual performance. As the grid starts to send
frequent status updates on all critical and non-critical assets, the only way to capture this data will likely be
with advanced big-data tools.
As such, GTM Research believes that legacy RDBMS will be unable to meet the comprehensive future needs
of the smart grid. Up until now, systems integrators and middleware players have been able to patch new
solutions onto legacy systems, but at a certain point, big data will begin to overwhelm spaghetti architecture.
FIGURE 4-2: COMPARATIVE VENDOR RANKINGS FOR THE DATA MANAGEMENT AND MOVEMENT LAYER
GTM RESEARCH
M
A
R
K
E
T

B
R
E
A
D
T
H
U
T
I
L
I
T
Y

R
E
L
A
T
I
O
N
S
H
I
P
S
F
U
T
U
R
E

N
E
E
D
S

E
X
T
E
N
S
I
B
I
L
I
T
Y
F
L
E
X
I
B
I
L
I
T
Y
S
C
A
L
A
B
I
L
I
T
Y
R
E
P
O
R
T
I
N
G

&

E
N
T
E
R
P
R
I
S
E

T
O
O
L
S
C
O
S
T
S
E
C
U
R
I
T
Y
W
E
I
G
H
T
E
D

A
V
E
R
A
G
E
SAS 3 4 5 5 5 5 5 4 4 4.44
Teradata 4 3 5 5 5 5 5 4 4 4.44
IBM 5 5 4 4 4 4 5 3 5 4.33
EMC/
Greenplum
5 5 4 4 4 3 5 3 5 4.22
Oracle 5 5 3 4 4 3 5 4 5 4.22
Cisco 5 5 4 4 3 3 4 4 5 4.11
SAP 5 5 4 4 4 3 5 3 4 4.11
Versant 3 3 5 4 5 5 4 4 4 4.11
Hortonworks 2 2 5 5 5 5 4 5 3 4.00
OSIsoft 3 3 4 4 4 5 4 4 5 4.00
Cloudera 2 2 5 5 5 5 3 5 3 3.89
Hadapt 2 2 5 5 5 5 3 5 3 3.89
5=highest score 1=lowest score
4.2.2 Analytics and Applications Layer
The analytics and applications layer covers the new and necessary solutions that vendors are bringing to the
market. In the utility/smart grid space, there are four domains that will increasingly rely on analytics: the
enterprise, grid operations (T&D), consumer-oriented oerings, and energy portfolio management and trading.
The following diagram demonstrates where three of these domains sit relative to the physical grid infrastructure.
FIGURE 4-3: UTILITIES THREE PRIMARY DOMAINS FOR ANALYTICS
GTM RESEARCH
Virtually all smart grid vendors are competing in the analytics and applications layer. It is dicult to provide an apples-to-
apples comparison of these vendors and their product and service oerings, as the solutions that each are oering are
often unique. The following list identies the leading solutions that vendors are targeting in the 2012-2015 timeframe.
FIGURE 4-4: LEADING UTILITY SMART GRID ANALYTICS FOR 2012 -2015
Geospatial and visual analytics that oer a
centralized view of multiple technologies
Outage restoration analytics
Grid optimization and power quality (including
voltage control and conservation)
Peak load management (via demand-side
management analytics) and energy portfolio
management analytics
Asset protection analytics and predictive asset
maintenance
Service quality analytics
Vegetation management analytics
Revenue protection (including theft and non-
technical loss analytics)
Analytics to correct legacy system errors (such as
CIS and MDM)
Consumer behavioral analytics (including
comparison to neighbors/peers)
Home signature and thermostat control analytics
Time-of-use pricing analytics
Renewable energy and storage analytics
GENERATION
UTILITY INFRASTRUCTURE CONSUMER
TRANSMISSION
SUBSTATION
DISTRIBUTION
HOME / BUILDING /
DATA CENTERS
DISTRIBUTED GENERATION
AND STORAGE
POWER LAYER INFRASTRUCTURE
COMMUNICATION LAYER END-TO-END COMMS PLATFORM
Moving from Traditional, Historical Analytics
to Real-Time Predictive Analytics
Complete Situational Awareness
Business Intelligence (BI)
Trading with live look at the Grid
Simulation/Visualization
ENTERPRISE ANALYTICS CONSUMER ANALYTICS
Behavioral Analytics
Tiered Pricing - Trading, Selling Megawatts (DR)
Building Energy Management
Power Analytics (Load Flow)
Social Media Data Intergration
DG/EV/Microgrid Analytics
GRID OPERATIONS ANALYTICS
Grid Optimization and Operational Intelligence
Asset Management Analytics
Crisis Management Analytics
DMS Analytics
Outage Management Analytics/Fault
Detection and Correction
Weather/Location data
Mobile Workforce Management
Energy Theft
FIGURE 4-5: COMPARATIVE VENDOR RANKINGS FOR DATA ANALYTICS AND APPLICATION LAYER
M
A
R
K
E
T

B
R
E
A
D
T
H
U
T
I
L
I
T
Y

R
E
L
A
T
I
O
N
S
H
I
P
S
I
N
D
U
S
T
R
Y
-
L
E
A
D
I
N
G

S
O
L
U
T
I
O
N
V
A
L
U
E

F
O
R

S
E
R
V
I
C
E
S
/
S
O
L
U
T
I
O
N
S
F
U
T
U
R
E

N
E
E
D
S

O
F

C
U
S
T
O
M
E
R
S

E
X
T
E
N
S
I
B
I
L
I
T
Y

O
F

F
E
A
T
U
R
E
S
S
O
P
H
I
S
T
I
C
A
T
I
O
N

O
F

A
N
A
L
Y
T
I
C
S
E
N
D

U
S
E
R

E
X
P
E
R
I
E
N
C
E
S
T
R
A
T
E
G
Y
W
E
I
G
H
T
E
D

A
V
E
R
A
G
E
SAS 4 4 5 5 5 5 5 4 4 4.60
IBM 5 5 5 4 4 4 4.5 4 5 4.53
Opower 3 5 5 5 5 4 4 4 5 4.45
Space-Time Insights 3 3 5 5 5 4 4.5 5 5 4.43
EcoFactor 3 3 4 5 5 5 4.5 5 5 4.38
GE 5 5 4.5 4 4 4 3 4 5 4.23
Siemens 5 5 4 4 4 4 4 4 4 4.20
eMeter (a Siemens co.) 3 5 5 4 4 4 3.5 4 5 4.18
Accenture 4 5 4.5 4 5 3 3 4 5 4.13
ABB/Ventyx 5 5 4.5 4 4 3 3.5 4 4 4.10
Landis+Gyr 4 5 4.5 4 4 4 3.5 4 4 4.10
Aclara 4 4.5 4.5 4 4 4 3.5 4 4 4.05
Tendril 3 4 4 4 4 4 5 4 4 4.05
Ecologic Analytics
(a Landis+Gyr company)
3 4.5 4.5 4 4 4 3.5 4 5 4.05
Silver Spring Networks 4 5 4 4 4 4 3.5 4 4 4.03
Echelon 4 4 4 4 5 4 4 4 3 4.00
DataRaker 3 3 4 4 4 4 4 4 5 3.90
Telvent
(a Schneider Electric co.)
4 5 4 4 3 4 3.5 4 3 3.83
EnerNOC 3 4 4 4 4 4 4 4 3 3.80
Itron 4 5 4.5 4 3 4 3 3 3 3.73
Tableau Software 3 2 4 4 4 4 4 4 4 3.70
Energate 3 3.5 4 4 4 4 3.5 4 3 3.68
Grid Net 3 3 3.5 4 4 4 3.5 4 4 3.65
Power Analytics 3 3 3.5 4 4 4 3 4 4 3.58
ECOtality 2 2 4 4 4 3 3.5 4 4 3.43
5=highest score 1=lowest score
THE SOFT GRID 2013-2020, SAS RESEARCH EXCERPT
FOR MORE INFORMATION ON BIG DATA & UTILITY ANALYTICS
Additional Resources
High-Performance Analytics for The Smart Grid
This white paper presents results from a survey of more than 70 North American utility executives. The
research reveals how utilities are dening, conceptualizing and understanding both big data and analytics.
The paper also explores some of the barriers utilities face in both day-to-day use and enterprisewide
adoption of analytics.
http://bit.ly/SASGTM
SAS Digital Magazine on Energy Transformation
This multi-media asset covers the hottest topics in todays energy industry, including Dodd-Frank regulatory
impacts, analytics for distribution/asset optimization, and exploration of unconventional oil and gas
resources. To download the interactive magazine type this URL into the browser: http://bit.ly/energymag
Explore SAS Visual Analytics
SAS Visual Analytics provides unique insights that allow utilities to understand how customer and market
behaviors inuence drive protable growth opportunities. With new online demos, a utility can experience
how SAS Visual Analytics provides an in-depth knowledge of customers, assets and operations. Log on for a
test drive today: http://bit.ly/utilityVA
Contact SAS
SAS is the leader in business analytics software and services. For over 35 years, our solutions have enabled
utilities to nd hidden patterns in data and create intelligence from disparate data sources for eective
decision-making. Find out more at sas.com/utilities.
Tim Fairchild
Director, SAS Global Energy Practice
Tim.Fairchild@sas.com +1.919.531.0981

Soft Grid 2013 2020 Big Data Utility Analytics Smart Grid PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Soft Grid 2013 2020 Big Data Utility Analytics Smart Grid PDF

Uploaded by

Copyright:

Available Formats

A Greentech Media Company

THE SOFT GRID 2013-2020:

You might also like