T E C H N O L O G Y W H I T E P A P E R

Power utilities supporting bulk electric systems (BES) must comply with the Critical
Infrastructure Protection (CIP) requirements specified by the North American Electric
Reliability Corporation (NERC). Specifically, network endpoints, such as routers and
switches that access communications networks at BES locations, are critical cyber assets
(CCAs) and must be protected within electronic security perimeters (ESPs).
This paper shows that MPLS-based networks provide secure, reliable, efficient, flexible
and cost-effective communication between CCAs at different BES locations, as well as
between CCAs and other smart grid elements. Even if utilities decide to take advantage
of the currently available exemption that does not require systems using non-routable
protocols to be protected within ESPs, MPLS networks can be used to emulate all
necessary non-routable protocols over a single networking infrastructure.
We provide an analysis of the current state of the NIST Smart Grid Cyber Security Strategy
and Requirements and discuss how they can be applied to MPLS endpoints in order to
satisfy the NERC CIP cyber security requirements. We also demonstrate how the ITU-T
X.805 security standard can be used to depict the compliance level of a CCA, as well as
the entire ESP.
Alcatel-Lucent offers a family of MPLS routers with a broad range of security features necessary
to provide the defense-in-depth mandated by the NERC CIP cyber security requirements.
Achieving NERC CIP
*
Compliance
with Secure MPLS Networks
A Bell Labs Memorandum
Ahmet Akyamac, Ph.D., Jayant Deshpande, Ph.D., Andrew McGee, CISSP, GREM, GCIH
* Critical Infrastructure Protection (CIP) requirements from the North American Reliability Corporation (NERC) standards (Sections
CIP-001 through CIP-009). The NERC standards are available at www.nerc.com/files/Reliability_Standards_Complete_Set_2009Dec3.pdf
Table of contents
1 1. Introduction
2 2. Reference Architecture
2 2.1 Key Definitions
2 2.2 Basic Communication Architecture
3 2.3 Extended Reference Architecture
4 3. Communication over MPLS networks
4 3.1 MPLS Architecture
5 3.2 Converged MPLS Networks
6 3.3 Additional MPLS Features
6 4. Interim NERC CIP Compliance with MPLS-based Non-Routable Protocol
8 5. MPLS is the Right Choice with or without the Exemption
8 6. ESP Security Implementation
8 6.1 Requirements Overview
9 6.2 ESP Identification and Protection
9 6.3 System Security Management
9 6.4 Technical Guidance for Compliance with NERC CIP Requirements
10 7. Using ITU-Ts X.805 Security Standard to Secure the Smart Grid
11 8. Threats to the Electronic Security Perimeter
12 9. Potential Vulnerabilities in the ESP
12 10. Mitigations for ESP Vulnerabilities and NERC CIP Compliance
14 11. Conclusions
15 12. References
16 13. Acronyms
17 Appendix A. MPLS Architecture
19 Appendix B. Additional MPLS Features
21 Appendix C. Technical Guidance for Compliance with NERC CIP Requirements
24 Appendix D. The X.805 Security Dimensions
25 Appendix E. Potential Vulnerabilities in the Power Grid
1 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
1. Introduction
The Reliability Standards for the Bulk Electric Systems of North America
1
, specified by the North
American Electric Reliability Corporation (NERC), includes requirements for Critical Infrastructure
Protection (CIP) for compliance by electric power utilities in protecting the critical cyber assets
(CCA) of bulk electric systems (BES). All hardware, software, data systems, and network elements
at bulk generation stations, transmission substations, and utility data and control centers must
comply with the NERC CIP requirements.
This paper shows that MPLS-based networks provide secure, reliable, efficient, flexible and cost-
effective communication between the CCAs at different BES locations, as well as between the
CCAs and other smart grid network elements.
NERC requirements [1] define the Electronic Security Perimeter (ESP) as a logical border
surrounding a network to which CCAs are connected and access must be controlled. In most cases,
an ESP will include CCAs at a single BES location connected over a LAN. Any system (e.g. a
router) that uses a routable protocol (such as IP) is, by definition, considered a CCA and must be
included in an ESP. Consequently, a communication system in the BES that does not use a routable
protocol would not be considered a CCA. This loosened requirement is referred to as NERC CIPs
non-routable protocol exemption (Examples of non-routable protocols include PDH/SONET,
Ethernet or Frame Relay). Therefore, networking systems providing connectivity with a non-
routable protocol can reside outside of an ESP, and are not subject to NERC CIP requirements.
We also show that MPLS networks can natively and effectively support communication over many
non-routable protocols; therefore a utility does not need to deploy multiple networks with different
non-routable protocols.
It is believed by many that NERC CIPs non-routable protocol exemption (called the exemption
throughout this paper) has been deliberately allowed by NERC to facilitate timely NERC CIP
compliance without substantial immediate investment. Future revisions of the NERC CIP requirements
may require all communication systems at a BES location to be CCAs, removing the current (implied)
exemption of systems with non-routable protocols.
In addition, this paper will show that MPLS networks facilitate secure implementation and NERC
CIP compliance with or without the exemption.
The reference architecture relevant to the NERC CIP requirements is presented in Section 2.
In Section 3, we describe key features of MPLS infrastructure and emulation of communication
protocols and services. Section 4 establishes the applicability and advantages of supporting non-
routable protocols over MPLS infrastructure, leading to compliance of the current requirements
with the exemption on non-routable protocols. Section 5 details MPLS network essentials that
support utility applications and NERC CIP compliance, even when the exemption is removed from
the specifications.
The remainder of this paper discusses the impact of removing the non-routable protocol exemption
would have on compliance requirements. An overview of the nine NERC CIP requirements is
provided in Section 6, along with guidance for satisfying the requirements technical aspects.
Section 7 describes ITU-T Standard X.805 [2] and how it can be used to measure compliance levels
of a cyber asset or entire ESP. Sections 8 and 9 lists locations, threat types, and potential
vulnerabilities to the bulk electric system. Section 10 describes countermeasures that can mitigate
those vulnerabilities. Finally, a summary and our conclusions are presented in Section 11.
For convenience, several Appendices at the end of the document present additional information on
MPLS features, X.805, and other security aspects related to the main body of the document.
1
http://www.nerc.com/files/Reliability_Standards_Complete_Set_2009Dec3.pdf
2. Reference Architecture
Before presenting the reference architecture, a few relevant terms from the NERC CIP standard [1]
are introduced.
2.1 Key Denitions
The NERC CIP requirements [1] (more correctly the Regional Reliability Organization) define a
Bulk Electric System (BES) as the electrical generation resources, transmission lines, interconnections
with neighboring systems, and associated equipment, generally operated at voltages of 100 kV or
higher. Radial transmission facilities serving only load with one transmission source are generally
not included in this definition. Thus, cyber assets at most distribution substations, and the
distribution feeders, are not covered by NERC CIP requirements.
As defined in the standards, Critical Assets are defined as facilities, systems, and equipment
which, if destroyed, degraded, or otherwise rendered unavailable, would affect the reliability or
operability of the BES. Critical Cyber Assets (CCAs) are programmable electronic devices and
communication networks including hardware, software, and data that are essential to reliable
operation of critical assets. Hardware, software, data systems, and networks at utility control
centers, bulk power stations, and transmission substations are examples of CCAs
2
. Distribution
systems, AMI systems, and their interconnection networks are not.
The Electronic Security Perimeter (ESP) is a logical border surrounding a network to which
Critical Cyber Assets are connected and access is controlled. As a practical matter, an ESP will be
confined to a physically protected building or space within. Communication links/networks
connecting discrete ESPs are not considered part of the ESPs
3
, so routers and switches in these
connecting networks are not CCAs. However, network endpoints on equipment within an ESP
functioning as access points to the ESP are considered a CCA and must be secured.
2.2 Basic Communication Architecture
The network architecture in Figure 1 illustrates concepts applicable to NERC CIP requirements.
Figure 1. Example Reference - Communication Architecture for a Bulk Power System
2 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
ESP
(Electronic Security
Perimeter)
(Utility) Data and Control Center
(Transmission) Substation Bulk Power Station
LAN
D
P
Routable Protocol ( ie, IP)
T
COMMUNICATION
NETWORK
2
See Requirement CIP-002-B.R1.2 in [1]
3
See Requirement CIP-005-B.R1.3 in [1]
The host systems (those attached to protection elements, Supervisory Control and Data Acquisition
(SCADA) systems, control and monitoring servers, etc.), switches, routers, and other communication
gear at a BES location are essential for reliable operation of the BES, thus constituting the CCAs.
These systems, with their interconnecting LAN, are bound by the ESP at that location, and are
subject to NERC CIP requirements. The CCAs within an ESP may use routable protocols for
communicating amongst themselves within the ESP.
For communication between systems in different ESPs, the routers in the corresponding ESPs may
use routable protocols between them. (e.g., between routers D and P in Figure 1)
2.3 Extended Reference Architecture
For a general smart grid communication architecture, see [3]. In a general smart grid environment,
the CCAs in the bulk electric system may need to communicate with systems in distribution
substations or other locations as illustrated in Figure 2.
Figure 2. Extending Basic Reference Architecture to Include Other Smart Grid Elements
Smart grid systems outside of the bulk electric system are mostly not considered CCAs
4
. Examples
of such systems are AMI meters at customer locations, meter concentrators/collectors, IEDs
(Intelligent Electronic Devices) and RTUs (Remote Terminal Units) of the distribution SCADA
systems. Please note that some of these systems may not necessarily be located at a distribution
substation; they may be deployed at feeder or consumer locations.
As indicated above, the communication network between distinct ESPs, and between an ESP and
outside system, is not included in any ESP thus not subject to the CIP requirements of [1].
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 3
ESP
(Electronic Security
Perimeter)
(Utility) Data and Control Center
(Transmission) Substation Bulk Power Station
LAN
RTU
D
P
Routable Protocol ( ie, IP)
T
COMMUNICATION
NETWORK
IED
and other smart
grid elements
Meter
Collector
Meter
4
NERC CIP requirements do not address connectivity with the systems outside of the bulk electric system. However, connectivity of utility
and smart grid systems to the secure CCAs is shown here for a complete presentation of a secure utility communications architecture
3. Communication over MPLS networks
Before presenting advantages of MPLS networks towards NERC CIP compliance, communication
services over an MPLS network are briefly described in this section. Many standards-based services
and protocols at Layer 1, Layer 2, and Layer 3 (of the OSI framework) can be independently emulated
over the MPLS infrastructure with little impact on functionality or performance. In short, there is
no need to deploy, operate and manage multiple networks for multiple functions.
3.1 MPLS Architecture
In an MPLS network, packets are assigned labels and transported end-to-end in logical tunnels or
connections called label switched paths (LSP). Packet forwarding decisions are based on the MPLS
label, rather than on the protocol-specific destination information field read from the packet (such
as Ethernet MAC address, Frame Relay DLCI, IP address, etc). This (outer) label (added as part
of a shim header) together with the original packet (payload) constitutes the MPLS packet (See
Figure 3). Additional in-depth background information on MPLS architecture is given in
Appendix A.
Figure 3. Packet forwarded on an LSP through an MPLS network
For purposes of this discussion, it is important to note that packets are forwarded based on the
label, rather than the destination address as defined in the native protocol of that payload. The
end-to-end path of the LSP is pre-determined there is no change in this path as a function of the
destination address while the packet is traversing the network so the MPLS network is used to
support non-routable protocols. Additional detail and discussion is provided in Appendix A.
Switches or routers participating in the MPLS network are called label edge routers or label
switched routers, depending on their location in the network. Figure 3 illustrates a packet being
forwarded on an LSP through an MPLS network.
A packet is received at the ingress MPLS router (called ingress Label Edge Router ingress LER or
iLER), an initial label is added and the new MPLS packet is forwarded to a determined interface
on the LER where the LSP was configured. Label Switched Routers (LSRs) traversed by the LSP
extract the label, swap with the next label and continue forwarding the packet to the egress LER
(eLER), where the final label is removed.
4 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
MPLS NETWORK
LSP
LSR
Switching in the core MPLS network
-Match on incoming label
-Lookup outgoing label and interface
-Swap labels and forward out
LSR eLER
Packet
MPLS Labels
iLER
Pkt Pkt Pkt
Pkt Pkt
LSP configuration is independent of the end to end protocols of the packets carried in the LSP or
the relationship between the end users exchanging these packets. It is possible that an LSP may
carry traffic between several sets of end users with different networking protocols. Multiplexing of
multiple end-to-end virtual connections (VCs) with differing protocols can be achieved by
providing another (inner) label in the packet identifying the corresponding connection. This
additional label has only an end-to-end (LER-LER) significance and is not changed at the LSR
hops. (See Figure 4).
Figure 4. Multiplexing Virtual Connections in an LSP
3.2 Converged MPLS Networks
End to end Layer 1 and Layer 2 communications services over MPLS are shown in Figure 5.
Figure 5. Communication Services over MPLS
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 5
MPLS NETWORK
LSR eLER
Core LSP Tunnel - Uses outer label
Psuedo wires or VCs - Use inner label
Customer Nodes
iLER
Services can be dened and shared over
MPLS network emulating conventional
L1, L2, and L3 connections
Independence and isolation of MPLS
network operations and management
from individual service operations
and management
Point-to-point MPLS tunnels between
pairs of network access points
Trafc between the endpoints
transparently carried in tunnel
Network supports tunnel management
including QoS, reliability, and security
Ethernet broadcast domain over multiple
MPLS tunnel emulating
- An Ethernet switch
- A switched Ethernet network
Point-to-point connection through a
single MPLS tunnel emulating
- PDH, SDH/SONET
- (Point-to-point) Ethernet
- Frame relay
- ATM
VPLS
Virtual Private LAN Service
Non-routable protocol
VPWS
(Virtual Psuedo-wire Service)
Non-routable protocol
End-to-End
Services over
MPLS Network
3.2.1 VPWS
One of the most common implementations of MPLS-based services is Virtual Pseudo-Wire
Service (VPWS). Also known as Virtual Leased Line (VLL) service, this service is defined by
RFC 4447 [4]. A pseudo-wire is point-to-point connection between two end points at sites such
as substations, data centers, corporate sites, etc. (i.e., a tunnel).
Legacy non-routable protocols such as PDH/SONET, Frame Relay, and Ethernet can be carried
over MPLS pseudo wires. Since utilities may have numerous legacy networks in place, emulation
of these connections in the converged MPLS network continue providing these services without
having to maintain multiple backbone infrastructures (much like a telecommunications service
provider does). The ability to combine multiple protocols in one network greatly simplifies and
reduces the cost of operating a network.
3.2.2 VPLS
Another implementation of MPLS-based services is Virtual Private LAN Service (VPLS), as defined
by RFC 4762 [5]. VPLS is a Layer 2 VPN service and is used to provide multi-point to multi-point
Ethernet connectivity between substations, data centers, corporate sites, etc. and emulates an
Ethernet bridge connecting these endpoints. In a converged MPLS network, this Layer 2 VPN is
achieved using a full mesh of LSPs between the participating sites. Conceptually, a number of
secure tunnels are constructed, allowing multipoint connectivity.
3.3 Additional MPLS Features
MPLS based services provide utilities with the ability to support closed user groups (Layer 1,
Layer 2 VPNs identified above) for communication among systems associated with similar application
requirements and/or the associated users. The isolation of traffic of one closed user group from
other groups adds to the network security and facilitates flexibility in implementing security and
other requirements individually for each group. The protocols used between endpoints within a
closed user group can be different from the protocol used within another group.
With MPLS, protection mechanisms ensure that reliability requirements are met and that failures
can be recovered within specified time limits. With the combination of MPLS and DiffServ (or
class-aware traffic engineering), service differentiation can be implemented for traffic on a per-
class basis (QoS classes). See the Appendices for details of some of these features.
4. Interim NERC CIP Compliance with MPLS-based Non-Routable Protocol
For some utilities, it may be difficult to implement ESP requirements since inclusion of IP routers
in an ESP may be considered too costly for timely compliance of NERC CIP requirements. With
the exemption, network connectivity from systems outside of ESP can support compliance with
communication over non-routable protocols.
Figure 6 is an illustration of the basic architecture with the routers/switches residing outside of the
ESPS and communicating over non-routable protocols.
6 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
Figure 6. Example Communication Architecture with Exemption
A non-routable protocol provides Layer 1 or Layer 2 connection between two end points, or in the
case of multipoint Ethernet, connection among multiple endpoints. Even if the MPLS infrastructure
is a mesh network, traffic between the two endpoints follows a pre-determined physical or logical
path and there is no change (no routing) as a function of the protocol-specific destination information
(address). Since there is no routing, packets move through their own secure tunnel. Consequently,
MPLS networks provide cost effective, efficient, and flexible point-to-point and point-to-multipoint
connectivity emulating a variety of non-routable protocols
5
. Consequently, switch/routers can be
located outside the ESP, as shown in Figure 6.
The non-routable protocols may be extended to the general smart grid network as shown in Figure 7
Figure 7. Extending Non-routable Protocols to Include Other Smart Grid Elements
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 7
ESP
(Electronic Security
Perimeter)
(Utility) Data and Control Center
(Transmission) Substation Bulk Power Station
LAN
LAN LAN
D
C
P
Q S
Routable Protocol ( ie, IP)
T
COMMUNICATION
NETWORK
(over MPLS supporting eg,
VPWS, VPLS)
Non-routable Protocol
ESP
(Electronic Security
Perimeter)
(Utility) Data and Control Center
(Transmission) Substation Bulk Power Station
LAN
LAN LAN
D
C
P
Q S
Routable Protocol ( ie, IP)
T
COMMUNICATION
NETWORK
(over MPLS supporting eg,
VPWS, VPLS)
Non-routable Protocol
and other smart
grid elements
RTU IED
Meter
Collector
Meter
5
Leased line services procured from service providers are often implemented using MPLSs VPWS or VPLS service on the
service providers own MPLS networks.
5. MPLS is the Right Choice with or without the Exemption
In the near future, it is expected that the "exemption" of systems with non-routable protocols from
being CCAs may be removed from NERC CIP requirements. In that case, all networking elements
at a BES location will be required to be in the ESP. Irrespective of whether a routable or non-
routable protocol is used, MPLS technology is still the appropriate technology for communication
between endpoints in different ESPs, as well as between an endpoint in an ESP and other smart grid
elements not in an ESP. In the case of VPWS and VPLS implementations, the tunnels inherent to
these services, and defined by the relevant standards, provide secure links between critical nodes.
Should IP services be necessary, these can also be supported securely over MPLS using RFC 4364
[6]. These are known as VPRN (Virtual Private Routed Network) services, or MPLS VPNs. The
same label switching described above is utilized with a VPRN to create secure tunnels.
MPLS provides utilities with the ability to support closed user groups for communication among
systems associated with similar application requirements and/or the associated users. Such network
separation helps facilitate implementation of network security. MPLS protection mechanisms such
as Fast Re-route (FRR), described in Appendix B, can ensure that reliability requirements can be
met and failures recovered within specified time limits. This facility can be made available to
certain types of grid applications using class-based approaches. Furthermore, added security features
of MPLS-based vendor products also support efficient implementation of ESP security for NERC
CIP compliance.
MPLS-based converged communication networks increase operational efficiency and reduce
capital expense (CAPEX). As seen in Figure 5 and 6, several types of MPLS-based services can
be implemented on a shared core. In addition to providing Layer 1 and Layer 2 connections
using the VPWS and VPLS technologies, L3 routable protocols (such as IP) can be implemented
using VPRNs. Utilities can use individual VPWS, VPLS, and VPRN services for independent and
separate end-to-end application connections over the same MPLS network, providing traffic
segregation for each connection. Specifically, VPWS and VPLS services are isolated not only from
VPRN IP traffic, but also Internet traffic; packets from the Internet cannot enter VPWS or VPLS
services based VPNs. This key feature allows MPLS to support network-based logical access
control to protect power utility CCAs.
6. ESP Security Implementation
Removal of the non-routable exemption will result in the need to apply security to communications
endpoints previously exempt from NERC CIP requirements. These systems will now be considered
critical cyber assets that must be contained within an ESP. Referring to Figure 6, the electronic
security perimeter will now extend to include MPLS end systems C, Q and S in the Data and
Control Center, Bulk Power Station, and Transmission Substation respectively. In other words,
the ESP boundary will now be the interface between these switches and the power utility's
communication network. The communication network will remain out of scope for NERC CIP
requirements (See Requirement CIP-005-B.R1.3 in [1] ), but measures must be put in place to
ensure that the new ESP boundary remains intact, and that the MPLS end systems comply with
those requirements.
6.1 Requirements Overview
The nine NERC CIP requirements [1] provide auditable standards that must be in place for the
protection of bulk electric systems.
CIP-001 Sabotage Reporting requires guidelines and procedures for reporting disturbances or
unusual occurrences suspected, or determined to be caused by, sabotage to the appropriate
systems, governmental agencies, and regulatory bodies, as well as informing operating personnel.
CIP-002 Critical Cyber Asset Identification requires the identification and documentation of
Critical Cyber Assets (CCAs) associated with critical assets that support the reliable operation
of the bulk electric system.
8 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
CIP-003 Security Management Controls requires the implementation and documentation of a
cyber security program for the secure management of critical cyber assets.
CIP-004 Personnel and Training requires that personnel having authorized cyber or authorized
unescorted physical access to critical cyber assets, including contractors and service vendors,
have an appropriate level of personnel risk assessment, training, and security awareness.
CIP-005 Electronic Security Perimeter(s) requires the identification and protection of
Electronic Security Perimeters (ESPs) within which all critical cyber assets reside as well as all
access points on the perimeter.
CIP-006 Physical Security of Critical Cyber Assets mandates the implementation of a physical
security program for the protection of critical cyber assets.
CIP-007 Systems Security Management requires the definition of processes, methods and
procedures for securing critical cyber assets as well as non-critical assets that are within an ESP.
CIP-008 Incident Reporting and Response Management ensures the identification,
classification, response and reporting of cyber security incidents related to critical cyber assets.
CIP-009 Recovery Plans for Critical Cyber Assets requires that recovery plans are put in place
for critical cyber assets and that these plans follow established business continuity and disaster
recovery techniques and practices.
The NERC CIP-002 through NERC CIP-009 standards provide a cyber security framework for the
identification and protection of critical cyber assets supporting the reliable operation of the bulk
electric system. The majority of these standards consist of policies, procedures and best practices
that are required to be put in place. CIP-002, CIP-005 and CIP-007 are the most technical of
these NERC critical infrastructure protection standards. CIP-002 has been discussed at length
throughout this paper. The technical aspects of CIP-005 and CIP-007 deserve further treatment.
6.2 ESP Identication and Protection
CIP-005 prescribes requirements for ensuring that every critical cyber asset resides within an
electronic security perimeter. The set of CCAs within an ESP can be thought of as a security
enclave - a grouping of critical assets by function or role that can be isolated as much as possible
from unauthorized access. CIP-005 discusses the technical mechanisms for controlling the
external, electronic access points to an ESP. The access points are the ports and protocols on CCAs
and non-critical assets within the ESP that provide access from outside. CIP-005 also requires
electronic or manual logging of all access attempts and an annual cyber security vulnerability
assessment of the external electronic access points to the ESP.
6.3 System Security Management
CIP-007 is concerned with maintaining and verifying the security of CCAs and non-critical cyber
assets within an ESP. CIP-007 revisits the issue of ensuring that unused ports and services are
disabled and requires the use of anti-malware software where technically feasible. CIP-007 requires
technical controls that enforce access authentication and accountability for all user actions and
contains technical controls providing password security. Other technical control requirements
include controls to monitor and generate alerts for system events related to cyber security and an
annual vulnerability assessment which includes a review of the controls for default accounts.
6.4 Technical Guidance for Compliance with NERC CIP Requirements
The NERC CIP cyber security standards identify requirements that must be met to secure the bulk
electric system; they provide guidance regarding what needs to be done, but do not specify how it
should be done. The National Institute of Standards and Technology (NIST) draft standard
NISTIR 7628 and the International Electrotechnical Commission (IEC) 62351 series of standards
are sources that provide technical guidance for compliance.
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 9
The second draft of NISTIR 7628, Smart Grid Cyber Security Strategy and Requirements [7] was
issued in February 2010 and describes the overall cyber security strategy for the smart grid. It
contains technical guidance for securing the smart grid, based on a set of general purpose security
requirements found in NIST SP 800-53 Recommended Security Controls for Federal Information
Systems and Organizations [8].
The NISTIR 7628 smart grid cyber security requirements provide comprehensive, detailed technical
guidance for securing cyber assets within an ESP and are organized according to the DHS Catalog
of Control Systems Security [9] categories. The technical requirements in those categories relevant
to communications systems within the bulk energy system are summarized below.
System and Communication Protection consists of steps taken to protect systems and the
communication links between them from cyber intrusions.
System Development and Maintenance includes technical requirements that ensure secure backups
and remote maintenance.
Incident Response includes technical requirements that enable the examination of a system as
well as its recovery and/or reconstitution after a disruption or failure.
System and Information Integrity ensures that sensitive data is not modified or deleted in an
unauthorized manner.
Access Control ensures that resources are only accessed by authorized personnel and that those
personnel are accurately identified.
Audit and Accountability ensures the existence and availability of system logs that are used to
detect breaches of system security, anomaly detection, and forensic analysis.
The NISTIR 7628 Smart Grid Cyber Security Strategy and Requirements standard provides comprehensive
technical guidance for securing critical cyber assets as well as non-critical cyber assets within an
ESP. The IEC 62351 series of standards (62351-1 through 62351-7) [10] is concerned with securing
the unique communication protocols used by the power utility industry. The IEC standards address
the end-to-end security of the communication protocols, so that intervening communications
equipment is transparent to these security standards.
7. Using ITU-Ts X.805 Security Standard to Secure the Smart Grid
Extending the ESP boundary to include previously excluded communication systems necessitates a
standards-based security assessment of these systems to ensure compliance with NERC CIP
requirements and ESP security. The ITU-T X.805 [2] security standard provides a comprehensive
framework that can be used to ensure that the management, control, and end-user plane of these
systems are secure relative to eight dimensions (access control, authentication, non-repudiation,
data confidentiality, data integrity, availability, communication security, and privacy). The assessment
will identify features within the communication systems that satisfy NERC CIP requirements. It
also identifies new security features that need to be developed or compensating controls that need
to be deployed to ensure that the security of the ESP remains intact.
NISTIR 7628, Smart Grid Cyber Security Strategy and Requirements, contains security requirements
that lead to NERC CIP compliance for cyber assets within an ESP. Each of the NISTIR 7628
requirements is a control that resides in one or more X.805 security dimensions. Therefore, the
compliance level of the critical cyber asset with respect to NISTIR 7628 and NERC CIP corresponds
to how well it addresses X.805 security dimensions.
Figure 8 shows how the X.805 security dimensions can be used to measure the compliance level of
the CCA with respect to NERC CIP in an easy-to-understand pictorial manner. This format can
10 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
also be used to depict the ESP's overall compliance level with NERC CIP requirements. In order
for the critical cyber asset to fully satisfy the NISTIR 7628 requirements, the solid green area
would have to completely cover the red area. The white area depicts the implementation level of
additional security requirements that a power utility may want to deploy.
Figure 8. Depicting the Security Compliance Level of an Example CCA
8. Threats to the Electronic Security Perimeter
Power utilities are increasingly reliant on information technologies to manage the power grid,
resulting in an integration with telecommunications networks. These networks must be managed
to the same reliability as the power grid since threats to telecommunications networks now
become threats to the power grid.
IEC TS 62351-1, Power Systems Management and Associated Information Exchange Data and
Communications Security Part 1: Communication Network and System Security Introduction to
Security Issues, lists the four cyber security threats to the power grid as:
1. Unauthorized access to information,
2. Unauthorized modification or theft of information,
3. Denial of service,
4. Repudiation/unaccountability.
All of these threats directly relate to the CCAs and other cyber assets within an ESP. Therefore,
precautions must be taken to protect communications equipment once the ESP is extended to
include them.
The motivations for attacking the power grid include industrial espionage, vandalism, cyber
hacking and terrorism. Figure 9 indicates the principal location of threats to the bulk electric
system. Protection against denial of service is of paramount importance to the power utility
industry, a critical infrastructure, since the primary mission of every power utility is to provide
uninterrupted electric service to its customers.
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 11
Authentication
Non
Repudiation
Data
Condentiality
Communication
Security
Data
Integrity
Availability
Privacy
Access
Control
77%
0%
24%
31%
29% 13%
68%
42%
35%
35%
87%
88%
Figure 9. Threats to the Bulk Electric System
9. Potential Vulnerabilities in the ESP
Threats and threat agents will always be present in any environment and nothing can be done to
eliminate them. In order to secure the power grid, vulnerabilities must be focused on. NISTIR
7628 contains an extensive list of vulnerabilities that could be present in power grid equipment
and is organized into several vulnerability classes that illustrate the four threats to the ESP and
how they would be realized in the form of an attack.
Vulnerabilities in communications systems typically reside in the management or control plane as
end-user traffic does not terminate on this type of equipment. Machine-to-machine interactions
occur in the control plane, whereas both human-to-machine and machine-to-machine interactions
occur in the management plane. An attacker can forge messages associated with any of these
communication types to cause denial of service, unauthorized access, unauthorized modification
or theft of information, and repudiation/unaccountability of actions.
10. Mitigations for ESP Vulnerabilities and NERC CIP Compliance
Security must be designed in at the architectural level to provide the most complete and cost
effective solution. To provide defense in depth, security functionality must be implemented using
a tiered approach; an integral part of systems as well as networks. Defining power grid security
requirements in advance will ensure that they are implemented, as retroactive deployment of
security updates or compensating controls may be cost prohibitive.
Security mechanisms found in the X.805 Access Control security dimension are often the first
line of defense to mitigate communications system vulnerabilities. If an attacker cannot access
the system, he/she cannot compromise it, take it out of operation, or exhaust its resources.
Unauthorized access can be prevented in many ways, including secure user IDs and passwords,
access control lists, firewalls, intrusion detection/prevention systems, etc.
12 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
(Utility) Data and Control Center
(Transmission) Substation Bulk Power Station
LAN
LAN LAN
RTU
D
P T
COMMUNICATION
NETWORK
(Preferably MPLS-based
IED
and other smart
grid elements
Unauthorized access to information
Unauthorized modication or
theft of information
Denial of Service
Repudiation/Unaccountability
Denial of Service
Repudiation/Unaccountability
Denial of Service
Repudiation/Unaccountability
Meter
Collector
ESP
(Electronic Security
Perimeter)
Meter
The communication network in the above architecture diagram raises concern as a potential
conduit for unauthorized access to critical cyber assets. MPLS VPNs can be used to provide
network-based access control, as well as traffic isolation across shared or converged networks.
Since they inherently provide traffic isolation capability at Layer 2, an MPLS VPN can be created
for each security domain to deploy closed user groups as the foundation for a secured network
infrastructure between ESPs that contribute to NERC CIP compliance.
Controls contained in the remaining X.805 security dimensions must also be deployed in order to
provide the comprehensive security required for NERC CIP compliance. After access control,
availability controls are probably the most important for the power utility industry. These controls
include per-flow queuing, per-peer queuing, and business continuity/disaster recovery/continuity of
operations procedures.
Attention must also be paid to authentication, data integrity, and non-repudiation security
dimensions in order to ensure that administrative actions on CCAs are performed only by
authorized personnel, who can be held accountable. This also ensures that messages transmitted
between CCAs and across ESPs have not been forged or tampered with.
Communication security, which ensures that information travels between authorized end-points
without being diverted or intercepted, is also provided by MPLS VPNs. Data confidentiality and
privacy security dimensions are less important in context of the bulk electric systems, especially
the communications system CCAs that have been focused on in this paper.
Alcatel-Lucent 77x0 SR MPLS routers offer a broad range of security features to provide the defense-
in-depth necessary to protect critical cyber assets within an electronic security perimeter. SR-OS
features that enable secure system operation include support for SSH and SCP, support for SNMPv3,
logging of system events and access requests, TCP wrappers and IP tables, session timeouts for
remote management sessions, and login banners. The SR-OS also supports RADIUS and TACACS+
password management, as well as password profiles for separation of duties assignments.
To protect against DoS attacks, SR-OS provides several types of ACLs, as well as CPM queues and
per peer queuing. SR-OS supports CPM filters and Management Access Filters (MAFs) in addition
to typical ACLs that are applied to logical ports. It also provides anti-spoofing filters, unicast
Reverse Path Forwarding (uRPF), Secure MAC Learning, and MAC Learning Protection to
protect against spoofed IP and MAC addresses. The uRPF feature discards packets with unverifiable
IP source addresses, and Secure MAC Learning prevents the registration of another MAC address
on a VPLS service access point (SAP) or service distribution point (SDP) after one is initially
registered. MAC Learning Protection discards MAC update requests for protected addresses
that originate from unprotected service access points.
The combination of DHCP Snooping and SR-OS provided ARP Reply Agent features can be used
to protect against DHCP starvation and ARP spoofing attacks. Using the ARP Reply Agent to
receive and generate requests from subscribers can restrict the MAC-IP pairs registering on the
network. The ARP Reply Agent uses DHCP Snooping to listen to the DHCP exchange and
populate its table of known IP, MAC address pairs. When new ARP requests are attempted, they
are compared to known values to prevent an attacker from spoofing an ARP request or response.
SR-OS provides MD5 hashes for supported routing protocols, as well as BGP TTL protection to
protect the integrity of routing protocol messages. BGP TTL protection leverages the fact that the
TTL field of received BGP updates should never be less than 254 since they always originate from
a neighboring router.
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 13
11. Conclusions
This paper establishes that MPLS based networks provide secure, reliable, efficient, flexible and
cost-effective communication between the Critical Cyber Assets at different locations of the Bulk
Electric System, as well as between CCAs and other smart grid network elements.
We have shown that MPLS networks natively support communication with non-routable protocols.
With the current requirement exemption, Bulk Electric System endpoints connecting to the
MPLS networks that are not currently considered to be CCAs, and allowed to be outside the ESP
boundary, can communicate over the MPLS network with non-routable protocol.
It is further shown that, if necessary, MPLS networks can be extended for secure communication
between CCAs of the bulk electric systems and other smart grid network elements in compliance
with NERC CIP. The traffic isolation capabilities inherent in MPLS-based services provide
network-based access control for bulk energy system CCAs and other smart grid network elements.
Several types of MPLS services can be configured to establish closed user groups. IP packets
originating from external endpoints, including the Internet, cannot enter these closed user groups,
thus preventing many types of external attacks.
If the non-routable protocol exemption is removed, all communication endpoints providing external
access into an ESP, including MPLS endpoints, will be considered critical cyber assets and
therefore subject to NERC CIP compliance. This paper has provided guidance for securing these
communication endpoints and pointed out several security features present in Alcatel-Lucent
MPLS switches that comply with the NERC CIP requirements.
14 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
References
[1] North American Electric Reliability Corporation, Reliability Standards for the Bulk Electric
Systems of North America, NERC, December 2009.
www.nerc.com/files/Reliability_Standards_Complete_Set_2009Nov2.pdf
[2] International Telecommunication Union Telecommunication Standardization Sector (ITU-T),
Security Architecture for Systems Providing End-to-End Communications, ITU-T Rec. X.805,
October 2003.
//www.itu.int/rec/T-REC-X.805-200310-I
[3] K. C. Budka, J. G. Deshpande, T. L. Doumi, M. Madden, T. Mew, Communication Network
Architecture and Design Principles, to appear in Bell Labs Technical Journal special issue on
Eco-Sustainability and Green Information and Communication Technology, Summer 2010.
[4] L. Martini, Ed., E. Rosen, T. Smith, G. Heron Pseudo-wire Setup and Maintenance Using the
Label Distribution Protocol (LDP), IETF RFC 4447, April 2006.
[5] M. Lassers, Ed., V. Kampala, Ed., Virtual Private LAN Service (VPLS) Using Label Distribution
Protocol (LDP) Signaling IETF RFC 4762, January 2007.
[6] E. Rosen, Y. Rekhter BGP/MPLS IP Virtual Private Networks (VPNs), IETF RFC 4363
February 2006.
[7] National Institute of Standards and Technology, Smart Grid Cyber Security Strategy and
Requirements, DRAFT NISTIR 7628, February, 2010.
csrc.nist.gov/publications/drafts/nistir-7628/draft-nistir-7628.pdf
[8] National Institute of Standards and Technology, Recommended Security Controls for Federal
Information System and Organizations, NIST SP 800-53, Revision 3, August 2009.
csrc.nist.gov/publications/nistpubs/800-53-Rev3/sp800-53-rev3-final-errata.pdf
[9] Department of Homeland Security, National Cyber Security Division, Catalog of Control Systems
Security: Recommendations for Standards Developers, September 2009.
www.us-cert.gov/control_systems/pdf/
Catalog_of_Control_Systems_Security_Recommendations.pdf
[10] International Electrotechnical Commission, Power Systems Management and Associated
Information Exchange Data and Communications Security(all parts), IEC TS 62351-1 62351-7,
2005 2009.
www.iec.ch
[11] L. Anderson, T. Madsen, Provider Provisioned Virtual Private Network (VPN) Terminology, IETF
RFC 4026, March 2005.
[12] P. Pan, G. Swallow, A. Atlas, Fast Reroute Extensions to RSVP-TE for LSP Tunnels, IETF RFC
4090, May 2005.
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 15
Acronyms
ACL Access Control List
AMI Advanced Metering Infrastructure
ARP Address Resolution Protocol
ATM Asynchronous Transfer Mode
BES Bulk Electric System
BGP Border Gateway Protocol
CCA Critical Cyber Asset
CES Circuit Emulation Service
CIP Critical Infrastructure Protection
CPM Control Processor Module
DER Distributed Energy Resource
DHCP Dynamic Host Conguration Protocol
DNP3 Distributed Network Protocol version 3
DoS Denial of Service
ESP Electronic Security Perimeter
FR Frame Relay
FRR Fast Re-Route
FTP File Transfer Protocol
ICCP Inter-Control Center Communications Protocol
IDS Intrusion Detection System
IEC International Electrotechnical Commission
IED Intelligent Electronic Device
IETF Internet Engineering Task Force
IP Internet Protocol
IPS Intrusion Prevention System
IPSec IP Security protocol
ITU-T International Telecommunication Union
Telecommunication Standardization Sector
LAN Local Area Network
LER Label Edge Router
LSP Label Switched Path
LSR Label Switching Router
MAC Media Access Control
MAF Management Access Filter
MD5 Message Digest version 5
NAT Network Address Translation
NIST National Institute of Standards and Technology
NERC North American Electric Reliability Corporation
OSI Open Systems Interconnection
PDH Plesiochronous Digital Hierarchy
PKI Public Key Infrastructure
RADIUS Remote Access Dial-In User Service Protocol
RFC Request For Comments
RTU Remote Terminal Unit
SAP Service Access Point
SCADA Supervisory Control and Data Acquisition
SCP Secure Copy
SDP Service Distribution Point
SHA-1 Secure Hash Algorithm version 1
SNMPv3 Simple Network Management Protocol version 3
SONET Synchronous Optical Network
SSH Secure Shell
SSL Secure Sockets Layer
TACACS+ Terminal Access Controller Access Control
System Plus Protocol
TASE.2 Telecontrol Application Service Element version 2
TCP Transmission Control Protocol
TDM Time Division Multiplexed
TLS Transport Layer Security
TTL Time To Live
uRPF unicast Reverse Path Forwarding
VC Virtual Channel
VLAN Virtual Local Area Network
VLL Virtual Leased Line
VoIP Voice over IP
VPLS Virtual Private LAN Service
VPN Virtual Private Network
VPWS Virtual Pseudo-Wire Service
16 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
Appendix A. MPLS Architectures
In an MPLS network, packets are assigned labels and transported end-to-end in logical tunnels or
connections called label switched paths (LSP). Packet forwarding decisions are made based on the
MPLS label rather than on the contents of the destination address field from the packet. The label
is added as part of a shim header of a packet considered the payload of the MPLS packet. An
illustration of the MPLS shim header is shown in Figure 10.
Figure 10. High level MPLS packet structure showing the shim header
On the right hand side of the figure is the MPLS payload (eg, SONET, Ethernet, ATM or Frame
Relay frame, IP packet). The MPLS payload contains its native protocol header and its own payload,
in this case corresponding to the data being transmitted. On the left hand side of the figure is the
MPLS shim header that includes a 20-bit label.
Packets are forwarded based on the label, rather than routed based on the destination address
defined in the native protocol of that payload. The end-to-end path of the LSP is pre-determined
and there is no change in this path as a function of the destination address while the packet is
traversing the network. Thus, MPLS networks support non-routable protocols.
LSPs are uni-directional and established end-to-end; the labels associated with an LSP are
known to each incident switch or router involved in its establishment. The switches or routers
participating in the MPLS network are called label edge routers or label switched routers, depending
on their location in the network. Figure 3 shows an illustration of a packet being forwarded on an
LSP through an MPLS network.
When a packet is received at the ingress MPLS router (called a Label Edge Router LER), an initial
label is added and the new MPLS packet is forwarded to a determined interface on the LER where
the LSP was configured. An appropriate interface was identified when the LSP was created. The
Label Switched Routers (LSRs) traversed by the LSP extract the label, swap with the next label and
continue forwarding the packet to the egress LER where the final label is removed. The LSP
configuration is independent of the end-to-end protocols of the packets carried in the LSP or the
relationship between the end users exchanging these packets. It is possible that an LSP may carry
traffic between several sets of end users with different networking protocols. MPLS provides
another important capability called label stacking. More than one MPLS shim header can be added
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 17
Payload MPLS Shim Header
Data Protocol
Header
Label
(20 bits)
Other
Fields
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 18
to the packet, thereby stacking labels. This provides two important facilities. The first is the
concept of a pseudo wire or virtual channel [11], which allows different service types (e.g.
different sets of end users using different protocols) to be carried over a common LSP in the MPLS
core while maintaining separation between them. The second is the concept of service protection,
which we discuss in Appendix B. Each traffic stream is confined to its own pseudo wire or VC
[11], as identified by its VC identifier. The label stacking capability of MPLS can then be used to
embed the VC identifier as a label in the MPLS packet. The VC label is often called the inner label
[11] and does not change as the packet traverses through the LSP. The outer label corresponds to
the LSP in the core network (which we referred to as the MPLS label earlier), and its value may
change as the packets pass through the LSRs
18 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
Appendix B. Additional MPLS Features
In this section, we describe additional MPLS features that can be deployed to ensure reliability
requirements can be met through protection mechanisms, and allow service differentiation to be
implemented for traffic on a per-class basis.
MPLS Protection
For an LSP tunnel that is established between two endpoints or LERs (at the substations or sites),
two types of protection mechanisms can be employed (individually or in combination). The first is
the use of a backup or standby LSP that uses a secondary path that is a link and/or node disjoint
from the first one. This ensures that if there is a single link or node failure in the core network that
affects the LSP, a secondary LSP is available as a backup path to carry the data. Since the secondary
LSP also consumes bandwidth, this type of protection is typically used for traffic with very stringent
protection requirements. In the following figure, we show an example of a backup/standby LSP.
Figure 11. Primary LSP and backup/standby LSP with disjoint secondary path
The primary and secondary LSPs are shown in Figure 11. Secondary LSPs can also be established in
the network deployment phase along with the primary LSPs and no new LSP provisioning needs to
be performed after a failure has occurred. This also maintains the non-routable nature of the network.
The second type of available protection is referred to as fast reroute (FRR) or facility bypass (RFC
[12]). FRR uses the label encapsulation method to bypass single link or node failures in the network.
However, when FRR is active, there may be temporary congestion in parts of the network where
the bypass has been established. The following figure shows an example where bypass tunnels are
established to protect against node failures.
Figure 12. Primary LSP and FRR bypass tunnels to protect against router failures
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 19
LSR
LSR
LSR
LSR
eLER
LSP- Secondary Path
LSP- Primary Path
iLER
IP
IP
LSR LSR LSR
R2
R3
LSR
R4
eLER
Bypass tunnel to
protect R2 failure
Protected (Primary) LSP
iLER
R1
R6
R7 R9
R8
R5
Bypass tunnel to
protect R4 failure
Bypass tunnel to
protect R3 failure
In the figure above, the primary LSP is shown as a solid red line. The LSPs shown with dotted lines
represent bypass tunnels used to forward traffic around failed routers. Note that the protection
mechanism consists of stacking a new label on a packet and forwarding along the bypass tunnel.
Thus, SONET-like protection times in the range of 50 ms. can be achieved. The FRR tunnels can
be designed and deployed along with the primary LSPs, again maintaining the non-routable nature
of the network.
MPLS Class-Based Quality of Service Differentiation
DiffServ or class-aware MPLS implementations allow differentiation of quality of service (QoS) on
a per-class basis. Service differentiation can be performed by assigning packets of a particular
application to a specific traffic class. In the MPLS core network, LSPs can be set up to carry packets
of one or more specific class types. At each of the routers in the core network, bandwidth allocation
and priorities (hence, the treatment of the packets) can be set according to the different classes.
This allows great flexibility in how applications are delivered in the network. For example, video
surveillance applications can be allocated specific bandwidth, but policed to make sure the bandwidth
used does not starve other applications; VoIP traffic can be assigned to low latency paths; network
control data and incident-related communications can be assigned highest priority, etc.
While many application types exist, the number of traffic classes is technically limited to 8; these
typically represent 4 classes with 2 priority levels each. When the converged utility network is set
up, the allocation of application types to the traffic or QoS classes must be performed. In this mapping,
applications sharing similar characteristics would typically be assigned to the same class type.
In addition to class-based differentiation, MPLS networks also incorporate the concept of traffic
engineering. Unlike traditional IP networks, where traffic takes simple shortest path routes, MPLS
traffic engineering allows LSP paths to be established that are optimal according to criteria other
than the shortest path and can take into account the available bandwidth on individual links.
Thus, traffic engineering can result in more efficient use of bandwidth resources in IP network, and
helps to further reduce the overall cost of operations. When class-aware traffic engineering is
implemented, high priority/low latency traffic can be carried on shorter, less congested paths to
ensure the QoS criteria are being met. The following figure shows an example.
Figure 13. Two LSPs established from R2 to R5 using MPLS class-aware traffic engineering
In Figure 13, high priority critical data from R2 to R5 (e.g. control data) is sent on the high priority
(red with solid line) LSP, whereas non-critical operations data is sent on the low priority (blue with
dotted line) LSP. Class-based traffic engineering is very important in meeting the QoS requirements
of a diverse set of applications.
20 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
R2
R3
R4
Low Priority LSP
R1
R6
R7 R9
R8
R5
High Priority LSP
Appendix C. Technical Guidance for Compliance with NERC CIP Requirements
NERC CIP requirements provide a comprehensive set of best practices that must be followed to
secure the bulk electric system. They provide guidance regarding what needs to be done, however
they do not specify how it should be done. The National Institute of Standards and Technology
(NIST) draft standard NISTIR 7628 and the International Electrotechnical Commission (IEC)
62351 series of standards are sources that provide technical guidance for compliance with the
NERC CIP requirements.
The second draft of NISTIR 7628, Smart Grid Cyber Security Strategy and Requirements [7] was issued
in February 2010 and describes the overall cyber security strategy for the smart grid. It also contains
technical guidance for securing the smart grid that is based on a set of general purpose security
requirements found in NIST SP 800-53 Recommended Security Controls for Federal Information
Systems and Organizations [8].
The NISTIR 7628 smart grid cyber security requirements provide comprehensive, detailed technical
guidance for securing cyber assets within an ESP and are organized according to the DHS Catalog
of Control Systems Security [9] categories. The technical requirements in those categories relevant
to communications systems within the bulk energy system are summarized below.
System and Communication Protection consists of steps taken to protect systems and the
communication links between them from cyber intrusions. This category includes technical
requirements involving:
partitioning management traffic,
isolating security functions,
preventing DoS attacks,
prioritizing the use of system resources,
protecting the authenticity, integrity and confidentiality of communicated information,
establishing and providing trusted communications paths,
Managing cryptographic keys,
using validated cryptographic algorithms,
using PKI certificates,
identifying and protecting external communications connections,
establishing security roles for all users.
System Development and Maintenance. This category includes technical requirements involving:
performing secure backups of critical software, applications and data,
authorizing, managing and monitoring remote maintenance.
Incident Response. This category includes technical requirements involving:
system operation in a safe/limited mode that allows the examination of logs and configuration
information, as well as resetting, enabling and disabling the system,
mechanisms to enable recovery and/or reconstitution of the system by authorized personnel
after a disruption or failure.
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 21
System and Information Integrity ensures that sensitive data is not modified or deleted in an
unauthorized manner. The technical requirements in this category involve:
system and information integrity procedures,
protection against malicious code,
network intrusion detection systems,
access control lists (ACLs),
dynamic packet filtering,
system hardening,
logging and reporting of security events and system activities,
detection of unauthorized changes to software and information.
Access Control ensures that resources are only accessed by authorized personnel and that those
personnel are accurately identified. The technical requirements in this category involve:
access control policies and procedures for management tasks that are commensurate with the
criticality of the task,
identification and authentication controls, including the authentication of communications
between systems,
account management including establishment, activation, modification, reviewing, disabling
and removing accounts (e.g., default accounts and passwords, temporary accounts, emergency
accounts, inactive accounts),
auditing account management activities,
managing user identifiers and authenticators: associating a unique identifier with each user or
process, disabling unused identifiers, authorizing user identifiers,
support for individual, role-based, group-based, and device-based user identification
and authentication,
authenticator management: defining authenticator content, distributing authenticators,
periodic changing of authenticators, changing default authenticators,
enforcing assigned authorizations for controlling access to the system, restricting access to
privileged functions and security-relevant information to authorized personnel,
access control support for separation of duties and least privilege,
prohibition of anonymous, guest and public accounts,
obfuscation of authentication input (e.g., displaying asterisks when the password is typed),
controlling the flow of information between interconnected systems in accordance with
applicable policy,
password security including password complexity, password expiration,
acceptable system use notification,
limiting the number of concurrent sessions for a user,
notifying a user, after successful logon, of the date and time of the last successful logon and
the number of intervening unsuccessful logons,
limiting the number of unsuccessful logon attempts,
locking sessions (remote and local) after a period of inactivity,
securing remote access,
preventing access to the system from the operator's enterprise network,
recording and reporting unauthorized access attempts to the system.
22 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
Audit and Accountability ensures the existence and availability of system logs that are used to
detect breaches of system security, anomaly detection, and forensic analysis. The technical
requirements in this category involve:
generation of audit records for security events, control events, configuration changes,
transmission of audit records and logs to a centralized log management system for long-term
storage and correlation,
content of audit records,
local storage capacity for audit records on the system,
alerting when the audit system fails (e.g., the audit storage utilization exceeds a previously
defined percentage of audit storage capacity),
automated detection and alerting mechanisms for inappropriate, unusual or suspicious activity
or security violations,
providing time stamps in audit log records,
protection of audit log information and tools from unauthorized access.
The NISTIR 7628 Smart Grid Cyber Security Strategy and Requirement standard provides comprehensive
technical guidance for securing critical cyber assets as well as non-critical cyber assets within an ESP.
The IEC 62351 series of standards (62351-1 through 62351-7) [10] is concerned with securing the
unique communication protocols used by the power utility industry. These protocols are:
IEC 60870-5 is widely used in Europe and other non-US countries for SCADA system to RTU
data communications. It is used both in serial links (IEC 60870-5-101) and TCP/IP networks
(IEC 60870-5-104). DNP3 was derived from IEC 60870-5 for use in the USA and is now widely
used in many other countries as well, primarily for SCADA system to RTU data communications.
IEC 60870-6 (also known as TASE.2 or ICCP) is used internationally for communications
between control centers and often for communications between SCADA systems and other
engineering systems within control centers.
IEC 61850 is used for protective relaying, substation automation, distribution automation,
power quality, distributed energy resources (DERs), substation to control center, and other
power industry operational functions. It includes profiles to meet the ultra fast response times of
protective relaying and for the sampling of measured values, as well as profiles focused on the
monitoring and control of substation and field equipment.
IEC 62351 series of standards addresses the end-to-end security of these protocols, as such, the
intervening communications equipment is transparent to these security standards. Modbus, Fieldbus
and proprietary communication protocols are used by legacy systems. These protocols typically
provide serial communication between RTUs and SCADA systems and must also be protected
end-to-end.
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 23
Appendix D. The X.805 Security Dimensions
The eight X.805 security dimensions provide a framework to understand the types of measures that
need to be applied to the management plane, control plane, and end-user plane in order to protect
cyber assets within an ESP. (See [2].)
1. The Access Control dimension ensures that only authorized personnel or devices are allowed to
access the communications system. Example access control measures that can be put in place to
maintain an ESP include ACLs, firewalls, IDS/IPS, access filters, user profiles.
2. The Authentication dimension confirms the identities of communicating entities. Example
authentication measures that can be put in place to maintain an ESP include passwords,
two-factor authentication, anti-spoofing, digital signatures, digital certificates.
3. The Non-Repudiation dimension prevents deniability of an action or activity. Example non-
repudiation measures that can be put in place to maintain an ESP include logging and
digital signatures.
4. The Data Confidentiality dimension protects information from unauthorized disclosure; it ensures
that data content cannot be understood by unauthorized individuals. Data encryption and file
system/database access controls are two example data confidentiality measures that can be put
in place to maintain an ESP.
5. The Communication Security dimension ensures that information flows between authorized
end-points without being diverted or intercepted. Example communication security measures
that can be put in place to maintain an ESP include tunneling protocols, IPsec, VLANs, VPNs,
SSH, SSL/TLS.
6. The Data Integrity dimension protects against unauthorized modification, creation, deletion
and replication of data that is in-transit or at rest. It also provides an indication of these
unauthorized activities. Example data integrity measures that can be put in place to maintain an
ESP include MD5 and SHA-1 hashes, message authentication codes, message digests, anti-
virus/anti-malware software.
7. The Availability dimension ensures that there is no denial of authorized access to network
resources. Example availability measures that can be put in place to maintain an ESP include
packet filtering, per-flow queuing, per-peer queuing, business continuity/disaster
recovery/continuity of operations procedures.
8. The Privacy dimension provides for the protection of information that might be derived from
the observation of network activities (e.g., traffic analysis). Example privacy measures that
can be put in place to maintain an ESP include private IP addresses, NAT, web proxies, web
anonymizer services.
24 Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper
Appendix E. Potential Vulnerabilities in the Power Grid
NISTIR 7628 contains an extensive list of vulnerabilities that could be present in power grid
equipment. The vulnerability list is organized into several vulnerability classes that would allow the
four threats to the ESP to be realized in the form of an attack. Since the focus of this paper is the
effect of extending the electronic security perimeter to include previously excluded communications
equipment, we will only consider vulnerability classes applicable to the security functional
capabilities of communication systems.
Code quality vulnerabilities that allow an attacker to stress the system in unexpected ways.
Authentication bypass or other circumvention/manipulation of the authentication process.
Authorization vulnerabilities that allow authenticated entities to perform actions the security
policy does not allow.
Cryptographic vulnerabilities that allow an attacker to view, modify or forge encrypted data as
well as impersonate another party by using compromised digital signatures.
Logging and auditing vulnerabilities that aid an attack or increase its likelihood of success by
allowing the attacker to cover his/her tracks.
Password management vulnerabilities that allow an attacker to obtain or guess passwords.
Use of insecure protocols for which security was not sufficiently considered during the
development process (e.g., telnet, ftp).
Installed security capabilities not enabled by default.
Un-needed services running.
Insufficient log management, inadequate security monitoring and event logging, no centralized
log server.
Inadequate integrity checking of messages; the integrity of protocol messages and data messages
should be verified before routing or processing.
Inadequate network segregation to control traffic between security zones.
Weakness in authentication process or in authentication keys; the authentication mechanism
does not sufficiently authenticate devices or exposes authentication keys to attack.
Achieving NERC CIP Compliance with Secure MPLS Networks | Technology White Paper 25
www.alcatel-lucent.com Alcatel, Lucent, Alcatel-Lucent and the Alcatel-Lucent logo
are trademarks of Alcatel-Lucent. All other trademarks are the property of their respective owners.
The information presented is subject to change without notice. Alcatel-Lucent assumes no responsibility
for inaccuracies contained herein. Copyright 2010 Alcatel-Lucent. All rights reserved.
EPG1806100705 (07)