WHITE PAPER

MPLS WAN Explorer

Enterprise Network Management

Visibility through the
MPLS VPN “Cloud”
MPLS WAN Explorer

Executive Summary
Increasing numbers of enterprises are outsourcing their backbone WAN routing to MPLS
VPN Service Providers. MPLS VPN WAN services have been gaining in market traction
against Frame Relay due to availability of higher bandwidth links, and their price advantage
when delivering “full mesh” application traffic between many sites in the network, as
opposed to simple hub and spoke. However, MPLS VPN WAN services come with some
serious network management liabilities that can be quite costly. Once MPLS VPNs are
deployed, IT loses end-to-end routing and traffic visibility across the WAN backbone. This
loss of visibility makes it more difficult to keep Service Providers accountable for service
quality, causing costly finger-pointing when problems occur. More importantly, the lack of
end-to-end routing and traffic visibility greatly impairs key network operations and
engineering processes, which increases the cost of managing the network while causing
application delivery to suffer.

Packet Design’s MPLS WAN Explorer restores much-needed network-wide routing visibility
to enterprises that utilize MPLS VPN services for their WAN backbone. MPLS WAN Explorer
extends Packet Design’s industry-leading route analytics technology, which leverages the
network’s live routing protocols as a source of network management information. With
MPLS WAN Explorer, enterprises can now see beyond the traditional borders of their
internal networks and understand their end-to-end network, even across MPLS VPNs.
MPLS WAN Explorer greatly improves network monitoring and troubleshooting processes
with network-wide routing visualization, Layer 3 network reachability monitoring and
alerting, re-windable troubleshooting history, end-to-end path tracing, and detailed analysis
tools.

This paper reviews how Layer 3 MPLS VPNs work, and explores the network management
challenges introduced by deploying them. The paper then introduces MPLS WAN Explorer,
the route analytics technology that powers it, how it works across MPLS VPNs, and
illustrates how MPLS WAN Explorer can help enterprises increase the efficiency and
accuracy of key network management processes, keep Service Providers accountable for
service quality, and more successfully deliver end-users’ application traffic.

Layer 3 MPLS VPNs—A Brief Overview

Layer 3 MPLS VPNs are delivered by Service Provider IP/MPLS networks that are organized
into a core of Provider or P routers, and a layer of customer-facing Provider Edge or PE
routers. PE routers are configured to handle multiple VPNs through separate virtual routing
and forwarding (VRF) tables. Each customer’s VPN is handled by dedicated VRFs on various
PEs located in different geographies across the Service Provider’s network.
Interconnectivity between VRFs is delivered by a mesh of MPLS tunnels, with a special
extension of BGP providing control plane mapping of tunnels to VRFs.

When using MPLS-based VPN services, enterprise customers are responsible for
connections from each site to the Service Provider network, by connecting their Customer

 2009 Packet Design, Inc.

1
MPLS WAN Explorer

Edge (CE) router to a PE router and enabling routing, typically using the Border Gateway
Protocol (BGP).

MPLS VPNs are an IP Routing Service—the Implications

While Service Providers often position MPLS VPNs as simply an “IP-enabled” version of the
Frame Relay WANs that enterprises have been familiar with for years, the reality is that
MPLS VPNs are very different from Frame Relay and have a much bigger network
management impact. While Frame Relay is a Layer 2 service over which enterprises
manage and have visibility into WAN routing, MPLS VPNs are an IP routing service. In other
words, the Service Provider not only takes responsibility for providing a Layer 2 “link” and
getting traffic across it, but also for delivering the enterprises’ internal IP routing updates
properly and privately across its shared VPN network. Because this routing aspect of the
MPLS VPN service is delivered via a standards-based MPLS VPN service architecture that
blocks enterprises from seeing into the Service Provider network, IT completely loses end-
to-end visibility across the enterprise network.

Technically speaking, the lack of end-to-end network visibility means that IT engineers are
blind to a key function of IP networks—routing reachability. The role of routing protocols in
IP networks is to ensure that IP subnets (represented by routed prefixes) attached to
routers across the network can communicate with (or reach) each other. With a MPLS VPN
obscuring backbone routing, IT engineers can no longer tell if the network is operating
correctly at an IP routing level. This poses a fundamental monitoring challenge, since
SNMP management systems can show all devices and interfaces being “up”, while
application traffic may be dropped or delayed due to routing-layer issues that are occurring
within the Service Provider network “cloud”, or at the complex BGP peering interface
between PE and CE routers. Without any detailed information on end-to-end routing
reachability, troubleshooting the network aspect of an application problem also becomes
even more of a challenge than normal, often getting stuck in finger-pointing between IT and
the Service Provider. Finally, it becomes even easier to introduce errors into the network
during routine network changes since engineers don’t have any detailed insight into the
actual state of network operations.

The bottom-line impact of the lack of visibility into routing reachability is that key operations
and engineering processes such as monitoring, troubleshooting and planning the network
to ensure application delivery become much more time-consuming, and much less
accurate. Ultimately, these inefficiencies cause operations costs to rise in the face of ever-
increasing demands for networked applications needed to drive business automation.

MPLS WAN Explorer—Visibility through the MPLS VPN Cloud

Packet Design offers a unique solution called MPLS WAN Explorer, which is designed
to help enterprises regain end-to-end network routing and traffic visibility across MPLS
VPNs. MPLS WAN Explorer utilizes Packet Design’s industry-leading route analytics
technology that is deployed in hundreds of large enterprise, government and Service
Provider networks worldwide. Route analytics solutions listen passively to routing

 2009 Packet Design, Inc.

2
MPLS WAN Explorer

protocol exchanges on the network and deliver a “router’s eye view” of Layer 3
connectivity and reachability, providing network engineers with previously unavailable
intelligence on the end-to-end Layer 3 operation of an IP network. Route analytics
works by forming passive (listen-only) peerings with key routers in the network using
standards-based routing protocols such as BGP, OSPF, IS-IS and EIGRP, recording every
routing protocol update, and creating a model of the network that is as accurate as the
routers themselves understand it. In the case of MPLS VPNs, MPLS WAN Explorer extends
route analytics by peering via IBGP with the CE routers and receiving all the routing updates
that the CE routers exchange with other CE routers via the MPLS VPN PE routers. By
combining route analytics understanding of both BGP and IGP, MPLS WAN Explorer
provides visibility into the end-to-routing topology across MPLS VPNs, significantly improving
the accuracy and efficiency of key enterprise IT processes. MPLS WAN Explorer provides a
variety of monitoring, troubleshooting, and other analysis tools that help network managers
make sense of what is happening to their WAN

MPLS VPN Reachability Monitoring, Alerting and Visualization

One of the key missing ingredients in MPLS VPN SLA’s is any provision for guaranteeing IP
reachability. MPLS WAN Explorer helps IT ensure that the backbone routing managed by
the Service Providers is working properly by creating and maintaining a moving window
baseline of per-VPN and per-site routing reachability. Based on user-defined thresholds, it
can monitor and alert on any loss of routing reachability across one or more (redundant)
MPLS VPNs. An intuitive network-wide topology view including the VPN “cloud” provides at
a glance detection of sites that have lost reachability or are experiencing other problems
such as routing policy violations where sites are connected to a VPN that they aren’t
supposed to be.

Figure 1: MPLS WAN Explorer provides end-to-end WAN topology visualization

 2009 Packet Design, Inc.

3
MPLS WAN Explorer

Easy to use monitoring and analysis reports provide detailed reachability information on a
per VPN, site and prefix basis, as seen in Figure 2.

Figure 2: The Reachability from Other Sites report shows a list of VPN sites, their
announced prefixes and percentage of reachability to those prefixes from other sites

Fast Detection of MPLS VPN Routing Outages and Instabilities

MPLS WAN Explorer not only monitors and alerts on per site and per VPN prefix reachability
issues, but can also monitor and alert on VPN site routing outages and instabilities. Watch-
lists of paths between data centers and their satellite user sites can be monitored and
alerts triggered if any path fails or changes. User-set thresholds to monitor for excessive
routing protocol activity (churn) as well as prefix and link flapping can trigger alerts if the
network experiences harmful instabilities that can impact application traffic.

Often times, when enterprises utilize two MPLS VPN Service Providers for fault tolerance, IT
managers have no idea if the primary has failed and the secondary VPN is active, simply
because there has historically been no way to monitor the level of redundancy in the
network. MPLS WAN Explorer provides early warning of increased continuity risk in the
network by alerting on per-VPN loss of reachability. This early warning system helps
network managers quickly alert their Service Provider of problems so that redundancy can
be restored in order to avert a potentially disastrous failure of the network should the
secondary VPN experience a problem. Knowledge of these failures also helps network
managers keep their Service Providers accountable and can even aid enterprises during
contract renegotiations.

Rewindable Troubleshooting History

One of the biggest challenges with managing complex, redundant IP networks is
understanding precisely what happened in the past, whether five minutes or five days ago.

 2009 Packet Design, Inc.

4
MPLS WAN Explorer

This is no less true of trying to troubleshoot what happened in a MPLS VPN service problem.
Fortunately, MPLS WAN Explorer continuously records all routing events and provides a
History Navigator that allows engineers to “rewind the network” back to the point in time
when a problem was occurring to understand the network operation at that moment. MPLS
WAN Explorer even allows historical analysis on a per-site basis.

Figure 3: Engineers can “rewind the network” for more effective troubleshooting using the
History Navigator

End-to-End Path Tracing and Detailed Routing Analyses

Once engineers have rewound the network to the time that a problem was occurring, they
can utilize MPLS WAN Explorer’s end-to-end path tracing to localize the portion of the
network that carried the application traffic, and thus should be examined during the
troubleshooting process. MPLS WAN Explorer’s path tracing provides visibility between
sites across the VPN, including ingress and egress PE routers, and can even traverse
multiple VPNs.

 2009 Packet Design, Inc.

5
MPLS WAN Explorer

Figure 4: MPLS WAN Explorer provides path tracing across MPLS VPNs

MPLS WAN Explorer provides a variety of reports to aid troubleshooting analysis. Detailed
routing analyses with flexible drill-down views allow engineers to further pinpoint the source
of problems within the network. An example troubleshooting scenario is shown in Figures
5-7. In this case, several sites have lost prefix reachability to the Chicago-1 site.

 2009 Packet Design, Inc.

6
MPLS WAN Explorer

Figure 5: A summary reachability report shows reachability problems to Chicago-1

In Figure 6, a drill-down report on site reachability shows that there is variable reachability
to the Chicago-1 site. For example, Atlanta-1 has lost all reachability to Chicago-1. Since
most other sites have retained most of their reachability to Chicago-1, its most likely that
the source of Atlanta-1’s problems are local to Atlanta-1, perhaps due to a down condition
or instability in the EBGP peering between its CE router and the Service Provider’s PE
router.

Figure 6: Flexible drill-downs such as the site reachability report allow engineers to identify
the per-site location of problems in the network

A more complex task is to understand what has happened to sites such as Boston-1, which
have partially lost reachability. With MPLS WAN Explorer, engineers can utilize further drill-
down reports to look at prefix-level reachability and see if individual prefixes can be reached
by any other sites or not, as seen in Figure 7. In this case, one Chicago-1 prefix is reachable
by 17 sites, but not by Boston-1, which means that the source issue is problem at Boston-1.
By contrast, another prefix is not reachable from any sites, meaning that the problem is
local to Chicago-1.

 2009 Packet Design, Inc.

7
MPLS WAN Explorer

Figure 7: Detailed routing reachability analyses allow engineers to further localize the
source of reachability issues on a per-prefix basis.

Powerful BGP Troubleshooting Tools for VPN Peering Problems

As with all other router to router connections, problems in the BGP peerings between
Service Provider PE routers and per-site CE routers can and sometimes do occur. Since
these issues occur at the boundary between two networks, and because BGP is a very
difficult routing protocol to understand, enterprise engineers need powerful tools to
understand what has happened. MPLS WAN Explorer’s rewindable history and topology
visualization allows engineers to easily tell if there was a peering issue by visually verifying if
any CE site was isolated from the VPN. Per-CE router BGP event history tables also allow
engineers to verify if a peering was reset by the CE or the PE router.

Monitoring and Alerting on Breaches in MPLS VPN Privacy

MPLS WAN Explorer can help network managers ensure the privacy and integrity of an
enterprise’s MPLS VPN backbone by alerting on significant changes in the number of
prefixes in the MPLS WAN VPN. Since an enterprise WAN should be relatively stable in the
number of its advertised prefixes, if a large number of prefixes are advertised into the
network in an unexpected manner, then it is possible that the Service Provider has
inadvertently mixed customer VPNs. Once an increase of prefixes has been detected,
engineers can look at the History Navigator’s histogram of levels in advertised prefixes in
the network and find the time when the prefixes entered the network by looking for a jump
in the prefix graph. MPLS WAN Explorer also provides a list of all known prefixes, which can
filtered to show any routes that are advertised but not in the baseline, as seen in Figure 9.

 2009 Packet Design, Inc.

8
MPLS WAN Explorer

Figure 9: Detailed views of new, non-baseline BGP prefixes help detect “foreign” routes

In addition, in cases where the Internet routing table isn’t being advertised into the network,
engineers can also see whether there are unknown BGP Autonomous Systems associated
with routes in the network. When connecting to a provider’s layer 3 VPN service using BGP,
each of the enterprise’s sites must have a unique Autonomous System Number (ASN),
typically private ASNs assigned by the Service Provider. These ASNs in effect represent the
list of VPN sites. The Service Provider’s network should never inject routes into the
customer’s VPN that are from an unknown ASN, as this would indicate that another
customer’s VPN has inadvertently been connected into the VPN. MPLS WAN Explorer
provides a Routing Information Base (RIB) Browser tool that can analyze BGP routing based
on a number of attributes including ASN, and thus show if there are any unknown ASNs in
the network, as seen in Figure 10. Drill-down analyses to historical event details show
when and where the “foreign routes” were introduced to the VPN.

Figure 10: Listing of ASNs and their respective advertised route counts. If an unknown ASN
appears in this listing, then network managers know that the privacy and integrity of their
VPN service has been compromised.

Monitoring of Remote Site IGP Routing Issues

Some CE sites have extensive IGP routed networks behind them, perhaps with multiple
OSPF/IS-IS areas or EIGRP AS. In these cases, network managers also need to be able to
get insight into routing issues within those IGP domains, especially in cases where WAN

 2009 Packet Design, Inc.

9
MPLS WAN Explorer

reachability issues are traced to the IGP domain behind the CE. MPLS WAN Explorer
provides extensive OSPF, IS-IS, and EIGRP monitoring, historical analysis and even scenario
modeling. For more details on how route analytics can be used for a variety of network
management purposes, please visit Packet Design’s white paper library at:
http://www.packetdesign.com/technology/wp.htm

Scalable Monitoring of Satellite Sites

MPLS WAN Explorer can also monitor thousands of smaller, satellite WAN sites that consist
of a CE router with perhaps a single routed prefix and no IGP domain, without requiring a
BGP peering to each CE. Monitoring of paths to important satellite sites allows real-time
alerting to reachability issues to those sites. In addition, MPLS WAN Explorer provides a
real-time updated satellite site reachability report to aid monitoring and analysis.

Integrated Routing and Traffic Analysis across MPLS VPNs

MPLS WAN Explorer not only provides network-wide, end-to-end understanding of routing
and IP reachability dynamics, but when combined with Packet Design’s Traffic Explorer,
enables integrated routing and traffic monitoring, historical analysis, network modeling and
capacity planning across MPLS VPNs. When enabled by MPLS WAN Explorer, Traffic
Explorer provides MPLS VPN specific site-to-site traffic reports that work hand in hand with
MPLS WAN Explorer’s site reachability analysis reports. For more information on Traffic
Explorer, please visit Packet Design’s website at http://www.packetdesign.com

MPLS WAN Explorer Benefits

MPLS WAN Explorer offers enterprise IT managers a number of benefits when deployed to
help manage MPLS VPN services and ensure application delivery across the WAN:

• More responsive monitoring due to real-time alerting on critical network events.

Unlike SNMP, routing protocols operate with milli-second response times. Since
MPLS WAN Explorer’s route analytics “sees” network events at the same speed as
routers, network managers get the benefit of real-time alerting on critical network
issues such as CE to PE peering outages or lost redundancy, lost site reachability
and suspicious additions of routes to the VPN.
• Faster troubleshooting and higher network quality. MPLS WAN Explorer increases
IT engineers’ ability to localize the network problem domain and reduce finger
pointing. Rather than wasting time wondering who’s to blame for a problem or
waiting for the provider to respond, network managers can now proactively find the
source of issues that impact application delivery. Historical problem analysis
prevents past or intermittent problems from continuing to plague application
delivery over time.
• Intelligence to keep Service Providers accountable. Without MPLS WAN Explorer,
enterprises have no visibility to understand whether Service Providers are providing
the level of routing service quality that is needed to support critical networked
applications. In the case where a provider has caused a reachability problem,

 2009 Packet Design, Inc.

10
MPLS WAN Explorer

network managers now have a complete forensic history and powerful visualization
and reporting tools to aid them in holding their provider accountable for service
outages and instabilities.

Conclusion
MPLS WAN Explorer provides enterprise IT managers with the intelligence needed to ensure
that MPLS VPN deployments don’t impede key network operations and engineering
processes and cause costly application delivery problems. With network managers
increasingly being “graded” on application delivery and cost savings rather than just basic
infrastructure availability, MPLS WAN Explorer’s Layer 3 visibility is a must-have capability to
ensure successful and cost-effective WAN management. To learn more about Packet
Design, MPLS WAN Explorer and route analytics, please visit us online at
http://www.packetdesign.com, email us at info@packetdesign.com or call us at 408-490-
1000.

 2009 Packet Design, Inc.

11