DMVPN

BRKSEC-3052
Advanced DMVPN Designs

Alex HONOR
Cisco TAC
ahonore@cisco.com
BRKSEC-3052
2013 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Session Agenda
DMVPN refresher
Review of Phase 3 logic

Per-Tunnel Quality of Service
DMVPN virtualization
MPLS over DMVPN
Multicast over DMVPN
DMVPN with IPv6
Tunnel protection sharing
IKE Profile-based tunnel selection
Migrating from DMVPN to FlexVPN

BRKSEC-3052
Cisco Public
DMVPN refresher
BRKSEC-3052
Cisco Public
Terminology
Core Network
192.168.128.0/17
192.168.101.0/24
Overlay Addresses
192.168.102.0/24
Tunnel Address
Hub 1
Hub 2
Tunnel:
10.0.0.101
Physical: 172.16.101.1
Tunnel:
10.0.0.102
Physical: 172.16.102.1
NBMA Address
Tunnel:
10.0.0.1
Physical: 172.16.1.1
Tunnel:
10.0.0.2
Physical: 172.16.2.1
GRE/IPsec
Tunnels
Spoke 1
Spoke 2
192.168.1.0/24
192.168.2.0/24
Transport Network
BRKSEC-3052
Overlay Network
Cisco Public
Feature History
IOS on 7301, 7200 series, ISR, ISR-G2:

Phases 1 & 2 since 12.3(17), 12.3(14)T6, 12.4M, 12.4(4)T
Phase 3 since 12.4(6)T, available in all 15.x M/T releases
IOS-XE on ASR1k:
Phase 3 support since Release 3S
Cat6500, C7600 with VPN-SPA + Sup720:

No Phase 3 ever
Not recommended for DMVPN (all muscle, little brain)
Recommended combos today:

ASR1k + IOS-XE 3.4.2S (or above): full Phase 3 support, best scalability
ISR-G2 + IOS 15.0M (or above)
BRKSEC-3052
Cisco Public
Base Topology
Core Network
192.168.128.0/17
192.168.101.0/24
192.168.102.0/24
Neighborship
Hub 1
Hub 2
Spoke 1
Spoke 2
192.168.1.0/24
192.168.2.0/24
Spokes can be configured with 1 or more hubs

Each spoke registers (NHRP) to each of the configured hubs
Hub records NBMA and Tunnel address for each registering spoke
BRKSEC-3052
Cisco Public
Phase 3 Synopsis
Core Network
192.168.128.0/17
192.168.101.0/24
192.168.102.0/24
4: Resol. Request
(Dest 192.168.2.2)
192.168.0.0/16 via Hub1
next hop = 10.0.0.101
Hub 1
Hub 2
1: Initial packet flow
6: Tunnel initiation
(S2 to S1)
7: Resol. Reply
(NBMA 172.16.2.1)
Spoke 1
S1 LAN to S2 LAN
Spoke 2
8: Direct packet flow

Tunnel: 10.0.0.1
NBMA: 172.16.1.1
192.168.1.0/24
Tunnel: 10.0.0.2
NBMA: 172.16.2.1
192.168.2.0/24
Spokes receive a summary for the overlay network with the hub(s) as next hop
Hubs send NHRP indirection to source spokes when a more direct path exists
Spokes send resolution requests to resolve destination addresses into NBMA

addresses, reply generated by destination spoke over direct spoke-spoke tunnel
BRKSEC-3052
Cisco Public
Review of Phase 3 logic
BRKSEC-3052
Cisco Public
Configuration
Core Network
192.168.128.0/17
Routing table
C 172.16.101.0/30 Gig0/0 (WAN)

C 192.168.101.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C1 (Core)
NHRP cache
Core 1
192.168.101.0/24
Tunnel: 10.0.0.101
NBMA: 172.16.101.1
Hub 1
Stat 10.0.0.102 172.16.102.1 (H2)
interface Tunnel0
description DMVPN Hub 1
ip address 10.0.0.101
ip nhrp network-id 1
ip nhrp map multicast dynamic
ip nhrp map multicast 172.16.102.1
ip nhrp map 10.0.0.102 172.16.102.1
ip nhrp redirect
tunnel source GigabitEthernet0/0
tunnel mode gre multipoint
tunnel protection ipsec profile TP
Core 2
192.168.102.0/24
Tunnel: 10.0.0.102
NBMA: 172.16.102.1
Hub 2
Static mapping
for Hub 2 (peer)
NHRP mcast
Stat 172.16.102.1 (H2)
Spoke 1
Tunnel: 10.0.0.1
NBMA: 172.16.1.1
192.168.1.0/24
Routing table
C 172.16.1.0/30 Gig0/0 (WAN)

C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
NHRP cache
Stat 10.0.0.101 172.16.101.1 (H1)
interface Tunnel0
description DMVPN Spoke 1
ip address 10.0.0.1
ip nhrp map 10.0.0.101 172.16.101.1
ip nhrp nhs 10.0.0.101
ip nhrp shortcut
tunnel protection ipsec profile TP
Static mapping
for Hub 1 (NHS)
NHRP mcast
Stat 172.16.101.1 (H1)

BRKSEC-3052
Cisco Public
10
1: Spoke Registration
Core Network
192.168.128.0/17
Routing table
C 172.16.101.0/30 Gig0/0 (WAN)

C 192.168.101.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C1 (Core)
NHRP cache
Core 1
192.168.101.0/24
Routing table
Core 2
192.168.102.0/24
Tunnel: 10.0.0.102
NBMA: 172.16.102.1
Tunnel: 10.0.0.101
NBMA: 172.16.101.1
Hub 1
C 172.16.102.0/30 Gig0/0 (WAN)

C 192.168.102.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C2 (Core)
Hub 2
NHRP cache
Stat 10.0.0.102 172.16.102.1 (H2)

Dyn 10.0.0.1 172.16.1.1 (S1)
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 10.0.0.2 172.16.2.1 (S2)
NHRP mcast
NHRP mcast
Stat 172.16.102.1 (H2)

Dyn 172.16.1.1 (S1)
Stat 172.16.101.1 (H1)

Dyn 172.16.2.1 (S2)
Spoke 1
Spoke 2
Tunnel: 10.0.0.1
NBMA: 172.16.1.1
Tunnel: 10.0.0.2
NBMA: 172.16.2.1
192.168.1.0/24
Routing table
C 172.16.1.0/30 Gig0/0 (WAN)

C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
192.168.2.0/24
NHRP cache
Stat 10.0.0.101 172.16.101.1 (H1)
Routing table
NHRP cache
C 172.16.2.0/30 Gig0/0 (WAN)

C 192.168.2.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
Stat 10.0.0.102 172.16.102.1 (H2)
NHRP mcast
NHRP mcast
Stat 172.16.101.1 (H1)

BRKSEC-3052
Stat 172.16.102.1 (H2)

Cisco Public
11
2: Routing Neighborship Establishment

Core Network
192.168.128.0/17
Routing table
C 172.16.101.0/30 Gig0/0 (WAN)

C 192.168.101.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C1 (Core)
B 192.168.102.0/24 10.0.0.102 (H2)
B 192.168.1.0/24 10.0.0.1
B 192.168.2.0/24 10.0.0.102
NHRP cache
Core 1
192.168.101.0/24
Core 2
192.168.102.0/24
Tunnel: 10.0.0.101
NBMA: 172.16.101.1
Hub 1
Routing table
iBGP
(10.0.0.101 .102)
Tunnel: 10.0.0.102
NBMA: 172.16.102.1
Hub 2
C 172.16.102.0/30 Gig0/0 (WAN)

C 192.168.102.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C2 (Core)
B 192.168.101.0/24 10.0.0.101 (H1)
B 192.168.2.0/24 10.0.0.2
B 192.168.1.0/24 10.0.0.101
NHRP cache
Stat 10.0.0.102 172.16.102.1 (H2)

Dyn 10.0.0.1 172.16.1.1 (S1)
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 10.0.0.2 172.16.2.1 (S2)
NHRP mcast
NHRP mcast
Stat 172.16.102.1 (H2)

Dyn 172.16.1.1 (S1)
Stat 172.16.101.1 (H1)

Dyn 172.16.2.1 (S2)
Spoke 1
Spoke 2
Tunnel: 10.0.0.1
NBMA: 172.16.1.1
Tunnel: 10.0.0.2
NBMA: 172.16.2.1
192.168.1.0/24
Routing table
C 172.16.1.0/30 Gig0/0 (WAN)

C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.101
192.168.2.0/24
NHRP cache
Stat 10.0.0.101 172.16.101.1 (H1)
NHRP mcast
Routing table
NHRP cache
C 172.16.2.0/30 Gig0/0 (WAN)

C 192.168.2.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.102
Stat 172.16.101.1 (H1)

BRKSEC-3052
Stat 10.0.0.102 172.16.102.1 (H2)
NHRP mcast
Stat 172.16.102.1 (H2)

Cisco Public
12
3: Spoke-Hub Traffic
4: Route Lookup
Routing table
C 172.16.101.0/30 Gig0/0 (WAN)

C 192.168.101.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C1 (Core)
B 192.168.102.0/24 10.0.0.102 (H2)
B 192.168.1.0/24 10.0.0.1
B 192.168.2.0/24 10.0.0.102
NHRP cache
Core Network
192.168.128.0/17
Core 1
192.168.101.0/24
5: Forwarding
Routing table
Core 2
192.168.102.0/24
Tunnel: 10.0.0.102
NBMA: 172.16.102.1
Tunnel: 10.0.0.101
NBMA: 172.16.101.1
Hub 1
Hub 2
C 172.16.102.0/30 Gig0/0 (WAN)

C 192.168.102.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C2 (Core)
B 192.168.101.0/24 10.0.0.101 (H1)
B 192.168.2.0/24 10.0.0.2
B 192.168.1.0/24 10.0.0.101
NHRP cache
Stat 10.0.0.102 172.16.102.1 (H2)

Dyn 10.0.0.1 172.16.1.1 (S1)
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 10.0.0.2 172.16.2.1 (S2)
NHRP mcast
NHRP mcast
Stat 172.16.102.1 (H2)

Dyn 172.16.1.1 (S1)
Stat 172.16.101.1 (H1)

Dyn 172.16.2.1 (S2)
3: Forwarding
Spoke 1
S1 LAN to Core
dest: 192.168.200.1
Tunnel: 10.0.0.1
NBMA: 172.16.1.1
192.168.1.0/24
NHRP cache
1: Route Lookup
BRKSEC-3052
Tunnel: 10.0.0.2
NBMA: 172.16.2.1
192.168.2.0/24
Routing table
C 172.16.1.0/30 Gig0/0 (WAN)

C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.101
Spoke 2
CEF switching: follow

adjacency on Tun0 for
next hop 10.0.0.101
Stat 10.0.0.101 172.16.101.1 (H1)
2: NHRP Lookup (*)

NHRP mcast
Routing table
NHRP cache
C 172.16.2.0/30 Gig0/0 (WAN)

C 192.168.2.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.102
Stat 172.16.101.1 (H1)

Stat 10.0.0.102 172.16.102.1 (H2)
NHRP mcast
Stat 172.16.102.1 (H2)

Cisco Public
13
4: Spoke-Spoke Traffic: Initial Flow & Indirection

4: Route Lookup
Routing table
C 172.16.101.0/30 Gig0/0 (WAN)

C 192.168.101.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C1 (Core)
B 192.168.102.0/24 10.0.0.102 (H2)
B 192.168.1.0/24 10.0.0.1
B 192.168.2.0/24 10.0.0.102
NHRP cache
Core Network
192.168.128.0/17
Core 1
192.168.101.0/24
Routing table
Core 2
192.168.102.0/24
Tunnel: 10.0.0.101
NBMA: 172.16.101.1
6: Forwarding
Hub 1
Tunnel: 10.0.0.102
NBMA: 172.16.102.1
Hub 2
C 172.16.102.0/30 Gig0/0 (WAN)

C 192.168.102.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C2 (Core)
B 192.168.101.0/24 10.0.0.101 (H1)
B 192.168.2.0/24 10.0.0.2
B 192.168.1.0/24 10.0.0.101
NHRP cache
Stat 10.0.0.102 172.16.102.1 (H2)

Dyn 10.0.0.1 172.16.1.1 (S1)
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 10.0.0.2 172.16.2.1 (S2)
5: NHRP Lookup
NHRP mcast
NHRP mcast
Stat 172.16.102.1 (H2)

Dyn 172.16.1.1 (S1)
Stat 172.16.101.1 (H1)

Dyn 172.16.2.1 (S2)
Spoke 1
Tunnel: 10.0.0.1
NBMA: 172.16.1.1
192.168.1.0/24
NHRP cache
1: Route Lookup
BRKSEC-3052
Tunnel: 10.0.0.2
NBMA: 172.16.2.1
2: NHRP Lookup
Routing table
C 172.16.1.0/30 Gig0/0 (WAN)

C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.101
Spoke 2
S1 LAN to S2 LAN
dest: 192.168.2.10
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 192.168.2.10 (incomplete)
8: Cache Update +
NHRP
mcast
Resolution
Request
192.168.2.0/24
Routing table
NHRP cache
C 172.16.2.0/30 Gig0/0 (WAN)

C 192.168.2.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.102
Stat 172.16.101.1 (H1)
Stat 10.0.0.102 172.16.102.1 (H2)
NHRP mcast
Stat 172.16.102.1 (H2)

Cisco Public
14
5: Spoke-Spoke Traffic: Resolution Request

4: Route Lookup
Routing table
C 172.16.101.0/30 Gig0/0 (WAN)

C 192.168.101.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C1 (Core)
B 192.168.102.0/24 10.0.0.102 (H2)
B 192.168.1.0/24 10.0.0.1
B 192.168.2.0/24 10.0.0.102
NHRP cache
Core Network
192.168.128.0/17
Core 1
Routing table
Core 2
192.168.101.0/24
192.168.102.0/24
Tunnel: 10.0.0.102
NBMA: 172.16.102.1
Tunnel: 10.0.0.101
NBMA: 172.16.101.1
Hub 1
6: Resolution Request
Hub 2
C 172.16.102.0/30 Gig0/0 (WAN)

C 192.168.102.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C2 (Core)
B 192.168.101.0/24 10.0.0.101 (H1)
B 192.168.2.0/24 10.0.0.2
B 192.168.1.0/24 10.0.0.101
NHRP cache
(192.168.2.2/32)
Stat 10.0.0.102 172.16.102.1 (H2)

Dyn 10.0.0.1 172.16.1.1 (S1)
7: Route Lookup
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 10.0.0.2 172.16.2.1 (S2)
5: NHRP Lookup
8: NHRP Lookup
NHRP mcast
NHRP mcast
Stat 172.16.102.1 (H2)

Dyn 172.16.1.1 (S1)
Stat 172.16.101.1 (H1)

Dyn 172.16.2.1 (S2)
Generate NHRP
Resolution Request
Spoke 1
Tunnel: 10.0.0.1
NBMA: 172.16.1.1
1: Route Lookup
192.168.1.0/24
2: NHRP Lookup
Routing table
C 172.16.1.0/30 Gig0/0 (WAN)

C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.101
NHRP cache
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 192.168.2.10 (incomplete)
NHRP mcast
Tunnel: 10.0.0.2
NBMA: 172.16.2.1
192.168.2.0/24
Routing table
C 172.16.2.0/30 Gig0/0 (WAN)

C 192.168.2.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.102
Stat 172.16.101.1 (H1)

BRKSEC-3052
Spoke 2
DMVPN exit point for

destination address
10: Route Lookup +

Resolution Cisco
Reply
Public
NHRP cache
Stat 10.0.0.102 172.16.102.1 (H2)
NHRP mcast
Stat 172.16.102.1 (H2)

15
5: Spoke-Spoke Traffic: Resolution Reply & Shortcut Creation

Core Network
192.168.128.0/17
Routing table
C 172.16.101.0/30 Gig0/0 (WAN)

C 192.168.101.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C1 (Core)
B 192.168.102.0/24 10.0.0.102 (H2)
B 192.168.1.0/24 10.0.0.1
B 192.168.2.0/24 10.0.0.102
NHRP cache
Core 1
192.168.101.0/24
Routing table
Core 2
192.168.102.0/24
Tunnel: 10.0.0.102
NBMA: 172.16.102.1
Tunnel: 10.0.0.101
NBMA: 172.16.101.1
Hub 1
Hub 2
C 172.16.102.0/30 Gig0/0 (WAN)

C 192.168.102.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.128.0/27 C2 (Core)
B 192.168.101.0/24 10.0.0.101 (H1)
B 192.168.2.0/24 10.0.0.2
B 192.168.1.0/24 10.0.0.101
NHRP cache
Stat 10.0.0.102 172.16.102.1 (H2)

Dyn 10.0.0.1 172.16.1.1 (S1)
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 10.0.0.2 172.16.2.1 (S2)
2: Tunnel Initiation
(S2 to S1)
NHRP mcast
Stat 172.16.102.1 (H2)

Dyn 172.16.1.1 (S1)
NHRP mcast
Stat 172.16.101.1 (H1)

Dyn 172.16.2.1 (S2)
3: Resolution Reply
(192.168.2.0/24, NH 10.0.0.2, NBMA 172.16.2.1)
Spoke 1
Tunnel: 10.0.0.1
NBMA: 172.16.1.1
192.168.1.0/24
C 172.16.1.0/30 Gig0/0 (WAN)
C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.101
H 192.168.2.0/24 10.0.0.2
NHRP cache
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 192.168.2.0/24 172.16.2.1
NH = 10.0.0.2
NHRP mcast
Tunnel: 10.0.0.2
NBMA: 172.16.2.1
Generate NHRP
Resolution Reply
4: Cache Update
Routing table
Spoke 2
192.168.2.0/24
Routing table
NHRP cache
C 172.16.2.0/30 Gig0/0 (WAN)

C 192.168.2.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.102
Stat 172.16.101.1 (H1)

Stat 10.0.0.102 172.16.102.1 (H2)

Dyn 10.0.0.1 172.16.1.1 (S1)
Dyn Loc 192.168.2.0/24 172.16.2.1
NHRP mcast
S 172.16.102.1 (H2)
5: RIB Update (15.2T+, IOS-XE)

BRKSEC-3052
1: Cache Update
Cisco Public
16
Phase 3 Shortcuts: Old & New CEF Forwarding

Hub 1
Forwarding table
Forwarding table
B 192.168.0.0/16 10.0.0.101 (H1)
B 192.168.0.0/16 10.0.0.101 (H1)

H 192.168.1.0/24 10.0.0.1 (S1)
NH: 10.0.0.101
NHRP
Shortcut
Switching
NH: 10.0.0.2
NH: 10.0.0.1
NHRP cache
NHRP cache
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 192.168.2.0/24 172.16.2.1
NH = 10.0.0.2 (S2)
Adjacencies
Tu0 10.0.0.101 GRE 172.16.101.1

Tu0 10.0.0.2 GRE 172.16.2.1
Stat 10.0.0.101 172.16.101.1 (H1)

Dyn 192.168.1.0/24 172.16.1.1
NH = 10.0.0.1 (S1)
Spoke 1
(15.1M)
Spoke 2
(15.2T)
Adjacencies
Tu0 10.0.0.101 GRE 172.16.101.1

Tu0 10.0.0.1 GRE 172.16.1.1
# show cef int Tunnel0 | i Output features

Output features: NHRP Shortcut Switching
FIB lookup for S2 LAN next hop = H1
FIB lookup for S2 LAN next hop = S1
Packet hits NHRP Shortcut output feature

NHRP lookup for S2 LAN next hop = S2
No need for NHRP Shortcut output feature

(next hop & adjacency are already correct)
Next hop rewritten to S2
Follow adjacency for S1 on Tunnel ifc
Follow adjacency for S2 on Tunnel ifc

BRKSEC-3052
Cisco Public
17
Per-tunnel Quality of Service
BRKSEC-3052
Cisco Public
The need for QoS
QoS is crucial on DMVPN for:

Sharing network bandwidth
Marshaling bandwidth usage of applications
Meeting application latency & speed requirements
The greedy spoke problem:

Spoke 1
(greedy)
CE 1
Hub
Interface w/ limited downstream rate
Crypto engine or WAN link
Packets are lost, AND

other spokes are starved
BRKSEC-3052
Packets are lost

Spoke 2
Spoke 3
Most common problem

Cisco Public
19
Per-tunnel QoS overview
Per-tunnel QoS will apply dynamic per-spoke QoS policy on hub

Spokes are split into groups (spoke sends group ID during NHRP registration)
Each group is mapped to a QoS template
HQF / CCE framework is used

The feature applies to FlexVPN, DMVPN and EzVPN with dVTI
Not supported for crypto map based designs
Hub crypto engine and WAN link overruns are rare

WAN link overrun could be addressed with aggregate QoS
Spoke downlink overruns are more frequent

Nothing could be done
This is the primary goal of per-tunnel QoS
BRKSEC-3052
Cisco Public
20
Per-tunnel QoS groups
Spokes register as part of a specific group during NHRP registration

Each spoke tunnel inherits the QoS policy for the corresponding group
interface Tunnel0
ip nhrp map group gold service-policy output gold
ip nhrp map group silver service-policy output silver
!
policy-map gold
class class-default
! offer 5Mbps to each spoke in the group
shape average 5000000
policy-map silver
class class-default
! offer 1Mbps to each spoke in the group
Spoke 1
interface Tunnel0
ip nhrp group gold
BRKSEC-3052
Hub
hub# sh ip nhrp group-map

Interface: Tunnel0
NHRP group: gold
QoS policy: gold
Tunnels using the QoS policy:
Tunnel destination overlay/transport address
10.0.0.1/172.16.1.1
NHRP group: silver
QoS policy: silver
Tunnels using the QoS policy:
Tunnel destination overlay/transport address
10.0.0.2/172.16.2.1
10.0.0.3/172.16.3.1
Spoke 2
interface Tunnel0
ip nhrp group silver
Spoke 3
interface Tunnel0
ip nhrp group silver
Cisco Public
21
Hierarchical shaper
Tunnel bandwidth parent policy

Each tunnel is allocated a maximum bandwidth
A shaper provides the backpressure mechanism
Protected packets processed by the client policy

Reserved bandwidth, LLQ, etc.
class-map control
match ip precedence 6
class-map voice
match ip precedence 5
!
policy-map sub-policy
class control
bandwidth 20
class voice
priority percent 60
!
policy-map gold
class class-default
service-policy sub-policy
!
policy-map silver
class class-default
service-policy sub-policy
Reserved BW
Low-latency queuing
Fair queuing
Aggregate shaper
BRKSEC-3052
Cisco Public
22
Per-tunnel QoS sequence
Classification happens at the tunnel level (before encaps & crypto engine)
Policing (dropping) & marking also applied at the tunnel level
Queuing & scheduling happen at the physical interface
BRKSEC-3052
Hierarchical queuing per tunnel
Tunnel 1 Data
Data
Tunnel 1 Voice
Voice
Tunnel 2 Data
Tunnel 2 Voice
Crypto
Engine
Data
Voice
Tunnel 3 Data
Data
Tunnel 3 Voice
Voice
Tunnel 1
Policy
Tunnel 2
Policy
Tunnel 3
Policy
Cisco Public
Derived
Interface QoS Policy
SA Classification
Classification, policing & marking
Physical
Interface
23
DMVPN virtualization
BRKSEC-3052
Cisco Public
Quick review: Virtual Routing/Forwarding
Router maintains separate L3 forwarding information for each

VRF instance (RIB, FIB, routing protocols)
Two variants: VRF with MPLS, and VRF-Lite
Each interface belongs to a single VRF
For ip unnumbered, referenced interface must belong to the same VRF
If no VRF specified, interface belongs to the global VRF
VRF definition and assignment:

Old CLI:
BRKSEC-3052
ip vrf red
rd 1:1
!
interface Ethernet0/0
ip vrf forwarding red
ip address 10.0.0.1 255.255.255.0
New CLI:
vrf definition red

rd 1:1
address-family ipv4
exit-address-family
!
vrf forwarding red
ip address 10.0.0.1 255.255.255.0
Cisco Public
25
Quick review: forwarding & tunneling with VRF-Lite

Blue RIB/FIB
Routing
Eth0/0
Red RIB/FIB
Routing
Eth0/1
Global RIB/FIB
Red RIB/FIB
Routing
Eth1/0
Eth1/1
Routing
Eth2/0
Tunnel1
interface Eth0/0
vrf forwarding blue
!
interface Eth0/1
vrf forwarding blue
Inside VRF (iVRF)

BRKSEC-3052
interface Eth1/0
vrf forwarding red
!
interface Eth1/1
! no VRF = global
!
interface Tunnel1
vrf forwarding red
tunnel source Eth1/1
Orange RIB/FIB
Routing
Eth2/1
Tunnel2
Front VRF
(fVRF)
Cisco Public
interface Eth2/0
vrf forwarding green
!
interface Eth2/1
vrf forwarding orange
!
interface Tunnel2
vrf forwarding green
tunnel vrf orange
tunnel source Eth2/1
iVRF
26
DMVPN virtualization with VRF-Lite (1)
Tunnel interface can be part of only one iVRF

one DMVPN Tunnel per iVRF needed
Spokes can be single-tenant or multi-tenant
(single-tenant not necessarily VRF-aware)
Hub
Spoke-spoke direct communication

One pair of IPsec SAs per peer per iVRF
Spokes
BRKSEC-3052
Cisco Public
27
DMVPN virtualization with VRF-Lite (2)
Convenient if only a few iVRFs

Main drawbacks:
Major configuration overhead if many iVRFs
One hub-spoke routing protocol neighborship per iVRF
Tunnel address ranges cannot overlap if hubs use the
BGP Dynamic Neighbors feature to peer with spokes (CSCtw69765)
If separate authentication is needed for each DMVPN:

Different ISAKMP profiles required (different IKE credentials)
Different IPsec profiles required
Different source interfaces required (same source requires shared profile)
BRKSEC-3052
Cisco Public
28
DMVPN virtualization with MPLS VPN
Single Tunnel interface in global VRF

MPLS VPN labels identify which iVRF
the tunneled traffic belongs to
Hub
Hub-spoke only, no spoke-spoke direct

BGP must be used as the routing protocol
between hubs & spokes
Separate IKE authentication not possible
Single pair of IPsec SAs per peer
BRKSEC-3052
Spokes
Cisco Public
29
MPLS over DMVPN
BRKSEC-3052
Cisco Public
Part 1:
MPLS VPN review
Quick review of MPLS VPN (1)

MPLS basics
MPLS switches packets based solely on labels, protocol agnostic

Label = integer between 0 and (220 1)
Label stack inserted between L2 header and payload
Each MPLS router (LSR):

Binds a label to each IGP prefix
Exchanges prefixes & labels with neighboring LSRs (using LDP or BGP)
Builds a Label Information Base (LIB) with all prefix/label mappings
Builds a Label Forwarding Information Base (LFIB) to forward labeled packets
Updates the CEF Forwarding Information Base (FIB) to label packets as needed
BRKSEC-3052
Cisco Public
32

Forwarding & Penultimate Hop Popping
Packet forwarding on LSR:

FIB lookup for non-labeled packets (possible actions: label packet, forward, )
LFIB lookup for labeled packets (possible actions: push, swap, pop, un-label)
LSRs advertise locally connected prefixes with an implicit-null label

Instructs neighboring LSRs to pop the label on packets to those prefixes
Saves an LFIB lookup (would anyway result in a second lookup in FIB)
This is called Penultimate Hop Popping (PHP)
BRKSEC-3052
Cisco Public
33
Sample MPLS core

10.0.0.1/32
10.0.0.2/32
10.0.0.3/32
LDP
LDP
PE1
P1 Prefixes
10.0.0.4/32
LDP
P2
P1
PE2
10.0.1.0/24
.1
.2
10.0.2.0/24
.1
.2
10.0.3.0/24
.1
.2
OSPF
OSPF
OSPF
= mpls ip configured
P1 LIB
P1 RIB/FIB
P1 LFIB
Type
Via
Interface
Action
Local
On PE1
On P2
Label
Action
Interface Next Hop
10.0.0.1/32
10.0.1.1 (PE1)
Eth0/0
Forward
18
NULL
20
18
POP
Eth0/0
10.0.1.1 (PE1)
10.0.0.2/32
connected
Loop0
Receive
NULL
17
19
10.0.0.3/32
10.0.2.2 (P2)
Eth1/0
Forward
20
20
NULL
20
POP
Eth1/0
10.0.2.2 (P2)
10.0.0.4/32
10.0.2.2 (P2)
Eth1/0
PUSH 18
19
19
18
19
SWAP 18 Eth1/0
10.0.2.2 (P2)
10.0.1.0/24
connected
Eth0/0
Glean
NULL
NULL
21
10.0.2.0/24
connected
Eth1/0
Glean
NULL
18
NULL
10.0.3.0/24
10.0.2.2 (P2)
Eth1/0
Forward
21
21
NULL
21
POP
10.0.2.2 (P2)
Next hop &

output interface
BRKSEC-3052
Local label
Eth1/0
Peer expects packet without a label

Cisco Public
34
Forwarding on MPLS core

10.0.0.1/32
Penultimate hop
10.0.0.2/32
PE1
10.0.0.3/32
10.0.0.4/32
P2
P1
10.0.1.0/24
.1
.2
10.0.2.0/24
.1
.2
PE2
10.0.3.0/24
.1
.2
10.0.0.1 10.0.0.4
Label pushed
10.0.0.1 10.0.0.4
Ethertype: 0x8847
Ethertype: 0x8847
Label: 19
10.0.0.1 10.0.0.4
Label: 18
10.0.0.1 10.0.0.4
Ethertype: 0x0800
10.0.0.1 10.0.0.4
Label popped
Label swapped
PE1 FIB
P1 LFIB
PE2 FIB
P2 LFIB
Prefix
Adjacency
Label
Action
Ifc. NH
Label
Action
Ifc. NH
Prefix
Adjacency
10.0.0.1/32
Receive
18
POP
E0
PE1
18
POP
E0
PE2
10.0.0.1/32
E0 P2 TAG 20
10.0.0.2/32
E0 P1 IP
19
SWAP 18
E1
P2
19
POP
E1
P1
10.0.0.2/32
E0 P2 TAG 19
10.0.0.3/32
E0 P1 TAG 20
20
POP
E1
P2
20
SWAP 18
E1
P1
10.0.0.3/32
E0 P2 IP
10.0.0.4/32
E0 P1 TAG 19
21
POP
E1
P2
21
POP
E1
P1
10.0.0.4/32
Receive
10.0.1.0/24
Glean
10.0.1.0/24
E0 P2 TAG 21
10.0.2.0/24
E0 P1 IP
10.0.2.0/24
E0 P2 IP
10.0.3.0/24
E0 P1 TAG 21
10.0.3.0/24
Glean
BRKSEC-3052
Cisco Public
35

MPLS Virtual Private Networks
Terminology:
Provider (P), Provider Edge (PE), Customer Edge (CE), Customer (C) routers
P and PE routers run an IGP and exchange core labels with LDP
PE routers maintain one VRF instance per connected VPN
and connect to MPLS core within the global VRF (global routing table)
PE routers run an IGP/EGP within the VRF to exchange prefixes with CE
CE routers not necessarily VRF-aware (required if multi-tenant)
PE routers exchange VPN prefixes & labels across the core with MBGP
PE1
CE1
LDP
P1
LDP
P2
LDP
CE2
PE2
VRF red
VRF red
IGP/EGP
C1
IGP (core prefixes only)

Global VRF
IGP/EGP
C2
MBGP (VPNv4 prefixes for VRF red)

BRKSEC-3052
Cisco Public
36
Sample MPLS VPN topology

10.0.0.1/32
PE1
VRF red
10.0.0.2/32
10.0.1.0/24
.1
.2
P1
10.0.0.3/32
10.0.2.0/24
.1
.2
VRF blue
EIGRP
EIGRP
10.10.0.0/30
dot1Q 10
P2
10.0.0.4/32
10.0.3.0/24
.1
.2
VRF red
EIGRP
R1 LAN
10.10.10.0/24
BRKSEC-3052
VRF blue
EIGRP
10.20.0.0/30
dot1Q 20
EIGRP
10.10.0.4/30
dot1Q 10
CE1
10.10.1.0/24
PE2
10.20.0.4/30
dot1Q 20
CE2
10.20.1.0/24
EIGRP
B1 LAN
10.20.10.0/24
10.10.2.0/30
EIGRP
R2 LAN
10.20.20.0/24
Cisco Public
10.20.2.0/30
EIGRP
B2 LAN
10.20.20.0/24
37
Exchanging VPNv4 routes

10.0.0.1/32
PE1
10.0.0.2/32
10.0.1.0/24
.1
.2
P1
LDP
10.0.0.3/32
10.0.2.0/24
.1
.2
P2
10.0.0.4/32
10.0.3.0/24
.1
.2
Core prefixes
& labels via LDP
PE2
VRF red
EIGRP
10.10.0.0/30
dot1Q 10
10.10.1.0/24
EIGRP
R1 LAN
10.10.10.0/24
BRKSEC-3052
iBGP (AS 65100)

router bgp 65100
neighbor PE peer-group
neighbor PE remote-as 65100
neighbor 10.0.0.4 peer-group PE
!
address-family vpnv4
neighbor PE send-community extended
neighbor 10.0.0.4 activate
exit-address-family
!
address-family ipv4 vrf red
redistribute connected
redistribute eigrp 1
exit-address-family
!
address-family ipv4 vrf blue
exit-address-family
VPNv4 prefixes & labels via MBGP (SAFI 128)

65100:10:10.10.0.0/30 via 10.0.0.1 label 22
65100:10:10.10.1.0/24 via 10.0.0.1 label 23
65100:10:10.10.10.0/24 via 10.0.0.1 label 24
Extended Community: RT:65100:10
65100:20:10.20.0.0/30 via 10.0.0.1 label 25
65100:20:10.20.1.0/24 via 10.0.0.1 label 26
65100:20:10.20.10.0/24 via 10.0.0.1 label 27
vrf definition red
rd 65100:10
route-target both 65100:10
vrf definition blue
rd 65100:20
route-target both 65100:20
Cisco Public
38
Forwarding across MPLS VPN

10.0.0.1/32
10.0.0.2/32
Transport
label
VRF red
CE1
10.10.0.0/30
PE1
10.0.1.0/24
.1
.2
10.0.0.3/32
VRF red
10.0.2.0/24
.1
.2
P1
10.0.0.4/32
10.0.3.0/24
.1
.2
P2
PE2
10.10.0.4/30
10.10.2.0/24
10.10.1.0/24
Ethertype: 0x8847
Ethertype: 0x0800
10.10.1.2 10.10.2.2
Ethertype: 0x8847
Label: 18 (top)
Label: 23 (bottom)
10.10.1.2 10.10.2.2
Label: 19 (top)
Label: 23 (bottom)
10.10.1.2 10.10.2.2
PE1 FIB (VRF red)

Prefix
Adjacency
10.10.0.0/30
Glean
10.10.0.4/30
10.0.0.4 22
10.10.1.0/24
E1.10 CE1 IP
10.10.2.0/24
10.0.0.4 23
10.10.10.0/24
E1.10 CE1 IP
10.10.20.0/24
10.0.0.4 24
VPN label
Ethertype: 0x8847
Label: 23
10.10.1.2 10.10.2.2
P1 LFIB
Adjacency
...
10.0.0.4/32
E0 P1 TAG 19
BRKSEC-3052
Ethertype: 0x0800
10.10.1.2 10.10.2.2
Top label popped
Top label swapped

P2 LFIB
VPN label popped

PE2 LFIB
Label
Action
Ifc. NH
Label
Action
Ifc. NH
Lbl. Action
18
POP
E0
PE1
18
POP
E0
PE2
17
19
SWAP 18
E1
P2
19
POP
E1
P1
20
POP
E1
P2
20
SWAP 18
E1
P1
22
No Label
red
aggregate
21
POP
E1
P2
21
POP
E1
P1
23
No Label
red
E1.10 CE2
24
No Label
red
E1.10 CE2
25
No Label
blue
aggregate
26
No Label
blue
E1.20 CE2
27
No Label
blue
E1.20 CE2
PE1 FIB (global)

Prefix
CE2
LFIB entry contains adjacency

no need for FIB lookup
VRF
POP
Ifc.
NH
E0
P2
...
Cisco Public
39
Part 2:
MPLS over DMVPN
Synopsis
Tunnel0
10.0.0.1/24
MBGP
Tunnel0
10.0.0.101/24
VRF red
VRF red
Spoke1
(PE)
VRF blue
192.168.11.11 192.168.12.12
VPNv4 prefix from Hub1:

red:192.168.0.0/16 label 16
Global VRF
VRF blue
Hub1
(PE)
172.16.1.1 172.16.101.1
IPsec (ESP transport mode)
GRE (protocol: 0x8847)
Label: 16
192.168.11.11 192.168.12.12
192.168.11.11 192.168.12.12
Tunnel interface in global, not part of any customer VRF
Hub and spokes act as PE routers, exchange VRF prefixes over MBGP
mGRE Tunnel creates a back-to-back connection
spoke LSR is the penultimate hop only the VPN label is pushed
LDP still required for a supported design
BRKSEC-3052
Cisco Public
41
Encapsulation sequence (1)

VRF blue
Global
Output interface
Input interface
IP Data
Encryption
GRE encapsulation
IP
GRE
IP Data
IP ESP
GRE
IP Data ESP
VRF-Lite
MPLS over DMVPN
VRF red
Global
Output interface
Input interface
IP Data
Label imposition
MPLS
IP Data
Encryption
GRE encapsulation
IP GRE MPLS
BRKSEC-3052
IP ESP GRE MPLS
IP Data ESP
IP Data
Cisco Public
42
Encapsulation sequence (2)

VRF blue
Global
VRF red
GRE encapsulation
VRF-Lite
Encryption
MPLS over DMVPN

VRF blue
VRF red
Label imposition
BRKSEC-3052
Global
Single IPsec tunnel

Single GRE Tunnel
Cisco Public
43
VRF & LAN interfaces (spoke & hub)
Set extended community 1:1 on

outgoing VPNv4 routes from VRF red
Import incoming VPNv4 routes with
extended community 1:1 into VRF red
Enable IPv4 within VRF red
(new CLI; not required with old CLI)
Place Eth1/1 within VRF red
Place Eth1/2 within VRF blue
BRKSEC-3052
vrf definition red

rd 1:1
route-target export 1:1
route-target import 1:1
!
address-family ipv4
exit-address-family
!
vrf definition blue
rd 2:2
route-target export 2:2
route-target import 2:2
!
address-family ipv4
exit-address-family
!
vrf forwarding red
ip address 192.168.10.101 255.255.255.0
!
vrf forwarding blue
ip address 192.168.220.101 255.255.255.0
Cisco Public
44
mGRE tunnel (spoke & hub)

No vrf forwarding statement
cleartext traffic enters mGRE
tunnel from the global VRF
No tunnel vrf statement
GRE traffic encrypted & routed
using the global routing table
No ip nhrp shortcut and

ip nhrp redirect statements
(no spoke-to-spoke tunnels)
Enable MPLS on interface
using labels received from BGP
BRKSEC-3052
interface Tunnel0
ip address 10.0.0.1 255.255.255.0
no ip redirects
ip nhrp map 10.0.0.101 172.16.101.1
mpls bgp forwarding
mpls ip
tunnel source Ethernet0/0
tunnel protection ipsec profile dmvpn
interface Tunnel0
ip address 10.0.0.101 255.255.255.0
no ip redirects
ip nhrp map 10.0.0.102 172.16.102.1
mpls bgp forwarding
mpls ip
tunnel protection ipsec profile dmvpn
Cisco Public
Spoke
Hub
45
Spoke BGP configuration

Neighborships, VRF, redistribution & VPNv4
Use iBGP for spoke-hub peerings

Define hub neighborships statically
Send RT extended community

along with VPNv4 routes
Redistribute connected networks

that belong to VRF red into BGP
Redistribute LAN routes learned from
EIGRP within VRF red into BGP
BRKSEC-3052
router bgp 65100

bgp router-id 172.16.1.1
bgp log-neighbor-changes
neighbor hubs peer-group
neighbor hubs remote-as 65100
neighbor hubs update-source Tunnel0
neighbor 10.0.0.101 peer-group hubs
!
neighbor hubs send-community extended
exit-address-family
!
exit-address-family
!
exit-address-family
Cisco Public
46
Hub BGP configuration

Listener, neighborships, VRF, redistribution & VPNv4
Listen for incoming BGP connections
from all spoke tunnel addresses
Use iBGP for hub-spoke
and hub-hub peerings
Define hubs neighborships statically

Send RT extended community
along with VPNv4 routes
Reflect spoke routes to other hubs
Set next hop to self on iBGP routes sent to other hubs
Redistribute connected networks
that belong to VRF blue into BGP
Redistribute LAN routes learned from
EIGRP within VRF blue into BGP
BRKSEC-3052
router bgp 65100

bgp listen range 10.0.0.0/24 peer-group spokes
bgp listen limit 10
neighbor spokes peer-group
neighbor spokes remote-as 65100
neighbor spokes update-source Tunnel0
!
neighbor spokes activate
neighbor spokes send-community extended
neighbor hubs send-community extended
neighbor hubs route-reflector-client
neighbor hubs next-hop-self all
exit-address-family
!
exit-address-family
exit-address-family
Cisco Public
47
Route summarization (1)

Option 1: VPNv4 default routes
VRF red
VRF blue
Spoke
Hub
1:1:0.0.0.0/0 via 10.0.0.101 label 16

2:2:0.0.0.0/0 via 10.0.0.101 label 17
Import static Null0 default for

VRF blue and advertise it into BGP
Ensure that:
spokes will only receive one default per VRF
hubs will receive spoke prefixes but no defaults
BRKSEC-3052
router bgp 65100

!
neighbor spokes prefix-list default-only out
neighbor hubs prefix-list no-default out
exit-address-family
!
network 0.0.0.0
default-information originate
exit-address-family
!
network 0.0.0.0
default-information originate
exit-address-family
!
ip route vrf red 0.0.0.0 0.0.0.0 Null0
ip route vrf blue 0.0.0.0 0.0.0.0 Null0
!
ip prefix-list default-only seq 5 permit 0.0.0.0/0
!
ip prefix-list no-default seq 5 deny 0.0.0.0/0
ip prefix-list no-default seq 10 permit 0.0.0.0/0 le 32
Cisco Public
48
Route summarization (2)

Option 2: VPNv4 overlay summaries
VRF red
VRF blue
Spoke
Hub
1:1:192.168.0.0/17
via 10.0.0.101 label 16
2:2:192.168.128.0/17 via 10.0.0.101 label 17
Ensure that:
spokes will only receive the summary routes
hubs will receive spoke prefixes but no summaries
Match on RT value 2:2 in outbound VPN routes

and compare with summary prefix for VRF blue
BRKSEC-3052
router bgp 65100

neighbor spokes route-map summary-only out
neighbor hubs route-map no-summary out
network 192.168.0.0 mask 255.255.128.0
network 192.168.128.0 mask 255.255.128.0
!
ip route vrf red 192.168.0.0 255.255.128.0 Null0
ip route vrf blue 192.168.128.0 255.255.128.0 Null0
!
ip prefix-list sum-red seq 5 permit 192.168.0.0/17
ip prefix-list sum-blue seq 5 permit 192.168.128.0/17
!
ip extcommunity-list 11 permit rt 1:1
ip extcommunity-list 22 permit rt 2:2
!
route-map no-summary deny 11
match ip address prefix-list sum-red
match extcommunity 11
route-map no-summary deny 22
match ip address prefix-list sum-blue
route-map no-summary permit 65535
!
route-map summary-only permit 11
match ip address prefix-list sum-red
route-map summary-only permit 22
match ip address prefix-list sum-blue
Cisco Public
49
Packet forwarding (1)
Hub1
VRF red
VRF red
192.168.12.0/24
192.168.11.0/24
192.168.111.0/24
192.168.111.2
192.168.112.2
.2
Spoke1
Spoke2
.2
192.168.112.0/24
172.16.1.1 172.16.101.1
Label: 36
192.168.111.2 192.168.112.2
spoke1#show ip route vrf red

Gateway of last resort is 10.0.0.101 to network 0.0.0.0
B*
0.0.0.0/0 [200/0] via 10.0.0.101, 00:00:10
192.168.11.0/24 is variably subnetted, 2 subnets, 2 masks
C
192.168.11.0/24 is directly connected, Ethernet1/1
L
R
192.168.111.0/24 [120/1] via 192.168.11.11, 00:00:08, Ethernet1/1
spoke1#show ip cef vrf red 0.0.0.0 0.0.0.0
0.0.0.0/0
nexthop 10.0.0.101 Tunnel0 label 36
BRKSEC-3052
FIB lookup leads to label imposition
Cisco Public
50
Detailed FIB entry & Tunnel adjacency

spoke1#show ip cef vrf red 0.0.0.0 0.0.0.0 internal
0.0.0.0/0, epoch 0, flags rib defined all labels, default route, RIB[B], refcount 5, per-destination sharing
output chain: label 36 TAG midchain out of Tunnel0, addr 10.0.0.101 F33D7CB0
IP adj out of Ethernet0/0, addr 172.16.1.2 F33D7F10
spoke1#show adjacency 10.0.0.101 Tunnel0 detail
Packets following default route will be labeled
Protocol Interface
Address
IP
Tunnel0
10.0.0.101(7)
Encap length 24
4500000000000000FF2FFDABAC100101
AC10650100000800
Tun endpt
Specific TAG adjacency for labeled packets
Next chain element:
IP adj out of Ethernet0/0, addr 172.16.1.2
TAG
Tunnel0
10.0.0.101(5)
0x2F = 47 = GRE
Encap length 24
4500000000000000FF2FFDABAC100101
172.16.1.1 = mGRE tunnel source
AC10650100008847
Tun endpt
0x8847 = MPLS unicast
Next
chain
element:
172.16.101.1 = Hub1 NBMA address
IP adj out of Ethernet0/0, addr 172.16.1.2
BRKSEC-3052
Cisco Public
51
Packet forwarding (2)
Hub1
VRF red
VRF red
192.168.12.0/24
192.168.11.0/24
192.168.111.0/24
192.168.111.2
192.168.112.2
.2
Spoke1
Spoke2
172.16.1.1 172.16.101.1
Label: 36
192.168.111.2 192.168.112.2
hub1#show mpls forwarding-table labels

Local
Outgoing
Prefix
Label
Label
or Tunnel Id
36
No Label
0.0.0.0/0[V]
36
Bytes Label
Switched
0
hub1#show ip cef vrf red 192.168.112.2

192.168.112.0/24
nexthop 10.0.0.2 Tunnel0 label 19
spoke2#show mpls forwarding-table labels 19
Local
Outgoing
Prefix
Bytes Label
Label
Label
or Tunnel Id
Switched
19
No Label
192.168.112.0/24[V]
\
114
BRKSEC-3052
.2
192.168.112.0/24
172.16.101.1 172.16.2.1
Label: 19
192.168.111.2 192.168.112.2
Outgoing
Next Hop
interface
aggregate/red
192.168.111.2
192.168.112.2
Aggregate of multiple prefixes

FIB lookup needed
FIB lookup yields Spoke2 and new label

Spoke2 LFIB contains adjacency to LAN next hop router
Outgoing
interface
Next Hop
Et1/1
192.168.12.12
Cisco Public
52
Multicast over DMVPN
BRKSEC-3052
Cisco Public
NHRP Replication: Local Multicast

interface Tunnel0
ip nhrp map 10.0.0.102 172.16.102.1
(LAN) Gig0/1
Locally generated multicast
hub1# show ip nhrp multicast

I/F
NBMA address
Tunnel1
172.16.102.1
Flags: static
Tunnel1
172.16.1.1
Flags: dynamic
Hub 1
Tun0 10.0.0.101/24
Replication by NHRP
(WAN) Gig0/0
NHRP always performs replication of

local multicast (224.0.0.0/24)
To Spoke 1
(NBMA 172.16.1.1)
To Hub 2
(NBMA 172.16.102.1)
NHRP multicast mappings can be static

or dynamic (spoke registration)
Each packet gets replicated to all NBMA
addresses in the NHRP multicast table
BRKSEC-3052
Cisco Public
54
PIM: Hub-Spoke & Spoke-Hub Multicast

Source
interface Tunnel0
ip pim nbma-mode
ip pim sparse-mode
ip nhrp map 10.0.0.102 172.16.102.1
Multicast traffic (225.1.1.1)

(LAN) Gig0/1
Transit multicast traffic is always

replicated in the forwarding path
Hub 1
Tun0 10.0.0.101/24
Selective replication by CEF
PIM neighborship established

thanks to NHRP multicast replication
(WAN) Gig0/0
Only PIM-SM and PIM-SSM are supported
To Spoke 1
(Tunnel 10.0.0.1)
ip pim nbma-mode instructs PIM

to set up multicast traffic replication
NBMA-mode required:
To Hub 2
(Tunnel 10.0.0.102)
Receiver
on hubs and IOS-XE spokes (always)
on IOS spokes (if hosting a source)
No receiver
(*, 225.1.1.1), 00:19:28/00:03:26, RP 1.1.1.1, flags: S

Incoming interface: Null, RPF nbr 0.0.0.0
Outgoing interface list:
Tunnel0, 10.0.0.1, Forward/Sparse, 00:19:28/00:03:26
Group OIL
BRKSEC-3052
Cisco Public
55
PIM-SM: Spoke-Spoke Multicast (1)

Unicast RIB
C 172.16.1.0/30 Gig0/0 (WAN)
C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.101 (summary)
B 1.1.1.1/32 10.0.0.101 (anycast RP)
Core 1
192.168.101.0/24
Tunnel: 10.0.0.101
NBMA: 172.16.101.1
Anycast RP:
1.1.1.1
Core 2
192.168.102.0/24
Tunnel: 10.0.0.102
NBMA: 172.16.102.1
MSDP
Hub 1
Multicast RIB
C 172.16.1.0/30 Gig0/0 (WAN)
C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B + 192.168.0.0/16 10.0.0.101 (summary)
B + 1.1.1.1/32 10.0.0.101 (anycast RP)
Core Network
192.168.128.0/17
Hub 2
PIM Join Shared Tree
Spoke 2
Spoke 1
IP mroutes
(*, 225.0.0.12), , RP 1.1.1.1, flags: SJC
Incoming interface: Tunnel0, RPF nbr 10.0.0.101
Gig1/0, Forward/Sparse, 05:09:41/00:02:59
Tunnel: 10.0.0.1
NBMA: 172.16.1.1
Tunnel: 10.0.0.2
NBMA: 172.16.2.1
192.168.1.0/24
192.168.2.0/24
IGMP Join 225.0.0.12

Receiver (.2)
(Group: 225.0.0.12)
BRKSEC-3052
Source (.2)
(Group: 225.0.0.12)
Cisco Public
56

Unicast RIB
C 172.16.1.0/30 Gig0/0 (WAN)
C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.101 (summary)
B 1.1.1.1/32 10.0.0.101 (anycast RP)
Core 1
192.168.101.0/24
Tunnel: 10.0.0.101
NBMA: 172.16.101.1
Core Network
192.168.128.0/17
Anycast RP:
1.1.1.1
Core 2
192.168.102.0/24
Tunnel: 10.0.0.102
NBMA: 172.16.102.1
Hub 1
Multicast RIB
C 172.16.1.0/30 Gig0/0 (WAN)
C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B + 192.168.0.0/16 10.0.0.101 (summary)
B + 1.1.1.1/32 10.0.0.101 (anycast RP)
Hub 2
PIM Prune Shared Tree
Source Tree
PIM Join Source Tree
Spoke 2
Spoke 1
IP mroutes
(*, 225.0.0.12), , RP 1.1.1.1, flags: SJC
(192.168.2.2, 225.0.0.12), , flags: JT
BRKSEC-3052
Shared Tree
Tunnel: 10.0.0.1
NBMA: 172.16.1.1
Tunnel: 10.0.0.2
NBMA: 172.16.2.1
192.168.1.0/24
192.168.2.0/24
Traffic to 225.0.0.12
Receiver (.2)
(Group: 225.0.0.12)
Source (.2)
(Group: 225.0.0.12)
Cisco Public
57

Unicast RIB
C 172.16.1.0/30 Gig0/0 (WAN)
C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B 192.168.0.0/16 10.0.0.101 (summary)
B 1.1.1.1/32 10.0.0.101 (anycast RP)
H 192.168.2.0/24 10.0.0.2 (S2)
Core Network
192.168.128.0/17
Core 1
192.168.101.0/24
Core 2
192.168.102.0/24
Anycast RP:
1.1.1.1
Tunnel: 10.0.0.101
NBMA: 172.16.101.1
Tunnel: 10.0.0.102
NBMA: 172.16.102.1
Hub 1
Hub 2
Multicast RIB
C 172.16.1.0/30 Gig0/0 (WAN)
C 192.168.1.0/24 Gig0/1 (LAN)
C 10.0.0.0/24 Tun0 (DMVPN)
B + 192.168.0.0/16 10.0.0.101 (summary)
B + 1.1.1.1/32 10.0.0.101 (anycast RP)
H + 192.168.2.0/24 10.0.0.2 (S2)
Spoke 1
IP mroutes
(*, 225.0.0.12), , RP 1.1.1.1, flags: SJC
(192.168.2.2, 225.0.0.12), , flags: JT
Incoming interface: Tunnel0, RPF
RPFnbr
nbr10.0.0.101
10.0.0.2
BRKSEC-3052
Shared Tree
PIM Prune Shared Tree
Spoke 2
PIM Join Source Tree
Tunnel: 10.0.0.1
NBMA: 172.16.1.1
192.168.1.0/24
No PIM
neighborship !
Tunnel: 10.0.0.2
NBMA: 172.16.2.1
192.168.2.0/24
Traffic to 225.0.0.12
Receiver (.2)
(Group: 225.0.0.12)
Similar issue with PIM-SSM

(cannot join (S,G) at all)
Cisco Public
Source (.2)
(Group: 225.0.0.12)
58
Why did this happen ?
A mechanism exists in PIM-SM to prevent this situation: the T flag

T flag set on (S,G) entry if traffic was received over the source-based tree
Prune shared tree & join source-based tree only if T flag is set on (S,G) entry
Not effective for DMVPN

RP and source are behind the same mGRE interface
Traffic from RP comes in through mGRE Tunnel
mGRE Tunnel = RPF interface towards the source
T flag is set while it should not

(multicast believes that it has seen traffic
over the source-based tree, while it was
actually coming from the RP)
BRKSEC-3052
(*, 225.0.0.12), , RP 1.1.1.1, flags: SJC

(192.168.2.2, 225.0.0.12), , flags: JT
Cisco Public
59
Possible solutions (1)

Option 1: SPT-threshold infinity
Simplest solution: never switch to source-based tree

ip pim vrf <ivrf> spt-threshold infinity
Drawbacks:
Not applicable for source-specific multicast (no shared tree)
Must be configured on all multicast-enabled routers on spoke LANs
(so that none of the last-hop routers will try to join the SPT)
Not selective, applies to all multicast traffic within the iVRF
Prevents creation of (S,G) entries reduced granularity in show commands
Alternative: cancel adverse effect of NHRP route on RPF check

RPF check must keep pointing towards the hub
NHRP route must no longer be replicated into multicast RIB
BRKSEC-3052
Cisco Public
60
Possible solutions (2)

Option 2: push multicast default route
Best solution: push multicast default route from hubs via MBGP (SAFI 2)
Overrides replication of BGP routes
Hub 1
Compatible with anycast RP

Caution: SAFI 129 support required
Multicast [vrf lan]:
if iVRF is not global/default
0.0.0.0/0 via Hub1
IOS-XE: 3.5S (IPv4), 3.7S (IPv6)
Spoke 1
IOS: 15.3(1)T (IPv4 & IPv6)

Routing Table: lan:multicast
Gateway of last resort is 10.0.0.101 to network 0.0.0.0
B*
0.0.0.0/0 [200/0] via 10.0.0.101 (lan), 00:00:00
C
10.0.0.0/24 is directly connected, Tunnel0
L
10.0.0.1/32 is directly connected, Tunnel0
C
L
BRKSEC-3052
router bgp 65100

address-family ipv4 multicast [vrf lan]
neighbor spokes peer-group
neighbor spokes remote-as 65100
neighbor spokes update-source Tunnel0
neighbor spokes default-originate
exit-address-family
router bgp 65100
address-family ipv4 multicast [vrf lan]
exit-address-family
Unicast routes replicated from

BGP & NHRP have disappeared
Cisco Public
61
DMVPN with IPv6
BRKSEC-3052
Cisco Public
IPv6 support in DMVPN
IPv6 overlay supported since IOS 12.4(20)T and IOS-XE 3.7S

Transport/NBMA is still IPv4-only
Dual-stack: IPv6 & IPv4 data packets over the same mGRE tunnel
Complete set of NHRP commands for IPv6
NHRP registers both global and link-local IPv6 addresses
Phase 3 designs only
Spoke
Hub
interface Tunnel0
ipv6 address fe80::2001 link-local
ipv6 address 2001::1/64
ipv6 nhrp network-id 1
ipv6 nhrp map multicast dynamic
interface Tunnel0
ipv6 address fe80::2002 link-local
ipv6 address 2001::2/64
ipv6 nhrp map 2001::1 172.17.0.1
ipv6 nhrp map multicast 172.17.0.1
ipv6 nhrp nhs 2001::1
ipv6 nhrp network-id 1
IPv6 transport support added in IOS 15.2(1)T and IOS-XE 3.8S

BRKSEC-3052
Cisco Public
63
Shared tunnel protection
BRKSEC-3052
Cisco Public
Reminder: static crypto map vs. tunnel protection

crypto ipsec transform-set tset
mode transport
!
crypto map cmap 10 ipsec-isakmp
set peer 172.16.100.1
set transform-set tset
match address gre-tun0
!
(172.16.1.1) Gi0/0
interface GigabitEthernet0/0
ip address 172.16.1.1 255.255.255.0
crypto map cmap
!
Must be repeated for every
interface Tunnel1
new GRE/IPsec tunnel
ip address 10.1.0.1 255.255.255.252
tunnel destination 172.16.100.1
!
ip access-list extended gre-tun1
(172.16.100.1) Gi0/0
permit gre host 172.16.1.1 host 172.16.100.1
Tun1 (10.1.0.1)
IPsec protection
GRE tunneling
Tun1 (10.1.0.2)
crypto ipsec transform-set tset
mode transport
IPsec profile = crypto map
!
crypto ipsec profile tp
template (no peer, no ACL)
Makes
!
interface Tunnel1
ip address 10.1.0.1 255.255.255.252
Single config statement for
every new GRE/IPsec tunnel
tunnel protection ipsec profile tp
BRKSEC-3052
Cisco Public
mGRE possible
65
Background: IPsec data structures
Security Policy Database (SPD):

Describes traffic that should be protected with IPsec
Describes which parameters to use (algorithms, tunnel/transport mode)
Control plane-only concept
show crypto map
Security Association Database (SAD):

Describes traffic that we are currently protecting with IPsec
Control & data plane concept
show crypto ipsec sa
SPD and SAD maintained together within SP/SA DB

BRKSEC-3052
Cisco Public
66
Scope of SP/SA DB: static crypto map (1)
One SP/SA DB per IKE/IPsec endpoint

Single crypto map on single interface
one SP/SA DB for the crypto map
Gig1/0
Tun1
Single crypto map on two interfaces

two separate SP/SA DBs
Each interface becomes a separate
IKE/IPsec endpoint
SPDs are distinct but identical
(protecting the same traffic)

set peer 172.16.100.1
!
ip address 172.16.1.1 255.255.255.0
crypto map cmap
Gig0/0
= crypto map
Single SP/SA DB
= SP/SA DB scope
Gig1/0
Tun1
SADs are maintained separately

(tunnels can be up at the same time)
IKE & IPsec traffic terminates
on each interfaces IP address
BRKSEC-3052
Gig0/0
Gig0/1
Multiple SP/SA DBs

set peer 172.16.100.1
!
ip address 172.16.1.1 255.255.255.0
crypto map cmap
!
ip address 172.16.2.1 255.255.255.0
crypto map cmap
Cisco Public
67
Scope of SP/SA DB: static crypto map (2)

= crypto map
With crypto map local-address
Gig1/0
Tun1
Defines a single IKE/IPsec endpoint for the map
Loop0
Can be an interface with no crypto map applied
Single SP/SA DB:

Attached to the configured local address
Shared by all interfaces where map is applied
router#show crypto map
Crypto Map: "cmap" idb: Loopback0 local address: 172.16.3.1
Crypto Map IPv4 "cmap" 10 ipsec-isakmp
Peer = 172.16.100.1
Extended IP access list gre-tun0
access-list gre-tun0 permit gre host
Current peer: 172.16.100.1
Interfaces using crypto map cmap:

Ethernet0/0
Ethernet0/1
BRKSEC-3052
= SP/SA DB scope
Gig0/0
Gig0/1
Single
SP/SA DB
Root: Loop0
interface Loopback0
ip address 172.16.3.1 255.255.255.255
!
crypto map cmap local-address Loopback0
set peer 172.16.100.1
!
ip address 172.16.1.1 255.255.255.0
crypto map cmap
!
ip address 172.16.2.1 255.255.255.0
crypto map cmap
Cisco Public
68
Tunnel Protection & Crypto Secure Sockets (1)
Tunnel Protection feature

Signals to Crypto Secure Sockets
Crypto Secure Sockets subsystem

!
interface Tunnel1
ip address 10.1.0.1 255.255.255.252
Creates ad hoc crypto map based on IPsec profile

Creates new SP/SA DB for this tunnel
Binds new crypto map and SP/SA DB to protected interface
router#show crypto sockets
Number of Crypto Socket connections 1
IKE endpoints
Tu1 Peers (local/remote): 172.16.1.1/172.16.100.1

Local Ident (addr/mask/port/prot): (172.16.1.1/255.255.255.255/0/47)
Remote Ident (addr/mask/port/prot): (172.16.100.1/255.255.255.255/0/47)
IPSec Profile: "tp"
Socket State: Open
GRE endpoints
Client: "TUNNEL SEC" (Client State: Active)
Crypto Sockets in Listen state:
Client: "TUNNEL SEC" Profile: "tp" Map-name: "Tunnel1-head-0"
BRKSEC-3052
Tunnel
Protection
Crypto
Sockets
Listener for incoming connections

Cisco Public
69
Tunnel Protection & Crypto Secure Sockets (2)

Crypto Map IPv4 "Tunnel1-head-0" 65536 ipsec-isakmp
Profile name: tp
Security association lifetime: 4608000 kilobytes/3600 seconds
Responder-Only (Y/N): N
PFS (Y/N): N
Transform sets={
tset: { esp-256-aes esp-sha-hmac } ,
}
Entry 65536:
crypto socket listener
(similar to dynamic-map)
Crypto Map IPv4 "Tunnel1-head-0" 65537 ipsec-isakmp

Map is a PROFILE INSTANCE.
Each new socket triggers the creation of a
Peer = 172.16.100.1
profile instance entry in the tunnel map-head
Extended IP access list
access-list permit gre host 172.16.1.1 host 172.16.100.1
Current peer: 172.16.100.1
Security association lifetime: 4608000 kilobytes/3600 seconds
Entries 65537 & higher:
Responder-Only (Y/N): N
crypto sockets
PFS (Y/N): N
Transform sets={
(open or closed)
tset: { esp-256-aes esp-sha-hmac } ,
}
Interfaces using crypto map Tunnel1-head-0:

Tunnel1
BRKSEC-3052
Cisco Public
70
Scope of SP/SA DB: tunnel protection
Point-to-point GRE:
Single remote endpoint
Remote endpoint is known
crypto map Tunnel1-head-0

listening crypto socket
crypto socket to 172.16.100.1
socket created automatically
Multipoint GRE + NHRP:
Gig1/0
= Tunnel1-head-0 crypto map
Tun1
= Tunnel1 SP/SA DB scope
Some remote endpoints are

known via NHRP mappings
sockets created automatically
Incoming dynamic connections
will hit the listening socket
Outgoing dynamic connections
based on NHRP resolutions
Gig0/0
crypto map Tunnel1-head-0

sockets created dynamically

From NHRP mappings
BRKSEC-3052

P-P GRE
!
interface Tunnel1
ip address 10.1.0.1 255.255.255.252

mGRE
!
interface Tunnel1
ip address 10.1.0.1 255.255.255.0
ip nhrp map 10.1.0.100 172.16.100.1
ip nhrp map 10.1.0.200 172.16.200.1
Cisco Public
71
Gig1/0
Quick Mode proposal processing (1)

Example 1: dual mGRE, different tunnel sources
SP/SA DB root: Gig0/0

SPD: crypto map Tunnel1-head-0
Profile: tp1 (tset, red-peers)

Profile: tp2 (tset, blue-peers)
Select SP/SA DB based on local address (Gig0/0)

Look for local/remote proxies in SPD
Best match: 0.0.0.0/0 0.0.0.0/0 (listening socket)
Check for matching transforms
Check for matching IKE profile (if configured)
All OK create new crypto socket and profile instance
BRKSEC-3052
Tun2
(mGRE)
(mGRE)
Tun1 SP/SA DB
Root: Gig0/0
(172.16.1.1)
ESP with AES-128/SHA, transport mode

GRE 172.16.1.1/32 172.16.10.1/32
Tun1
Gig0/0
QM request from peer 172.16.10.1 on Gig0/0:
Gig1/1
Gig0/1
Tun2 SP/SA DB
Root: Gig0/1
(172.16.2.1)
! tset = esp-aes 128 sha-hmac trans.

crypto ipsec profile tp1
set isakmp-profile red-peers
set isakmp-profile blue-peers
!
interface Tunnel1
ip address 10.1.0.1 255.255.255.0
vrf forwarding red

tunnel protection ipsec profile tp1
!
interface Tunnel2
ip address 10.2.0.1 255.255.255.0
vrf forwarding blue

Cisco Public
72
Gig1/1
Gig1/0
Quick Mode proposal processing (2)
Tun1
Tun2
Example 2: dual mGRE, same tunnel source

QM request from peer 172.16.10.1 on Gig0/0:
Gig0/0
(172.16.1.1)
ESP with AES-128/SHA, transport mode

GRE 172.16.1.1/32 172.16.10.1/32
Profile: tp1 (tset, red-peers)

Profile: tp2 (tset, blue-peers)
Ambiguity: multiple SP/SA DB rooted on Gig0/0

Only works if we use different transforms (not just
separate identical transform sets !) under tp1 & tp2
Using different IKE Profiles in this configuration
looks appealing but is not supported
BRKSEC-3052
! tset = esp-aes 128 sha-hmac trans.

set isakmp-profile red-peers
set isakmp-profile blue-peers
!
interface Tunnel1
ip address 10.1.0.1 255.255.255.0
vrf forwarding red

!
interface Tunnel2
ip address 10.2.0.1 255.255.255.0
vrf forwarding blue

Cisco Public
73
Shared tunnel protection (1)
Tun1
tp-head-0 crypto map
tunnel protection ipsec profile <profile> shared
All tunnels protected by the shared IPsec

profile will share the same SP/SA DB and
crypto map: <profile>-head-1
Socket for Tunnel1

Socket for Tunnel2
GRE traffic multiplexed based on tunnel key

Removes the ambiguity, but introduces
many restrictions
BRKSEC-3052
Tun2
Gig0/0
Shared tunnel protection:

SPD: crypto map tp-head-0
Profile: tp (tset, all-peers)
Gig1/1
Gig1/0
Profile tp SP/SA DB
Root: Gig0/0
(172.16.1.1)

set isakmp-profile all-peers
!
interface Tunnel1
ip address 10.1.0.1 255.255.255.0
vrf forwarding red

tunnel key 1
tunnel protection ipsec profile tp shared
!
interface Tunnel2
ip address 10.2.0.1 255.255.255.0
vrf forwarding blue

tunnel key 2
tunnel protection ipsec profile tp shared
Cisco Public
Gig1/1
Gig1/0
Shared tunnel protection (2)
Tun1
Tun2
router#show crypto sockets

Shd Peers (local/remote): 172.16.1.1/172.16.100.1
Local Ident (addr/mask/port/prot): (172.16.1.1/255.255.255.255/0/47)
Remote Ident (addr/mask/port/prot): (172.16.100.1/255.255.255.255/0/47)
Flags: shared
IPSec Profile: "tp"
Socket State: Open
Client: "TUNNEL SEC" (Client State: Active)
Gig0/0
Crypto Sockets in Listen state:

Client: "TUNNEL SEC" Profile: "tp" Map-name: "tp-head-1"
Crypto Map: "tp-head-1" idb: GigabitEthernet0/0 local address: 172.16.1.1
Crypto Map IPv4 "tp-head-1" 65536 ipsec-isakmp
Profile name: tp
Crypto Map IPv4 "tp-head-1" 65537 ipsec-isakmp

Crypto is applied to
Map is a PROFILE INSTANCE.
Peer = 172.16.100.1
Tun1 and Tun2
Extended IP access list
access-list permit gre host 172.16.200.201 host 172.16.200.202
but local address

is set to Gig0/0
Interfaces using crypto map tp-head-1:

Tunnel1
Tunnel2
BRKSEC-3052
Cisco Public
75
Guidelines & restrictions (1)

Tunnel1
Tunnel2
Situation
TP shared
Differentiator(s)
P-P GRE
or mGRE
P-P GRE
or mGRE
Different sources
Not required
Tunnel source
(IPsec profiles must be different)
P-P GRE
P-P GRE
Same source
Different destinations
Not required
Tunnel destination
P-P GRE
P-P GRE
Same source
Same destination
Required
Tunnel key
mGRE
mGRE
Same source
Required
Tunnel key
P-P GRE
Same source
P-P GRE initiator only
Required
Tunnel key
P-P GRE
Same source
P-P GRE responder
Need fix for

Tunnel key
CSCub95247 (*) IPsec profile (**, as workaround)
mGRE
mGRE
(*) fix available in: IOS 15.1(4)M6, 15.2(4)M3, 15.3(2)T & IOS-XE 3.7.3S/15.2(4)S3, 3.8.2S/15.3(2)S
(**) workaround: use different IPsec profiles with different transforms and no shared keyword
mode & algorithms act as differentiator during QM (two separate identical transform sets will not work)
BRKSEC-3052
Cisco Public
76
Guidelines & restrictions (2)

Shared tunnel protection
Summary:
Keyword shared always required if tunnel source is shared
Exception: all point-point GRE with different destinations (no ambiguity)
Special case: mGRE & point-point GRE responder (CSCub95247)
Prevents the use of multiple IKE Profiles (due to single IPsec profile)
If TP shared is used for a given tunnel source:
All tunnels with that tunnel source must use TP shared and the same IPsec profile
Other tunnels with different sources may not use the same IPsec profile
Always use the interface name as tunnel source, never the IP address
interface Tunnel0
tunnel source 172.16.200.201
interface Tunnel0
BRKSEC-3052
Incorrect !
Correct
Cisco Public
77
BRKSEC-3052
Cisco Public
The future of tunnel protection (1)

IPsec-protected Tunnel interface linked to IKEv1 or IKEv2 profile:

Through IPsec profile (current behavior)
Through extended tunnel protection command:
tunnel protection ipsec profile <ipsec-prof>
[shared|{isakmp|ikev2}-profile <ike-prof>]
Each IPsec profile can have either an IKEv1 or IKEv2 profile, not both
Tunnel interface selection in Phase 2 based on IKE profile
IKE profile on Tunnel must match the one derived by IKE from peer ID
TP shared still supported (mutually exclusive per local address)

BRKSEC-3052
Cisco Public
79
The future of tunnel protection (2)

Major improvements
Configuration:
Allows different IPsec profiles to coexist on the same local address
Allows IKEv1 & IKEv2 tunnels to coexist with the same source address
IKEv1 profile remains optional on initiator & responder
IKEv2 profile becomes mandatory on responder (already mandatory on initiator)
dVTI responder: profile derived by IKE assigned to Virtual-Access upon creation
Behavior changes:
Tunnel with IKEv1 profile no longer accepts all IKEv2 connections (& vice-versa)
Makes Tunnel selection deterministic, removes ambiguities for good
Currently planned for IOS-XE 3.10S & IOS 15.3(3)T

BRKSEC-3052
Cisco Public
80
Migrating from DMVPN to FlexVPN
BRKSEC-3052
Cisco Public
Why migrate to FlexVPN ?
Includes all capabilities of DMVPN Phase 3 (and many more):

Dynamic peers, spoke-spoke direct tunnels, redundancy
Key advantages over DMVPN:

Point-to-point interfaces all features can be configured per peer
AAA integration most parameters can be stored on a RADIUS server
Advanced initiator logic (tracking-based) via FlexVPN client block
Much more concise configuration in complex setups (e.g. multi-tenant)
IKEv2 routes can complement or replace a dynamic routing protocol
Brings all advantages of the IKEv2 protocol
Upgrading the hardware ?

opportunity to move to next-gen crypto as well
BRKSEC-3052
Cisco Public
82
Why NOT migrate to FlexVPN ?
If something is not broken, dont fix it

DMVPN still fully supported (Phase 3 = the only recommended design)
Hardware constraints (FlexVPN only supported on ASR1k & ISR-G2)
Constraints due to migration scenarios
BRKSEC-3052
Cisco Public
83
Sample configuration Hub

FlexVPN with dynamic mesh
Match peers with a certificate where
DN O field is cisco
Populate Virtual-Access attributes based on

RADIUS profile named after DN OU field
Default IPsec profile (pre-configured) points

to default IKEv2 profile (not pre-configured)
IP unnumbered, iVRF, NHRP network-id,

GRE tunnel key, will be populated
based on AAA authorization attributes
BRKSEC-3052
aaa new-model
aaa authorization network rad group radius
!
crypto pki certificate map cisco-map 10
subject-name co o = cisco
!
crypto ikev2 name-mangler ou
dn organization-unit
!
crypto ikev2 profile default
match address local interface Ethernet0/0
match certificate cisco-map
identity local dn
authentication remote rsa-sig
authentication local rsa-sig
pki trustpoint root-ca
aaa author group cert list radius name-mangler ou
virtual-template 2
!
interface Virtual-Template2 type tunnel
ip nhrp redirect
tunnel protection ipsec profile default
Cisco Public
84
Sample configuration Spoke

FlexVPN with dynamic mesh
Match peers with a certificate where
DN O field is cisco and OU is eng
Use default local IKEv2 authorization policy

(route set interface, route accept any)
Tunnel interface required on initiator, tunnel
destination can be hardcoded or dynamic
(driven by Flex client configuration block)
Spoke of eng type interface parameters

are hardcoded (on hub side, will be derived
from AAA authorization based on OU field)
A separate Virtual-Access is instantiated for

every spoke-spoke direct connection
BRKSEC-3052
aaa new-model
aaa authorization network here local
!
crypto pki certificate map cisco-eng 10
subject-name co o = cisco
subject-name co ou = eng
!
crypto ikev2 profile default
match certificate cisco-eng
...
aaa authorization group cert list here default
virtual-template 2
!
interface Tunnel2
ip unnumbered Loopback2
ip nhrp shortcut virtual-template 2
tunnel key 2
!
interface Virtual-Template2 type tunnel
ip unnumbered Loopback2
ip nhrp shortcut virtual-template 2
tunnel key 2
Cisco Public
85
Migration scenarios (1) stating the obvious

Premise: spoke router is not being replaced
DMVPN Hub
FlexVPN Hub
1) One step approach:
Pre-configure FlexVPN on all spokes

(tunnel remains shut down)
Bring up FlexVPN hub
Turn DMVPN off & FlexVPN on,

on all spokes at once
DMVPN
WAN/MPLS/
FlexVPN
2) Progressive approach:
Bring up FlexVPN alongside DMVPN
Make the FlexVPN routes less preferred (change metrics on the hubs)
Use e.g. a test prefix to ensure spoke-spoke direct works OK
Make the FlexVPN routes preferred (change metrics on the hubs)
BRKSEC-3052
Spoke
Cisco Public
86
Migration scenarios (2)

One step approach
Advantages:
No need for DMVPN and FlexVPN to coexist (no need to share tunnel source)
Tunnel addressing scheme can be reused
Disadvantages:
Only practical for small number of spokes
Potentially long maintenance window required
Downtime is unavoidable
BRKSEC-3052
Cisco Public
87
Migration scenarios (3)

Progressive approach
Advantages:
Spoke sites can be prepared in sequence
FlexVPN fully brought up (incl. routing) while DMVPN remains in production
Short maintenance window required for switchover, easy rollback
Virtually no downtime (just re-routing)
Disadvantage:
IKEv1 and IKEv2 must coexist on the same device
BRKSEC-3052
Cisco Public
88
Option 1: separate source addresses

DMVPN & FlexVPN both use tunnel protection
WAN/MPLS/
DMVPN IPsec profile requires IKEv1 profile

FlexVPN IPsec profile requires IKEv2 profile
Only possible if tunnel source is different
Need additional routable WAN address
(not always possible or practical)
crypto ipsec profile DMVPN
DMVPN
set isakmp-profile DMVPN
!
ip address 172.16.1.1 255.255.255.252
!
interface Tunnel1
...
tunnel protection ipsec profile DMVPN
BRKSEC-3052
Loopback0
172.16.100.1/32
Ethernet0/0
172.16.1.1/30
Spoke
crypto ipsec profile default

FlexVPN
set ikev2-profile default
!
interface Loopback0
ip address 172.16.100.1 255.255.255.255
!
interface Tunnel2
...
tunnel source Loopback0
Cisco Public
89
Option 2: IKE Profile-based tunnel selection

Tunnel interface becomes IKEv1- or IKEv2-only

Different IPsec profiles allowed on single tunnel source
Currently planned for IOS-XE 3.10S & IOS 15.3(3)T
WAN/MPLS/
Ethernet0/0
172.16.1.1/30
Spoke
crypto ipsec profile DMVPN
DMVPN
set isakmp-profile DMVPN
!
interface Tunnel1
...
tunnel protection ipsec profile DMVPN
BRKSEC-3052
crypto ipsec profile default

FlexVPN
set ikev2-profile default
!
interface Tunnel2
...
Cisco Public
90
Before we part
BRKSEC-3052
Cisco Public
Call to Action
Attend these recommended sessions:
Advanced IPSec with FlexVPN and IKEv2 (BRKSEC-3013, Frederic Detienne)
IPv6 Security Threats and Mitigations (BRKSEC-2003, Eric Vyncke)
Get hands-on experience with the following Walk-in Labs:

Deploying DMVPN (LABSEC-2031)
Advanced DMVPN (LABSEC-2035)
Meet the Engineer:

Alex Honore & Olivier Pelerin (TAC Engineers)
Frederic Detienne (Distinguished Engineer)
Discuss your projects challenges

at the Technical Solutions Clinics
Visit the Cisco Campus at the World of Solutions
BRKSEC-3052
Cisco Public
92
BRKSEC-3052
Cisco Public
93

DMVPN

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DMVPN

Uploaded by

Copyright:

Available Formats

BRKSEC-3052

Advanced DMVPN Designs

2013 Cisco and/or its affiliates. All rights reserved.

Review of Phase 3 logic

Migrating from DMVPN to FlexVPN

2013 Cisco and/or its affiliates. All rights reserved.

2013 Cisco and/or its affiliates. All rights reserved.

2013 Cisco and/or its affiliates. All rights reserved.

IOS on 7301, 7200 series, ISR, ISR-G2:

Cat6500, C7600 with VPN-SPA + Sup720:

Recommended combos today:

2013 Cisco and/or its affiliates. All rights reserved.

Spokes can be configured with 1 or more hubs

2013 Cisco and/or its affiliates. All rights reserved.

8: Direct packet flow

Spokes send resolution requests to resolve destination addresses into NBMA

2013 Cisco and/or its affiliates. All rights reserved.

Review of Phase 3 logic

2013 Cisco and/or its affiliates. All rights reserved.

C 172.16.101.0/30 Gig0/0 (WAN)

Stat 10.0.0.102 172.16.102.1 (H2)

Stat 172.16.102.1 (H2)

C 172.16.1.0/30 Gig0/0 (WAN)

Stat 10.0.0.101 172.16.101.1 (H1)

Stat 172.16.101.1 (H1)

2013 Cisco and/or its affiliates. All rights reserved.

C 172.16.101.0/30 Gig0/0 (WAN)

C 172.16.102.0/30 Gig0/0 (WAN)

Stat 10.0.0.102 172.16.102.1 (H2)

Stat 10.0.0.101 172.16.101.1 (H1)

Stat 172.16.102.1 (H2)

Stat 172.16.101.1 (H1)

C 172.16.1.0/30 Gig0/0 (WAN)

Stat 10.0.0.101 172.16.101.1 (H1)

C 172.16.2.0/30 Gig0/0 (WAN)

Stat 10.0.0.102 172.16.102.1 (H2)

Stat 172.16.101.1 (H1)

2013 Cisco and/or its affiliates. All rights reserved.

Stat 172.16.102.1 (H2)

2: Routing Neighborship Establishment

C 172.16.101.0/30 Gig0/0 (WAN)

C 172.16.102.0/30 Gig0/0 (WAN)

Stat 10.0.0.102 172.16.102.1 (H2)

Stat 10.0.0.101 172.16.101.1 (H1)

Stat 172.16.102.1 (H2)

Stat 172.16.101.1 (H1)

C 172.16.1.0/30 Gig0/0 (WAN)

Stat 10.0.0.101 172.16.101.1 (H1)

C 172.16.2.0/30 Gig0/0 (WAN)

Stat 172.16.101.1 (H1)

2013 Cisco and/or its affiliates. All rights reserved.

Stat 10.0.0.102 172.16.102.1 (H2)

Stat 172.16.102.1 (H2)

C 172.16.101.0/30 Gig0/0 (WAN)

C 172.16.102.0/30 Gig0/0 (WAN)

Stat 10.0.0.102 172.16.102.1 (H2)

Stat 10.0.0.101 172.16.101.1 (H1)

Stat 172.16.102.1 (H2)

Stat 172.16.101.1 (H1)

C 172.16.1.0/30 Gig0/0 (WAN)

CEF switching: follow

Stat 10.0.0.101 172.16.101.1 (H1)

2: NHRP Lookup (*)

C 172.16.2.0/30 Gig0/0 (WAN)