IPSI Signaling Bandwidth Requirements for

WAN IP Connected G650 Port Networks

Presented by Ken Wnuck

© 2005 Avaya Inc. All rights reserved.

 IPSI Signaling Bandwidth

What is IPSI Signaling?

How we Tested
Bandwidth Provisioning
Layer 3 Traffic Classification
Cisco CBWFQ and LLQ
Extreme QoS Profiles
Conclusions
Phase II Initiatives

© 2005 Avaya Inc. All rights reserved. 2

 IPSI CONTROL TRAFFIC

The IPSI circuit pack (TN2312AP/BP)

provides enterprises with the capability
to IP-connect Port Networks over
LAN/WAN links in simplex and high
availability configurations.
The IPSI signaling traffic is
encapsulated AVAYA proprietary CCMS
(Control Channel Message Set)
messages inside TCP/IP packets.
CCMS Call Signaling and System
Maintenance traffic is passed between
S87XX/S8500 call control servers and
the IPSI circuit packs in a port network.
The CCMS messages are used for:
– H.323, H.225 RAS, and Q.931
• Register IP endpoints via local
CLANs to the S8XXX server.
• Setup and Teardown calls
• Periodic testing of the hardware.
• KeepAlives
The encapsulated CCMS messages
are critical to the stability of a port
network and delivery of CCMS
messages must be guaranteed.

© 2005 Avaya Inc. All rights reserved. 3

 System Test Model

© 2005 Avaya Inc. All rights reserved. 4

 Methodology

CM 4.0
150 TARTS IP endpoints registered per region, where each of the two PNs had its own
isolated region.
The originating TARTS IP endpoints were used to generate traffic over two ISDN PRI
trunks that then terminated to a second set of 150 TARTS IP endpoints on the second PN.
This provided our outbound and inbound simulated PSTN trunk traffic respectively
– no inter-port network VoIP traffic, all bearer traffic traversed via the ISDN PRI trunks
– Poisson Distribution for random call patterns
Call durations were 10 seconds
BHCC call rates were 1K, 2.5K, 5K, 7.5K, and 10K
Bandwidth was limited and delay introduced by a PacketStorm WAN Emulator
Up-to 300 msec delay, 0.1% Packet Loss for IPSI traffic and an IPSI bandwidth limitation
of 128Kbps were introduced.
An Radcom and an Ethereal Packet Sniffer was used to collect and analyze TCP/IP
Packet data.

© 2005 Avaya Inc. All rights reserved. 5

 Example of Ethereal Statistics for 5K BHCC,
0.1% Packet Loss, and BW limits of 128K

© 2005 Avaya Inc. All rights reserved. 6

 5K BHCC 0.1%Packet Loss and BW limits of 128K

X-axis is time and Y-axis is bytes

Depending on which PN was under test, we could measure IPSI BW for incoming
ISDN PRI traffic terminating to local IP endpoints or measure IPSI BW for
outgoing ISDN PRI traffic being generated from local IP endpoints
© 2005 Avaya Inc. All rights reserved. 7
 BHCC vs IPSI Bandwidth

The simulated call scenario is a

general business case.
IPSI Call Signaling Packet Traffic
BHCC IPSI bandwidth is based
100
on 150 IP endpoints originating
and answering 10 second 80
duration ISDN trunk calls. 60

Kbps
40
20
0
1K 2.5K 5K 7.5K 10K
BHCC

© 2005 Avaya Inc. All rights reserved. 8

 BHCC Bandwidth

BHCC Usage Per Station Average IPSI Bandwidth Average IPSI TCP/IP
Per PN (Kbps) full duplex packets per second

1K Light Traffic 17.3 Kbps 21

2.5K 30.5 Kbps 37

5K Moderate Traffic 52.2 Kbps 61

7.5K 73.8 Kbps 85

10K Heavy Traffic 83.5 Kbps 107

© 2005 Avaya Inc. All rights reserved. 9

 Provisioning must include the Layer 2 overhead, which
includes preambles, headers, flags, CRCs and ATM cell
padding and encryption overhead.
Ethernet adds a 18 byte header, plus a 4 Protocol Overhead per
byte CRC plus an optional 4-byte 802.1Q
packet
Tag plus a 8 byte preamble for a total of
up to 34 bytes per packet. Ethernet 34 bytes
Point-to-Point Protocol (PPP) adds 12
bytes of layer 2 overhead per packet. PPP 12 bytes
Multilink PPP adds 13 bytes per packet.
Frame Relay adds 6 or 7 bytes per MLPPP 13 bytes
packet.
ATM adds varying amounts of overhead
depending on cell padding.
Frame Relay 6 or 7 bytes
IPSI encryption adds up-to 23 bytes
(AES) for the encryption header and ATM Varies because
padding in addition to Layer 2 overhead. of padding
AES Encryption 23 bytes

© 2005 Avaya Inc. All rights reserved. 10

 IPSI bandwidth calculations should include the
additional overhead on a per packet basis
depending on the type of WAN link.

A moderate general business traffic rate is 5K BHCC

The L2 overhead for a PPP link would be 12 bytes/packet X 61 PPS X 8

bits/byte equals 5.9 Kbps for L2 overhead

5.9 Kbps for L2 overhead + 52.2 Kbps for TCP/IP CCMS Packets equals a
minimum average bandwidth requirement of 58.1 Kbps.

IPSI encryption adds up-to 23 bytes/packet X 61 PPS x 8 bits/byte equals

11.2 Kbps

Total bandwidth minimum bandwidth requirement is 58.1 Kbps + 11.2 for

encryption = 69.3 Kbps

© 2005 Avaya Inc. All rights reserved. 11

 IPSI Bandwidth Provisioning
A general rule of thumb for IPSI Control traffic bandwidth allocation is to add an
additional 64Kbps of signaling bandwidth to the minimum required bandwidth in
order to manage peak (burst) traffic loads and either round up or down to nearest
DS0.

For example; for 5K busy hour calls using encrypted PPP links to control remote
port networks, as described in the previous example, you would guarantee
128Kbps (69.3Kbps + 64Kbps) for IPSI signaling bandwidth across the WAN link.
BHCC Ethernet PPP MLPPP Frame Relay
1K 64Kbps 64Kbps 64Kbps 64Kbps
1K w/ encryption 64Kbps 64Kbps 64Kbps 64Kbps
2.5K 128Kbps 128Kbps 128Kbps 128Kbps
2.5K w/ encryption 128Kbps 128Kbps 128Kbps 128Kbps
5K 128Kbps 128Kbps 128Kbps 128Kbps
5K w/ encryption 128Kbps 128Kbps 128Kbps 128Kbps
>=7.5K 192Kbps 192Kbps 192Kbps 192Kbps
>=7.5K w/ encryption 192Kbps 192Kbps 192Kbps 192Kbps

© 2005 Avaya Inc. All rights reserved. 12

 Standby IPSI Signaling Traffic

• In the above figure, the green line represents traffic from the standby IPSI
which in turn overlaps the blue line with traffic from the standby server. As
you can see there is very little traffic.
• 2.4 Kbps bandwidth is consumed by the standby IPSI.
© 2005 Avaya Inc. All rights reserved. 13
 Where does IPSI Signaling Fit?
8 Class QoS Model Voice
Higher Priority Traffic can starve Video
equal and lower priority traffic
– Voice Bearer Traffic and IPSI
Signaling Traffic currently share H323 Call Signaling
the same queue and compete
for bandwidth within this queue Network Management
– Voice Bearer cannot Tolerate
Delay or Jitter Critical Data
– IPSI can tolerate up to 300
msec Delay
Bulk Data
– Voice Quality tolerates 3%
Packet Loss
Best Effort
– A Port Network will become
unstable with greater than 3%**
Packet Loss across the link Scavenger

** AVAYA IP VOICE QUALITY NETWORK REQUIREMENTS White Paper,

Issue 3.1, April 2006.
© 2005 Avaya Inc. All rights reserved. 14
 Layer 3 Traffic Classification
IP Precedence and DSCP

The IPV4 Header Version ToS

Len …rest of the header…………….
contains a ToS Byte Length 1 Byte
The first 3 bits are
used for Priority.

1 0 1 0 0 0 0 0 TOS = 10100000 = 160

P P P D T R M R IP Precedence = 101 = 5
Precedence
7 - 111 Network Ctl DSCP uses an additional 3 bits of the ToS Byte to
6 - 110 Internetwork Ctl
5 - 101 Critical differentiate traffic into 64 classes
4 - 100 Flash Override
3 - 011 Flash Delay Throughput Reliability Monetary Reserved
2 - 010 Immediate
1 - 001 Priority Cost
0 - 000 Routine 0 Normal 0 Normal 0 Normal 0 Normal *ALWAYS* set
1 Minimize 1 Maximize 1 Maximize 1 Minimize to zero

© 2005 Avaya Inc. All rights reserved. 15

 RFC 2597 Expedited Forwarding, Assured Forwarding and Best
Effort Classes
Expedited Forwarding (EF) DSCP 46
Provides low loss, low latency, low jitter, and assured bandwidth
Assured Forwarding Classes:
Within each class, an IP packet is assigned 3 different levels of drop precedence; low,
medium and high. AF41 (drop me last) is less likely to be dropped than AF13 (drop me after BE)
Drop Preference CLASS 1 CLASS 2 CLASS 3 CLASS 4

LOW 001010 010010 011010 100010

AF11 AF21 AF31 AF41
DSCP 10 DSCP 18 DSCP 26 DSCP 34

MEDIUM 001100 010100 011100 100100

AF12 AF 22 AF32 AF42
DSCP 12 DSCP 20 DSCP 28 DSCP 36

HIGH 001110 010110 011110 100110

AF13 AF23 AF33 AF43
DSCP 14 DSCP 22 DSCP 30 DSCP 38

Best Effort DSCP 0 - Drop me first

© 2005 Avaya Inc. All rights reserved. 16

 Congestion Avoidance and Management
CBWFQ , LLQ , WRED

CBWFQ provides support for user-defined traffic classes.

Traffic classes are defined based on match criteria including protocols, IP DSCP
values and access control lists (ACLs)
CBWFQ allows you to specify the exact amount of bandwidth to be allocated for
a specific class of traffic. Taking into account available bandwidth on the
interface, you can configure up to 64 classes and control distribution among
them. Bandwidth can be assigned a percentage of total link speed or in Kbps.
A FIFO queue is reserved for each class. Optional WRED can selectively
discard lower priority traffic when the interface begins to get congested.
The bandwidth assigned to a class is the guaranteed bandwidth delivered to the
class during congestion.
LLQ supports a strict priority queue for a traffic class
– LLQ gives the priority queue absolute preferential treatment over lower
priority queues;
– The strict priority LLQ queue allows more than one class of traffic to be
served

© 2005 Avaya Inc. All rights reserved. 17

 Cisco AutoQoS and Proposed IPSI AF Value

Voice Voice EF 46

Video Video AF41

Call Signaling IPSI Control AF42

Call Signaling AF31 or CS3

Network Management Network Management CS6

Critical Data Critical Data AF21

Bulk Data Bulk Data AF11

Best Effort Best Effort 0

Scavenger Scavenger CS1

© 2005 Avaya Inc. All rights reserved. 18

 CBWFQ

There are 3 steps to configure CBWFQ

– Define your classes of traffic (classify)

– Define your policies using those classes
– Apply to one or more interfaces (in, out)

© 2005 Avaya Inc. All rights reserved. 19

 Classification

It is proposed to implement QoS policies that provide:

DSCP 36 (AF42) for IPSI signaling traffic.

– DSCP 34 (AF41) (CS 4 Lowest Drop Preference) already reserved for
Video in Cisco AutoQoS
– IPSI signaling can be assigned another DCSP value but we must
guarantee bandwidth to minimize Packet Loss
Expedited Forwarding - DSCP 46 - like behavior for the real-time voice
Assured Forwarding (AF31) like behavior for H.323 Call Signaling Traffic

IPSI traffic classification can be performed by an S87XX/S8500 via

the change ipsi-server interface command.

© 2005 Avaya Inc. All rights reserved. 20

 ipserver-interface command

change ipserver-interface 1 Page 1 of 1

IP SERVER INTERFACE (IPSI) ADMINISTRATION - PORT NETWORK 1

IP Control? y Socket Encryption? y

Ignore Connectivity in Server Arbitration? n Enable QoS? y

Primary IPSI QoS Parameters

------------ --------------
Location: 1A02 Call Control 802.1p: 6
Host: 172.19.27.202 Call Control DiffServ: 36
DHCP ID: ipsi-A01a IPADMIN]: set diffserv 36
[IPADMIN]: show qos
QoS values currently in use:
The SAT command marks traffic Server to IPSI. VLAN tagging : on
VLAN id : 0
Login to the IPSI; The IPSI Firmware CLI is used VLAN user priority : 6
Diffserv value : 55
to mark traffic from IPSI to Server. QoS values to be used after next reset:
VLAN tagging : on
VLAN id : 0
VLAN user priority : 6
Diffserv value : 36

© 2005 Avaya Inc. All rights reserved. 21

 Example Policy Configuration

Configuring QoS for VoIP on a WAN Link

class-map VoIP-Bearer
match ip dscp 46
class-map H.323-VoIP-Control Classify the traffic of
match ip dscp af31 interest for QoS Policy
class-map IPSI-Control
match ip dscp af42
! Define QoS Policy
Treatment
policy-map QoS-Policy As Percent or Kbps
class IPSI-Control
bandwidth 128
class H.323-VoIP-Bearer
priority percent 60
class VoIP-Control
bandwidth percent 5
class class-default
random-detect
!

© 2005 Avaya Inc. All rights reserved. 22

 Class-Based WFQ—QoS Guarantees Plus Bandwidth Efficiency

IPSI – 128 K AF42 10%

Voice – LLQ,
Voice EF EF EF EF EF 60%

Voice signaling AF31 5% IPSI Signaling, Voice Signaling, Data:

CBWFQ
Data – default classData
BE BE 25%

Define QoS Applications Define Bandwidth percentage (example)

Each queue is separately configured for QoS

LLQ guarantees no Packet Delay due to congestion
Weights (CBWFQ) guarantees minimum bandwidth based on percentage of
the link speed or in Kbps for congested links
Unused capacity is shared among the default classes and should be at least
25%
WRED drops lower priority data traffic
– BE, AF13, AF12, AF11, AF23 ... AF31, AF43, AF42, AF41

© 2005 Avaya Inc. All rights reserved. 23

 Extremeware QoS Profiles

Queue 802.1p DiffServ

IPSI and Bearer will default Priority Code Point
to the same queue QP6
Value
Traffic can be scheduled by:
Queue Weight QP1 - Low 0 0-7
Minimum Bandwidth QP2 - LowHi 1 8 - 15
Maximum Bandwidth
QP3 - Normal 2 16 - 23
Priority
– 802.1p QP4 - NormalHi 3 24 - 31
– DSCP QP5 - Medium 4 32 - 39
QP6 - MediumHi 5 40 - 47
QP7 - High 6 48 - 55
QP8 - HighHi 7 56 - 63

You can change the QoS profile assignment for each of the 64 code points using the following command:

configure diffserv examination code-point <code-point> {qosprofile} <qosprofile>

© 2005 Avaya Inc. All rights reserved. 24

 Remoted PN Design

QoS can help protect critical traffic on occasionally congested links

– QoS is not a cure for really inadequate bandwidth
Good stable Hierarchical WAN Network with a redundant, high-availability design
– Router Protocols configured for quick failover
– Minimize short, intermittent outages < 3 sec
– Converged Network Analyzer
Plan for long duration outages
– Back-up Servers (ESS)
– Redundant Links
Network management in place
– For bandwidth reporting and capacity planning
Call Admission Control (CAC) can be administered to limit Voice Bearer bandwidth
on congested links

© 2005 Avaya Inc. All rights reserved. 25

 Voice Bearer and IPSI Signaling

Voice bearer traffic should be marked with DSCP 46 (EF)

Voice bearer loss should not be greater than 3% and average one-way jitter
should be less than 30 milliseconds.
One-way delay should be no more than 150 milliseconds.
H.323 Signaling Traffic should be marked DSCP 26 (AF31)

IPSI Signaling Traffic should be marked DSCP 36 (AF42)

IPSI Signaling Packet loss should not be greater than 3%
IPSI Signaling traffic should be prioritized and given a guaranteed
bandwidth on WAN links based on BHCC.
IPSI Links should be reliable and redundant

© 2005 Avaya Inc. All rights reserved. 26

 Phase II

H.248 Gateways bandwidth requirements

Recovery testing with high number of IP phones off of one
PN.
Filesync and translations pushes.
Steady state Annex H and other signaling impact –
calculation only (unless time permits for testing).

© 2005 Avaya Inc. All rights reserved. 27

 Acknowledgements

Co-Pilot
– Steve Regini
System Test
– Andy Cornejo
– David James
– Ana Kesselring
– Rob Pospisil
ATAC
– Tim Kaye
– Ken Lin

© 2005 Avaya Inc. All rights reserved. 28

 Additional Slides

© 2005 Avaya Inc. All rights reserved. 29

 Why is IPSI Intolerant to Packet Loss

Older VxWorks stack have “issues”

related to packet retransmission
Global Synchronization Issue where
IPSI backs-off
Server requests retransmission of
lost packet
IPSI backs off and waits 500 msec,
then 1sec, then 2 sec > 3 sec
AKA TCP Slow Start for
congestion avoidance

© 2005 Avaya Inc. All rights reserved. 30

 QoS Design

Bearer, 1% PL, 150ms,

delay, 30ms jitter
2-way 1,150,30
1-way 5%, 5 sec, ++jittter
Delay Sensitive TCP
RIP, OSPG, EIGRP,BGP
SNMP,NTP,Syslog,NFS
Locally defined – revenue driven
SAP, Oracle, Financial, SQL

Database sync, email, ftp

Non-critical, http requires
25% allocation

1% to minimally
differentiate Napster,
KaZaa, Gaming from BE

© 2005 Avaya Inc. All rights reserved. 31