IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART C: APPLICATIONS AND REVIEWS, VOL. 33, NO.

3, AUGUST 2003

297

Optimizing QoS Routing in Hierarchical

ATM Networks Using Computational
Intelligence Techniques
Athanasios Vasilakos, M. P. Saltouros, A. F. Atlassis, and Witold Pedrycz, Fellow, IEEE

AbstractIn this paper, the use of a computational intelligence

approach a Reinforcement Learning Algorithm (RLA)for
optimizing the routing in asynchronous transfer mode (ATM)
networks based on the private network-to-network interface
(PNNI) standard is proposed. This algorithm which is specially
designed for the quality of service (QoS) routing problem, aims
at maximizing the network revenue (allocating efficiently the
network resources) while ensuring the QoS requirements for each
connection. In this study, large-scale networks are considered
where it becomes necessary to be organized hierarchically so that
a scale in terms of computation, communication and storage requirements will be achieved. A comparative performance study of
the proposed and other commonly used routing schemes is demonstrated by means of simulation on existing commercial networks.
Simulation results over a wide range of uniform, time-varying and
skewed loading conditions show the effectiveness of the proposed
routing algorithm, and disclose the strength and weakness of the
various schemes.
Index TermsATM QoS routing, computational intelligence, reinforcement learning algorithm.

I. INTRODUCTION

HE implementation of the emerging high-speed communication networks, such as broadband ISDN, has been
proposed by the ITU-T [1] to be based on the ATM technology.
In ATM networks, various kind of sources (such as voice,
video, or data) with different traffic features and quality of
service (QoS) requirementssuch as cell loss ratio (CLR),
maximum cell transfer delay (maxctd) and cell delay variation (CDV)are statistically multiplexed at very high rates.
However, the integrated character of ATM networks causes a
number of serious congestion problems characterized by cell
losses and excessive delays making the statistical multiplexing
ineffective without control.
In ATM networks, the QoS of a connection is closely linked
to the allocated network resources. A connection request for a
Manuscript received August 31, 2002; revised April 13, 2003 and June 17,
2003. This paper was recommended by Guest Editor S. Karnouskos.
A. Vasilakos is with the Institute of Computer Science, Foundation for
Research and TechnologyHELLAS (FORTH), Crete 15410, Greece (e-mail:
vasilako@ath.forthnet.gr).
M. P. Saltouros and A. F. Atlassis are with the National Technical University of Athens, Electrical and Computer Engineering Department, Heroon Polytechniou 9, GR-157 73, Athens, Greece (e-mail: saltouro@telecom.ntua. gr;
atlasis@telecom.ntua. gr; theolog@cs.ntua. gr).
W. Pedrycz is with the Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, T6G 2G7, Canada (e-mail:
pedrycz@ee.ualberta. ca).
Digital Object Identifier 10.1109/TSMCC.2003.817354

given call is accepted only when sufficient resources are available to carry the new connection through the entire network at its
requested QoS while maintaining the agreed QoS of the already
established connections. It is the connection admission control
(CAC) which is responsible for deciding whether a new connection request should be accepted or not. To operate efficiently,
CAC takes into account the connection traffic descriptor and the
requested QoS for both the incoming call and the already established connections, as well as the available network resources.
Once the connection has been established, the requested QoS
should be provided as long as the connection is compliant with
the negotiated traffic contract.
Part of the CAC actions is the routing algorithm [1], which
identifies paths that meet QoS constraints between the sourcedestination pair of an incoming call, and selects one that leads
to high overall network efficiency. Without an efficient QoS
based routing algorithm, network may fail to find a route and
reject a request for the call connection, though there are enough
resources to successfully establish that call. A routing algorithm aims at improving network efficiency which is usually
expressed in terms of increase in revenue, where revenue is typically a function of the number flows or the amount of bandwidth
carried by the network, while guaranteeing the QoS constraints.
Thus, for a QoS-based routing algorithm, not only is it important to select a route that satisfies all the QoS requirements, but
also it is significant to manage efficiently the network resources,
e.g., buffer space and bandwidth. So, it is clear that an efficient
scheme should aim at reducing the consumption of resources
and at balancing the traffic load across the network when necessary. The later property is particularly helpful for maximizing
the ability to accommodate future calls. Furthermore, a high performance ATM routing protocol should be dynamic, being capable of selecting the routes in response to variation of the traffic
load and network failure, contributing in this way to the better
exploitation of the network resources.
In this paper, a computational intelligence approach to
the ATM flat and hierarchical (PNNI) QoS routing issue is
presented based on the Stochastic Estimator Learning Algorithm (SELA), an efficient Reinforcement Learning Algorithm
(RLA). RLA as simple propabilistic models performs robustly
in the face of uncertainty conducting safety -based routing
(inaccurate bandwidth feedback from aggregated topology)
and thus, moving from deterministic to propabilistic models
when dealing with networking complexities, is encouraged.
The learning approach presented in this paper, treats routing

1094-6977/03$17.00 2003 IEEE

298

Fig. 1.

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 3, AUGUST 2003

Two-level PNNI network hierarchy model.

as a stochastic allocation problem amenable to a dynamic decentralized approach. Based on real-time feedback of network
status, Learning Algorithms (LAs) operating at various network
switches determine how the traffic is to be routed.
The proposed routing methodology is evaluated by means of
simulation on existing commercial topologies and simulation
results demonstrate its effectiveness for the cases of connections with specific CLR and maxCTD requirements. This effectiveness is highlighted when compared in terms of average
cell transfer delay, bandwidth rejection ratios, bandwidth acceptance ratios, network throughput, and network revenue with
other commonly used routing algorithms over a wide range of
uniform, time-varying and skewed loading conditions. Moreover, the proposed algorithm is simple and can be implemented
in hardware easily according to the procedures described in [2].
The rest of the paper is structured as follows. Since the route
optimization methodology is proposed for ATM networks based
on the PNNI specification, at first background information on
the PNNI standards is provided, and prior work on hierarchical
routing is summarized in Section II. In Section III, we describe
briefly how the RLAs operate and justify their use in routing
problems in networks. In Section IV, a concise description of the
SELA algorithm is given. The proposed routing methodology
based on SELA is developed in Section V, while in Section VI
the simulation model is described and the extracted numerical
results from the comparative performance study between the
various routing schemes are exhibited. Finally, Section VII concludes the paper and intimates some interesting aspects of future
work.
II. BACKGROUND
In this section, at first a brief overview of the PNNI standards
for ATM networks is presented, and then prior work on hierarchical routing is summarized.

A. General Survey of PNNI Standards

PNNI protocols [3] construct a logical hierarchy atop a physical ATM network. In a global network it is unpractical for
each node to contain a complete description of the entire network because the costs of storing and updating this information are prohibitive. Thus, a hierarchical scheme allows nodes
to have reduced views of the network (aggregated topology
information), since it organizes the network nodes into levels.
According to this concept, the network nodes and links are organized hierarchically. At the lowest level of the hierarchy, each
node represents an ATM switch and each link represents a physical link or an ATM virtual path (VP). The nodes and links of
each level can be recursively aggregated into higher levels, such
that a high-level node represents a collection of one or more
lower level nodes, and a high-level link represents a collection
of one or more lower level links. The ATM PNNI is a hierarchical, dynamic link-state routing protocol, designed to scale to
the largest possible ATM networks, encompassing thousands of
switches. It may support therefore a maximum of 105 hierarchy
levels.
To support this hierarchy, PNNI defines a uniform network
model at each level, with a set of logical nodes connected by
logical links. In PNNI a collection of nodes at the same hierarchical level, geographically collocated and/or belonging to a
group, is organized into a PNNI peer group (PG) which is assigned a unique peer group identifier (Pgid). At the lowest level
of hierarchy a PG consists of physical nodes connected via physical links. At the next level of hierarchy the PG consists of a set
of logical group nodes (LGNs), which are abstractions of the
lower-level peer groups. Two LGNs are interconnected by a logical link which corresponds to a logical connection that consists
of one or more links of the lower level in tandem or/and in parallel. LGNs summarize and advertise reachability information
for their respective lower-level PGs. The functions associated

VASILAKOS et al.: OPTIMIZING QOS ROUTING IN HIERARCHICAL ATM NETWORKS

with an LGN are actually performed by the peer group leader

(PGL). The PGL is determined by means of a distributed election process, executed by all the LGNs in the PG.
When neighboring nodes sense an active link they exchange
hello messages which contain their PGids. If the PGids are
alike, they belong to the same PG. After that each node ties its
state information, including status of its links to its immediate
neighbors, to a PNNI topology state element (PTSE), which is
flooded through the entire PG. The collection of PTSEs received
by a node constitute the nodes current view of its PNNI routing
domain. PTSEs are sent both horizontally across a peer group,
and downward from a parent LGN to a child PG. Consequently,
each node has complete routing information (connectivity and
state) about the nodes and links in its own lowest-level PG, but
partial and approximate routing information about the nodes and
links that belong to different PGs. Moreover, when neighboring
nodes conclude on the basis of exchanged hello messages that
they belong to different PGs, they become border nodes. In other
words, a border node in a peer group is a logical node that has
at least one link that crosses the peer group boundary. Fig. 1
shows a schematic of the PNNI network hierarchy model with
two levels.
In PNNI, the first (originating) switching system that receives
the connection setup request, routes the incoming call to the
destination (source routing) according to the topology information that each one of the candidate switching systems advertises.
This information can be the state of the links attached to these
switching systems (either physical or logical), the QoS parameters that can be guaranteed, as well as the ability to transmit
various types of traffic. In other words, the originating switching
system determines the path to the destination comprising all the
detail of the hierarchy known to it. This is called a hierarchically
complete source route. Such a path is not a fully detailed source
route because some portions of it are abstracted as a sequence
of logical group nodes to be traversed. Thus, this source route
does not hold the details of the path outside the originators peer
group and is considered as a logical path. The actual selection of
a physical path outside the source switching system peer group
has to be made by each border node of peer groups along the
logical path. When the call setup arrives at the border node of a
peer group, this node is responsible for selecting a lower level
source route describing the crossing of that peer group.
There are significant reasons that induce the compression
and the aggregation of the information that is exchanged in a
peer-group and is advertised to the higher level (parent PG).
First of all, a decrease in the amount of the exchanged information decreases the resources that are consumed for their distribution, which is very important for WAN. Besides, it is not
practical for every entity of the network (e.g., every node) to
maintain detailed information about the state of all the others
entities. Moreover, there are cases that an internal topology of a
network should be concealed from nodes outside it for security
reasons. However, the compression and the aggregation of the
information (which is called topology aggregation) should be
implemented in such a manner that the real network topology be
represented satisfactorily so that efficient routing and resource
allocation will be achieved.

299

The problem regards what information each switching system

should advertise and also how often this information should
be distributed to the other switching systems. The problem of
having fast updating so that each switching system will have
up-to-date information about all the other switching systems
is particularly difficult due to the large propagation delay that
large-scale ATM networks introduce. An increase in the frequency of updating and in the amount of the exchanged information increases the probability of choosing a more suitable route,
but it also increases both the processing complexity and the resources that are consumed, and vice versa.
Several schemes for topology aggregation have been proposed; the best known are the star, the full-mesh, and the
simple-node approaches. The star approach, which has been
adopted by PNNI [3], is an efficient compromise between
the other two. Full-mesh approach uses a full graph for the
representation of the connections between the border nodes that
belong to a particular peer-group, while simple-node approach
represents the whole peer-group with a single node. A fully
detailed and a brief description of these topology aggregation
schemes can be found in [3] and [5], respectively.
B. Prior Work on Hierarchical Routing
There have been several studies in the literature dealing with
hierarchical routing. Some interesting approaches can be found
in [4][13]. The authors in [3] study some general QoS path
selection problems when the routing information is inaccurate
due to the aggregation process that occurs in hierarchically
interconnected networks. Lee examines some issues of topology
aggregation for hierarchical PNNI routing in ATM networks in
[5], while in [6] he proposes a spanning tree method for link state
aggregation in large communication networks. In [7], Awerbuch
and Slavitt present a distortion-bounded solution for state aggregation of a sub-network. The authors also show how this solution
can be applied to PNNI. In [8], they give a comparison of the
influence of some different aggregation schemes on network
performance, by means of simulation. However, in the model
used only one source-destination pair is considered. In [9] and
[10], Baras et al. present a method based on a hierarchical reduced
load approximation to evaluate PNNI routing. Yet, no simulation
results are presented. Orda suggests approximation methods
for the fundamental problem of constrained path optimization
(typical of QoS routing) in [11], which exploit the structure of
hierarchical aggregated topologies to give -optimal solutions.
In [12], the same author proposes routing algorithms that base
their decision on a probability vector (probabilistic routing) for
networks characterized by uncertain parameters. Kim et al. suggest a scalable inter-domain routing scheme which addresses the
issue of selecting a path with multiple metrics for building QoS in
a homogeneous high speed WAN in [13]. The route computation
load imposed by the PNNI routing scheme is investigated and
proved to be unevenly distributed among the network nodes in
[14]. Finally, the authors in [15] evaluate the performance of
topology aggregation techniques applied in hierarchical routing
and investigate the interaction of this topology aggregation with
factors such as routing update frequency, routing algorithms, and
network configuration.

300

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 3, AUGUST 2003

III. REINFORCEMENT LEARNING ALGORITHMS AND THEIR

APPLICATION TO THE ROUTING PROBLEM IN NETWORKS
An RLA is defined by the sextuple
.
number of possible actions;
action ;
: set of the actions
offered by the environment;
action selected at time ;
action that has the highest mean reward;
input set of possible environmental responses,
;
environmental response at time ,
;
set of possible internal states of the LA;
Pn
: choice probability;
: pmf over the set of actions;
updating scheme that modifies at each iteration by
using the response of the environment:
;
: output function;
;
;
pdf
;
;
;
(prime) implies an estimate;
mean of the
last environmental responses to ;
learning window: a predefined small integer;
;
: internal state of the automation;
resolution parameter (a constant, internal to the LA);
(a constant);
The RLA is a sequential machine characterized by the
following:
1) set of actions;
2) an action probability distribution;
3) a reinforcement learning scheme.
It is connected in a feedback loop to the random environment
(the function to be optimized, the process to be controlled, etc.,).
At every sampling instant (see Fig. 2) LA chooses an action
from a finite set of actions on the basis of a probability distribution. The selected action causes a response from the environment, which in turn is the input signal for the LA. With a reinforcement scheme, the RLA recursively updates its probability
distribution, and modifies its state by means of a transition function, trying to learn the optimal action the environment offers.
The objective in the design of the LA is to determine how the
choice of the action at any stage should be guided by past actions
and environment responses. The important point to note here
is that the decisions must be made with very little knowledge
concerning the nature of the environment. The latter may have
time-varying characteristics. Alternately, the uncertainty may be
due to the fact that the output of the environment is influenced
by the actions of other agents unknown to the decision maker.

Fig. 2.

RLAs that interacts with a stochastic environment.

In all cases the LA must be designed to improve some overall

performance function. The decision maker, in such a feedback
configuration of decision maker (or LA) and environment, is referred to as a learning automaton.
The power of the LAs approach is best realized in complex
systems when several automata operate in a distributed fashion
with each LA having a small number of actions. Such situations arise frequently in large-scale systems, where practical
constraints make distributed decentralized control mandatory.
Traffic routing in communication networks is an example of
such systems where the RLAs can operate at various network
switches determining how the traffic is to be routed. Besides, the
need for learning in identification or control problems depends
on the prior information that is available regarding the system,
the characteristics of the noise present, and the constraints that
exist on the inputs. For low levels of uncertainty, learning may
not be the most effective approach. But for highly uncertain systems it may be essential for adequate system performance.
Consequently, the effectiveness of learning routing techniques on telecommunication networks is justified by the fact
that the states of these networks, which vary with time, are
regarded as nonstationary environments and the information
concerning these states at a particular instant may no longer
be valid at the subsequent instant. Moreover, in large-scale
networks it is impractical for any single entity (i.e., nodes and
links) to have access to detailed state information about all
the other network entities. This leads to the need of topology
aggregation for reasons of scaling in terms of computation,
communication and storage requirements, but also for security
reasons since the internal topology of a network may have
to be hidden. As a result, the selection of a route across
such large networks will typically be performed based on
partial or inaccurate information derived from the process of
topology aggregation performed. In general, the accuracy of
the provided state information decreases as the amount of the
aggregation that is carried out increases. This inaccuracy can
lead to wrong routing decisions [4]. Learning algorithms techniques, choosing a path probabilistically, attempt to minimize
the effects of inaccuracies on decision making. The greatest
potential of a RLAs methodology is that it permits the analysis
of very complex dynamic systems and enables an optimum
global operation. Even in the case of an incomplete access to
the state information of the environment stochastic policies can
yield higher performance than deterministic policies and tend
to stabilize a nonstationary system to predict its behavior [16].

VASILAKOS et al.: OPTIMIZING QOS ROUTING IN HIERARCHICAL ATM NETWORKS

()

IV. STOCHASTIC ESTIMATOR LEARNING ALGORITHM

The SELA used in the proposed methodology is a powerful
and flexible ergodic RLA, especially when operating in a nonstationary stochastic environment [17]. Ergodic means that it
converges at the optimal action with a distribution independent
of the initial state. SELA has already been successfully used in
control-related problems in networks, e.g., Routing [18], connection admission control (CAC) [19] and UPC [20], [35].
where ,
The SELA is defined as
, , , and
are as defined in section C, and E is the
environments estimator defined below.
is the set of the n actions
offered by the environment. The action selected at the time
is symbolized as
.
is the set of the possible environment responses (feedback). The feedback of action
at time instant is symbolized as
.
the probability vector
We denote by
is the probability of choosing
of choosing each action, i.e.,
contains at any time instant
action . The estimator
the estimated environment characteristics. We define:
where
is the true estimate vector. The true estimate
of the selected
is the mean reward which this action received the last
action
times that it was selected. It is computed as

(1)
is the total reward received by the Learning
where
times that action
was selected.
Algorithm during the last
(where
is an inThe parameter
teger internal LA parameter called learning window and is
used for ignoring old -and probably invalid- environmental responses.
is the Oldness Vector;
of action at any time instant is a nonnegative integer
number which expresses the time passed (counted in number of
iterations) from the last time that action was selected.
is the stochastic estimator
of action is defined as
vector. the stochastic estimate
(2)
symbolizes a random number selected with
where
a normal probability distribution, with mean equal to 0 and
proportional to
standard deviation
the time passed from the last time each action was selected.
Specifically a is an internal LA parameter that determines how
rapidly the Stochastic Estimates become independent from the
is the maximum permitted standard
True Estimates, while
deviation of the stochastic estimates, that bounds the standard
deviation.
A. SELA Algorithm

Initialization: Set Pi =
ui (t) = 0, 8i 2 f1; 2; . . . ; ng.

1=n

() =

Step 1: Select an action a t

the probability vector.

() =

and di t

mi (t)

301

ak according to

Step 2: Receive the feedback bk t of action ak

from the environment.
Step 3: Compute the new True Estimate dk t of
the selected action ak according to (1).
Step 4: Update the Oldness Vector by setting
mk t
and mi t
mi t 0
8i 6 k.
Step 5: Compute the new Stochastic Estimate
ui t 8i according to (2).
Step 6: Sort the set of the n actions in increasing order of their stochastic estimate
of mean reward so that the first element of
that classification (1st-optimal action am )
will be the one with the highest value and the
last one (nth-optimal action ar ) that with
the lowest value. Thus, um t
fui t g and
ur t
fui t g.
Step 7: Update the probability vector in
the following way: For every action ai i
set:
; ; m 0 ;m
; ; n with Pi t
>
Pi t
Pi t 0 =N where N is a parameter
called resolution parameter and determines
the step size
=N of the probability
updating. For the optimal action am set:
Pm t
0 i=m Pi t
, with
 Pi  and
n Pi
.
i=1
Step 8: Go to step 1. A complete formal description of SELA can be found in [17]. It
should be noted that the choice of LAs parameters a; max ; N is a critical issue, relative to the LAs performance under various
switching environments. An RLA is called
" 0 optimal [21], if there is an internal patg
.
rameter N such that:
N E fPm
(The symbolism E f1 1 1g stands for the expected
value). We have:

()

()=0

( ) = ( 1) + 1

()

( ) = max ( )

( ) = min ( )

1 2 ...
1
( + 1) =

+ 1 ... )
() 1

()

1(1 = 1 )

( + 1) = 1
=1
(

( + 1)

lim(lim !1

( ) )=1

Theorem: The SELA RLA is -optimal [21] in every stochastic environment that offers symmetrically distributed noise.
be the mean rewards offered by the enviLet
, respectively. If action
ronment to the actions ,
is the optimal one (
for
)
, then for every value
and
of the resolution parameter there is a time instant
such that for every
it holds that
.
The proof of the theorem is given in [34] and also for completeness and the reader convenience in Appendix. The assumption of symmetrically distributed noise is not unreasonable. The
noise of all known stochastic environments is symmetrically
distributed about the mean rewards of the actions. The -optimality of the algorithm implies that in case that there is an
optimal action, SELA converge to it. Thus, corresponding each
possible route of a single source-destination pair to a SELA action, SELA converge to the optimal action and thus, to the optimal route. SELA, is able to be adapted dynamically in changes
that take place to the environment; thus, it succeeds in converging to the optimal action irrespective of the traffic or/and
network changes. This results in choosing each time the optimal
route the environment offers.

302

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 3, AUGUST 2003

V. ROUTING SCHEME BASED ON SELA

In this section a QoS-based, dynamic source routing algorithm is presented, in compliance with the PNNI routing protocol, aiming at an efficient resource allocation and at a maximization of the network revenue. In particular, each source
node maintains a topology database consisting of a collection
of all PTSEs received from its neighbors that represents that
nodes present view of the PNNI routing domain. Moreover,
as it has been shown in several studies dealing with nonhierarchical (flat) QoS routing (e.g., [22]), efficient resource exploitation can be realized when routing is restricted to short paths,
even though there are longer candidate paths that may satisfy
the QoS requirements. Hence, in this study such a restriction
on candidate paths is applied deliberately. Specifically, at each
source node a routing table (database) of pre-computed hierarchically complete k-shortest path routes connecting this source
node to every possible destination node is constructed and maintained. This routing table needs to be updated whenever the network topology is modified. One of the algorithms described in
[23] can be used off-line to compute the k-shortest path (minimum-hop) routes for each source-destination pair. Upon receiving a request to establish a new connection with specific
traffic parameters and QoS requirements, the source node selects one of the pre-computed k-shortest path routes that can be
accepted by the CAC using its path selection routing algorithm.
If no such a route exists, then the incoming call is rejected.
Below the definition of the SELA actions, the computation of
the environmental feedback, and finally the description of the
SELA routing algorithm are given.
A. Definition of the Actions of the SELA
In this study a dynamic decentralized learning approach is
used where one SELA LA operates at each source node, determining how each incoming call is to be routed from this
node to every possible destination node. The SELA algorithm
uses k actions that represent the k-shortest path (minimum-hop)
routes for each source-destination pair. k depends on the network topology size and the relative traffic flows.
B. Environmental Feedback Received by SELA
In the proposed methodology, the utilization of the links (either logical or physical) are used in order to compute the feedback of SELA. Therefore, the algorithm requires that the nodes
retain a current link utilization table (database) for the entire
network in their level of hierarchy. It is also reminded that path
selection is performed at the source node as well as at each entry
border node. Hence, the feedback of the algorithm at the source
node is computed using the actual metrics inside the source peer
group and the aggregated metrics outside the source peer group.
On the other hand, the feedback of the algorithm at each entry
border node is derived from the actual metrics inside its peer
group.
Furthermore, in order to enhance the performance of the
SELA routing algorithm in heavy loading conditions -where
uncontrolled alternate routing can lead to increased call
blocking rate [18] we developed it to include the concept of
trunk reservation which is a technique suitable for achieving

this objective. Its effectiveness is based on the fact that at heavy

traffic load, where the saving of network resources is very critical, the highly utilized links are used only for minimum-hop
routes resulting in even better usage of the network resources
and so, in increased throughput. When the expected route
(which is defined as the maximum expected
utilization
utilization of the links that comprise this route) is greater than a
, the call is established at
predefined utilization threshold
this route only if it is a minimum-hop one. Otherwise, another
route is tried. It is noted that the expected utilization of the
links that comprise a route is defined as the utilization the links
will have assuming that the route will be accepted by the CAC
and accommodate the connection request.
Thus, if n links comprise the route that corresponds to the
is the expected utilization
action ak chosen by SELA, and
for this action is
of the th link, the feedback of the LA
computed as
(3)
where
if
otherwise
and MAX_NO_OF_HOPS is the maximum number of hops that
a route between any source-destination pair can have.
It should be noted that the first term on the right hand side of
the (3) represents a constant, predefined, maximum theoretical
bound for the sum of the utilization of the links that a route between any source-destination pair can have. This term indicates
the maximum feedback that a route can take, which occurs when
all the links that comprise the specific route are not utilized at
.
all
Equation (3) indicates that as the sum increases, which means
that the previously selected route becomes congested, the feedback of SELA deteriorates; such operation seems to be quite
natural. Moreover, this sum also increases implicitly when the
number of the hops of the route increases, which implies an alternate (not a minimum-hop) route. Thus, the algorithm generally favors the shortest path routes. It favors alternate routes
only when the congestion level of the shortest routes is much
greater than the one of the alternate routes. In addition to this,
when an alternate route is selected, its corresponding environmental feedback strongly deteriorates if that route consists of
links that have expected utilization greater than the trunk reser. In that way the over-utilization of the alvation threshold
ternate routes is avoided when the network becomes congested
(inherent characteristic of the trunk reservation mechanism).
C. SELA Routing Algorithm
The SELA routing algorithm is summarized as follows:
Offline operation:
At each source node:
1) Find the k-shortest (minimum-hop) routes
for each source-destination pair using one
of the algorithms described in [23] and

VASILAKOS et al.: OPTIMIZING QOS ROUTING IN HIERARCHICAL ATM NETWORKS

inform the routing tables (databases) at

these nodes.
2) Correspond each candidate route to one of
the k actions of SELA.
Online operation:
1) Whenever there is a new call-request at
the source node, estimate the expected
utilization of the links of the candidate
routes.
2) Run the SELA algorithm computing the environmental feedback.
3) Select the 1st-optimal action.
4) IF that action corresponds to a shortest
(minimum-hop) route AND this route is accepted by the CAC, THEN choose that action
as the selected one and establish the call
over the corresponding route.
5) ELSE IF that action corresponds to an alternate route which is accepted by the CAC
route
AND whose utilization exp
 TRT , THEN
choose that action as the selected one and
establish the call over the corresponding
route.
6) ELSE IF nothing of the above stands, THEN
repeat Step 3 concerning the next optimal
action.
7) IF all actions have been tried without any
of them being chosen as the selected one,
THEN the call request is rejected.

D. Properties of the SELA Routing Algorithm

Excellent FaultTolerance: The estimator considers the history of the link. Misleading network feedback doew not seriously affect the SELA Routing performance. SELA routing is a
simple probabilistic model that performs robustly in the face of
uncertainty conducting safety-based routing (inaccurate bandwidth feedback from aggregated topology).
a) Delayed feedback. This is a fundamental problem in
adapting routing. Cases with delayed feedback were sim. With
ulated using dynamic environment
the same conditions of the experiments of Tables XI and
SELA routing
XII and with delayed feedback
achieves 7.6% less while all other algorithms more than
15% performance degradation.
b) Node/link failure. If a node/link fails then, the network
resets; i.e., the shortesst path (minimumhop) routing
[23] is called and finds the new minimumhop paths.
The SELA restarts and, due to its high speed of convergence, adapt, to the new topology quickly. The same procedure can be followed when the topology is changed
(new nodes/links are added). Because topology changes
can be interpreted (i.e., changing the offered load), we
simulated using timevarying (burst) traffic. The minimumhop algorithm must react quickly. However, minimumhop algorithms need a minimum time and the
minimumhop algorithm to be chosed is not very crucial because the faster the minimumhop algorithm, the

303

higher the complexity. So, the time needed for the minimumhop to recompute the shortest paths is not very
crucial in SELA routing compared with the time SELA
uses to find the best routes again.
Small Computational Overhead: A very important feature of
SELA routing is the very small computational overhead it has.
This is due to the
1) minimal amount of feedback information the SELA
needs;
2) simple updating procedures;
3) control messages are much fewer than classic routing
schemes.
LA are amenable to simple hardware implementaiton because
basic stochastic computing elements have long proved their succees in synthezing LA [2].
VI. SIMULATOR DESIGN
The simulator was written in C++ programming language and
executed on an Ultra Spark Sun workstation. The simulation
runs were carried out at cell level. The confidence intervals of
our measurements were 95% constructed with the method of
independent replications. The Welchs procedure was used to
determine the warm-up (initial transient) period so as to discard possible misleading data. The length of the runs was estimated using the replication/deletion approach [24] in order to
gather reliable statistics. All the routing algorithms were performed using the same stream of random numbers (used for the
generation of call requests), so that a fair comparison among the
algorithms can be achieved, because, in this way any differences
regarding the calls arrival pattern are eliminated.
The candidate paths that a source node or an entry border
restricted cannode can choose belong to the minimum-hop
set
didate paths set. It is clarified that the minimum-hop
contains the minimum-hop paths, and additionally these paths
which are only one and two hops longer. This is desirable because using a longer path to route a connection ties up resources
at more intermediate nodes, and subsequently decreases the network revenue, especially at heavy traffic load [25].
A. Network Model
Simulation runs for flat and hierarchical routing were performed on the vBNS (very high performance backbone network
service) and NSFnet backbone topologies (existing commercial
networks) respectively shown in Figs. 3 and 4. For hierarchical
routing the NSFnet network was arbitrarily divided into four
peer-groups, each with three or four switches, corresponding in
this way to a two level hierarchical network. The star topology
aggregation scheme was adopted and performed. Distances between two neighboring switches correspond to 1500 Km. Assuming high-speed networks, where the propagation delay depends on the distance between both endpoints, we choose the
propagation speed to be two thirds the speed of light. The networks parameters are summarized in Table I.
B. Topology Aggregation
At first the aggregated (logical) links between two adjacent
peer groups are defined as inter-group links, while the logical

304

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 3, AUGUST 2003

TABLE II
TRAFFIC CHARACTERISTICS

Fig. 3. NSFnet simulated network which consists of 14 nodes, 22

bi-directional links, average degree 3, and is divided into four PGs for a
two-level hierarchical routing.

the spokes. Specifically, in this simulation study the diameter

of a logical node for the hop-count, and the maxCTD metrics
is defined as the maximum number of hops and the maximum
end-to-end delay of the corresponding candidate low level
paths constituting all the intra-group links within the same peer
group. It is noted that a logical intra-group link may correspond
to a number of low level (physical) paths connecting the same
source and destination border nodes. As far as the diameter of
a logical node for the allocated bandwidth topology attribute is
concerned, we regarded it as the maximum allocated bandwidth
of the links that constitute the corresponding candidate low
level paths of all the intra-group links within the group.
Similarly, an inter-group link may correspond to several physical links between two adjoining peer groups and consequently
its aggregated allocated bandwidth and maxCTD can be defined
in the same manner.
C. Triggered Updates of Topology State Information

Fig. 4. Commercial VBNS backbone network that runs on a synchronous

optical network (Sonet) at the OC-12 rate of 622.08 Mb/s and interconnects
12 points of presence (POPs).
TABLE I
NETWORK PARAMETERS

links between every two border nodes in the same peer group
are defined as intra-group links. In general, the topology aggregation consists of the aggregation of nodal and link information.
In this simulation study, the aggregation of links hop-count,
maxCTD (topology metrics), and allocated bandwidth
(topology attribute) are considered. In particular, to accommodate traversing a logical node as well as routing to the inside
of the node, the symmetric star topology aggregation scheme
with a uniform radius is adopted and applied (default node
representation in the PNNI complex node representation). The
center of the star is the interior reference point of the logical
node, and is referred to as the nucleus. The logical connectivity
between the nucleus and a port of the logical node is referred to
as a spoke. For each topology state parameter associated with
a logical node, a radius is derived from the diameter of
the logical node. For a topology metric, the radius is simply
half the diameter. For a topology attribute, the radius is the
same as the diameter. For each topology state parameter a
radius has to be advertised to be used as the default value for

PNNI topology state information which has not changed

and has not been updated must be re-originated periodically
to prevent it from aging out of the network. Re-origination of
the PTSE is done using a timer-based update triggering policy
since it is easier to control update overhead. We implemented a
simple version of flooding in our simulator: each node update
its topology state information at regular time intervals (default
value of 5 s) and sends it to all its neighbors. When receiving
a state information update, a node updates its database and
forward the information to all its neighbors, except the one
from which the update was received. Duplicate routing updates
are discarded.
D. Traffic Model
As far as the traffic model is concerned, real-time Variable
Bit Rate (rt-VBR) traffic was used. Specifically, bursty (ON-OFF)
sources were assumed with exponentially distributed burst and
silent periods. So, when the source is active it generates cells
according to a Poisson process, while the burst size is assumed
to follow an exponential distribution. The duration of the established calls and the inter-arrival time of new call requests were
also assumed to be exponentially distributed (Poisson arrival
process). The traffic parameters of the offered calls are summarized in Table II. As far as their QoS requirements are concerned,
the CLR was set to 106, while the maxCTD constraint values
ranged from 15 ms to 65 ms.
The output buffer management scheme has been implemented. With the output buffer scheme, each switch has buffers
of finite size for each outgoing link. We assume output queuing
because it achieves optimal throughput/delay performance
among the various buffer schemes [26]. The queue memory size

VASILAKOS et al.: OPTIMIZING QOS ROUTING IN HIERARCHICAL ATM NETWORKS

requirements (buffer size) of a system based on fixed length

packets depend on the performance requirements of the system
(the acceptable cell loss rate, load, delay) and the selected
queuing principle [27]. Since it has been proven in [27] that a
switching element based on the output queuing principle has a
very well controlled behavior up to loads of 80 to 85%, we set
the buffer size for a certain cell loss ratio (CLR) requirement of
. This probability of losing a cell is a consequence of the
fact where more cells than a queue in the switch can store will
simultaneously compete for this queue. In ATM switches, this
probability must be kept within limits to ensure a high semantic
transparency.
In our study, we consider uniform, time-varying, and skewed
workloads. In uniform workload, the same call arrival rate is
used for each source-destination pair. In this case, the routing algorithms were examined for light, moderate, and heavy loading
conditions being determined by the values of the call arrival
rate. Moreover, the performance of the routing algorithms was
tested under changing traffic conditions. In this scenario, after
every passage of a predefined simulated time period (in seconds)
the network load varied by changing the call arrival rates for
each source-destination pair. For the vBNS network the call arrival rate was uniformly distributed in the range 0 to 200 calls/s
while for the NSFnet in the range 0 to 530 calls/s. Finally, in
skewed workload-applied only to the NSFnet- some switches
are selected as the destination of the majority of the calls and
consequently are regarded as hot-spots. Specifically, for each
calls/s call arrival rate was
source-destination pair the
used, except for the pairs where the hot-spots switches were
calls/s call
the destination nodes. In this later case, the
arrival rate was used. In this simulation study, the switches 1 and
13 were selected as the hot-spots switches.
E. Connection Admission Control Scheme
The CAC scheme we adopted is the one proposed by Guerin
et al. in [28], which is based on the fluid-flow approximation
model of [29]. In summary, the equivalent capacity of N multiplexed sources is given by

where
and
are the mean
is
idle and burst periods of source , respectively, and
its peak cell rate (PCR). Furthermore, for a buffer size and a
buffer overflow probability requirement (CLR) smaller than ,
is given by
the factor

Using the above equivalent bandwidth approximation, the expected utilization of the links that constitute the candidate route
can be easily estimated by the relation

where is the link capacity.

In order to satisfy the CLR requirements in a single node
in case of a new call set-up request, the CAC accepts the new

305

candidate connection only if

. Otherwise, the
CAC rejects the candidate connection.
As far as the maxCTD requirements are concerned, when a
connection is routed over a path which consists of links
is computed by
with link delay , the end-to-end delay
the following:
(4)
where
In (4)
and
are the propagation and
and
transmission delay respectively at link , while
are the processing and the queuing delay, respectively
that a cell experiences at the previous switch where link is attached. It is noted that, while the first three of them are constant, the queuing delay depends on the queue at the buffer of
the switch. In this study, for the estimation of the queuing delay,
we consider the worst-case where a cell enters a buffer with size
as the th cell and thus, it experiences the maximum possible
of (4) is an upper bound of
queuing delay. In this case,
the maximum end-to-end delay. Consequently, a candidate new
will be acconnection with end-to-end delay constraint
.
cepted by the CAC if
VII. SIMULATION PERFORMANCE STUDY
A. Compared Routing Algorithms
In this section, the routing algorithms that were compared in
the simulation study are briefly described.
SELA-routing Algorithm Described in Section V.
Competitive Call Routing (CompCR): Define the cost metric
, where and are the occupied and total
for link as
bandwidth for link and is a parameter. Define the cost of a
path as the sum of the link costs. Select the minimum cost path
provided it is feasible (i.e., it satisfies the QoS requirements of
the specific call request) and its cost does not exceed a threshold
. Selecting appropriate values for parameters and turns
and
out to be nontrivial [30]. In this paper, we set
after pilot simulation runs to achieve the best possible
performance.
Shortest Distance Routing (SDR): Define the cost metric for
, where
is the bandwidth available on link .
link as
Define the cost of a path as the sum of the link costs. Select the
minimum cost path provided it is feasible [22].
Least Loaded-Shortest Path Routing (LSR): A feasible path
(i.e., a path that satisfies the QoS requirements of the specific
call request) with the minimum hop count (minimum-hop path)
is tried. If there are several such paths, the one with the maximum available bandwidth (least loaded path) is chosen. If several such paths exist, one of them is randomly chosen. The call
request is rejected if there is not any minimum-hop path that satisfies the QoS requirements.
Shortest-Least Loaded Path Routing (SLR): A feasible path
with the maximum available bandwidth is tried. If there are several such paths, the one with the minimum hop count is chosen.
If several such paths exist, one of them is randomly chosen. A
call request is rejected only if there is not any feasible path.

306

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 3, AUGUST 2003

TABLE III
SELAS PARAMETER VALUES

Shortest-Least Loaded Path Routing With Trunk Reservation

(TSLR): A shortest-least loaded path routing algorithm that
uses the trunk reservation concept.
Least Loaded-Alternate Path Routing (LALT): A feasible
least loaded minimum-hop path is tried. If no feasible minimum-hop path exists, try the least loaded alternate (other than
the minimum-hop) paths in order of preference (i.e., first the
paths which are one hop longer, then the two hop longer and so
on). If several such paths exist, one is randomly chosen. A call
request is rejected only if there is not any feasible path.
Least Loaded-Alternate Path Routing With Trunk Reservation
(TLALT): A least loaded alternate path routing that uses the
trunk reservation concept.
It should be mentioned that the values of the SELA parameters that were used during the simulation are tabulated in
Table III. These parameters were set after pilot simulation runs
in order to achieve the best possible performance. Finally, the
trunk reservation threshold was set to 0.81 for all the routing
algorithms which make use of the trunk reservation concept.

TABLE IV
PERFORMANCE OF THE ROUTING ALGORITHMS IN TERMS OF THE EARNED
NETWORK REVENUE RATIOS UNDER VARIOUS UNIFORM LOADING
CONDITIONS FOR THE VBNS-BACKBONE TOPOLOGY

TABLE V
PERFORMANCE OF THE ROUTING ALGORITHMS IN TERMS OF THE BANDWIDTH
REJECTION RATIOS UNDER VARIOUS UNIFORM LOADING CONDITIONS
FOR THE VBNS-BACKBONE TOPOLOGY

TABLE VI
PERFORMANCE OF THE ROUTING ALGORITHMS IN TERMS OF THE EARNED
NETWORK REVENUE RATIOS UNDER TIME-VARYING LOADING CONDITIONS
FOR THE VBNS-BACKBONE TOPOLOGY

B. Performance Metrics
Consider the following notation. Let the network be represented as a capacitated (directed or undirected) graph
with n nodes and m links, where
represents
. Let
be the total number of
the capacity of the link
and
and the number of the
call requests
rejected calls where each call request is described by the tuple
. Let nodes and be the source and
the bandwidth (equivalent cadestination of the request,
denote the holding time (i.e.,
pacity) required by the and
duration) of the call. We define the performance metrics used in
the evaluation of the routing algorithms as follows: Bandwidth
Rejection Ratio (BRR). It is defined as the percentage of the
total requested bandwidth that was rejected

TABLE VII
PERFORMANCE OF THE ROUTING ALGORITHMS IN TERMS OF THE BANDWIDTH
REJECTION RATIOS UNDER TIME-VARYING LOADING CONDITIONS FOR THE
VBNS-BACKBONE TOPOLOGY

Bandwidth Acceptance Ratio (BAR). This metric introduced

in [22] and expresses the percentage of the total requested bandwidth that was routed successfully. Note that this measure ignores the duration of the calls and does not take into account
the length of paths or other measures of efficiency.

time (bandwidth-duration product). We introduce the measure

ENRR as the percentage of the mean total earned network revenue described by the following:

Earned network revenue ratio (ENRR). We define the revenue for each call to be proportional to its capacity and holding

Average cell transfer delay (ACTD). It is defined as the average delay to deliver a cell from origin to destination.

VASILAKOS et al.: OPTIMIZING QOS ROUTING IN HIERARCHICAL ATM NETWORKS

307

TABLE VIII
PERFORMANCE OF THE ROUTING ALGORITHMS IN TERMS OF THE AVERAGE CELL TRANSFER DELAY IN SECS (I.E., THE AVERAGE
DELAY REQUIRED TO DELIVER A CELL FROM ORIGIN TO DESTINATION) UNDER VARIOUS UNIFORM LOADING CONDITIONS
FOR THE NSFNET-BACKBONE TOPOLOGY CONFIGURED AS A TWO LEVEL HIERARCHICAL NETWORK

TABLE IX
PERFORMANCE OF THE ROUTING ALGORITHMS IN TERMS OF THE BANDWIDTH ACCEPTANCE RATIO (EXPRESSING THE NETWORK REVENUE) UNDER
VARIOUS UNIFORM LOADING CONDITIONS FOR THE NSFNET-BACKBONE TOPOLOGY CONFIGURED AS A TWO LEVEL HIERARCHICAL NETWORK

TABLE X
PERFORMANCE OF THE ROUTING ALGORITHMS IN TERMS OF THE ACHIEVED THROUGHPUT (EXPRESSED AS THE NUMBER OF THE SUCCESSFULLY ESTABLISHED
CALLS) UNDER VARIOUS UNIFORM LOADING CONDITIONS FOR THE NSFNET-BACKBONE TOPOLOGY CONFIGURED AS A TWO LEVEL HIERARCHICAL NETWORK

Network throughput (NT). We consider as a measure of the

network throughput the total number of the successfully established calls

C. Numerical Results
In this section the numerical results obtained from the comparative simulation performance study among the proposed
methodology and the routing algorithms mentioned above on

the vBNS and NSFnet backbone topologies are presented. The

performance results are tabulated in Tables IVXIII. Due to
limited space, the numerical results are not commented with
details, but, instead, some general observations are given
Giving priority to shortest paths, a routing scheme always
performs better. This is due to the fact that the use of long paths
is undesirable since they tie up network resources at more intermediate nodes than minimum hop paths. Thus, since the amount
of the consumed resources increases, the number of the connections that can be accepted decreases.

308

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 3, AUGUST 2003

TABLE XI
PERFORMANCE OF THE ROUTING ALGORITHMS IN TERMS OF THE AVERAGE
CELL TRANSFER DELAY IN SECS (I.E., THE AVERAGE DELAY REQUIRED TO
DELIVER A CELL FROM ORIGIN TO DESTINATION) UNDER TIME-VARYING AND
SKEWED LOADING CONDITIONS FOR THE NSFNET-BACKBONE TOPOLOGY
CONFIGURED AS A TWO LEVEL HIERARCHICAL NETWORK

TABLE XII
PERFORMANCE OF THE ROUTING ALGORITHMS IN TERMS OF THE BANDWIDTH
ACCEPTANCE RATIO (EXPRESSING THE NETWORK REVENUE) UNDER
TIME-VARYING AND SKEWED LOADING CONDITIONS FOR THE
NSFNET-BACKBONE TOPOLOGY CONFIGURED AS A TWO LEVEL
HIERARCHICAL NETWORK

TABLE XIII
PERFORMANCE OF THE ROUTING ALGORITHMS IN TERMS OF THE ACHIEVED
THROUGHPUT (EXPRESSED AS THE NUMBER OF THE SUCCESSFULLY
ESTABLISHED CALLS) UNDER TIME-VARYING AND SKEWED LOADING
CONDITIONS FOR THE NSFNET-BACKBONE TOPOLOGY CONFIGURED
AS A TWO LEVEL HIERARCHICAL NETWORK

rate is ). In this case SELA exhibits the second lower blocking

ratio. As far as the average cell delay performance metric is concerned, SELA manages to deliver the cells from origin to destination exhibiting the second lower value in all of the examined
loading conditions, since the LSR always achieves the lower average cell delay as it is expected.
Finally, it should be noted that the fact that as the load increases the average cell transfer delay decreases is not peculiar,
although it seems supprises at first. This happens because when
the load increases, fewer alternate routes are being selected,
thus lowering the average propagation delay and consequently
the average cell transfer delay (transmission delay and queuing
delay are much smaller than the propagation one and those, they
do not affect the overall cell transfer delay as significantly as the
propagation delay do.
Generally, it could be concluded that among the compared
routing schemes, the SELA routing algorithm always has
the best (or the second best) performance irrespective of the
loading conditions. Thus, it manages to admit more calls
into the network than the other algorithms, and deliver the
cells from origin to destination achieving a quite satisfactory
performance regarding the average cell delay metric, while
guaranteeing the QoS requirements (since the CAC algorithm
rejects the calls when the QoS constraints cannot be satisfied).
The SELA routing algorithm achieves to route the offered
traffic effectively because it avoids unnecessary use of alternate
routes while balancing the load when required. By predicting
an oncoming congestion, it routes the offered loads through
the less loaded paths. The simulation results demonstrate the
effective compromise that the SELA routing algorithm attains
between the use of minimum-hop routes and the load balancing,
which is the key of its efficiency.
VIII. CONCLUSIONS AND FUTURE WORK

In light loading conditions, even though a routing scheme

should favorably select shortest paths for use, it should also balance the load of the network since there are plenty of resources.
As the network load increases, it should use alternate routes
more selectively decreasing the load balancing, so that waste of
network resources and consequently, premature network congestion will be avoided.
Generally, the algorithms that use trunk reservation (SELA,
TSLR, TLALT) exhibit better performance than the others
(LSR, SLR, LALT) in every loading scenario except for the
case where the network is loaded very lightly. In this latter
case, LALT shows the best performance establishing more
connections than the other algorithms.
The SELA routing algorithm achieves the best performance
over all the other routing algorithms, in almost all the loading
conditions. Specifically, it shows the lower blocking ratio in
seven out of the eight examined loading scenarios, allowing
more connections to be established. The only case that one of the
other algorithms -specifically the LALT- blocks fewer calls than
SELA is under very light loading conditions (when the arrival

In this paper, a dynamic call routing methodology was proposed that uses an efficient RLA suitable for real-time application in ATM networks. RLAs as simple propabilistic models
performs robustly in the face of uncertainty conducting safety
-based routing (inaccurate bandwidth feedback from aggregated
topology). This algorithm uses hierarchical route information of
the connection, aggregated topology and loading information of
the network to select feasible paths that satisfy the QoS constraints while achieving high resource efficiency.
In particular, it is demonstrated that for the implementation
of the algorithm two sets of information are required. The first
one regards the topology information that is necessary for the
computation of the k-shortest path hierarchically complete
routes, while the other includes the attribute information of the
allocated bandwidth on the links that comprise these routes.
This latter information is used by the proposed methodology
for calculating and retaining a link utilization database, which
is used by the routing algorithm for the selection of the most
suitable among these k-shortest path routes. Since the PNNI
protocol adopted by the ATM forum [3] includes mechanisms for distributing the information as well as hierarchy
mechanisms, which provide an aggregated view of clusters
of switching elements, it is clear that the proposed routing

VASILAKOS et al.: OPTIMIZING QOS ROUTING IN HIERARCHICAL ATM NETWORKS

methodology can be implemented in the framework of the

PNNI.
A comparative performance study among the proposed
and other well-known path selection routing algorithms was
demonstrated by means of simulation on existing commercial
networks. The used performance metrics were the average cell
transfer delay, bandwidth rejection ratios, bandwidth acceptance ratios, network throughput, and network revenue that
the various algorithms exhibited under uniform, time-varying
and skewed traffic workloads. Simulation results showed the
effectiveness of the proposed routing algorithm and disclosed
the strength and weakness of the various schemes.
Our future research includes studying other aggregation
schemes and route selection policies. Our objective is to
compare the performance of the various aggregation schemes
and study the influence that other important factors exercise on that performance, such as routing update frequency,
routing algorithms, and network configuration. Besides, we
also plan to extend the scheme to handle point-to-multipoint
and multipoint-to-multipoint routing. Finally, a promising
approach seems to be the use of AntNets mobile distributed
agents(or completely fault-tolerant ant agents) that collect online information to improve/enrich the operation of the SELs
agents, statically connected to the network nodes and directly
responsible for routing decisions. This is a Reinforcement
Learning multiagent system showing many interesting features
i.e the routing traffic does not increase with the rate of change
in the network making these algorithms suitable for highly
dynamic networks (adaptability to user requirements, demand,
cost structure, and topology changes) [31].

309

Theorem 6.2.6.) that under these conditions

,
are
real valued.
. It is known
Now, lets define the random variable
([33], Theorem 13) that the characteristic function of is:
(6)
and
are real valued it is derived that
Since
is also real valued.
Thus, is symmetric with respect to 0.
Since
it is derived that is symmetric with respect to
. Lemma 1 has been proved.
to be (respectively) the means
Lemma 2: Let
to be the variances of the symmetrically
and
distributed (with respect to their means) rewards of the
, offered by the environment. Let
actions
be corresponding current stochastic
estimates of the mean rewards at the time instant t. Then,
are symmetrically distributed (with
respect to their means) random variables with means equal to
, respectively.
denotes that the
Proof: The symbolism
random variable is normally distributed with a mean and a
variance
Let
be the window size. Consider action
.
be the content of the th place of actions
Let
window at time instant .
was the last selected at time
and
If the action
we have

APPENDIX
Proof of Epsilon Optimality

(7)

(For Detailed Analysis of the SELA and its Epsilon Optimality the Reader Should Consult [34]). We shall first prove
some supplementary lemmas.
Lemma 1: Assume two statistically independent random
with mean equal to , respectively. Define
variables ,
. If the density functions
the random variable
,
of ,
are symmetric (with respect to their
means , ) then the random variable is also symmetric
).
(with respect to its mean
Obviously this lemma can be easily generalized for
where
are symmetric
random variables.
Proof: Define the random variables , as :
, and
with density functions
,
and characteristic functions
,
, respectively. Obviously the random variables
Proof: Define the random variables , as
(5)
,
and characteristic functions
with density functions
,
, respectively.
Obviously, the random variables and have means equal
to 0 and are symmetric with respect to 0. It is known ([32],

for
By definition, the random variables
are symmetrically distributed with a mean and a variance
is also a symmetric random variable. Thus,
is a sum of
symmetric random variables. So, from Lemma 1 is derived that
is also a symmetric random variable. Its mean
and its variance are as follows :

In the same way if

then
Thus,
is a symmetrically distributed (with respect
to its mean) random variable with a mean equal to (for every
). Lemma 2 has been proved.
Lemma 3: For any two actions ai and aj we define:
. If
then there is a positive real number
such that
and consequently
for any time instant . We assume that
has a
nonzero limit.
.
Proof: Define the random variable:
is a symmetric (with
From Lemmas 1 and 2 is derived that
respect to its mean) random variable with a mean equal to
and a variance equal to
.

310

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 3, AUGUST 2003

Let
be the density function of
is symmetric about the line
lier,
If we define
.
Thus, we have

. As discussed ear.
we have:

SELA LA, for any two actions

, we have

(8)
and
is symmetric about
It is known that
. So, (4) can be written as
the line

such that

, and

(10)
In the same way

(11)
(9)
where
Since the variance of
it is derived that

by utilizing (6) and (7) we have

is bounded
. It

is obvious that
Thus
Thus we have that
for any time instant . Lemma 3 has been proved. Note that this
is not proven conditional on the state of the learning automata.
of the action set ,
Lemma 4: Assume any subset
. Thus,
such that
. For any action
define:
for
.
Then according to the SELA LA, for any two actions ,
such that
and
we have

for any time instant . (Where

is defined as in Lemma 3).
Proof: We shall prove Lemma 4 by utilizing mathematical
subset.
induction on the size of the
Basis step. We have to prove that the theorem holds
. Proof of basis step. For any subset
for
we have

Thus,
for any time
instant .
Induction hypothesis. Assume that the theorem holds for
. Thus, for any subset An of the action set A such
it holds that according to the LA of the SELA
that
such that
and
automaton, for any two actions ,
we have:
for
any time instant .
Induction step. We have to prove that the theorem holds
.
for:
of the action
Thus, we have to prove that for any subset
it holds that according to the
set such that

(12)
So (8) becomes

Thus, Lemma 4 has been proved.

for
. Thus
Lemma 5: Let
is an optimal one. If
action
then according to the SELA
for
LA for any time instant t we have
.
be an optimal one. Thus,
Proof: Let action
for
.
From Lemma 3 is derived that for every
it holds that:
. Thus, for every
it holds that:
(13)
)
Now, by combining (9) with Theorem 3 (for subset
it holds
is derived that for every
that

Thus,
for
. Lemma 5 has
been proved.
Theorem 1: The SELA learning automaton is -optimal in
every environment that offers symmetrically distributed noise.
is an optimal one (
for
Thus, if action
) and
, then for every value
of the resolution parameter there is a time
such that for every
it holds
.
instant
Proof: For every two actions , and every time instant
.
t define
As in Lemma 5 lets define the quality:

VASILAKOS et al.: OPTIMIZING QOS ROUTING IN HIERARCHICAL ATM NETWORKS

Assume that at a time instant we have

and
. If neither action or
has the highest estimated
for
mean reward at this instant (
) then both
and
are decreased by the same
(where is the resolution parameter of the SELA
quantity
automaton). Thus

If action or action
has the highest estimated mean reward at the time instant
, or
for
) the
T(
is an average increase by
quantity
where the quantities
,
are as defined above.
and
Thus, if at a time instant we have
then the average modification
of the quantity
is

(14)
For any pair of actions

If

and

define the quantity

as:

311

Lets define

We have:
(16)
with
From (12) is derived that for any pair of actions ,
, there is a time instant
such that:
(17)
If
Thus, before
ence between

then
.
will become equal to zero to average differand
is

There are two possible cases.

and
i)
and
ii)
If condition (13) is satisfied then Case ii) becomes

,
.

then:

If
then from Lemma 3 is derived that
for any time instant . In this case we have

Thus, if condition (13) is satisfied then in both case (i) and

.
(ii) we have
.
Thus,
We have proved that for every pair of actions , with
there is a time
such that:
(18)
be an optimal one (
Let action
).
we have
Thus, for any section
quently, from (14) is derived that for any action
that
is a time instant

If we select
and, subsequently,

(15)
for .
It is known (see proof Lemma 3) that
are two actions so that
Thus, from (11) is derived that if ,
then

then

for
and consethere

for every

Theorem 1 has been proved. The SELA learning automaton is

-optimal in every stationary stochastic environment.
REFERENCES
[1] Traffic Control and Congestion Control in B-ISDN ,, Geneva, ITU-T,
Recommendation 1.371, 1996.
[2] P. Mars and W. J. Poppelbaum, Stochastic and Deterministic Averaging
Processors. Stevenage, U.K.: Peregrinus, 1981.

312

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICSPART C: APPLICATIONS AND REVIEWS, VOL. 33, NO. 3, AUGUST 2003

[3] Private Network-Network Interface Specification Version 1.0

(PNNI 1.0), The ATM Forum Technical Committee, ATM Forum
af-pnni-0055.000, 1996.
[4] R. Guerin and A. Orda, QoS-Based Routing in Networks With Inaccurate Information: Theory and Algorithms,, IBM Res. Rep., RC 20515,
1996.
[5] W. Lee, Topology aggregation for hierarchical routing in ATM networks, Comput. Commun. Rev., pp. 8292, 1995.
, Spanning tree method for link state aggregation in large commu[6]
nication networks, in Proc. IEEE INFOCOM, Apr. 1995, pp. 297302.
[7] B. Awerbuch and Y. Shavitt, Topology Aggregation for Directed Graph,
to be published.
[8] B. Awerbuch, Y. Du, B. Khan, and Y. Shavitt, Routing Through Teranode
Networks with Topology Aggregation, to be published.
[9] H. Xie and J. Baras, Performance Analysis of PNNI Routing in ATM
Networks: Hierarchical Reduced Load Approximation, Univ. Maryland, Institute System Research Tech. Rep. TR 97-65, 1997.
[10] J. Baras and M. Corson, Tactical and strategic communication network
simulation and performance analysis, in Proc. 1st Annual Conf. on
Advanced Telecommunications and Information Distribution Research
Program (ATIRP), to be published.
[11] A. Orda, Routing with end-to-end QoS guarantees in broadband networks, IEEE/ACM T-Networking, vol. 7, no. 3, pp. 365374, June 1999.
[12] D. Lorenz and A. Orda, QoS routing in networks with uncertain parameters, IEEE/ACM T-Net, pp. 768778, Dec. 1998.
[13] S. H. Kim, K. Lim, and C. Kim, A scalable QoS-based inter-domain
routing scheme in a high speed wide area network, Comput. Commun.,
vol. 21, pp. 390399, Apr. 1998.
[14] E. Felstaine and R. Cohen, On the distribution of routing computation
in hierarchical ATM networks, IEEE/ACM Trans. Net., vol. 7, no. 6,
pp. 906916, Dec. 1999.
[15] F. Hao and E. Zegura, On scalable QoS routing: performance evaluation of topology aggregation, in Proc. IEEE INFOCOM, Mar. 2000, pp.
147156.
[16] Marie, S. Singh, T. Jaakkola, and M. Jordan, Learning without state
estimation in partially observable Markovian decision processes, in
Proc. 11th Machine Learning Conf., New Brunswick, NJ, July 1994,
pp. 284292.
[17] A. V. Vasilakos and G. Papadimitriou, Ergodic discretized estimator
learning automata with high accuracy and high adaptation rate for
nonstationary environments, Neurocomput., vol. 4, pp. 181196, May
1992.
[18] A. F. Atlasis, M. P. Saltouros, and A. V. Vasilakos, On the use of a
stochastic estimator learning algorithm to the ATM routing problem: A
methodology, in Proc. IEEE GLOBECOM, Dec. 1998.
[19] A. F. Atlasis, A. V. Vasilakos, and N. H. Loukas, The use of learning
algorithms in ATM networks call admission control problem: A
methodology, in Computer Networks Amsterdam, North Holland,
The Netherlands.
[20] A. F. Atlasis and A. V. Vasilakos, LB-SELA: Rated-based access control for ATM networks, in Proc.IEEE INFOCOM 94, Mar. 1994, pp.
15521569.
[21] K. Narendra and M. Thathachar, Learning automata: A survey, IEEE
Trans. Syst. Man, Cybern., vol. TSMC4, pp. 323334, July 1974.
[22] Q. Ma and P. Steenkiste, On path selection for traffic with bandwidth
guarantees, in Proc. IEEE Int. Conf. on Network Protocols, Atlanta,
GA, to be published.
[23] T. H. Cormen, L. Ch, R. L. Rivest, and C. Stein, Introduction to Algorithms, 2nd Edition ed. Cambridge, MA: MIT Press, 2001.
[24] A. M. Law and W. D. Kelton, Simulation Modeling and Analysis, second
ed, E. M. Munson and L. Margery, Eds. New York: McGraw-Hill,
1991.
[25] I. Matta and A. U. Shankar, Dynamic routing of real-time circuits, in
Proc. IEEE Int. Conf. on Network Protocols, to be published.
[26] M. Hluchyi and M. Carol, Queuing in high-performance packet
switching, IEEE J. Select. Areas Commun., vol. 6, Dec. 1988.
[27] D. P. Martin, Asynchronous Transfer Mode: Solution for Broadband
ISDN, 2nd ed. West Sussex, U.K.: Ellis Horwood, 1993.
[28] R. Guerin, H. Ahmadi, and M. Naghshineh, Equivalent capacity and
its application to bandwidth allocation in high-speed networks, IEEE
J. Select. Areas Commun., vol. 9, pp. 968981, Sept. 1991.
[29] D. Anick, D. Mitra, and M. M. Sondhi, Stochastic theory of a datahandling system with multiple sources, Bell Syst. Tech. J., vol. 61, pp.
18711894, Oct. 1982.
[30] L. Zhang, M. Andrews, S. Bhatt, and K. Krishnan, A performance comparison of competitive on-line routing and state-dependent routing, in
Proc. IEEE GLOBECOM, 1997, pp. 18131819.

[31] G. D. Caro and A. V. Vasilakos, AntNet combined to SELA for QoS

routing in ATM networks, in Proc. Ants2000, From ant Colonies Artificial Ants: 2nd International Workshop ant Algorithms, Brussels, Belgium, Sept. 89, 2000.
[32] K. l. Chung, A Course in Probability Theory. Orlando, FL: Harcourt
Brace Jovanovich, 1968.
[33] H. Cramer, Random Variables and Probability Distributions. Cambridge, MA: Cambridge Univ. Press, 1970.
[34] A. Vasilakos and G. Papadimitriou, A new approach learning automata:
Stochastic Estimator Learning Algorithms, in Neurocomputing , 1995,
vol. 7, pp. 275297.
[35] W. Pedrycz and A. Vasilakos, Computational Intelligence in Telecommunications Networks. Boca Raton, FL: CRC, 2000.

Athanasios Vasilakos has born in Pella, Greece, in

1959. He received the B.S. degree in electrical engineering from the University of Thrace, Greece, in
1983, the M.S. degree in computer engineering from
the University of Massachusetts, Amherst, 1986, and
the Ph.D. degree in computer engineering from the
University of Patras, Greece, in 1988.
From 1988 to 1991, he was with Computer
Technology Institute, University of Patras. From
1991 to 1995, he was Professor at the Hellenic Air
Force Academy, Greece. From 1995 to 2002, he was
Researcher at the Computer Science Institute, Foundation for Research and
Technology-Hellas(FORTH), Greece. Since 2002, he is a Professor at the University of Thessaly, Greece, and Manager in the Hellenic Aerospace Industry.
His research interests involve telecommunication networks (IP/ATM, mobile,
active nets) and computational intelligence. He is a co-editor (with W. Pedrycz)
of the volume entitled Computational Intelligence in Telecommunications
Networks, (Boca Raton, FL: CRC Press, 2001). He has published more than 90
peer-reviewed journal and conference papers.
Dr. Vasilakos is a member of the technical progeram committees of a number
of conferences and serves on editorial boards of several journals, including
Computer Communications (19972002), ACM Applied Computing Review,
Soft Computing, and IEEE Communications Magazine. He has been a Guest
Editor of several journals, such as IEEE TRANSACTIONS ON SYSTEMS, MAN,
AND CYBERNETICS, IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS,
Soft Computing, and the Journal of Interactive Learning Research. He is a
member of the ACM.

M. P. Saltouros was born in Xouthi, Greece in 1973. He graduated from the

Hellenic Air Force Academy in 1995 and received the Ph.D. degree from National Technical University, Athens, Greece, in 2001.
He has been an Officer in the Hellenic Air Force Academy since 1995.

A. F. Atlassis was born in Thessaloniki, Greece in 1974. He graduated from

the Hellenic Air Force Academy in 1993 and recieved the Ph.D. degree from
National Technical University, Athens, Greece in 1999.
He has been an Officer in the Hellenic Air Force Academy since 1993.

Witold Pedrycz (F99) is a Professor and Chair

in the Department of Electrical and Computer
Engineering, University of Alberta, Edmonton AB,
Canada. He is also a Canada Research Chair in
Computational Intelligence. He is actively pursuing
research in computational intelligence, fuzzy
modeling, knowledge discovery and data mining,
fuzzy control including fuzzy controllers, pattern
recognition, knowledge-based neural networks,
telecommunication networks, relational computation, and software engineering. He has published
numerous papers in this area. He is also an author of seven research monographs covering various aspects of computational intelligence and software
engineering.
Dr. Pedrycz has been a member of numerous program committees of IEEE
conferences in the area of fuzzy sets and neurocomputing. He currently serves
as an Associate Editor of IEEE TRANSACTIONS ON SYSTEMS, MAN, AND
CYBERNETICS and IEEE IEEE TRANSACTIONS ON FUZZY SYSTEMS.