Network Topology

ELEG 667-013 Spring 2003

Outline:
Why Network Topology is Important ? Modeling Internet Topology Complex Networks Scale-free Networks Power-laws of the Web Search in power-law networks: GNUTELLA, a P2P example.

Why Topology is Important ?

Design Efficient Protocols Solve Internetworking Problems: - routing - resource reservation - administration Create Accurate Model for Simulation Derive Estimates for Topological Parameters Study Fault Tolerance and Anti-Attack Properties

Modeling Internet Topology [1]:

Graph representation Router-level modeling - vertices are routers -edges are one-hop IP connectivity Domain- (AS-) level model (high degree of abstraction) - vertices are domains (ASes) - edges are peering relationships Nodes can be assigned numbers rep. e.g. buffer capacity Edges migth have weights rep. e.g. prop. delay, bandwidth capacity.

Modeling Internet Topology [1]:

transit domains domains/autonomous systems

exchange point
border routers

peering hosts/endsystems routers

stub domains access networks lowly worm

Barabasi Albert Model (BA Model):

Basis for most current topology generators Very simplistic model Network evolves in size over time. Preferential Connectivity Probability that a newly added node will attach to node i

ki (ki ) jk
Many extensions.

Waxman Model:
Router level model Nodes placed at random in 2D space with dimension L Probability of edge (u,v):

a*e(-d / (bL) ), where d is

Euclidean distance (u,v), a and b are constants Models locality - no sense of backbone or hierarchy - does not guarantee connected network - as #nodes the #links proportionally

d(u,v)

Transit-Stub Model:
Router level model Transit domains placed in 2D space populated with routers connected to each other Stub domains placed in 2D space populated with routers connected to transit domains Models hierarchy Edge count, guaranteed connectivity

Transit-Stub Model:
No concept of a host all nodes are routers. Two level hierarchy First generate a number of transit domains, then generate a set of stub networks. Given average edge-count, produce a random graph, making sure that it is connected.

Inet:
Generate degree sequence Build spanning tree over nodes with degree larger than 1, using preferential connectivity randomly select node u not in tree join u to existing node v with probability d(v)/d(w) Connect degree 1 nodes using preferential connectivity Add remaining edges using preferential connectivity

BRITE:

Generate small backbone, with nodes placed: randomly or concentrated (skewed) Add nodes one at a time (incremental growth) New node has constant # of edges connected using: preferential connectivity and/or locality

Complex Networks:
Two limiting-case topologies have been extensively considered in the literature [4],[5].: regular network (lattice), the chosen topology of innumerable physical models such as the Ising model or percolation. random graph, studied in mathematics and used both in natural and social sciences. Properties studied in detail by Pal Erdos. Most of Erdos work concentrated on the case in which the number of vertices is kept constant but the total number of links between vertices increases: the Erds-Rnyi result states that for many important quantities there is a percolation-like transition at a specific value of the average number of links per vertex.

Complex Networks:

random networks are used in: Physics: in studies of dynamical problems, spin models and thermodynamics, random walks, and quantum chaos. Economics and social sciences: to model interacting agents.

Complex Networks:
In contrast to these two limiting topologies, empirical evidence suggests that many biological, technological or social networks appear to be somewhere in between these extremes. many real networks seem to share with regular networks the concept of neighborhood, which means that if vertices i and j are neighbors then they will have many common neighbors --- which is obviously not true for a random network. On the other hand, studies on epidemics show that it can take only a few ``steps'' on the network to reach a given vertex from any other vertex. This is the foremost property of random networks, which is not fulfilled by regular networks.

Complex Networks:

Complex Networks:

The Watts-Strogatz model [5]. : To bridge the two limiting cases, Watts and Strogatz [Nature 393, 440 (1998)] have introduced a new type of network which is obtained by randomizing a fraction p of the links of the regular network. Initial structure (p=0) is the one-dimensional regular network where each vertex is connected to its z nearest neighbors. For 0 < p < 1, we denote these networks disordered. for the case p=1, we have a completely random network.

Complex Networks:
Watts and Strogatz report that for a small value of the parameter p, there is an onset of small-world behavior. It is characterized by the fact that the distance between any two vertices is of the order of that for a random network and, at the same time, the concept of neighborhood is preserved. The effect of a change in p is extremely nonlinear, where a very small change in the connectivity of the network leads to a dramatic change in the distance between different pairs of vertices.

Complex Networks:

The scientific question we are trying to answer is: Does the onset of the small-world behavior occurs at a given value of p or does it occur for a value of the system size n which depends on p? To investigate this question, we need to look at the behavior of the system as a function of p for different values of n.

Complex Networks:

Complex Networks:

The appearance of the small-world behavior is not a phasetransition but a crossover phenomena. The average distance l is: l (n,p) ~ n* F ( n / n* ) where: F(u << 1) ~ u, and F(u >> 1) ~ln u, and n* is a function of p.

When the average number of rewired links, pnz/2, is much less than one, the network should be in the large-world regime. On the other hand, when pnz/2 >> 1, the network should be a small-world.

Scale-free networks:
It was proposed by Barabsi and Albert that real-world networks in general are scale-free networks. Scale-free networks have a distribution of connectivities that decays with a power-law tail. Scale-free networks emerge in the context of a growing network in which new vertices connect preferentially to the more highly connected vertices in the network. Scale free networks are also small-world networks because (i) they have clustering coefficients much larger than random networks, and (ii) their diameter increases logarithmically with the number of vertices n.

What are Power Laws ?

Distribution that fits :

P(k ) k

Characteristic property of Scale free networks Occur very often in Complex Systems literature. Many complicated real world networks obey power laws

Implications of Power Laws:

Majority of nodes have small connectivity. Few nodes have very large connectivity. Good resistance to random failure. Small resistance to planned attack. Could imply existence of some hierarchy (all real world power law networks support this). However, it is not clear whether Power Law Hierarchy

Origin of Power Law:

Power laws are an observed (empirical) phenomenon. The mechanisms that produce these can only be guessed at (for now!) Very typical in self organizing systems and chaotic systems.

Scale-free networks:
Scale-free networks: (a) the neuronal network of the worm C. elegans. (b) world-wide web. (c) the network of citations of scientific papers.

Scale-free networks:
broad-scale networks: or truncated scale-free networks, characterized by a connectivity distribution that has a powerlaw regime followed by a sharp cut-off, like an exponential or Gaussian decay of the tail. single-scale networks: characterized by a connectivity distribution with a fast decaying tail, such as exponential or Gaussian Aging of the vertices: The vertex is still part of the network and contributing to network statistics, but it no longer receives links. The aging of the vertices thus limits the preferential attachment preventing a scale-free distribution of connectivities. Cost of adding links to the vertices or the limited capacity of a vertex: physical costs of adding links and limited capacity of a vertex will limit the number of possible links attaching to a given vertex.

Power-laws of the Web [2].:

How many links on a page (outdegree)? How many links to a page (indegree)? Probability that a random page has k other pages -2.1 pointing to it is ~k (Power law) Probability that a random page points to k other pages is -2.7 ~k (Power law)

In-degree Distribution

Out-degree Distribution

Search in power-law networks: GNUTELLA [3].

Most of the P2P networks display a power-law distribution in their node degree. This distribution reflects the existence of a few nodes with very high degree and many with low degree. In P2P networks, the name of the target file may be known, but due to the networks ad hoc nature, the node holding the file may not be known until a real-time search is performed. A simple strategy to locate files, implemented by NAPSTER, is to use a central server that contains an index of all the files every node is sharing as they join the network. GNUTELLA and FREENET do not use a central server.

Search in power-law networks: GNUTELLA [3].

GNUTELLA is a peer-to-peer file-sharing system that treats all client nodes as functionally equivalent and lacks a central server that can store file location information. This is advantageous because it presents no central point of failure. The obvious disadvantage is that the location of files is unknown. When a user wants to download a file, he sends a query to all the nodes within a neighborhood of size ttl, the time to live assigned to the query. Every node passes on the query to all of its neighbors and decrements the ttl by one. In this way, all nodes within a given radius of the requesting node will be queried for the file, and those who have matching files will send back positive answers.

Search in power-law networks: GNUTELLA [3].

This broadcast method will find the target file quickly, given that it is located within a radius of ttl. However, broadcasting is extremely costly in terms of bandwidth. Such a search strategy does not scale well. As query traffic increases linearly with the size of GNUTELLA graph, nodes become overloaded.

Search in power-law networks: GNUTELLA [3].

Typically, a GNUTELLA client wishing to join the network must find the IP address of an initial node to connect to. Currently, ad hoc lists of good GNUTELLA clients exist. It is reasonable to suppose that this ad hoc method of growth would bias new nodes to connect preferentially to nodes that are already fairly well connected, since these nodes are more likely to be well known. Based on models of graph growth where the rich get richer, the power-law connectivity of ad hoc peer-to-peer networks may be a fairly general topological feature.

Search in power-law networks: GNUTELLA [3].

By passing the query to every single node in the network, the GNUTELLA algorithm fails to take advantage of the connectivity distribution [3]. To take advantage of the power-law distribution, we can modify each node to keep lists of files stored in first and second neighbor. Instead of passing the query to every node, now we can pass it only to the nodes with highest connectivity. High degree nodes are presumably high bandwidth node that can handle the query traffic.

Outline:

Internet Structure &Organization

Internet Hierarchical Structure ISPs, interconnection and organization [ref. 7]. POP Architecture and Load Balancing ISP Architecture [ref. 7]. in detail Topology Mapping Tool: Rocketfuel[ref. 8] Discussion ELEG 667-013 Spring 2003

Basic Internet Architecture

Basic Architecture: NAPs and national ISPs

The Internet has a hierarchical structure. At the highest level are large national Internet Service Providers that interconnect through Network Access Points (NAPs). There are about a dozen NAPs in the U.S., run by common carriers such as Sprint and Ameritech, and many more around the world. Regional ISPs interconnect with national ISPs which provide services to local ISPs who, in turn, sell access to individuals.

Basic Architecture: MAEs and local ISPs

As the number of ISPs has grown, a new type of network access point, called a metropolitan area exchange (MAE) has arisen. There are about 50 such MAE around the U.S. today. Sometimes large regional and local ISPs also have access directly to NAPs.

Internet Packet Exchange Charges

ISP at the same level usually do not charge each other for exchanging messages. This is called peering. Higher level ISPs, however, charge lower level ones (national ISPs charge regional ISPs which in turn charge local ISPs) for carrying Internet traffic. Local ISPs, of course, charge individuals and corporate users for access.

Connecting to an ISP
ISPs provide access to the Internet through a Point of Presence (POP). Individual users access the POP through a dial-up line using the PPP protocol. The call connects the user to the ISPs modem pool, after which a remote access server (RAS) checks the userid and password. Once logged in, the user can send TCP/IP/[PPP] packets over the telephone line which are then sent out over the Internet through the ISPs POP.

Connecting to an ISP (contd.)

Corporate users might access the POP using a T-1, T-3 or ATM OC-3 connections provided by a common carrier. T-1 and T-3 lines connect to the ISP POPs CSU/DSU device. Channel Service Unit/Data Service Unit. The CSU is a device that connects a terminal to a digital line. The DSU is a device that performs protective and diagnostic functions for a telecommunications line. . Typically, the two devices are packaged as a single unit. You can think of it as a very high-powered and expensive modem. Such a device is required for both ends of a T-1 or T-3 connection, and the units at both ends must be set to the same communications standard.

Inside an ISP Point of Presence

Individual Dial-up Customers

ISP Point-of Presence

Modem Pool

ISP POP

ISP POP
Corporate T1 Customer T1 CSU/DSU Layer-2 Switch Corporate T3 Customer T3 CSU/DSU ATM Switch

ISP POP

Corporate OC-3 Customer ATM Switch

Remote Access Server

NAP/MAE

Internet Organization
CN

NAP
POP

ISP

CN

CN

CN

ISP POP POP BSP BSP BSP POP NAP

POP

CN

NAP
POP
CN
POP

ISP
CN

CN

ISP = Internet Service Provider BSP = Backbone Service Provider NAP = Network Access Point POP = Point of Presence CN = Customer Network

Customer Network
Clients LAN
Ethernet 10 Mb/s

Servers

Router
WAN
T1 Link 1.54 Mb/s

NAP Architecture
ISP ISP ISP Backbone Operator

Routers Route Server

High-Speed LAN (FDDI, ATM, GigE)

Routers

Backbone Operator

ISP

Backbone NAP Operator

Internet structure: network of networks

roughly hierarchical at center: tier-1 ISPs (e.g., UUNet, BBN/Genuity, Sprint, AT&T), national/international coverage treat each other as equals
Tier-1 providers interconnect (peer) privately Tier-1 providers also interconnect at public network access points (NAPs)

Tier 1 ISP

NAP

Tier 1 ISP

Tier 1 ISP

Tier-1 ISP: e.g., Sprint

Sprint US backbone network

Tier-1 IP backbone

POP
The backbone is a set of POPs (usually one per city) Point-of-Presence (POP) : A collection of routers and switches housed in a single location

Internet structure: network of networks

Tier-2 ISPs: smaller (often regional) ISPs

Connect to one or more tier-1 ISPs, possibly other tier-2 ISPs

Tier-2 ISP pays tier-1 ISP for connectivity to rest of Internet tier-2 ISP is customer of tier-1 provider

Tier-2 ISP

Tier-2 ISP

Tier 1 ISP

NAP

Tier-2 ISPs also peer privately with each other, interconnect at NAP
Tier-2 ISP

Tier 1 ISP
Tier-2 ISP

Tier 1 ISP
Tier-2 ISP

Internet structure: network of networks

Tier-3 ISPs and local ISPs

last hop (access) network (closest to end systems)

local ISP Tier 3 ISP Tier-2 ISP

Local and tier3 ISPs are customers of higher tier ISPs connecting them to rest of Internet

local ISP

local ISP Tier-2 ISP

local ISP

Tier 1 ISP

NAP

Tier 1 ISP

Tier 1 ISP
Tier-2 ISP local ISP

Tier-2 ISP local ISP

Tier-2 ISP local local ISP ISP

Internet structure: network of networks

a packet passes through many networks!
local ISP Tier 3 ISP Tier-2 ISP local ISP

local ISP

local ISP Tier-2 ISP

Tier 1 ISP

NAP

Tier 1 ISP
Tier-2 ISP local local ISP ISP

Tier 1 ISP
Tier-2 ISP local ISP

Tier-2 ISP local ISP

Architecture of a POP
Backbone links Backbone Router

Backbone Router

Peering

Access Router

Access Router

Access Router

Access Router

ISPs

Corporate networks

Web Servers

Dial-up

ISP Architecture
Access Network Architecture

Dial-up ISDN DSL Dedicated Leased lines Frame Relay Service

Dial-up Access Network

Central Office

Modem

Circuit Switch ISP POP Modem Pool Web Cache Router

Internet Backbone

ISDN
ISDN service access links terminate at the ISP POP Digital signal. Due to signal strength limitations, ISDN subscribers must be within 18000 feet of the CO At the customers end, an ISDN adapter card is required.

DSL
Central Office

Modem

DSLAM Circuit Switch ISP POP Modem Pool Web Cache Router

Internet Backbone

DSL Access
DSL typically provisioned at 1.5Mbps from ISP to customer and at 128kbs in the reverse direction. DSL Access Multiplexer (DSLAM) at CO terminates DSL signals from hundreds of customers. The IP data is multiplexed into a single ATM connection by DSLAM and forwarded to the ISP POP

Dedicated Access
Leased lines from 56Kbs to 155Mbps. No multiplexing of other customers traffic. Can lead to higher operational cost. Lines terminate at routers in the POP.

Frame Relay Service

Network resembles a star topology, with one leg of the star connected to ISP and other legs connected to different customers.
Router

Router
Router

Frame Relay Network

ISP Router

ISP Architecture: The Backbone

Backbone router ISP Backbone

The backbone of a large ISP is typically a WAN spread out across a large geographic area. Backbone routers connect the individual links composing the backbone .

ISP Architecture: Backbone Nodes

Backbone Node
Backbone Node

ISP Backbone

For reasons of robustness and load management, multiple backbone routers can be located in the same geographic location and connected via a LAN.

We consider all of the backbone routers and the connecting LAN to be a backbone node.
These backbone nodes, whether they contain one or more routers, will serve as the points of connection from the outside world to the backbone.

ISP Architecture: Access Routers

Customers such as smaller ISPs and enterprises (Downstream)

Access Router

ISP Backbone
Dial-in POP (Downstream)

Customers, including smaller ISPs, enterprise, are connected to backbone nodes via access routers. Access routers gain their connectivity to the backbone, because they are on the same LAN as one or more backbone routers.

Remember, the backbone nodes contain backbone routers, as well as these access routers.
Any backbone entry point is known as a point of presence (POP). Modem entry points are known as dial-in POPs or dial-in hubs. Entry points for other types of networks are known as broadband POPs.

ISP Architecture: In Practice

Broadband POP

Access Router

ISP Backbone
Large dial-in POP (Downstream) Backbone Router

In practice, only the largest customers connect directly to access routers. Other customers are aggregated at broadband points of presence (broadband POPs). These are basically LANs. The customers connect to routers on these LANs, and then these LANs connect to the access nodes Additionally, some very large dial-in POPs do connect directly to backbone routers. These typically service very large corporate offices.

ISP Architecture: Gateways

Upstream ISP

ISP Backbone
Peer ISP Gateway Router

Gateway routers, which are also connected via LANs to backbone routers, connect ISPs to each other. The router is known as a gateway router, if it connects a peer or upstream ISP. Downstream ISPs generally connect via an access router, or directly to a backbone Router. So, a gateway router leads to a peer or upstream provider, whereas an access router leads to a downstream network.

Measuring ISP Topologies with Rocketfuel[8]: Rocketfule internet topology mapping engine The goal is to obtain realistic, router-level maps of ISP networks. Important influence on: - The dynamics of routing protocols - The scalability of multicast - The efficacy of proposals for denial-of-service tracing and response - Other aspects of protocol performance (Internet path selection) Real topologies are not publicly available - Confidential

Mapping techniques
Three categories of mapping techniques:

Selecting Measurements
Directed probing

Path reduction

Alias Resolution
IP identifier

Router identification and Annotation

Selecting Measurements
Directed probing

To employ BGP tables to identify relevant traceroutes and prune the remainder To identify redundant traceroutes
Only one traceroutes needs to be taken when

Path reduction

two traceroutes enter and leave the ISP network at the same point

Alias resolution
Mercator method

Sending traceroute-like probe(to a highnumbered UDP port but with a TTL of 255) directly to the potentially aliased IP address
Requirement: routers need to be configured to

send the UDP port unreachable response with the address of the outgoing interface as the source address: Two aliases should respond with the same source

Alias method
Proposed methods by Spring etc.

Mercators IP address-based method Comparing IP identifier field of the responses

IP identifier hints
IP identifier helps to identify a packet for reassembly after fragmentation IP identifier is commonly implemented using a counter that is incremented after sending a packet

Alias resolution by IP identifier

Process of alias resolution by IP identifier:

Ally, a tool for alias resolution, sends a probe packet to the two potential aliases Port unreachable responses, including the IP identifiers x and y Ally sends a third packet to the address that responded first

Router Identification & Annotation

Using DNS to determine routers owned by mapped ISP, their geographical location and role in the topology

Rocketfuel includes modules:

Mapping engine: Rocketfuel

BGP table from RouteViews Egress discovery: To find egress routers Tasklist generation: To generate a list of directed probes Path reductions: To apply ingress and next-hop AS reductions, and generate jobs for execution Public traceroute servers Alias resolution: Using IP identifier technique to resolve alias problem Database

References:
[1] Kenneth Calvert, Matthew Doar, Ellen Zegura, Modeling Internet Topology. [2]. Michalis Faloudsos, Petros Faloudsos, Christos Faloudsos, On Power-law Relationships of the Internet Topology [3]. Lada A. Adamic,1, Rajan M. Lukose,1, Amit R. Puniyani,2, and Bernardo A. Huberman1, Search in power-law networks. [4]. L. A. N. Amaral, A. Scala, M. Barthlmy, & H. E. Stanley, 1997, Classes of small-world networks. http://polymer.bu.edu/~amaral/Content_network.html [5]. Ellen Zegura, Kenneth Calvert, How to model an Internetwork [6]. Stefan Bornholdt, Holger Ebel, World Wide Web scaling exponent from Simons 1955 model [7]. S. Halabi and D. McPherson, Internet Routing Architectures, 2nd ed., Cisco Press, Indianapolis, 2000. [8]. Neil Spring Ratul Mahajan David Wetherall, Measuring ISP Topologies with Rocketfuel