Web Caching

based on:
Web Caching, Geoff Huston

Web Caching and Zipf-like Distributions:

Evidence and Implications, L. Breslau, P.
Cao, L. Fan, G. Phillips, S. Shenker

On the scale and performance of

cooperative Web proxy caching, A.
Wolman, G. Voelker, N. Sharma, N.
Cardwell, A. Karlin, H. Levy

Web Caching Page 1 of 24

 Hypertext Transfer Protocol (HTTP)
based on an end-to-end model (the network is
a passive element)
Pros:
The server can modify the content;
The server can track all content requests;
The server can differentiate between
different clients;
The server can authenticate the client.
Cons:
Popular servers are under continuous high
stress
high number of simultaneously opened
connections
high load on the surrounding network
The network may not be efficiently utilized

Web Caching Page 2 of 24

 Why Caching in the Network?
reduce latency by avoiding slow links between
client and origin server
low bandwidth links
congested links
reduce traffic on links
spread load of overloaded origin servers to
caches

Web Caching Page 3 of 24

 Caching points
content provider (server caching)

ISP
more than 70% of ISP traffic is Web based;
repeated requests at about 50% of the traffic;
ISPs interested in reducing the transmission
costs;
provide high quality of service to the end
users.

end client

Web Caching Page 4 of 24

 Caching Challenges
Cache consistency
identify if an object is stale or fresh
Dynamic content
caches should not store outputs of CGI
scripts (Active Cache?)
Hit counts and personalization
Privacy concerned user
how do you get a user to point to a cache
Access control
legal and security restrictions
Large multimedia files

Web Caching Page 5 of 24

 Web cache access pattern - (common
thoughts)

the distribution of page requests follows a

Zipf-like distribution: the relative probability
of a request for the i-th most popular page is

proportional to 1 i with 1

for an infinite sized cache, the hit-ratio grows

in a log-like fashion as a function of the client
population and of the number of requests
the hit-ratio of a web cache grows in a log-like
fashion as a function of cache size
the probability that a document will be
referenced k requests after it was last
referenced is roughly proportional to 1 k
(temporal locality)

Web Caching Page 6 of 24

 Web cache access pattern - (P. Cao
Observations)

distribution of web requests follow a a Zipf-

like distribution 0.64, 0.83 (fig. 1)

the 10/90 rule does not apply to web accesses

(70% of accesses to about 25%-40% of the
documents) (fig. 2)

low statistical corellation between the

frequency that a document is accessed and its
size (fig. 3)

low statistic correlation between a documents

access frequency and its average modifications
per request.

request distribution to servers does not follow

a Zipf law.; there is no single server
contributing to the most popular pages (fig. 5)

Web Caching Page 7 of 24

 Implications of Zipf-like behavior
Model:
A cache that receives a stream of requests
N = total number of pages in the universe
P N ( i ) = probability that given the arrival of
a page request, it is made for page
i, ( i = 1, N )
Pages ranked in order of their popularity,
page i is the i-th most popular
Zipf-like distribution for the arrivals:

PN ( i ) = i ,
1
N

= 1i
i = 1
no pages are invalidated in the cache.

Web Caching Page 8 of 24

 Implications of Zipf-like behavior
infinite cache, finite request stream:
= log ( R ), ( = 1 )
H (R) 11
= ( ( 1 ) ) ( R ) , (0 < < 1)
finite cache, infinite request stream:
= log C, ( = 1 )
H (C ) 1
= C ,0<<1
page request interarrival time:
1 1 k 1 k
d(k) --------------- 1 ----------------- 1 ------------
k log N N log N log N

H(R) = hit ratio

d(k) = probability distribution that the next
request for page iis followed by k-1 requests for
pages other than i

Web Caching Page 9 of 24

 Cache Replacement Algorithms
(fig. 9)
GD-Size:
performs best for small sizes of cache.
Perfect-LFU
performs comparably with GD-Size in hit-
ratio and much better in byte hit-ratio for
large cache sizes.
drawback: increased complexity
In-Cache-LFU
performs the worst

Web Caching Page 10 of 24

 Web document sharing and proxy
caching
What is the best performance one could
achieve with perfect cooperative caching?

For what range of client populations can

cooperative caching work effectively?

Does the way in which clients are assigned to

caches matter?

What cache hit rates are necessary to achieve

worthwhile decreases in document access
latency?

Web Caching Page 11 of 24

 For what range of client populations can
cooperative caching work effectively?
90
80

Request Hit Rate (%)

70
60
50
40
30 Ideal (UW)
20 Cacheable (UW)
Ideal (MS)
10 Cacheable (MS)
0
0 5000 10000 15000 20000 25000 30000
Population

for smaller populations, hit rate increases

rapidly with population (cooperative caching
can be used effectively)
for larger population cooperative caching is
unlikely to bring benefits
Conclusion:
use cooperative caching to adapt to proxy

assignments made for political or geographical

reasons.

Web Caching Page 12 of 24

 Object Latency vs. Population

Mean No Cache
Mean Cacheable
Mean Ideal
Median No Cache
Median Cacheable
2000 Median Ideal
Object Latency (ms)

1500

1000

500

0
0 5000 10000 15000 20000
Population

Latency stays unchanged when population

increases
Caching will have little effect on mean and
median latency beyond very small client
population (?!)

Web Caching Page 13 of 24

 Byte Hit Rate vs. Population

60

50
Byte Hit Rate (%)

40

30

20 Ideal
Cacheable
10

0
0 5000 10000 15000 20000
Population

same knee at about 2500 clients

shared documents are smaller

Web Caching Page 14 of 24

 Bandwith vs. population

Mean Bandwidth (Mbits/s) 5 No Cache

Cacheable
4 Ideal

0
0 5000 10000 15000 20000
Population

while caching reduces bandwidth consumption

there is no benefit to increased client
population.

Web Caching Page 15 of 24

 Proxies and organizations
100 Ideal Cooperative
90 Ideal Local
Cacheable Cooperative
80
Cacheable Local
70

Hit Rate (%)

60
50
40
30
20
10
0
978
735
480
425
392
337
251
246
239
228
222
219
213
200
192
15 Largest Organizations

70 UW
Random
60

50
Hit Rate (%)

40

30

20

10

0
978
735
480
425
392
337
251
246
239
228
222
219
213
200
192

15 Largest Organizations

there is some locality in organizational membership, but the impact is

not significant

Web Caching Page 16 of 24

 Impact of larger population size

100 Cooperative
Local
Hit Rate (%) 80

60

40

20

0
UW
Ideal

UW
Cacheable

MS
Ideal

MS
Cacheable
unpopular documents are universally
unpopular => unlikely that a miss in one large
group of population to get a hit in another
group.

Web Caching Page 17 of 24

 Performance of large scale proxy
caching - model
N = clients in population
n = total number of documents ((aprox. 3.2
billions)
pi = fraction of all requests that are for the i-th
1
most popular document ( p i ---- , = 0.8 )
i
exponential distribution of the requests done
by client with parameter N , where = 590
reqs/day = average client request rate
exponential distribution of time for document
change, with parameter (unpopular
document u = 1 186 days, popular
documents p = 1 14 days)
pc = 0.6 = probability that the documents are
cacheable
E(S) = 7.7KB = avg. document size
E(L) = 1.9 sec = last-byte latency to server

Web Caching Page 18 of 24

 Hit rate, latency and bandwidth
100

Cacheable Hit Rate (%)

80

60

40

Slow (14 days, 186 days)

20 Mid Slow (1 day, 186 days)
Mid Fast (1 hour, 85 days)
Fast (5 mins, 85 days)
0
10^2 10^3 10^4 10^5 10^6 10^7 10^8
Population (log)

Hit rate = f(Population) has three areas:

1. the request rate too low to dominate the rate

of change for unpopular documents

2. marks a significant increase in the hit rate of

the unpopular documents

3. the request rate is high enough to cope with

the document modifications

Latency lat req = ( 1 H N ) E(L) + H N lat hit
Bandwidth B N = H N E(S)

Web Caching Page 19 of 24

 Document rate of change
100

Cacheable Hit Rate (%)

A
80
B
60

40
C
mu_p
20 mu_u

0
1 2 5 10 30 1 2 6 12 1 2 7 14 30 180
min hour day
Change Interval (log)
100
B A
Cacheable Hit Rate (%)

80

60

40
C
mu_p
20 mu_u

0
1 2 5 10 30 1 2 6 12 1 2 7 14 30 180
min hour day
Change Interval (log)

Proxy cache hit is very sensitive to the change

rates of both popular and unpopular documents
with a decrease on the time scales at which hit
rate is sensitive once the size of the population
increases.

Web Caching Page 20 of 24

 Comparing cooperative caching
scheme
Three types of cache schemes: hierarchical, hash
based and summary caches;
Conclusions:
the achievable benefits in terms of latency are

achieved at the level of city-level cooperative

caching;
a flat cooperative scheme is no longer effective

if the served area increases after a limit

in a hierarchical caching scheme the higher

level might be a bottleneck in serving the

requests.

Web Caching Page 21 of 24

 Hierarchical Caching (Squid):
a hierarchy of k levels
a client request is forwarded up in the
hierarchy until a cache hit occurs.
a copy of the requested object is then stored in
all caches of the request path

Web Caching Page 22 of 24

 Hash-based caching system:
assumes a total of m caches cumulatively
serving a population of size N
upon a request for an URL the client
determines the cache in charge and forwards
the request to it.
if the cache has the URL the page is returned
to the client, otherwise the request is
forwarded to the server

Web Caching Page 23 of 24

 Directory based system
(Summary cache)
a total of m caches serving cumulatively a
population of size N
the population is partitioned into
subpopulations of size N/m and each
population is served by a single cache
each cache maintains a directory that
summarizes the documents stored at each of
the other cache in the system
when a client request can not be served by a
local cache, it is randomly forwarded to a
cache which is registered to have the
document. If no document has the document
the request is forwarded to the original server

Web Caching Page 24 of 24