Outline

1 2

Computer Science 331

Asymptotic Notation

Properties and Application Types of Asymptotic Notation Big-Oh Notation Big-Omega Notation Big-Theta Notation Little-oh Notation Little-omega Notation Useful Properties and Functions Recommended Reading

Mike Jacobson
Department of Computer Science University of Calgary

Lecture #7

3 4

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

1 / 19

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

2 / 19

Properties and Application

Types of Asymptotic Notation

Big-Oh Notation

Properties and Application

Asymptotic Notation . . . provides information about the relative rates of growth of a pair of functions (of a single integer or real variable) ignores or hides other details, including
behaviour on small inputs results are most meaningful when inputs are extremely large multiplicative constants and lower-order terms which can be implementation or platform-dependent anyway

Big-Oh Notation
Suppose f , g : R0 R0 . f O(g) : There exist constants c > 0 and N0 0 such that f (n) c g(n) for all n N0 . Intuition: growth rate of f is at most (same as or less than) that of g Eg. 4n + 3 O(n) denition is satised using c = 5 and N0 = 3

permits classication of algorithms into classes (eg. linear, quadratic, polynomial, exponential, etc...) is useful for giving the kinds of bounds on running times of algorithms that we will study in this course

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

3 / 19

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

4 / 19

Types of Asymptotic Notation

Big-Oh Notation

Types of Asymptotic Notation

Big-Oh Notation

Example: 4n2 + 2 O(n2 )

90 80 70 60 50 40 30 20 10 0

Proof that 4n2 + 2 O(n2 )

4*x**2 + 2 x**2 5*x**2

Theorem 1
4n2 + 2 O(n2 )

Proof.
Let f (n) = 4n2 + 2 and g(n) = n2 . Then: f (n) = 4n2 + 2 4n2 + n2 = 5n2 whenever n2 2 n2 2 holds if n 2 1.414 f (n) cg(n) for all n N0 when c = 5 and N0 = 2. 0 0.5 1 1.5 2 2.5 3 3.5 4 By denition, f O(g) as claimed.

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

5 / 19

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

6 / 19

Types of Asymptotic Notation

Big-Omega Notation

Types of Asymptotic Notation

Big-Omega Notation

Big-Omega Notation
Suppose f , g : R0 R0 . f (g) : There exist constants c > 0 and N0 0 such that f (n) c g(n) for all n N0 . Intuition: growth rate of f is at least (the same as or greater than) that of g 4n + 3 (n) denition is satised using c = N0 = 1

Example: n2 (4n2 + 2)
90 80 70 60 50 40 30 20 10 0

x**2 4*x**2 + 2 (4 * x**2 + 2)/5

0.5

1.5

2.5

3.5

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

7 / 19

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

8 / 19

Types of Asymptotic Notation

Big-Omega Notation

Types of Asymptotic Notation

Big-Theta Notation

Transpose Symmetry
Theorem 2
Suppose f , g : R0 R0 . Then f O(g) if and only if g (f ).

Big-Theta Notation
Suppose f , g : R0 R0 . f (g) :

Proof.
If f O(g) : by defn c R>0 and N0 R0 such that f (n) cg(n) for all n N0 . implies cg(n) f (n) for all n N0 implies g(n) (1/c)f (n) for all n N0 thus g (f ) by denition If g (f ), . . .
Mike Jacobson (University of Calgary) Computer Science 331 Lecture #7 9 / 19

There exist constants cL , cU > 0 and N0 0 such that cL g(n) f (n) cU g(n) for all n N0 . Intuition: f has the same growth rate as g 4n + 3 (n) denition is satised using cL = 1, cU = 5, N0 = 3

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

10 / 19

Types of Asymptotic Notation

Big-Theta Notation

Types of Asymptotic Notation

Big-Theta Notation

Example: 4n2 + 2 (n2 )

90 80 70 60 50 40 30 20 10 0

An Equivalent Denition

4*x**2 + 2 5*x**2 4*x**2

Theorem 3
Suppose f , g : R0 R0 . Then f (g) if and only if f O(g) and f (g) Exercise: Prove that the two denitions of f (g) are equivalent. How To Solve This:

0.5

1.5

2.5

3.5

Work from the denitions, as in previous example!

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

11 / 19

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

12 / 19

Types of Asymptotic Notation

Little-oh Notation

Types of Asymptotic Notation

Little-oh Notation

Little-oh Notation
Suppose f , g : R0 R0 . f o(g): For every constant c > 0 there exists a constant N0 0 such that f (n) c g(n) for all n N0 .

Example: x o(x 2 )
10 8 6 4 2

x x**2 (x**2)/2 (x**2)/4 (x**2)/6

Intuition: f grows strictly slower than g

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

13 / 19

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

14 / 19

Types of Asymptotic Notation

Little-omega Notation

Little-omega Notation
Suppose f , g : R0 R0 . f (g) : For every constant c > 0 there exists a constant N0 0 such that f (n) c g(n) for all n N0 .

Example: x ( x)
50 40 30 20 10

Types of Asymptotic Notation

Little-omega Notation

x sqrt(x) 2*sqrt(x) 4*sqrt(x) 6*sqrt(x)

Intuition: f grows strictly faster than g

10

20

30

40

50

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

15 / 19

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

16 / 19

Types of Asymptotic Notation

Little-omega Notation

Useful Properties and Functions

Useful Properties
Suppose f , g : R0 R0 . Useful properties: f o(g) f O(g) f (g) f (g) Transpose Symmetry: f o(g) g (f ) f (x) f o(g) lim =0 x+ g(x) Limit Test: f (x) f (g) lim = + x+ g(x) Limit Test:

Some Standard Functions

Polynomial (degree d): p(n) = ad nd + ad1 nd1 + + a1 n + a0 p(n) (nd ) Exponentials: an , a R0 (increasing if a > 1) if a > 1, then an (p(n)) for every polynomial p(n) Logarithms: loga n, a R0 (loga n)k o(p(n)) whenever a > 1, k R0 , and p(n) is a polynomial with degree at least one

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

17 / 19

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

18 / 19

Recommended Reading

Recommended Reading
Please read Section 2.8 of the textbook. Chapter 3 of Cormen, Leiserson, Rivest and Steins Introduction to Algorithms is also highly recommended. Especially Useful in Introduction to Algorithms: Additional Properties and Exercises (pp. 4950) Standard Notation and Common Functions (Section 3.2):
Floors and Ceilings Modular Arithmetic Standard Functions: Polynomials, Exponentials, Logarithms, and Their Properties

Mike Jacobson (University of Calgary)

Computer Science 331

Lecture #7

19 / 19