Sorting

Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement & Questions
Sorting Algorithms
Dr. Iftikhar Ahmad
ia@giki.edu.pk
Faculty of Computer Science and Engineering, GIKI, Topi
Dr. Iftikhar Ahmad
Sorting Algorithms
1 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Acknowledgement
Questions
Dr. Iftikhar Ahmad
Sorting Algorithms
2 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Introduction
Theoretical Results
Can we do better?
Introduction
Sorting :: Definition
Sorting is the process of arranging the data in some particular
(ascending/descending) order.
Assumption
For the remainder of this lecture, we assume that we want to order
the list in ascending order.
Given
A sequence of n numbers < a1 , a2 , . . . , an >
Output
A re-ordering < a10 , a20 , . . . , an0 > of the input sequence such
that a10 a20 an0
Dr. Iftikhar Ahmad
Sorting Algorithms
3 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Introduction
Theoretical Results
Can we do better?
Comparison based Sorting Algorithms
Definition
A comparison based sorting algorithm takes as input an array
[a1 , a2 , . . . , an ] of n items, and can only gain information about the
items by comparing pairs of them. Each comparison ( is ai > aj )
returns YES or NO and counts a 1 time-step. The algorithm may
also for free reorder items based on the results of comparisons
made. In the end, the algorithm must output a permutation of the
input in which all items are in sorted order.
Dr. Iftikhar Ahmad
Sorting Algorithms
4 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Introduction
Theoretical Results
Can we do better?
Comparison based Sorting Algorithms :: Examples

Quick Sort
Merge Sort
Insertion Sort
Dr. Iftikhar Ahmad
Sorting Algorithms
5 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Introduction
Theoretical Results
Can we do better?
Theorem
Any deterministic-based sorting algorithm must perform (n log n)
comparisons to sort n elements in the worst case. Specifically, for
any deterministic comparison-based sorting algorithm A, n 2
there exists an input I of size n such that A makes at least
(n log n) comparisons to sort I .
Dr. Iftikhar Ahmad
Sorting Algorithms
6 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Introduction
Theoretical Results
Can we do better?
Can we do better?
NO
Lets relax our assumption
Assume that all keys are in some specific range , say l and u
Can we do better than n log n?
YES
Dr. Iftikhar Ahmad
Sorting Algorithms
7 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis
Counting Sort :: Basic Idea
Selection Sort :: Basic Idea

Assume that the keys are in the range 1, . . . , m, where
m = O(n), n is the number of elements to be sorted.
Find the frequency of each key
Use the information to determine the position of each element
in the sorted array.
Dr. Iftikhar Ahmad
Sorting Algorithms
8 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis
Counting Sort :: Example
Index
Data
1
3
2
8
3
1
4
3
5
9
6
4
7
5
8
3
9
9
10
7
11
6
12
2
Exampled
Sort the array.
Dr. Iftikhar Ahmad
Sorting Algorithms
9 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis
Index
Data
1
3
2
8
3
1
4
3
5
9
6
4
7
5
8
3
9
9
10
7
11
6
9
2
10
0
12
2
Step 1
Find frequency of each item
Index
Frequency
1
1
2
1
3
3
4
1
5
1
Dr. Iftikhar Ahmad
6
1
7
1
8
1
Sorting Algorithms
10 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis

Index
Data
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
3
5
9
6
4
7
5
8
3
9
9
4
1
5
1
6
1
7
1
8
1
4
6
5
7
6
8
7
9
8
10
10
7
11
6
9
2
10
0
12
2
Step 2
Re-use for Distribution
Index
Frequency
1
1
2
2
3
5
Dr. Iftikhar Ahmad
Sorting Algorithms
9
12
10
12
11 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis

Index
Data
1
3
Index
Frequency
2
8
1
1
3
1
2
2
4
3
3
5
5
9
4
6
6
4
7
5
5
7
8
3
9
9
6
8
7
9
8
10
10
7
9
12
11
6
12
2
10
12
Next
Find an appropriate location for 2
Index
Temp
2
2
Dr. Iftikhar Ahmad
10
Sorting Algorithms
11
12
12 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis

Index
Data
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
5
5
9
4
6
6
4
5
7
7
5
6
8
8
3
9
9
7
9
8
10
8
6
10
7
9
12
11
6
12
2
10
12
Next
Index
Temp
2
2
Dr. Iftikhar Ahmad
10
Sorting Algorithms
11
12
13 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis

Index
Data
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
5
5
9
4
6
6
4
5
7
7
5
6
7
8
3
9
9
7
9
8
10
8
6
9
7
10
7
9
12
11
6
12
2
10
12
Next
Index
Temp
2
2
Dr. Iftikhar Ahmad
10
Sorting Algorithms
11
12
14 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis

Index
Data
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
5
5
9
4
6
6
4
5
7
7
5
6
7
8
3
9
9
7
8
8
10
8
6
9
7
10
7
9
12
11
6
12
2
10
12
Next
Index
Temp
2
2
Dr. Iftikhar Ahmad
10
Sorting Algorithms
11
12
9
15 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis

Index
Data
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
5
5
9
4
6
6
4
7
5
5
7
8
3
9
9
6
7
7
8
8
10
8
6
9
7
10
7
9
11
11
6
12
2
10
12
Next
Index
Temp
2
2
5
3
Dr. Iftikhar Ahmad
10
Sorting Algorithms
11
12
9
16 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis

Index
Data
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
4
5
9
4
6
6
4
7
5
5
7
8
3
9
9
6
7
7
8
8
10
7
5
8
6
9
7
10
7
9
11
11
6
12
2
10
12
Next
Index
Temp
2
2
5
3
Dr. Iftikhar Ahmad
10
Sorting Algorithms
11
12
9
17 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis

Index
Data
1
3
Index
Frequency
2
8
1
1
3
1
2
1
4
3
3
4
5
9
4
6
6
4
7
5
5
6
8
3
9
9
6
7
7
8
8
10
7
5
8
6
9
7
10
7
9
11
11
6
12
2
10
12
Next
Index
Temp
2
2
5
3
6
4
Dr. Iftikhar Ahmad
10
Sorting Algorithms
11
12
9
18 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis

Index
Data
1
3
2
8
Index
Frequency
1
1
3
1
2
1
4
3
3
4
5
9
4
5
6
4
5
6
7
5
6
7
8
3
9
9
7
8
8
10
8
6
9
7
10
7
9
11
11
6
12
2
10
12
Next
Index
Temp
2
2
5
3
6
4
7
5
10
11
9
12
9
and so on ...
Dr. Iftikhar Ahmad
Sorting Algorithms
19 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis
Algorithm
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
function CountingSort(A[1 . . . n])

Initialize array Count[1 . . . m]
for i = 1 to n do
Count[A[i]] = Count[A[i]] + 1
end for
for i = 2 to m do
Count[i] = Count[i] + Count[i 1]
end for
for i = n downto 1 do
j = A[i]
Temp[Count[j]] = A[i]
Count[j] = Count[j] 1
end for
Return Temp
end function
Dr. Iftikhar Ahmad
Sorting Algorithms
20 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis
Analysis
The loops in line 3 5 and 9 13 all require O(n)

Line 2 and lines 6 7 requires time O(m)
Thus overall run time is O(n + m)
This is linear in the number of elements if m = O(n)
CountingSort is STABLE
After sorting, 2 items with the same key have their initial relative
order preserved.
Dr. Iftikhar Ahmad
Sorting Algorithms
21 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Algorithm
Analysis
Stable Sorting Algorithms
Definition
A sorting algorithm is stable if it always leaves elements with equal
keys in their original order.
Examples
Counting Sort, Merge Sort, Insertion Sort
Quick Sort is NOT stable.
Dr. Iftikhar Ahmad
Sorting Algorithms
22 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Analysis
Radix Sort
Basic Idea
Sort the keys digit by digit, starting with the least significant digit.
Dr. Iftikhar Ahmad
Sorting Algorithms
23 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Analysis
Radix Sort :: Example
Figure : Radix Sort :: Example

Dr. Iftikhar Ahmad
Sorting Algorithms
24 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Analysis
Radix Sort :: Another Example
170, 45, 75, 90, 802, 2, 24, 66

Sorting by least significant digit (1s place) gives:
170, 90, 802, 2, 24, 45, 75, 66
Sorting by next digit (10s place) gives:
802, 2, 24, 45, 66, 170, 75, 90
Sorting by most significant digit (100s place) gives:
2, 24, 45, 66, 75, 90, 170, 802
Dr. Iftikhar Ahmad
Sorting Algorithms
25 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Example
Analysis
Analysis
Worst-Case Complexity
O(nk)
n is the number of elements
k is the (average) length of elements
Question
Is RadixSort stable?
Dr. Iftikhar Ahmad
Sorting Algorithms
26 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Algorithm
Example
Quick Sort :: Basic Idea

Quick Sort :: Basic Idea
Given : An unsorted list data[left, right]
3 Basis Steps as follows;
Divide :
Partition the data into two sub-arrays, data[left, pivot 1] and
data[pivot + 1, right] such that elements of
data[left, pivot 1] are to data[pivot] and elements of
data[pivot + 1, right] are to data[pivot]
Conquer:
Sort the two sub-arrays recursively
Combine:
The array data[left, right] is in sorted form.
Dr. Iftikhar Ahmad
Sorting Algorithms
27 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Algorithm
Example
Quick Sort :: C++ Code

v o i d quickSort ( i n t data [] , i n t left , i n t right )
{
i f ( left < right )
{
i n t p = partition ( data , left , right ) ;
quickSort ( data , left , p -1) ;
quickSort ( data , p +1 , right ) ;
}
}
i n t partition ( i n t data [] , i n t left , i n t right )
{
i n t pivot = data [ right ];
i n t i = left - 1;
f o r ( i n t j = left ; j < right ; j ++)
{
i f ( data [ j ] <= pivot )
swap ( data [ i ] , data [ j ]) ; / / A u s e r d e f i n e d
function
}
swap ( data [ i +1] , data [ right ]) ;
r e t u r n ( i +1) ;
}
Time Complexity
Worst Case : O(N 2 )
Ave Case : O(N log N)
Dr. Iftikhar Ahmad
Sorting Algorithms
28 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Basic Idea
Algorithm
Example
Example Run
Given:
An unsorted list (array) with the following data;
28713564
Figure : Quick Sort :: Example Run :: First Partition

Dr. Iftikhar Ahmad
Sorting Algorithms
29 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Sorting :: Time Complexity
Sorting Type
Selection Sort
Bubble
Quick Sort (Worst Case)
Quick Sort (Ave Case)
Counting Sort
Radix Sort
Dr. Iftikhar Ahmad
Time Complexity
O(n2 )
O(n2 )
O(n2 )
O(n log n)
O(n + m)
O(nk)
Sorting Algorithms
30 / 32
Overview
1
5
6
Sorting
Introduction
Theoretical Results
Can we do better?
Counting Sort
Basic Idea
Example
Algorithm
Analysis
Radix Sort
Basic Idea
Example
Analysis
Quick Sort
Basic Idea
Algorithm
Example
Summary
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement
Questions
Acknowledgement
Source Acknowledgement
The material (including figures and algorithms) presented in this
lecture is adopted from the following books;
Cormen, Thomas, Charles Leiserson, Ronald Rivest, and
Clifford Stein. Introduction to Algorithms.
Leviten, Anany. Introduction to the Design and Analysis of
Algorithms.
www.inf.ed.ac.uk/teaching/courses/ads/Lects/lecture8.pdf
Radix Sort Example is from Wikipedia
Dr. Iftikhar Ahmad
Sorting Algorithms
31 / 32
Sorting
Counting Sort
Radix Sort
Quick Sort
Summary
Acknowledgement
Questions
Questions
Questions?
Dr. Iftikhar Ahmad
Sorting Algorithms
32 / 32

Sorting

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sorting

Uploaded by

Copyright:

Available Formats

Sorting

Dr. Iftikhar Ahmad

Comparison based Sorting Algorithms

Dr. Iftikhar Ahmad

Comparison based Sorting Algorithms

Comparison based Sorting Algorithms :: Examples

Dr. Iftikhar Ahmad

Comparison based Sorting Algorithms

Dr. Iftikhar Ahmad

Dr. Iftikhar Ahmad

Counting Sort :: Basic Idea

Selection Sort :: Basic Idea

Dr. Iftikhar Ahmad

Counting Sort :: Example

Dr. Iftikhar Ahmad

Counting Sort :: Example

Dr. Iftikhar Ahmad

Counting Sort :: Example

Dr. Iftikhar Ahmad

Counting Sort :: Example

Dr. Iftikhar Ahmad

Counting Sort :: Example

Dr. Iftikhar Ahmad

Counting Sort :: Example

Dr. Iftikhar Ahmad

Counting Sort :: Example

Dr. Iftikhar Ahmad

Counting Sort :: Example

Dr. Iftikhar Ahmad

Counting Sort :: Example

Dr. Iftikhar Ahmad

Counting Sort :: Example

Dr. Iftikhar Ahmad

Counting Sort :: Example

function CountingSort(A[1 . . . n])

The loops in line 3 5 and 9 13 all require O(n)

Dr. Iftikhar Ahmad

Stable Sorting Algorithms

Dr. Iftikhar Ahmad

Dr. Iftikhar Ahmad

Radix Sort :: Example

Figure : Radix Sort :: Example

Radix Sort :: Another Example

170, 45, 75, 90, 802, 2, 24, 66

Dr. Iftikhar Ahmad

Dr. Iftikhar Ahmad

Quick Sort :: Basic Idea

Quick Sort :: C++ Code

Dr. Iftikhar Ahmad

Figure : Quick Sort :: Example Run :: First Partition

Sorting :: Time Complexity

Dr. Iftikhar Ahmad

Dr. Iftikhar Ahmad

Dr. Iftikhar Ahmad

You might also like