1 complexity lecture 17-19 ref. handout p 66 - 77
TRANSCRIPT
![Page 1: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/1.jpg)
1
Complexity
Lecture 17-19
Ref. Handout p 66 - 77
![Page 2: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/2.jpg)
2
Measuring Program Performance
How can one algorithm be ‘better’ than another?
Time taken to execute ? Size of code ? Space used when running ? Scaling properties ?
![Page 3: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/3.jpg)
3
Scaling
Amount of data
Time
![Page 4: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/4.jpg)
4
Time and Counting
initialisefor (i = 1; i<=n; i++) {
do something}finish
Time needed:
t1set up loop t2each iteration t4
t5
The total ‘cost’ is t1+t2+n∗t4+t5 Instead of actual time, ti can be treated
as how many instructions needed
![Page 5: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/5.jpg)
5
The computation cost can be estimatedas a function f(x) In the above example,
cost is f(x) = t1+t2+x∗t4+t5
= c + m∗x ( c = t1+t2+t5, m = t4 )
f(x) =c+mx
x (size of data)
c
m = slope
a linear function
![Page 6: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/6.jpg)
6
The computation cost can be estimatedas a function f(x) – more examples
cost functions can be any type depending on the algorithm, e.g.f(x) = c ( c is a constant ), h(x) = log(x), or g(x) = x2
f(x) =c
x (size of data)
c
cost stays same
h(x) = log(x)
cost growsvery slowg(x) =x2
quadratic growth
![Page 7: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/7.jpg)
7
The Time Complexity
A cost function for an algorithm indicates how computation time grows when the amount of data increases
A special term for it – TIME COMPLEXITYcomplexity = time complexity/space complexitySpace complexity studies the memory usage (but mostly we pay our attention to time complexity)
“Algorithm A’s time complexity is linear” = A’s computation time is proportional to the size of A’s input data
![Page 8: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/8.jpg)
8
The Big-O Notation
How to compare (evaluate) two cost functions
Is n+100*n2 better than 1000*n2
It has been agreed to use a standard measurement – an order of magnitude notation – the Big-O notation
![Page 9: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/9.jpg)
9
The Big-O Notation - definition
A function f(x) is said to be the big-O of a function g(x) if there is an integer N and a constant C such that
f(x) ≤ C∗g(x) for all x ≥ N
We write
f(x) = O(g(x))
![Page 10: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/10.jpg)
10
The Big-O Notation - Examples
f(x) = 3∗xg(x) = xC = 3, N =0f(x) ≤ 3∗g(x)
f(x) = 4+7∗xg(x) = xC = 8, N =4f(x) ≤ 8∗g(x)
g(x) = x
f(x) = 3x = 3g(x)f(x) = 4+7x
g(x) = x
8*g(x)
3 x∗ = O(x) 4+7 x∗ = O(x)N
![Page 11: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/11.jpg)
11
The Big-O Notation - Examples
f(x) = x+100∗x2
g(x) = x2
C = 101, N = 1f(x) ≤ 101∗g(x)
f(x) = 20+x+5∗x2+8∗x3
g(x) = x3
C = 15, N =20f(x) ≤ 9∗g(x)
when x > 1x+100∗x2 < x2+100∗x2
x2+100∗x2 = 101*g(x)
x+100∗x2= O(x2)
when x > 2020+x+5∗x2+8∗x3 <x3+x3+5∗x3+8x3
= 15∗x3 = 15*g(x)20+x+5∗x2+8∗x3= O(x3)
![Page 12: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/12.jpg)
12
Power of n
A smaller power is always better eventually.for larger enough n
The biggest power counts most.for larger enough n
1000000000 * n < n2
10 *n4 + 23*n3 + 99*n2 + 200*n + 77 < 11*n4
![Page 13: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/13.jpg)
13
Using Big-O Notation for classifyingalgorithms
23*n3 + 11
1000000000000000*n3 +100000000*n2 + ...
or even 100*n2 + n
All cost functions above is O(n3)
Performance is no worse than C*n3 eventually
![Page 14: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/14.jpg)
14
Working Out Big-O
Ignore constants in multiplication - 10*n is O(n)
Leave products – n*log(n) is O( n*log(n) )
Use worst case in sums - n2 + 100*n + 2 is O(n2)
![Page 15: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/15.jpg)
15
Memo for In-class test 17 [ /5]
questions my answers
correct answers
comments
1
2
3
4
5
![Page 16: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/16.jpg)
16
Some Typical Algorithms - Search
Search an array of size n for a particular item
Sequential search
Time taken is O( )
Binary search
Time taken is O( )
![Page 17: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/17.jpg)
17
Some Typical Algorithms - Sorting
Insertion sort Use 2 lists – unsorted and sorted
insertionSort(input_list) {
unsorted = input_list (assume size = n)
sorted = an empty list
loop from i =0 to n-1 do {
insert (unsorted[i], sorted) }
return sorted
}
Time taken is O( )
![Page 18: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/18.jpg)
18
Some Typical Algorithms - Sorting
Bubble sort
Swap if xi < xi-1 for all pairs
Do this n times (n = length of the list)
x0 3
x1 5
x2 1
x3 2
x4 6
Time taken is O( )
![Page 19: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/19.jpg)
19
Some Typical Algorithms - Sorting
Quick sort
quickSort(list) {
If List has zero or one number return list
Randomly pick a pivot xi to create two partitions larger than xi and smaller than xi
return quickSort(SmallerList)
+ xi + quickSort(LargerList)
}
// ‘+’ means appending two lists into one
Time taken is O( )
![Page 20: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/20.jpg)
20
Worst Cases, Best Cases, and Average Cases
Complexity:performance on the average cases
Worst/Best cases:The worst (best) performance which an algorithm can get
E.g. 1 - bubble sort and insertion sort can get O(n) in the best case (when the list is already ordered)
E.g. 2 – quick sort can get O(n2) in the worst case (when partition doesn’t work well)
![Page 21: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/21.jpg)
21
Sorting a sorted list
123456789
![Page 22: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/22.jpg)
22
A Summary on Typical Algorithms
Sequential search O(n) Binary search (sorted list) O(log(n))
Bubble sort O(n2) best O(n) worst O(n2) Insertion sort O(n2) best O(n) worst O(n2)
Quick sort O(log(n)*n) best O(log(n)*n)
worst O(n2)
![Page 23: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/23.jpg)
23
Can you order these functions ?smaller comes first
log(n)
log(n)*n n4/3
√n
n 2n
(log(n))2
n2
![Page 24: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/24.jpg)
24
Memo for In-class test 18 [ /5]
questions my answers
correct answers
comments
1
2
3
4
5
![Page 25: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/25.jpg)
25
Speeding up the Machine
can solve problem of size N in one day machine speed
up
Complexity
X 1000 X 1000000
O( log(n) ) N1000 N1000000
O( n ) 1000 * N 1000000 * N
O( n2 ) 32 * N 1000 * N
O( n3 ) 10 * N 100 * N
O( 2n ) N + 10 N + 20
size
![Page 26: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/26.jpg)
26
Code Breaking
A number is the product of two primes
How to work out prime factors?
i.e. 35 = 7*5
5959 = 59*101
48560209712519 = 6850049 * 7089031
Complexity is O(10n) where n is the number of digits ( or O(s) where s is the number)
Typically n is about 150
![Page 27: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/27.jpg)
27
Cost of Speeding Up
For each additional digit multiply time by 10( as the complexity is O(10n) )
For 10 extra digitsincrease time from 1 second to 317 years
For 18 extra digitincrease time from 1 second totwice the age of the universe
![Page 28: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/28.jpg)
28
Where Different Formulas Appear
O(log(n)) find an item in an ordered collection
O( n ) look at every item
O( n2 ) for every item look at every other item
O( 2n ) look at every subset of the items
![Page 29: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/29.jpg)
29
Do Big-O Times Matter?
Is n3 worse than 100000000000000000*n ?
Algorithm A is O(n2) for 99.999% of casesand
O(2n) for 0.001% cases
Algorithm B is always O(n10)
Which one is better?
![Page 30: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/30.jpg)
30
Degree of Difficulty
log(n)
n*log(n)
nn, n (2^n)
n, n2, n3, ......
2n, 3n, ......
polynomial time
‘tractable’
exponential time
‘intractable’
![Page 31: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/31.jpg)
31
A Simple O(2n) Problem
Given n objects of different weights,
split them into two equally heavy piles
(or say if this isn’t possible)
![Page 32: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/32.jpg)
32
Hard Problem Make Easy
Magic Machine(for problem A)
Possible Datafor problem A
CorrectAnswer? Answer
checker
![Page 33: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/33.jpg)
33
NP Problems
A problem is in the class NP
(Non-deterministic Polynomial Time)
if the checking requires polynomial time
i.e. O(nc) where C is a fixed number)
Note: problems not algorithms
![Page 34: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/34.jpg)
34
Example
Given n objects of different weights,
split them into two equally heavy piles
{1,2,3,5,7,8}
Algorithm
1+2+3+7 = 8+5
Checkingok ✔
O(2n)
O(n)
![Page 35: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/35.jpg)
35
In Previous Example ...
We know there are algorithms takes O(2n)
But ......
How do we know there isn’t a better algorithm which only takes polynomial time anyway?
![Page 36: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/36.jpg)
36
Some Classes of Problems
P
NP
problems which canbe solved in polynomialtime using a magic box
problems which can be solved in polynomial time
![Page 37: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/37.jpg)
37
Problems in NP but not in P
NPP
NPP?=
is a subset of
but
Most problems forwhich the best knownalgorithm is believed to be O(2n) are in NP
P
NP
![Page 38: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/38.jpg)
38
The Hardest NP Problems
If a polynomial time algorithm is found for any problem in NP-Complete then every problem in NP can be solved in polynomial time.
i,e, P = NP
P
NP
NP-Complete
![Page 39: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/39.jpg)
39
Examples of NP-Complete Problems
The travelling salesman problem Finding the shortest common
superstring Checking whether two finite automata
accept the same language Given three positive integers a, b, and
c, do there exist positive integers such that a*x2 + b*y2 = c
![Page 40: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/40.jpg)
40
How to be Famous
Find a polynomial time algorithm for this problem
Given n objects of different weights,
split them into two equally heavy piles
(or say if this isn’t possible)
![Page 41: 1 Complexity Lecture 17-19 Ref. Handout p 66 - 77](https://reader030.vdocuments.net/reader030/viewer/2022032803/56649e235503460f94b1161b/html5/thumbnails/41.jpg)
41
Memo for In-class test 19 [ /5]
questions my answers
correct answers
comments
1
2
3
4
5