copyright © 2007 pearson addison-wesley. all rights reserved. a. levitin “ introduction to the...
DESCRIPTION
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch i. How to Design: Basic Patterns Brute force Divide and conquer Decrease and conquer Transform and conquer Space and time tradeoffs Greedy approach Dynamic programming Iterative improvement Backtracking Branch and boundTRANSCRIPT
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Chapter 2 Chapter 2
Fundamentals of the Analysis Fundamentals of the Analysis of Algorithm Efficiencyof Algorithm Efficiency
Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-2
Two main issues related to algorithmsTwo main issues related to algorithms
How to design algorithmsHow to design algorithms
How to analyze algorithm efficiencyHow to analyze algorithm efficiency
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-3
1.i. How to Design: Basic Patterns1.i. How to Design: Basic Patterns
Brute forceBrute force
Divide and conquerDivide and conquer
Decrease and conquerDecrease and conquer
Transform and conquerTransform and conquer
Space and time tradeoffsSpace and time tradeoffs
Greedy approachGreedy approach
Dynamic programmingDynamic programming
Iterative improvementIterative improvement
Backtracking Backtracking
Branch and boundBranch and bound
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-4
1. ii. How to Design: Basic Problem Types1. ii. How to Design: Basic Problem Types
sortingsorting
searchingsearching
string processingstring processing
graph problemsgraph problems
combinatorial problemscombinatorial problems
geometric problemsgeometric problems
numerical problemsnumerical problems
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-5
1. iii. How to Design: Basic Tools1. iii. How to Design: Basic Tools listlist
• arrayarray
• linked listlinked list
• string string stackstack queuequeue priority queuepriority queue
graphgraph treetree set and dictionaryset and dictionary
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1 1-6
2. i. Analysis of algorithms: Questions2. i. Analysis of algorithms: Questions
How good is the algorithm?How good is the algorithm?• time efficiencytime efficiency• space efficiencyspace efficiency
Does there exist a better algorithm?Does there exist a better algorithm?• lower boundslower bounds• optimalityoptimality
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
2. ii. Analysis of algorithms2. ii. Analysis of algorithms
Issues:Issues:• correctnesscorrectness• time efficiencytime efficiency• space efficiencyspace efficiency• optimalityoptimality
Approaches:Approaches: • empirical analysisempirical analysis• theoretical analysistheoretical analysis
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Empirical analysis of time efficiencyEmpirical analysis of time efficiency
Select a specific (typical) sample of inputsSelect a specific (typical) sample of inputs
Use physical unit of time (e.g., milliseconds)Use physical unit of time (e.g., milliseconds) oror Count actual number of basic operation’s executionsCount actual number of basic operation’s executions
Analyze the empirical dataAnalyze the empirical data
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Theoretical analysis of time efficiencyTheoretical analysis of time efficiency
Time efficiency is analyzed by determining the number of Time efficiency is analyzed by determining the number of repetitions of the repetitions of the basic operationbasic operation as a function of as a function of input sizeinput size
Basic operationBasic operation: the operation that contributes most : the operation that contributes most towards the running time of the algorithmtowards the running time of the algorithm
TT((nn) ) ≈≈ ccopopCC((nn))running time execution time
for basic operationNumber of times basic operation is
executed
input size
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Input size and basic operation examplesInput size and basic operation examples
ProblemProblem Input size measureInput size measure Basic operationBasic operation
Searching for key in a Searching for key in a list of list of nn items items ?? ??
Multiplication of two Multiplication of two matricesmatrices ?? ??
Checking primality of Checking primality of a given integer a given integer nn ?? ??
Typical graph problemTypical graph problem ?? ??
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
ProblemProblem Input size measureInput size measure Basic operationBasic operation
Searching for key in a Searching for key in a list of list of nn items items
Number of list’s items, Number of list’s items, i.e. i.e. nn Key comparisonKey comparison
Multiplication of two Multiplication of two matricesmatrices
Matrix dimensions or Matrix dimensions or total number of elementstotal number of elements
Multiplication of two Multiplication of two numbersnumbers
Checking primality of Checking primality of a given integer a given integer nn
n’n’size = number of digits size = number of digits (in binary representation)(in binary representation) DivisionDivision
Typical graph problemTypical graph problem #vertices and/or edges#vertices and/or edges Visiting a vertex or Visiting a vertex or traversing an edgetraversing an edge
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Best-case, average-case, worst-caseBest-case, average-case, worst-case
For some algorithms efficiency depends on form of input:For some algorithms efficiency depends on form of input:
Worst case: CWorst case: Cworstworst((nn) – maximum over inputs of size ) – maximum over inputs of size nn
Best case: CBest case: Cbestbest((nn) – minimum over inputs of size ) – minimum over inputs of size nn
Average case: CAverage case: Cavgavg((nn) – “average” over inputs of size ) – “average” over inputs of size nn• Number of times the basic operation will be executed on typical inputNumber of times the basic operation will be executed on typical input• NOT the average of worst and best caseNOT the average of worst and best case• Expected number of basic operations considered as a random variable Expected number of basic operations considered as a random variable
under some assumption about the probability distribution of all under some assumption about the probability distribution of all possible inputspossible inputs
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Example: Sequential searchExample: Sequential search
Worst caseWorst case
Best caseBest case
Average caseAverage case
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Types of formulas for basic operation’s countTypes of formulas for basic operation’s count
Exact formulaExact formula e.g., C(e.g., C(nn) = ) = nn((nn-1)/2-1)/2
Formula indicating order of growth with specific Formula indicating order of growth with specific multiplicative constantmultiplicative constant
e.g., C(e.g., C(nn) ) ≈≈ 0.5 0.5 nn22
Formula indicating order of growth with unknown Formula indicating order of growth with unknown multiplicative constantmultiplicative constant
e.g., C(e.g., C(nn) ) ≈≈ cncn22
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Order of growth Order of growth
Most important: Order of growth within a constant multiple Most important: Order of growth within a constant multiple as as nn→∞→∞
Example:Example:• How much faster will algorithm run on computer that is How much faster will algorithm run on computer that is
twice as fast?twice as fast?
• How much longer does it take to solve problem of double How much longer does it take to solve problem of double input size?input size?
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Values of some important functions as Values of some important functions as n n
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 1
Growth MattersGrowth Matters
Magic Computer – One Trillion Operations per Second Magic Computer – One Trillion Operations per Second • It would take approx 4*10It would take approx 4*101010 years to execute 2 years to execute 2100100 operations operations• This is more than 4.5 billion yearsThis is more than 4.5 billion years
Imagine how long it would take to execute 100! OperationsImagine how long it would take to execute 100! Operations
These are both fine examples of a somewhat small (100) These are both fine examples of a somewhat small (100) factor with 2factor with 2nn and n! complexity and n! complexity
1-17