FUNDAMENTALS OF ALGORITHMICS

Gilles Brassard and Paul Bratley

Departement d'informatique et de recherche operationeile

Universite de Montreal

Prentice-Hall International, Inc.

Contents

PREFACE xv

• / PRELIMINARIES 7

1.1 Introduct ion 1

1.2 Wha t is an algori thm? 1

1.3 Nota t ion for p rograms 6

1.4 Mathematical notat ion 7 1.4.1 Propositional calculus 7 1.4.2 Set theory 8 1.4.3 Integers, reals and intervals 8 1.4.4 Functions and relations 9 1.4.5 Quantifiers 10 1.4.6 Sums and products 11 1.4.7 Miscellaneous 12

1.5 Proof technique 1—Cont rad ic t ion 13

1.6 Proof technique 2 — Mathemat ica l induc t ion 16 1.6.1 The principle of mathematical induction 18 1.6.2 A horse of a different colour 23 1.6.3 Generalized mathematical induction 24 1.6.4 Constructive induction 27

1.7 Some reminders 31 1.7.1 Limits 31 1.7.2 Simple series 34 1.7.3 Basic combinatorics 38 1.7.4 Elementary probability 41

vii

viii Contents

1.8 Problems 48

1.9 References and further reading 55

• 2 ELEMENTARY ALGORITHMICS 57

2.1 Introduction 57

2.2 Problems and instances 58

2.3 The efficiency of algorithms 59

2.4 Average and worst-case analyses 61

2.5 What is an elementary operation? 64

2.6 Why look for efficiency? 66

2.7 Some examples 67 2.7.1 Calculating determinants 68 2.7.2 Sorting 68 2.7.3 Multiplication of large integers 70 2.7.4 Calculating the greatest common divisor 71 2.7.5 Calculating the Fibonacci sequence 72 2.7.6 Fourier transforms 73

2.8 When is an algorithm specified? 74

2.9 Problems 74

2.10 References and further reading 78

• 3 ASYMPTOTIC NOTATION 79

3.1 Introduction 79

3.2 A notation for "the order of" 79

3.3 Other asymptotic notation 85

3.4 Conditional asymptotic notation 88

3.5 Asymptotic notation with several parameters 91

3.6 Operations on asymptotic notation 91

3.7 Problems 92

3.8 References and further reading 97

• 4 ANALYSIS OF ALGORITHMS 98

4.1 Introduction 98

4.2 Analysing control structures 98 4.2.1 Sequencing 98 4.2.2 "For" loops 99 4.2.3 Recursive calls 101 4.2.4 "While" and "repeat" loops 102

Contents ix

4.3 Using a barometer 104

4.4 Supplementary examples 106

4.5 Average-case analysis 111

4.6 Amortized analysis 112

4.7 Solving recurrences 116 4.7.1 Intelligent guesswork 116 4.7.2 Homogeneous recurrences 118 4.7.3 Inhomogeneous recurrences 123 4.7.4 Change of variable 130 4.7.5 Range transformations 136 4.7.6 Asymptotic recurrences 137

4.8 Problems 139

4.9 References and further reading 146

• 5 SOME DATA STRUCTURES 147

5.1 Arrays, stacks and queues 147

5.2 Records and pointers 150

5.3 Lists 151

5.4 Graphs 152

5.5 Trees 154

5.6 Associative tables 159

5.7 Heaps 162

5.8 Binomial heaps 170

5.9 Disjoint set structures 175

5.10 Problems 181

5.11 References and further reading 186

• 6 GREEDY ALGORITHMS 1J37

6.1 Making change (1) 187

6.2 General characteristics of greedy algorithms 188

6.3 Graphs: Minimum spanning trees 190 6.3.1 Kruskal's algorithm 193

6.3.2 Prim's algorithm 196

6.4 Graphs: Shortest paths 198

6.5 The knapsack problem (1) 202

6.6 Scheduling 205 6.6.1 Minimizing time in the system 205 6.6.2 Scheduling with deadlines 207

x Contents

6.7 Problems 214 6.8 References and further reading 217

• 7 DIVIDE-AND-CONQUER 219

7.1 Introduction: Multiplying large integers 219 7.2 The general template 223

7.3 Binary search 226 7.4 Sorting 228

7.4.1 Sorting by merging 228 7.4.2 Quicksort 231

7.5 Finding the median 237 7.6 Matrix multiplication 242 7.7 Exponentiation 243 7.8 Putting it all together: Introduction to cryptography 247 7.9 Problems 250 7.10 References and further reading 257

• 8 DYNAMIC PROGRAMMING 259

8.1 Two simple examples 260 8.1.1 Calculating the binomial coefficient 260 8.1.2 The World Series 261

8.2 Making change (2) 263

8.3 The principle of optimality 265 8.4 The knapsack problem (2) 266

8.5 Shortest paths 268 8.6 Chained matrix multiplication 271 8.7 Approaches using recursion 274

8.8 Memory functions 276

8.9 Problems 278

8.10 References and further reading 283

• 9 EXPLORING GRAPHS 265

9.1 Graphs and games: An introduction 285

9.2 Traversing trees 291 9.2.1 Preconditioning 292

9.3 Depth-first search: Undirected graphs 294 9.3.1 Articulation points 296

9.4 Depth-first search: Directed graphs 298 9.4.1 Acyclic graphs: Topological sorting 300

Contents xi

9.5 Breadth-first search 302 9.6 Backtracking 305

9.6.1 The knapsack problem (3) 306 9.6.2 The eight queens problem 308 9.6.3 The general template 311

9.7 Branch-and-bound 312 9.7.1 The assignment problem 312 9.7.2 The knapsack problem (4) 315 9.7.3 General considerations 316

9.8 The minimax principle 317 9.9 Problems 319 9.10 References and further reading 326

• W PROBABILISTIC ALGORITHMS 328

10.1 Introduction 328 10.2 Probabilistic does not imply uncertain 329 10.3 Expected versus average time 331 10.4 Pseudorandom generation 331 10.5 Numerical probabilistic algorithms 333

10.5.1 Buffon's needle 333 10.5.2 Numerical integration 336 10.5.3 Probabilistic counting 338

10.6 Monte Carlo algorithms 341 10.6.1 Verifying matrix multiplication 341 10.6.2 Primality testing 343 10.6.3 Can a number be probably prime? 348 10.6.4 Amplification of stochastic advantage 350

10.7 Las Vegas algorithms 353 10.7.1 The eight queens problem revisited 355 10.7.2 Probabilistic selection and sorting 358 10.7.3 Universal hashing 360 10.7.4 Factorizing large integers 362

10.8 Problems 366 10.9 References and further reading 373

• 7/ PARALLEL ALGORITHMS 376

11.1 A model for parallel computation 376

11.2 Some basic techniques 379 11.2.1 Computing with a complete binary tree 379 11.2.2 Pointer doubling 380

11.3 Work and efficiency 383 11.4 Two examples from graph theory 386

11.4.1 Shortest paths 386 11.4.2 Connected components 387

xii Contents

11.5 Parallel evaluat ion of expressions 392 11.6 Parallel sor t ing ne tworks 397

11.6.1 The zero-one principle 399 11.6.2 Parallel merging networks 400 11.6.3 Improved sorting networks 402

11.7 Parallel sort ing 402 11.7.1 Preliminaries 403 11.7.2 The key idea 404 11.7.3 The algorithm 405 11.7.4 A sketch of the details 406

11.8 Some remarks on EREW and CRCW p- rams 406 11.9 Distr ibuted computa t ion 408 11.10 Problems 409 11.11 References and further read ing 412

• 12 COMPUTATIONAL COMPLEXITY 473

12.1 Introduct ion: A simple example 414 12.2 Information-theoretic a rgument s 414

12.2.1 The complexity of sorting 418 12.2.2 Complexity to the rescue of algorithmics 421

12.3 Adversa ry a rgument s 423 12.3.1 Finding the maximum of an array 424 12.3.2 Testing graph connectivity 425 12.3.3 The median revisited 426

12.4 Linear reduct ions 427 12.4.1 Formal definitions 430 12.4.2 Reductions among matrix problems 433 12.4.3 Reductions among shortest path problems 438

12.5 In t roduct ion to ЛГУ-comple teness 441 12.5.1 The classes T and ЖТ 441 12.5.2 Polynomial reductions 445 12.5.3 WT- complete problems 450 12.5.4 A few NT- completeness proofs 453 12.5.5 ^"P-hard problems 457 12.5.6 Nondeterministic algorithms 458

12.6 A menager ie of complexity classes 460 12.7 Problems 464 12.8 References and further reading 471

• 73 HEURISTIC AND APPROXIMATE ALGORITHMS 474

13.1 Heurist ic a lgor i thms 475 13.1.1 Colouring a graph 475 13.1.2 The travelling salesperson 477

Contents xiii

13.2 Approximate algorithms 478 13.2.1 The metric travelling salesperson 478 13.2.2 The knapsack problem (5) 480 13.2.3 Bin packing 482

13.3 JVy-hard approximation problems 484 13.3.1 Hard absolute approximation problems 486 13.3.2 Hard relative approximation problems 487

13.4 The same, only different 489 13.5 Approximation schemes 492

13.5.1 Bin packing revisited 493 13.5.2 The knapsack problem (6) 493

13.6 Problems 496 13.7 References and further reading 500

REFERENCES 501

INDEX 517