Algorithms and Theory of Computing

Lecture 1: Introduction, Basics of Algorithms

Xiaohui Bei

MAS 714

August 10, 2021

Nanyang Technological University MAS 714 August 10, 2021 1 / 17

Administration

LecturesI Tuesday 10:30am - 12:30pm SPMS-TR+12I Friday 10:30am - 12:30pm SPMS-TR+12

TutorialsI Friday 11:30am - 12:30pm biweekly

OnlineI Zoom Link: https://ntu-sg.zoom.us/j/98716177373 password: mas714I Week 1 & 2: fully onlineI Starting from week 3: online + offline

Website: http://personal.ntu.edu.sg/xhbei/MAS714/index.html

E-mail: xhbei@ntu.edu.sg Office: MAS-05-46

Nanyang Technological University MAS 714 August 10, 2021 2 / 17

Gradings

Homework: 25%, Midterm 25%, Final: 50%

HomeworkSubmitted via NTULearn (as PDF files obtained from LATEX or Word sources)Solutions will be discussed on tutorials.

Homework policy:You are allowed (and encouraged) to discuss with your peers on the questions andsolutions. But everyone needs to write and submit their own solutions.

NTU academic integrity policy can be found athttp://www.ntu.edu.sg/ai.

Nanyang Technological University MAS 714 August 10, 2021 3 / 17

Recommended Reading

Kleinberg, Tardos: Algorithm Design

Cormen, Leiserson, Rivest, Stein: Introduction to Algorithms

Sipser: Introduction to the Theory of Computation

Nanyang Technological University MAS 714 August 10, 2021 4 / 17

Course Structure1 Algorithm Design

I graph algorithmsI greedyI divide and conquerI dynamic programmingI linear programmingI network flow

2 Automata TheoryI regular laguagesI finite state machines

3 Computability TheoryI Turing machinesI undecidabilityI P and NPI NP-completeness

Nanyang Technological University MAS 714 August 10, 2021 5 / 17

Algorithms

An algorithm is a procedure for performing a computation.Start from an initial state and an input (perhaps empty), eventually produce an output.An algorithm consists of primitive steps/instructions that can be executed mechanically.What is a primitive step? Depends on the model of computation.

A computer is a device that can be programmed to carry out primitive steps.I can then implement an entire algorithm by keeping track of stateI model vs. device

Nanyang Technological University MAS 714 August 10, 2021 6 / 17

Computers can be Humans!

Figure: Women at work tabulating during World War II (Shorpy)

Nanyang Technological University MAS 714 August 10, 2021 7 / 17

Model of Computation

An “idealized mathematical construct” that describes the primitive instructions and otherdetails.

Turing MachinesCircuitsRandom Access Machine (RAM)etc.

Random Access Machine (RAM)read/write from registersarithmetic operation on registersindirect addressing

Within this course, basically, pseudocode/informal language.

Nanyang Technological University MAS 714 August 10, 2021 8 / 17

Complexity

How to measure the performance of an algorithm?

Correctness: a must!I the definition of “correctness” can be discussed

Two most important measures are time and space.

Time: number of primitive steps needed to perform the computation.

Space: the amount of storage needed during the computation.

Nanyang Technological University MAS 714 August 10, 2021 9 / 17

Algorithm Example

Number AdditionGiven two n-digit numbers x and y, compute x+ y.

Procedure

+

1 1

3 8 7 3 48 4 0 7 5

1 2 2 8 0 9

Algorithm explanined:write numbers under each otheradd number position by position moving a “carry” forward

Nanyang Technological University MAS 714 August 10, 2021 10 / 17

Algorithm Analysis

Primitive steps:add two digitsread and write

Time complexity:algorithm requires O(n) primitive steps.

Space complexity:algorithm requires O(n) storage space.

Nanyang Technological University MAS 714 August 10, 2021 11 / 17

Some Other Examples

Number MultiplicationGiven two n-digit numbers x and y, compute x · y.

Procedure8 7 34 7 5

4 3 6 56 1 1 1

3 4 9 24 1 4 6 7 5

×

Nanyang Technological University MAS 714 August 10, 2021 12 / 17

Number Multiplication

Number of primitive steps: O(n2).

Space: O(n2).

Can we do better? Yes, but highly nontrivial.I previous best time: O(n logn log logn) [Schonhage-Strassen 1971]I after 37 years... O(n logn · 2O(log∗ n)) [Fürer 2008]I big breakthrough: O(n logn) [Harvey and van der Hoeven 2019]

F published in Annals of Mathematics, 2021F requires n ≥ 21729

12

≈ 10214857091104455251940635045059417341952

We don’t even understand multiplication well.

Nanyang Technological University MAS 714 August 10, 2021 13 / 17

Independent Set

Independent SetInput: an undirected graph G = (V, E).A set of nodes S ⊆ V is independent if no two nodes in S are jointed by an edge.Problem: find an independent set that is as large as possible.

Harder: no efficient algorithm is known.conjecture: no such algorithm exists

On the good side: checking the validity of a solution is easy.

Nanyang Technological University MAS 714 August 10, 2021 14 / 17

Go

Go

one of the most popular game in the worldartificial intelligence and Go: great advances in recentyearsn× n Go: played on a n× n board

I with Japanese ko rules

Problem: determine whether a position is a winningposition.

Even harder: EXPTIME-Completehardest problem that can be solved in exponential timeeven checking a solution is hard

Nanyang Technological University MAS 714 August 10, 2021 15 / 17

Posts’ Correspondence Problem

Posts’ Correspondence ProblemInput: two sequences of strings.

I e.g. A = [a, ab, bba], B = [baa, aa, bb]Problem: Find a sequence of indices such that the corresponding concatenated stringsare the same.

I e.g. (3, 2, 3, 1) =⇒ bba ab bba a = bb aa bb baa

Even harder: UndecidableI no algorithm exists!

Nanyang Technological University MAS 714 August 10, 2021 16 / 17

Summary

Algorithm design is not a piece of cake.

Difficulty varies in different problems.

Nanyang Technological University MAS 714 August 10, 2021 17 / 17