1

Algorithms and Data

Structures(CS221)

Lecture 6

Abdisalam Issa-Salwe

Taibah University College of Computer Science & Engineering

Computer Science Department

2

Outline

• Computer Problem-Solving

• Efficiency of Algorithms

• Properties of algorithms

• Developing an Algorithm

• Complexity Analysis

• Big-O Notation

• Pseudocode

• Flowcharts

2

3

Algorithms

An algorithm is a finite set of precise instructions for performing a computation or for solving a problem.

A set of instructions for solving a problem or subproblem in a finite amount of time using a finite amount of data

The instructions must be unambiguous

4

Algorithm (cont…)

Algorithm is a systematic method of

instructing an agent how to accomplish a

task

Usually expressed in a step-wise

sequential form

May involve alternation, iteration or

recursion

Detail and language may depend on the

agent

3

5

Algorithm (cont…)

It is a set of instructions of how to carry out

a process.

A list of the steps that must be followed to

complete the process and to reach the

solution

An algorithm can be represented

diagrammatically in the form of a flow

chart.

6

Divide and Conquer

Problem solving: The act of finding a

solution to a perplexing, distressing,

vexing, or unsettled question

Break up a large problem into smaller

units that we can handle

Applies the concept of abstraction

The divide-and-conquer approach can be applied over and over again until each subtask is manageable

4

7

Computer Problem-Solving

8

Problem-Solving Phases

5

9

Task 1: Making A Cup of Tea!

The following list

represents the steps

needed in order to make

a cup of tea. Put the list in

the right order and

produce an algorithm in

POWERPOINT showing

the processes and

decisions you need to

make.

1) Take tea bag out of cup Boil the water

2) Put tea bag in cup

3) Add milk?

4) Pour boiling water in cup

5) Fill kettle with water

6) Stir

7) Ready

8) Add sugar ?

9) Fetch cup

Answers follow

10

The answers…1. Fill kettle

2. Boil the water

3. Fetch cup

4. Put tea bag in cup

5. Pour boiling water in cup

6. Take tea bag out of cup

7. Add sugar ?

8. Stir

9. Add milk?

10. Stir

11. Ready

START

Fill kettle

Boil the water

Fetch cup

Put tea bag in cup

Pour boiling water in cup

Take tea bag out of cup

Add sugar ?

Stir in sugar

Add milk?

Pour in milk Ready

N

N

Now add the shapes…

6

11

Description of an Algorithm

• Numbered steps with indentation commonly used

• Different descriptions possible for the same algorithm

• Difference between the algorithm description (design) and the completion of the task (implementation)

• More than one algorithm possible for the same task

12

Efficiency of Algorithms

• What resources are required to

accomplish the task

• How one algorithm compares with other

algorithms

7

13

Efficiency and Complexity

• Efficiency

• How much time or space is required

• Measured in terms of common basic

operations

• Complexity

• How efficiency varies with the size of the task

• Expressed in terms of standard functions of n

• e.g.: O(n), O(n2), O(log n), O(n log n)

14

Properties of algorithms

Input from a specified set,

Output from a specified set (solution),

Definiteness of every step in the computation,

Correctness of output for every possible input,

Finiteness of the number of calculation steps,

Effectiveness of each calculation step and

Generality for a class of problems.

8

15

Developing an Algorithm

The plan must be suitable in a suitable

form

Two methodologies that currently used

Top-down design

Object-oriented design

The plan must be suitable in a suitable

form

Two methodologies that currently used

Top-down design

Object-oriented design

16

Top-Down Design

Breaking the problem into a set of

subproblems called modules

Creating a hierarchical structure of

problems and subproblems (modules)

9

17

Testing the Algorithm

Desk checking Working through a design at a desk with a pencil and paper

Walk-through Manual simulation of the design by the team members, taking sample data values and simulating the design using the sample data

Inspection One person (not the designer) reads the design (handed out in advance) line by line while the others point out errors

18

Complexity Analysis

Number of CPU cycles it takes to run an algorithm depends on the computer on which the algorithm is run.

Count of how many instructions are executed depends on the programming language used to implement the algorithm and the way the programmer writes the program.

We want a measure of algorithm efficiency that is independent of the computer, the programming language, the programmer, and all the complex details of the algorithm such as incrementing of loop indices, setting of pointers, etc.

In general, the running time of an algorithm increases with the size of the input, and the total running time is roughly proportional to how many times some basic operation (such as a comparison instruction) is done.

10

19

Complexity Analysis (cont…)

We therefore analyze the algorithm’s efficiency by determining the number of times some basic operation is done as a function of the input size. This is called a time complexity analysis of an algorithm.

The basic operation may be a single instruction or a group of instructions; in some cases, we may want to consider more than one basic operation.

The input size may be easy to determine – such as the size of an array for Sequential or Binary Search -- or it may be more difficult.

In some algorithms, the basic operation is always done the same number of times for every instance of size N.

When this is the case, the every-case time complexity of the algorithm, T(n), is defined as the number of times the algorithm does the basic operation for an instance of size n.

20

Worst-Case Time Complexity Analysis

Basic operation: the comparison of an

item in the array with searchkey.

Input size: n, the number of items in the

array.

The basic operation is done at most n

times, which is the case if searchkey is the

last item in the array or if searchkey is not

in the array. Therefore, W(n) = n.

11

21

Average-Case Time Complexity Analysis

Basic operation: the comparison of an item in the array with searchkey.

Input size: n, the number of items in the array.

We first analyze the case in which it is known that searchkey is in the array, where the items in the array are all distinct, and where we have no reason to believe that searchkey is more likely to be in one array slot than it is to be in another. Based on this information, for 1 ≤ k≤ n, the probability that searchkey is in the kth array slot is 1/n. If searchkey is in the kth array slot, the number of times the basic operation is done to locate searchkey(and therefore, to exit the loop) is k. This means that the average time complexity is given by

22

Best-Case Time Complexity Analysis

Basic operation: the comparison of an

item in the array with searchkey.

Input size: n, the number of items in the

array.

Because n ≥ 1, there must be at least one

pass through the loop. If searchkey =

array[0], there will be one pass through the

loop regardless of the size of n. Therefore,

B(n) = 1.

12

23

Performance Analysis

Determining an estimate of the time and memory requirement of the algorithm.

Time estimation is called time complexity analysis

Memory size estimation is called space complexity analysis.

Because memory is cheap and abundant, we rarely do space complexity analysis

Since time is “expensive” , analysis now defaults to time complexity analysis

24

Big-O Notation

The most common method and notation

for discussing the execution time of

algorithms is "Big O”.

For the alphabetized dictionary the

algorithm requires O(log N) steps.

For the unsorted list the algorithm requires

O(N) steps.

Big O is the asymptotic execution time of

the algorithm.

13

25

Big-O Notation (In Practice)

When computing the complexity,

f(n) is the actual time formula

g(n) is the simplified version of f

Since f(n) stands often for time, we use

T(n) instead of f(n)

In practice, the simplification of T(n)

occurs while it is being computed by the

designer

26

Big-O Notation

(Common Complexities)

T(n) = O(1) // constant time

T(n) = O(log n) // logarithmic

T(n) = O(n) // linear

T(n) = O(n2) //quadratic

T(n) = O(n3) //cubic

T(n) = O(nc), c≥ 1 // polynomial

T(n) = O(logc n), c≥ 1 // polylogarithmic

T(n) = O(nlog n)

14

27

Other Algorithmic Analysis Tools

Big Omega T(N) is Ω( F(N) ) if there are

positive constants c and N0 such that

T(N) > cF( N )) when N > N0

Big O is similar to less than or equal, an upper bound.

Big Omega is similar to greater than or equal, a lower bound.

Big Theta T(N) is θ( F(N) ) if and only if

T(N) is O( F(N) )and T( N ) is Ω( F(N) ).

Big Theta is similar to equals.

28

Applying algorithm

When applying the theory of algorithm analysis, one must sometimes be aware of the time that it takes to execute the basic operation, the overhead instructions, and the control instructions on the actual computer on which the algorithm is implemented.

“Overhead instructions” includes things such as initialization instructions before a loop; the number of times these instructions execute does not increase with input size.

“Control instructions” means instructions such as incrementing an index to control a loop; the number of times these instructions execute increases with input size.

The basic operation, overhead instructions, and control instructions are all properties of an algorithm and the implementation of the algorithm; they are not properties of a problem.

15

29

Applying algorithm (cont…)

Algorithms with time complexities such as n and 100n are called linear-time algorithms, because their time complexities are linear in the input size n.

Algorithms with time complexities such as n2 and 0.01n2 are called quadratic-time algorithms, because their time complexities are quadratic in the input size n.

A fundamental principle: any linear-time algorithm is eventually more efficient than any quadratic-time algorithm.

Algorithms can be grouped into ordersaccording to their eventual behavior.

30

Pseudocode

Uses a mixture of English and formatting

to make the steps in the solution explicit

16

31

Pseudocode

Pseudocode is a kind of structured English for describing algorithms.

It allows the designer to focus on the logic of the algorithm without being distracted by details of language syntax.

At the same time, the pseudocode needs to be complete.

It describe the entire logic of the algorithm so that implementation becomes a rote mechanical task of translating line by line into source code.

32

Pseudocode (cont…)

The vocabulary used in the pseudocode

should be the vocabulary of the problem

domain, not of the implementation

domain.

The pseudocode is a narrative for

someone who knows the requirements

(problem domain) and is trying to learn

how the solution is organized

17

33

Constructs for flow of control (cont…)

1. SEQUENCE: A linear progression where one task is performed sequentially after another. Sequential control is indicated by writing one

action after another, each action on a line by itself, and all actions aligned with the same indent. The actions are performed in the sequence (top to bottom) that they are written.

2. WHILE: A loop (repetition) with a simple conditional test at its beginning. The WHILE construct is used to specify a loop with a

test at the top. The beginning and ending of the loop are indicated by two keywords WHILE and ENDWHILE.

34

Constructs for flow of control (cont…)

3. IF-THEN-ELSE: A decision (selection) in which a choice is made between two alternative courses of action. Binary choice on a given Boolean condition is

indicated by the use of four keywords: IF, THEN, ELSE, and ENDIF.

4. REPEAT-UNTIL: A loop with a simple conditional test at the bottom. This loop is similar to the WHILE loop except that the

test is performed at the bottom of the loop instead of at the top. Two keywords, REPEAT and UNTIL are used.

18

35

Constructs for flow of control (cont…)

5. CASE: A CASE construct indicates a multiway branch based on conditions that are mutually exclusive. Four keywords, CASE, OF, OTHERS, and

ENDCASE, and conditions are used to indicate the various alternatives.

FOR: This loop is a specialized construct for iterating a specific number of times, often called a "counting" loop. Two keywords, FOR and ENDFOR are used.

36

Constructs for flow of control (cont…)

NESTED CONSTRUCTS: The constructs

can be embedded within each other, and

this is made clear by use of indenting.

Nested constructs should be clearly indented from their surrounding constructs.

INVOKING SUBPROCEDURES: Using

the CALL keyword.

19

37

Flowcharts (cont…)

Step-form and pseudocode program

designs are both text-based, the

statements are written.

Flow charts are a graphical method of

designing programs and once the rules

are learned are very easy to draw.

38

Flowcharts

A well-drawn flow chart is also very easy to read since it basically uses just two symbols, two decision constructs. and two iteration constructs: the sequence symbol,

the decision symbol,

the decision construct if ... then

the decision construct if ... then ... else

the repetition construct - repeat

the repetition construct - while

20

39

Flowcharts (cont…)

40

Flowcharts (cont…)

Every flow chart has a START symbol and a STOP symbol

The flow of sequence is generally from the top of the page to the bottom of the page. This can vary with loops which need to flow back to an entry

point.

Use arrow-heads on connectors where flow direction may not be obvious.

There is only one flow chart per page

A page should have a page number and a title

A flow chart on one page should not break and jump to another page

A flow chart should have no more than around 15 symbols (not including START and STOP)

21

41

Flowcharts (cont…)

The major symbols are the DECISION (also known as selection) and the SEQUENCE (or process) symbols.

The START and STOP symbols are called the terminals.

The SUBPROCESS symbol is a variation on the sequence symbol.

Connectors are drawn between the symbols

There must be at least one other sequence symbol to represent input/output processes

42

Flowcharts (cont…)

Processes can have only one entry point and one exit point.

Decisions have only one entry point, one TRUE exit point and one FALSE exit point.

Repeat loop: repeat loop has the process preceding the decision; a repeat loop will always execute the process part at least once.

While loop is the reverse of the repeat loop; the decision comes first, followed by the process.

IF ... THEN construct is also known as the NULL ELSE, meaning that there is no ELSE part.

IF ... THEN ... ELSE ... construct has a process at each branch of the decision symbol.

22

43

Topic Questions

What is Algorithms and how it relates to

computerized programming?

Can you a problem with two solutions?

which one of the solution you think it is more efficient?

What is the difference between time

complexity and space complexity?

What is flowchart and how can be used to

solve a problem?

44

References

Michael Main, Data Structures and Other

Objects Using Java (Third Edition)

Abdisalam Issa-Salwe, Taibah University, Madinah, Saudi Arabia.