meljun cortes algorithm limitations of algorithm power

Limitations of algorithm Power

Design and Analysis of Algorithm

* Property of STI

of 25

The following are the topics to be discussed under

Limitations of algorithm power:

Definition of lower-bound arguments

Different algorithms using decision tree

Application of P, NP and NP-complete

problems



* Property of STI

of 25

A reasonable evaluation of algorithms as problem-

solving tools is inevitable.

These algorithms are considered as powerful

instruments, particularly when these are

processed by modern computers.

Sometimes there are problems that cannot be

solved by an algorithm or the problems seem not

to have efficient solutions.



* Property of STI

of 25

Lower bound :is a comparison-based algorithm

where the behavior of the algorithm is based only

on the comparison between elements

[www.cs.toronto.edu].

The lower bound deals with the complexity of a

problem instead of the algorithm.

Basically we have to prove that no algorithm, no

matter how complex, can do better than our

bound



* Property of STI

of 25

Examples:

number of comparisons needed to find the largest

element in a set of n numbers

number of comparisons needed to sort an array of

size n

number of comparisons necessary for searching in

a sorted array

number of multiplications needed to multiply two

n-by-n matrices



* Property of STI

of 25

Two ways to examine the algorithm efficiency:

Establish the asymptotic efficiency class similar to

the worst case

Single out where this class stands with respect to

the hierarchy of efficiency class



* Property of STI

of 25

Trivial lower bound: is based on counting the

number of items that must be processed in input

and generated as output [LEV07].

Examples:

finding max element -- n steps or n/2

comparisons

polynomial evaluation

sorting

element uniqueness

Hamiltonian circuit existence

Conclusions:

may and may not be useful

be careful in deciding how many elements

must be processed



* Property of STI

of 25

Information-theoretic arguments (decision trees) :

seek to establish a lower bound based on the

amount of information it has to produce [LEV07].

Example:

Deducing a positive integer between 1 and n

selected by somebody by asking that person

questions with yes/no answers.

Conclusions:

The approach is related to information theory

because it has proved to be useful for finding

the information-theoretic bounds in different

problem types concerning comparisons, such

as sorting and searching.

Its core concept can be apprehended more

precisely through decision trees.



* Property of STI

of 25

Adversary arguments: Malevolent simply means

that it will change a problem to force algorithm

down worst path and we can say that it is fair

because we must remain consistent with the work

that is already done for selection problems.

Examples:

Three-Card Monte

n-Card Monte

Finding Patterns in Bit Strings

Evasive Graph Properties

Conclusions:

The adversary should be thought of as a very

powerful, clever being that is trying to make

your algorithm run as slowly as possible.

The adversary cannot "read the algorithm's

mind", but it can try to be prepared for

anything the algorithm does.



* Property of STI

of 25

Problem reduction: proves a lower bound by

transforming a problem that already has an

established lower bound into the new problem.

Examples:

Reduction to linear programming

Algorithmic problem solving

Reduction to graph problems

Evasive Graph Properties



* Property of STI

of 25

Conclusion:

We must prove that an arbitrary instance of

problem Q can be transformed in a logically

efficient manner to an instance of problem P.

As a result, any algorithm solving P would solve Q

as well. Therefore, the lower bound for Q will also

be the lower bound for P.

The figure below are the list of problems used for

establishing lower bounds by problem reduction:



* Property of STI

of 25

Decision tree : is a convenient model of algorithms

involving comparisons in which internal nodes

represent comparisons and leaves represent

outcomes (or input cases).

The figure below represents the decision tree of

an algorithm for finding a minimum value of three

numbers:



* Property of STI

of 25

By studying the properties of decision trees for

comparison-based sorting algorithms, we can

derive important lower bounds on time efficiencies

of such algorithms.

Consider the following figure:



* Property of STI

of 25

Inequality implies that the height of a binary

decision tree for any comparison-based sorting

algorithm and the worst-case number of

comparisons made such an algorithm cannot be

less than [log2n!]:

Cworst (n) ≥ [log2n!]

We can also use the decision trees for analyzing

the average-case behavior of a comparison-based

sorting algorithm.



* Property of STI

of 25

We can compute the average number of

comparisons for a particular algorithm as the

average depth of its decision tree’s leaves like the

average path length from the root to the leaves.

The figure shows an example of a decision tree for

the three element in the insertion sort :



* Property of STI

of 25

An algorithm solves a problem in polynomial time

if its worst-case time efficiency belongs to O(p(n))

time, where p(n) is a polynomial of problem’s input

size n [LEV07].

Tractable : problems that can be solved in

polynomial time.

Intractable : problems that cannot be solved in

polynomial time.



* Property of STI

of 25

Reasons for drawing the intractability line :

We cannot solve arbitrary instances of

intractable problems in a reasonable amount

of time unless such instances are very small.

Although there might be a huge difference

between the running time in O(p(n)) for

polynomials of drastically different degrees,

there are very few useful polynomial time

algorithm with the degree of polynomial higher

than three.

Polynomial functions posses many convenient

properties, in particular both the sum and

composition of two polynomials are always

polynomials, too.

The choice of this class led to a development

of an extensive theory called computational

complexity.



* Property of STI

of 25

Class P: is a class of decision problems that can

be solved in polynomial time by deterministic

algorithms. This class of problems is called

polynomial [LEV07].

The restriction of P to decision problems can be

justified by the following:

It is sensible to exclude problems not solvable

in polynomial time because of their

exponentially large output.

Different important problems that are not

decision problems that are easier to study.



* Property of STI

of 25

There are many important problems showing the

polynomial time algorithm:

Hamilton circuit

Traveling salesman

Knapsack problem

Partition problem

Bin packing

Graph coloring

Integer linear programming



* Property of STI

of 25

NP (nondeterministic polynomial): the class of

decision problems whose proposed solutions can

be verified in polynomial time, which is solvable by

a nondeterministic polynomial algorithm.

A nondeterministic polynomial algorithm is an

abstract two-stage procedure that:

generates a solution of the problem (on some

input) by guessing

checks whether this solution is correct in

polynomial time



* Property of STI

of 25

NP (nondeterministic polynomial): the class of

decision problems whose proposed solutions can

be verified in polynomial time, which is solvable by

a nondeterministic polynomial algorithm.

A nondeterministic polynomial algorithm is an

abstract two-stage procedure that:

generates a solution of the problem (on some

input) by guessing

checks whether this solution is correct in

polynomial time

Sample NP or Nondeterministic algorithm

problems:

Guess a truth assignment

Substitute the values into CNF formula to see

if it evaluates to true



* Property of STI

of 25

Other Problems in NP

Class NP :is the class of decision problems that

can be solved by nondeterministic polynomial

algorithms. This class of problems is called

nondeterministic polynomial [LEV07].

All the problems in P can also be solved in this

manner (but no guessing is necessary), so we

have:

• P NP



* Property of STI

of 25

NP sample problems also includes:

Hamiltonian circuit existence

Partition problem: Is it possible to partition a

set of n integers into two disjoint subsets with

the same sum?

Decision versions of TSP, knapsack problem,

graph coloring, and many other combinatorial

optimization problems.



* Property of STI

of 25

NP-complete problem: is a problem in NP that is

as difficult as any other problem in this class,

because by definition, any problem in NP can be

reduced to it in polynomial time.

A decision problem D1 is said to be polynomially

reducible to a decision problem D2 if there exists a

function t that transforms instances of D1 to

instances of D2 such that:

t maps all yes instances of D1 to yes instances

of D2 and all no instances of D1 to all no

instances of D2

t is computable by a polynomial time

algorithm.

A decision problem D is NP-complete if it is as

hard as any problem in NP, i.e.,

D is in NP

every problem in NP is polynomial-time reducible

to D



* Property of STI

of 25

This definition immediately implies that if a

problem D is polynomially reducible to some

problem D that can be solved in polynomial time,

then problem D.

know n

NP-complete

problem

NP problems

candidate

for NP -

completeness



* Property of STI

of 25

Other NP-complete problems obtained through

polynomial-time reductions from a known NP-

complete problem, as illustrated in the figure

below.

know n

NP-complete

problem

NP problems

candidate

for NP -

completeness

meljun cortes algorithm limitations of algorithm power

Documents