Math 4408 Lecture 2

In the study of algorithms (procedures for the solution of problems), we often want to compare competing algorithms to determine which is best. At other times we wish to know if an algorithm can provide a solution in a "reasonable" amount of time or use of resources. In order to study questions like these we need to examine the concepts concerning computational complexity.

Complexity functions

Worst Case complexity

Average Case complexity

We use "big oh" notation for comparision of complexity functions, more precisely:

If f(n) and g(n) are functions of the discrete variable n (n varies over the positive integers), then we say that f is O(g) if there exists an r in Z, and a constant k > 0 such that
f(n) <= kg(n) for all n >= r.

Some important classes of complexity functions are:

O(n) [linear], O(n^s) [polynomial], O(log₂ n), O(nlog₂ n), O(cⁿ) [c > 1, exponential], O(n!).

Theorem: (a) If a constant c > 0, then f is O(cf) and cf is O(f).
(b) n is O(n² ), ..., n^p-1 is O(n^p), but n^p is not O(n^p-1).
(c) If f(n) is a polynomial of degree q with all coefficients non-negative, then f is O(n^q). (d) If c > 1, p non-negative, then n^p is O(cⁿ) and cⁿ is not O(n^p).

Deterministic vs. Non-Deterministic Algorithms

An algorithm is deterministic if at each step there is only one choice for the next step given the values of the variables at that step.

An algorithm is non-deterministic if there is a step that involves parallel processing.

A problem is said to be in the class P of problems if it can be solved by an algorithm which is deterministic and has a time complexity function which is polynomial.

An problem is said to be in the class NP of problems if it can be solved by an algorithm which is non-deterministic and has a time complexity function which is polynomial. NP problems are recognized by the fact that their solutions can be checked for correctness by a deterministic polynomial time algorithm.

Every problem in P is also in NP. The non-deterministic algorithm that can be used is "guess the answer". The guess can be checked in polynomial time by the algorithm which solves the problem.

A famous and long standing open problem is whether or not P = NP.

There is a collection of problems with the property that any polynomial time deterministic algorithm which solves one of them can be converted to a polynomial time algorithm which solves any other one of them (they are said to be polynomially equivalent problems) and if such an algorithm existed for any one of them, then P = NP. These problems are called NP- hard problems. NP-hard problems may or may not be NP problems. Those that are NP are called NP-complete problems.

An example of an NP-complete problem is the Travelling Salesman Problem.