Theory of Computation

Lecture #45
Sarmad Abbasi
Virtual University

Sarmad Abbasi (Virtual University)

Theory of Computation

1/1

Space Complexity

Today we will discuss what we have learnt in this course and go over
the major results and accomplishments of computability and
complexity theory.

Sarmad Abbasi (Virtual University)

Theory of Computation

2/1

Computability

The most important achievement of computability theory is a formal

definition of an algorithm.
The Church-Turing thesis asserts that the intuitive notion of algorithm
is captured by the mathematically precise notion of a Turing machine.
This allows us to formally talk about algorithms and prove things about
computations and algorithms.

Sarmad Abbasi (Virtual University)

Theory of Computation

3/1

Computability

Another extremely important theorem is the existence of a universal

TM

Theorem (Turing)
There exists a TM U such that
U(hM, xi) = M(x).
A major part of this proof is the realization that we can encode the
description of a TM. Another important idea was the fact that we can
make a TM that simulates another TM by reading its description.

Sarmad Abbasi (Virtual University)

Theory of Computation

4/1

Computability

We say that a language L is decidable if there exists an TM such that

halts on all inputs and accepts L. This gives us a precise definition of
the notion of algorithms. We realize that that the language accepted by
TM does not seem to be decidable.

Sarmad Abbasi (Virtual University)

Theory of Computation

5/1

Computability

We define
ATM = {hM, xi : M accepts x }
and a closely related problem
AH = {hM, xi : M halts on x }

Sarmad Abbasi (Virtual University)

Theory of Computation

6/1

Computability

We note that
ATM
is a the language accepted by the Universal TM. However, the
universal TM is not a decider. Thus the natural question that arises is
Is
ATM
decidable?
The main technique here was digitalization.

Sarmad Abbasi (Virtual University)

Theory of Computation

7/1

Computability

We proved the amazingly interested theorem of Turing.

Theorem (Turing)
ATM is not decidable.
The amazing thing about this theorem is that it points out to something
which probably would not have crossed our minds. The fact that well
defined computational questions can be undecidable.

Sarmad Abbasi (Virtual University)

Theory of Computation

8/1

Computability

Based on the undecidability of the halting problem. We were able to

prove that several other problems were undecidable. A generic result
called Rices theorem stated that all non-trivial semantic properties of
TMs are undecidable.

Sarmad Abbasi (Virtual University)

Theory of Computation

9/1

Computability

We looked at a very natural question that was undecidable. This

problem seemed to be a puzzle but turned out to be undecidable. That
was the Post Correspondence Problem. Thus problems which
naturally popup can also be undecidable.

Sarmad Abbasi (Virtual University)

Theory of Computation

10 / 1

Computability

We then started with the following puzzle.

Can one write a program that prints itself?

Sarmad Abbasi (Virtual University)

Theory of Computation

11 / 1

Computability

We were able to answer this in the affirmative and not only that we
were able to come up with a technique which told us that
A program can have access to its own description.
This statement was formalized in the recursion theorem.

Sarmad Abbasi (Virtual University)

Theory of Computation

12 / 1

Computability

The recursion theorem becomes a very nice tool to get undecidability

results.

Sarmad Abbasi (Virtual University)

Theory of Computation

13 / 1

Computability

We also looked at problems that arise from mathematical logic. In

particular we looked at the decidability of logical theories. We showed
using a very non-trivial algorithm that Presburger arithmetic is
decidable.

Sarmad Abbasi (Virtual University)

Theory of Computation

14 / 1

Computability

On the other hand we were able to show that Paeno Arithmetic (the
one that we are used to of) is undecidable. This was a shocking and
wonderful result.

Sarmad Abbasi (Virtual University)

Theory of Computation

15 / 1

Computability

We were able to also prove one of the most celebrated theorem in

mathematical logic: The Gdels Theorem. It states that there are true
statements in number theory that are not proveable.
Just understanding the statement of this theorem is a joy in itself!

Sarmad Abbasi (Virtual University)

Theory of Computation

16 / 1

Computability

We then looked at Kolomogrov or Descriptive complexity of strings.

This gave us another fundamental definition. It gave us a definition of
information. We proved several theorems in Kolmogrov complexity. We
defined incompressible strings. Furthermore we showed that most
strings are incompressible.

Sarmad Abbasi (Virtual University)

Theory of Computation

17 / 1

Computability

Lastly we showed that any property that holds for all most all strings
must be false for finitely many strings that are incompressible. Thus
incompressible strings look random from the point of view of any
computable property.

Sarmad Abbasi (Virtual University)

Theory of Computation

18 / 1

Complexity

Then we changed our focus and started studying Complexity Theory.

Thus now we were talking about
1

Problems that are solvable in practice.

Thus we study computable problems and ask what resources are

required to solve these problem.

Sarmad Abbasi (Virtual University)

Theory of Computation

19 / 1

Complexity

We precisely defined time and space complexity. We also noted that

k -tape TMs can be simulated by a 1-tape TM with quadratic delay.

Sarmad Abbasi (Virtual University)

Theory of Computation

20 / 1

Complexity

We defined non-deterministic and deterministic time and space

complexity. This lead us to two naturally defined classes. The class P
and the class NP.

Sarmad Abbasi (Virtual University)

Theory of Computation

21 / 1

Complexity

The class P captures the problem that can be solved in practice.

However, the class NP contains a lot of practical problem that we
would like to solve. We asked the question if

Question
Is P = NP?

Sarmad Abbasi (Virtual University)

Theory of Computation

22 / 1

Complexity

The P vs. NP question is one of the most celebrated open questions in

computer science. It is considered to be a deep question whose
resolution will enhance our understanding of computation in a
fundamental way.

Sarmad Abbasi (Virtual University)

Theory of Computation

23 / 1

Complexity

The Clay mathematical society has declared it as the problem of the

millennium.

Sarmad Abbasi (Virtual University)

Theory of Computation

24 / 1

Complexity

The study of P and NP has lead to the theory of NP-complexness.

When we prove a problem is NP-complete we show that if it can be
solved in polynomial time then all the problems in NP can be solved in
polynomial time. Thus it provides us with hard evidence that the
problem is hard. However, it does not provide us with a proof (at least
not till the P vs. NP conjecture is open).

Sarmad Abbasi (Virtual University)

Theory of Computation

25 / 1

Complexity

One of the most spectacular theorem in this area was the Cook-Levin
theorem.

Theorem (Cook-Levin)
SAT is NP-complete.

Sarmad Abbasi (Virtual University)

Theory of Computation

26 / 1

Complexity

This theorem can be read in two ways.

1

P = NP if and only if SAT P.

SAT in some sense captures the complexity of the entire class NP.

Sarmad Abbasi (Virtual University)

Theory of Computation

27 / 1

Complexity

Using the Cook-Levin theorem we were able to prove that several

other problems are NP-complete. These problems came from graph
theory (independent set, coloring, vertex cover), arithmetic (subset
sums), logic (sat) etc. Thus NP-complete is a rich class of problems.

Sarmad Abbasi (Virtual University)

Theory of Computation

28 / 1

Complexity

We defined non-deterministic and deterministic space complexity

classes. The class PSPACE and the class NPSPACE.

Sarmad Abbasi (Virtual University)

Theory of Computation

29 / 1

Complexity

The class PSPACE captures the problem that can be solved in

reasonable space. The class NPSPACE was a natural
non-deterministic analogue. We asked if

Question
Is PSPACE = NPSPACE?

Sarmad Abbasi (Virtual University)

Theory of Computation

30 / 1

Complexity

We proved

Theorem (Savitch)
PSPACE = NPSPACE.

Sarmad Abbasi (Virtual University)

Theory of Computation

31 / 1

Complexity

The study of P and PSPACE lead to the theory of

PSPACE-complexness. When we prove a problem is
PSPACE-complete we show that if it can be solved in polynomial time
then all the problems in PSPACE can be solved in polynomial time.
Thus it provides us with hard evidence that the problem is hard.
However, it does not provide us with a proof.

Sarmad Abbasi (Virtual University)

Theory of Computation

32 / 1

Complexity

We showed that

Theorem (Cook-Levin)
TQBF is SPACE-complete.

Sarmad Abbasi (Virtual University)

Theory of Computation

33 / 1

Complexity

We then showed that combinatorial games like FORMULA-GAME,

Generalized Geography is PSAPCE complete.

Sarmad Abbasi (Virtual University)

Theory of Computation

34 / 1

Complexity

In the last part of the course we studied the classes L and NL. These
were space complexity classes where the space requirement was
logrithmic. We again proved that path was NL-complete.

Sarmad Abbasi (Virtual University)

Theory of Computation

35 / 1

Complexity

We noticed that Savatichs techinique does not yield a result about L

and NL. However, the big surprise was the last theorem we proved was
NL = co NL
this gave us an example of a non-deterministic complexity class that is
closed under complementarily.

Sarmad Abbasi (Virtual University)

Theory of Computation

36 / 1

Complexity
What more to study? What have we missed? Theory of computation is
a wonderful subject and it is impossible to study it in one course. There
are many exciting things that you can look at in the future. These
include:
1

Randomized Complexity Classes.

The polynomial time hierarchy.

Hardness of Approximation.

Quantum computation and quantum complexity.

Cryptography.

and the list goes on...

Sarmad Abbasi (Virtual University)

Theory of Computation

37 / 1

Complexity

You can read papers in the following conference proceedings.

1

STOC

FOCS

CCC

Sarmad Abbasi (Virtual University)

Theory of Computation

38 / 1

Complexity

A few journals that you may want to look at to see what is going on in
complexity theory are
1

Journal of Computational Complexity

Theoretical Computer Science

Journal of ACM

Sarmad Abbasi (Virtual University)

Theory of Computation

39 / 1