9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

Share

0 More

NextBlog

dpiponi@gmail.com NewPost Design SignOut

ANeighborhoodofInfinity
Saturday,October06,2007

Howcanitpossiblybethatwecanexhaustivelysearchan infinitespaceinafinitetime?
There has been a bit of talk recently about how it is possible to exhaustively search certain infinite spaces in a finite time. Unfortunately the explanations seem to be couched in topological language making them a little inaccessible. So I thought it'd be good to attempt a rigorous, but elementary, proof so that is possible to see exactly where the 'magic' happens. Consider the type of infinite streams of booleans:
t y p eN a t u r a l=I n t e g e r S t r e a m: :N a t u r a l>B o o l

Can we write a function s e a r c h , that takes as argument a boolean predicate on S t r e a m , and tells us whether or not there is a stream that satisfies it?
s e a r c h: :( S t r e a m>B o o l )>B o o l

This sounds like an impossible task. There are uncountable many infinite streams of booleans, so how can we hope to search them all? There's one important thing in our favour: we're restricting ourselves to predicates that are computable and total. In other words, whatever stream we pass to the predicate, they have to return a boolean in a finite time. First, let's deal with an objection that make s e a r c hseem impossible to implement. Given a 4tuple of integers, we can encode these integers in a stream. For example, the integers (written in
blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html 1/11

9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

binary) (11,10,1,111) could be encoded as 11110011100011001111110100000000000000000... where we code the binary digit b as 1b, we separate the integers using 00 and terminate the sequence using 01. We could then write a predicate that checks to see if the 4-tuple gives a solution to xn+yn=zn with x,y,z>0 and n>=2. Plugging that into s e a r c hwe apparently have a way to decide the Fermat-Wiles theorem without doing any difficult number theory. Surely, therefore, s e a r c hcan't exist? But the predicate we've just described isn't total. Suppose we feed it the string 111111... as input. It's going to think that these represent an x with leading digits 111... and it's going to sit there collecting digits forever. It's never going to terminate. So the requirement that the predicate be total is such a strong condition that we can't actually implement the Fermat predicate, even though it uses nothing more than simple arithmetic. So maybe s e a r c his beginning to look a little more plausible. Here's the key thing: any predicate on boolean streams must terminate after a finite time, meaning it only reads a finite number of booleans. So given any predicate p, there is a (mathematical) function dp on the boolean streams with the property that dp(s) is the index of the last bit p looks at when given s as input. dp(s) is obviously always finite. Now at this point you might immediately jump to the opposite conclusion that of course s e a r c h is possible because you only need to check a finite number of bits. But even though dp(s) is always finite, it might be unbounded as s varies over the infinite set of streams. So the finiteness of dp on its own is not enough to bound our search. For example, consider predicates on 4tuples of integers again. Each of these predicates reads only a finite number of bits, and yet we don't expect a general purpose terminating search algorithm can tell us whether or not Fermat's equation has a solution. So to prove termination for the boolean stream case we need to exploit some more structure. Here's a first hint at what that structure might be. Suppose we try to make dp unbounded by arranging that

blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html

2/11

9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

dp(00000....) = 1 dp(10000....) = 2 dp(01000....) = 3 dp(11000....) = 4 and so on. You might notice that we immediately have a contradiction. If dp(00000...)=1 then the predicate must stop reading bits after it has read an initial 0. So actually, the sequence dp(01000...) is also 1. Lots of streams share prefixes in this way, so the question is, does the infinitude of the set of boolean streams win out, or does the amount of sharing of prefixes eventually tame the infinity? Consider the following infinite binary tree:

Each boolean stream corresponds to a single downward path through this tree. Fix a predicate p. Suppose, then, that this predicate, when fed the stream 011..., requires only 3 bits to be read, ie. dp(011...) = 3. Then we can cut off everything below that point on the tree because no
blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html 3/11

9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

predicate need read that far. We've removed every infinite path passing through 011:

Every stream s has a finite cutoff after dp(s) steps. And after we've hacked away all of the bits that our predicate never looks at what we're left with is a finitely branching tree with no infinite paths. Now comes the magic: we invoke Knig's Lemma which says "every tree that contains infinitely many vertices, each having finite degree, has at least one infinite simple path.". Conversely, if there are no infinite paths, the tree must be finite. In other words, for any predicate p we only need to search a finite tree. The rest is just details, such as the precise method by which we determine when we've finished searching. In the case of Martn Escard's paper the details are some of the most ingenious coding I've ever seen. Update: I fixed the type of s e a r c habove. Thanks to Cale for pointing this out. Unfortunately I deleted his comment by accident so I'm crediting him here. Posted by Dan Piponi at Saturday, October 06, 2007

15comments: dthurstonsaid...
blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html 4/11

9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

You wrote: Buteventhoughdp(s)isalwaysfinite,itmightbeunboundedassvariesovertheinfinitesetofstreams. Although you come to the right conclusion in the end, what you wrote above is not true: in fact you end up proving that for a given predicate there is a definite depth that you don't need to look beyond. Saturday, 06 October, 2007 sigfpesaid... dthurston, That's an epistemic "might". Ie. "might" = "I do not yet know that it is impossible that" rather than "there exists an example such that". Saturday, 06 October, 2007 Anonymoussaid... Thanks for the article, it was very informative. As an unrelated question, may I ask what program you used to produce those binary trees? Monday, 08 October, 2007 sigfpesaid... The trees were drawn with Inkscape. To misquote Chruchill, Inkscape is the worst free drawing application, apart from all the rest. Tuesday, 09 October, 2007 Anonymoussaid... Fascinating stuff. And thanks for your explanation, I can't follow that paper.
blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html 5/11

9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

I'm having a little trouble while exploring this problem more deeply. I "know" that there must not be a total function from infinite sequences of booleans onto the integers. I have devised this proof: If there were, then we could construct a function from booleans into booleans answering the question "such and such Turing machine halts in n steps", and then we could use the algorithm you just presented to solve the halting problem. But that doesn't give me any intuition for any deeper reasons. Could you explain a little more why there's no way to construct a total function from infinite sequences of booleans onto the integers? Thanks, Luke http://luqui.org/blog Sunday, 21 October, 2007 Anonymoussaid... It's interesting to note that one can't implement "search" as a pure recursive function of the type "(Stream > Bool) -> Bool". The problem is that its argument is a "black box", an opaque predicate. Although we know that the search tree (as described in the post) is finite, we don't know what it is and so we can't search it in finite time. However, you can implement "search", if you allow side effects, e.g. using "unsafePerformIO". This is left as an exercise for the reader :) I suspect that this "inevitability of impurity" is not accidental. Friday, 07 December, 2007 sigfpesaid... Anonymous, See here for a pure implementation: http://conway.rutgers.edu/~ccshan/wiki/blog/posts/Exhaustive/
blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html 6/11

9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

Friday, 07 December, 2007 Anonymoussaid... It seems that the complexity of your "search" program is not computable because the proof of its termination is (necessarily) not constructive. The proof tells you that the program will stop but it can't tell you when it will stop. So you believe that the program will terminate in all cases but in some cases you would never see the end of its execution (even if you were an eternal beeing): each day you could think that the program will stop tomorrow. There is no difference between the "search" program and any looping program with an undecidable termination. However, if the predicates eated by "search" are primitive recursive for example, then the complexity of "search" can probably be bounded by an Ackermann-like function (the predicates given as examples here are primitive recursive, although there is no stream in primitive recursion :o). Wednesday, 12 December, 2007 sigfpesaid... > There is no difference between the "search" program and any looping program with an undecidable termination. Not so. For any input, this program is guaranteed to terminate. For any input, there is a time t after which you will get a 'yes' or a 'no'. This is different from, say, doing a brute force search for a solution to Fermat's last equation. There is no time t at which such a loop would terminate and say "no solution". It can only answer "yes, here's a solution" or loop indefinitely. These are distinct classes of program. Wednesday, 12 December, 2007 Anonymoussaid... > For any input, this program is guaranteed to terminate. For any input, there is a time t after which you will get a 'yes' or a 'no'. Your proof says that "this program is guaranteed to terminate". It is provable, but is it true in the real world? Can you compute this t? In some metasystems of classical logic there is some decidable properties
blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html 7/11

9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

A(N) on integers such that: whe can prove "not(A(0))", "not(A(1))", "not(A(2))", "not(A(3))"... for all integers, and we can also prove "there is N such that A(N)", without incoherence. Despite the apparent contradiction, it's all coherent. Intuitively, we might chech the property A(N) for all integers to reach a contradiction, but no one can check an infinity of properties, even an eternal beeing, so we cannot reach a contradiction. Such a property A(N) is similar to the property "the program (search p) ends after N steps" with a well chosen p. This kind of example is similar to the coherent system S, defined by Gdel, which says that "there is an incoherent subsystem of S". So the "search" program is not garanteed to terminate, but we can believe in its termination without fear of contradiction, even if it's false in the real world. We don't have such paradoxes with constructive proofs because each proof generates a witness of reality: if we can prove that "there is a N such that A(N)", then we can prove "A(N)" for some N. This property does not hold in classical systems (because of the Knig's lemma in the case of "seach"). Thursday, 13 December, 2007 sigfpesaid... Anonymous, Reminds me of Goodstein's theorem. I think we're members of different mathematical cults. In my cult, an axiom system in which "not(A(0))", "not(A(1))", "not(A(2))", "not(A(3))"... for all integers, and we can also prove "there is N such that A(N)" is an interesting curiosity, but not an axiom system I'd use to describe the real world. :-) Thursday, 13 December, 2007 Anonymoussaid... Unfortunately, the pathological "A(N)" is not a vacuous statement: it states that the "search" program will terminate. It is more vicious than the Goodstein's theorem because it remains pathological in all coherent
blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html 8/11

9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

classical systems (the pathology arises because the Kolmogorov complexity of the predicates is not bounded). Moreover, the axiom system in which "not(A(0))", "not(A(1))", "not(A(2))", "not(A(3))"... and "there is N such that A(N)" are true is much more than a formal curiosity: it's a model of your axiomatic. That means it could be the true essence of the real world. Nobody can choose what is the real world and how it works. This is a valid instance of the real world: you coherently prove that "search" terminates in all cases and "search p" coherently loops with infinitely many p. It's even worse: it's validity is a consequence of your axiomatic, so you necessarily use it to describe the real world. This incongruity exists in all (big enough) classical systems, in all axiom systems you might believe in: there is no way to circumvent it... except being constructive. There is no cult in constructions. Constructions are facts. Classical statements are beliefs. You can believe in a model, but it's not logic, the "good" model is beyond logic (it's not based on a recursive axiomatic), it's just a sweat dream (and we can prove that). Friday, 14 December, 2007 sigfpesaid... Anonymous. Can you suggests some books on the flavour of mathematics you're espousing? It's very interesting. Sunday, 16 December, 2007 tertiumsaid... Re: pure implementation of "search" Oh, I must tie that knot on my too quick tongue. Thank you, Dan! Monday, 17 December, 2007 tertiumsaid... Anonymous, correct me if I'm wrong, but in a constructive setting shouldn't the argument of "search" be (the code of) a recursive function together with a constructive proof of its totality? Then, I guess, we can prove the termination of "search" in a constructive way, too.
blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html 9/11

9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

Monday, 17 December, 2007 Post a Comment Linkstothispost Create a Link Newer Post Subscribe to: Post Comments (Atom) Home Older Post

BlogArchive
2013 (2) 2012 (8) 2011 (13) 2010 (20) 2009 (21) 2008 (35) 2007 (37) December (1) November (5) October (2) Using Thermonuclear Pennies to Embed Complex Numbe... How can it possibly be that we can exhaustively se... September (3) July (3) June (2) May (1) April (3)
blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html 10/11

9/27/13

A Neighborhood of Infinity: How can it possibly be that we can exhaustively search an infinite space in a finite time?

March (7) February (6) January (4) 2006 (92) 2005 (53)

SomeLinks
The Comonad.Reader Rubrication Richard Borcherds: Mathematics and physics The n-Category Cafe Ars Mathematica

AboutMe
Dan Piponi
Follow

1.5k

Blog: A Neighborhood of Infinity Code: Github Twitter: sigfpe Home page: www.sigfpe.com View my complete profile

blog.sigfpe.com/2007/10/how-can-it-possibly-be-that-we-can.html

11/11