
Not
knowably sound

Sir
Roger Penrose is unique in offering something close to a proof
in formal logic that minds are not merely computers. There
is a kind of piquant appeal in an argument against the
power of formal symbolic systems which is itself
clothed largely in formal symbolic terms. Although it is this
'mathematical' argument, based on the famous proof by Gödel of the
incompleteness of arithmetic, which has attracted the
greatest attention, an important part of Penrose's theory is
provided by positive speculations about how consciousness might
really work. He thinks that consciousness may depend on a new
kind of quantum physics which we don't, as yet, have a theory for,
and suggests that the microtubules within brain cells might be the
place where the crucial events take place. I think it must be
admitted that his negative case against computationalism
is much stronger than these positive
theories.
Besides the direct arguments about
consciousness, Penrose's two books on the subject feature excellent
and highly readable passages on fractals, tiling the plane, and
many other topics. At times, it must be admitted, the relevance of
some of these digressions is not obvious  I'm still not convinced
that the Mandelbrot Set has anything to do with consciousness, for
example  but they are all fascinating and remarkably lucid pieces
in their own right. 'The Emperor's New Mind' is
particularly wideranging, and would be well worth reading even if
you weren't especially interested in consciousness, while a large
part of 'Shadows of the Mind' is
somewhat harder going, and focuses on a particular argument
which purports to establish that "Human mathematicians are not
using a knowably sound algorithm in order to ascertain mathematical
truth". 

I like the books
myself, mostly, but I don't find them convincing. Of course, people
find a lengthy formal argument intimidating, especially from someone
of Penrose's acknowledged eminence. But does anyone seriously think
this kind of highly abstract reasoning can tell us anything real
about how things actually work?


You don't think maths
tells us anything about the real world then? Well, let's start
with the Gödelian argument, anyway. Gödel proved the incompleteness
of arithmetic, that is, that there are true statements in arithmetic
which can never be proved arithmetically. Actually, the proof goes
much wider than that. He provides a way of generating a statement,
in any formal algebraic system, which we can see is true, but which
cannot be proved within the system. Penrose's point is that any
mechanical, algorithmic, process is based on a formal system of some
kind. So there will always be some truths that computers can't
prove  but which human beings can see are true! So human
thought can't be just the running of an
algorithm.


These unprovable truths
are completely uninteresting ones, of course: the sort of thing
Gödel produces are arid selfreferential statements of no wider
relevance. But in any case, the doctrine that people can always
see the truth of any such Gödel statement is a mere
assertion. In the simple cases Penrose considers, of course human
beings can see the truth of the statements, but there's no proof
that the same goes for more complex ones. If we actually defined the
formal system which brains are running on, I believe we might well
find that the Gödel statement for that system really was
beyond the power of brains to grasp.


I don't think that that could ever happen 
it just doesn't work like that. The complexity of the system in
question isn't really a factor. And in any case, brains are not
'running on' formal systems! 

Oh, but they
have to be! I'm not suggesting the 'program'
for any given brain is simple, but I can see three ways we
could in principle construct it.
One. If
we list all the sensory impressions and all the instructions to act
that go into or out of a brain during a lifetime, we can treat them
as inputs and outputs. Now there just must be some function, some
algorithm, which produces exactly those outputs for those
inputs. If nothing simpler is available (I'm sure it would be)
there is always the algorithm which just lists the inputs to date
and says 'given these inputs, give this
output'.
Two. If you don't like that
approach, I reckon the way neurons work is sufficiently clear for us
to construct a complete neuronal model of a brain (in principle 
I'm not saying it's a practical proposition); and then that would
clearly represent an implementation of a complex function for
the person in question.
Three. As a last resort, we just model
the whole brain in excruciating detail. It's a physical object, and
obeys the normal laws of physics, so we can construct a mechanical
description of how it works.
Any of these will do. The algorithms we
come up with might well be huge and unwieldy, but they
exist, which is all that matters. So we must be able to apply
Gödel to people, too. 

Nonsense! For a start, I don't believe
'inputs' and 'outputs' to human beings can be defined in those terms
 reality is not digital. But the whole notion of a person's
own algorithm is absurd! The point about computers is that their
algorithms are defined by a programmer and kept in a
recognised place, clearly distinguished from data, inputs, and
hardware, so it's easy to say what they are in advance. With a
brain, there is nothing you can point to in advance as the 'brain
algorithm'. If you insist on interpreting the brain as running an
algorithm, you just have to wait and see which bits of the brain and
which bits of the rest of the person and their environment turn
out to be relevant to their 'outputs' in what ways and
then construct the algorithm to suit. We
can never know what the total algorithm is until all the inputs and
outputs have been dealt with. In short, it turns out not to be
surprising that a person can't see the truth of their own Gödel
statement, because they have to dead before anyone can even decide
what it is!


Alright, well look at it this
way. We're only talking about things that can't be proved
within a particular formal system. Humans can see the truth
of these statements, and even prove them, because they go outside
the formal system to do so. There's no real reason why a computer
can't do the same. It may operate one algorithm to begin with, but
it can learn and develop more comprehensive algorithms for itself as
it goes. Why not? 

That's the whole point! Human beings can always find a new
way of looking at something, but an algorithm can't. You can't have
an algorithm which generates new algorithms for itself, because if
it did, the new bits would by definition be part of the original
algorithm. 

I think it
must be clear to anyone by now that you're just playing with
words. I still say that all this is simply too esoteric to
have any bearing on what is essentially a practical computing problem. If I
understand them correctly, both Dennett and your friend Searle agree
with me (in their different ways). The algorithms in practical AI applications
aren't about mathematical proof, they're about doing stuff. 

I was puzzled by Dennett's argument in
'Darwin's dangerous idea' in particular.
He's quite dismissive about the whole thing, but what he seems to
say is this. The narrow set of algorithms picked out by Penrose may
not be able to provide an arithmetical proof, but what about
all the others which Penrose has excluded from consideration? This
is strange, because the ones excluded from consideration, according
to Dennett, are: algorithms which don't do anything at all;
algorithms which aren't interesting; algorithms which aren't
about arithmetic; algorithms which don't produce proofs; and
algorithms which aren't consistent! Can we reasonably
expect proofs from any of these? Maybe not, says Dennett, but
some of them might play a good game of chess... This seems to miss
the point to me.
What I fear is that
this kind of reasoning leads to what I call the Roboteer's argument
(I've seen it put forward by people like Kevin Warwick and Rodney
Brooks). The Roboteer says, OK, so computers will never
work the way the human brain works. So what? That doesn't mean
they can't be intelligent and it doesn't mean they can't be
conscious. Planes don't fly the way birds do, but we don't say it
isn't proper flight because of that... 

Personally, I don't see anything wrong with that
argument. What about this quantum malarkey? You're not going to tell
me you go along with that? There is absolutely no reason to think
quantum physics has anything to do with this. It may be hard to
understand, but it's just as calculable and deterministic as any
other kind of physics. All there really is to this is that both
consciousness and quantum physics seem a bit
spooky.


It
isn't conventional, established quantum physics we're talking about.
Having established that human thought goes beyond the algorithmic,
Penrose needs to find a noncomputable process which can account for
it; but he doesn't see anything in normal physics which fits the
bill. He wants the explanation to be part of physics  you ought to
sympathise with that  so it has to be in a new physical theory, and
new quantum physics is the best candidate. Further strength is
given to the case by the ideas Stuart Hameroff and he have come up
with about how it might actually work, using the microtubules which
are present in the structure of nerve cells. 

They're present in most other kinds of cell, too,
if I understand correctly. Microtubules have perfectly ordinary jobs
to do within cells which have nothing to do with thinking. We don't
understand the brain completely, but surely we know by now that
neurons are the things that do the basic
work.


It
isn't quite as clear as that. There has been a tendency, right since
the famous McCulloch and Pitts paper of 1947, to see neurons as
simple switches, but the more we know about them the less plausible
that seems. Actually there is some highly complex chemistry
involved. Personally, I would also say that the way neurons are
organised looks very much like the sort of thing you might construct
if you wanted to catch and amplify the effects of very smallscale
events. One molecule  in the eye, one quantum, as Penrose points
out  can make a neuron fire, and that can lead to a whole chain of
other firings. 

At the end of the day,
the problem is that quantum physics just doesn't help. It
doesn't give us any explanatory resources we couldn't get
from normal physics.


That's too sweeping. There are actually several reasons,
in my view, to think that quantum physics might be relevant to
consciousness (although these are not Penrose's reasons). One is
that the way two different states of affairs can apparently be held
in suspense resembles the way two different courses of action can be
suspended in the mind during the act of choice. A related point is
the possibility that exploiting this kind of suspension could give
us spectacularly fast computing, which might explain some of the
remarkable properties of the brain. Another is the special role of
observation  becoming conscious of things  in causing the
collapse of the wavefunction. A third is that quantum physics puts
some limits on how precisely we can specify the details of the
world, which seems to militate against the kind of argument you were
making earlier, about modelling the brain in total detail. I know
all of these are open to strong objections: the
real reason, as I've already said, is just that quantum physics
is the most likely place to find the kind of new science which
Penrose thinks is needed.


I don't
see it. It seems to me inevitable that any new physics that may come
along is going to be amenable to simulation on a computer  if it
wasn't, it hardly seems possible it could be clear enough to be a
reasonable theory. 

In
other words, your mind is closed to any possibility except
computationalism. Consciousness seems to me to be such an important
phenomenon that I simply cannot believe it is something just
'accidentally' conjured up by a complicated
computation... 

Read:

"The Emperor's
New Mind"
An
essential text on the consciousness question, and contains a
great deal of fascinating stuff. The foreword is by the
late great Martin Gardner, and fans of his will find a similar
combination of deep thought with clear and entertaining
exposition here. 
"Shadows of the Mind "
Similar
ground is covered here, although in this case the details of
Penrose's argument about the nonalgorithmic nature of
mathematical thought is uppermost, and hence the going is just
a little harder. Still a very readable treatment of subjects
which inevitably require a degree of thought and
concentration to assimilate. 

Some
Links:

Biographical
details
 from the University
of St Andrews site 
Critical paper and response
 a critical
paper by Patricia Churchland with a response from Penrose and
Hameroff. 
Psyche  Symposium on 'Shadows of the
Mind' in the online
journal.   