BayesThe Max Planck Institute has set up a new project called the Bayesian Approach to Cognitive Systems, or BACS . Claiming to be inspired by the way real biological brains work, the project aims to create new artificial intelligence systems which can deal with high levels of real-world complexity; navigating through cluttered environments, tracking the fleeting expressions on a human face, and so on. There’s no lack of confidence here; the project’s proponents say they expect it to have a substantial impact on the economy and on society at large as the old dream of effective autonomous robots finally comes true.

It seems remarkable that the work of an eighteenth-century minister should now become such hot stuff technologically. Thomas Bayes is a rather mysterious figure, who published no scientific papers during his lifetime, yet somehow managed to become a member of the Royal Society: he must, I suppose, have been a great talker, or letter-writer. The theorem on which his fame rests, published only after his death, provides a way of calculating conditional probabilities. It gives a rational way of arriving at a probability based purely on what we know, rather than treating probability as an objective feature of the world which we can only assess correctly when we have all the data. If I want to know the odds of my taking out a green sock from my drawer, for example, I would normally want to know what socks were in there to begin with, but Bayes allows me to quantify the probability of green just on the basis of the half-dozen socks I have already pulled out. Philosophically, Bayes is naturally associated with the subjectivist point of view, which says that all probability is really just a matter of our limited knowledge – though given the lack of documentary evidence we can really only guess what his philosophical views might have been.

Universally accepted to begin with, then somewhat neglected, his ideas have been taken up in practical ways in recent years, and are now used by Google, for example.

Why are they so useful in this context? I think one reason is that they might offer a way round the notorious frame problem. This is one of the big barriers to progress with AI: the need to keep updating all your knowledge about the world every time anything changes. The problem as originally conceived was particularly bad because in addition to noting what had changed, you had to generate ‘no change’ statements for all the other items of knowledge about the world in your database. Daniel Dennett and others have reinterpreted this into a much wider philosophical problem about coping with real world knowledge.

Is there a solution? Well, the main reason the frame problem is so bad is because systems which rely on classical logic, on propositional and predicate calculus, cannot tolerate contradictions. They require every proposition in the database to be labelled either true or false: when a new proposition comes up, you have to test out the implications for all the existing ones in order to avoid a contradiction arising. It’s clearly a bad thing to have conflicting propositions in the database, but the problem is made much worse by the fact that in classical logic there is a valid inference from a contradiction to anything (one of the counterintuitive aspects of classical logic): this means any contradiction means complete disaster, with the system authorised to draw any conclusions at all. If you could have a different system, one that tolerated contradictions without falling apart, the problem would be circumvented, and that is why McCarthy, even as he was describing the frame problem for the first time, foresaw that the solution might lie in non-monotonic logics; that is, ones which don’t require that everything be either true or false.

Where can we find a good non-monotonic system? It’s not too difficult to extend normal logic to include a third value, neither true nor false, which we might call uncertain, or neutral: many would say that this is a more realistic model of reality. The snag with a trivalent logical system is that it sacrifices one of the main tactics available, namely that of deducing the falsity of a proposition from the fact that it leads to a contradiction. If there’s always a third possibility, contradictions are hard to derive, and as a result trivalent logics are much less powerful tools for drawing new conclusions than classical logic (which isn’t exactly a powerhouse of new insights itself).

Step forward, Bayesian methods. Now we can allocate a whole range of values to our propositions, representing the likelihood or the level of credence we assign to each. We don’t need to re-evaluate everything when a new piece of information comes up, because head-on contradictions no longer arise; and we’re no longer dealing in formal deductions anyway – instead we can use each new piece of evidence to make a rational adjustment in values. We don’t get an instant conclusion, but what may be better: a gradual refinement of our beliefs whose ratings get more accurate the longer we go on. We can start without much information at all, and still draw reasonable conclusions.

This sounds pretty good, but in addition Bayesian methods are well established in the field of neural networks and there’s some reason to think that this might be one of the ways real human brains work, especially in the case of perception. Rather than performing computations on visual data, it might well be that our brains use a Bayesian encoding, representing, say, the probability that we’re seeing a straight edge at a certain distance from us, and using new data to update the relevant values.

This all seems excitingly plausible so far as basic cognitive functions are concerned, but what about consciousness itself? It seems to me quite a reasonable hypothesis that our opinions and beliefs are held in a Bayesian kind of way – mostly with varying degrees of certainty, and with a degree of tolerance for inconsistency. Changes in what we believe about the world do generally seem to arrive as the result of a relatively gradual accumulation of evidence, rather than through a sudden deductive insight.

But what about our phenomenal experience, the famous ‘hard problem’? Here, as so often before, we seem to run aground a bit. I would have expected Bayesian perceptions to give us a rather cloudy, probabilistic view of the world, but instead we have a pretty clear and distinct one. What colour is the rose?

“Well, I’d say it looks 85% red, but it also looks 5% pink but in a poor light, and 4.5% orange. There’s actually a distinct possibility of picture or model about the rose itself, and I can see an outside chance of hologram.”

It very much isn’t like that, or so it seems to me: our senses present us with a pretty unambiguous world: if there are mists, they are generally external. In all the cases of ambiguous images I can think of, the brain enforces one interpretation at a time: you may be able to switch between seeing the shape as convex or concave, say, but you can’t see it as evens either way.


How much does that matter? We don’t have to solve all the problems of cognitive science at once, and however confident it may be, I don’t think the Max Planck Institute is attempting to. But I wonder why we are given this definite view of the world if the underlying mechanisms deal only in varying levels of probability? It can’t be that way for no reason: we can only assume that there was some survival value in our brains working that way. It might be that this sharp focus is somehow a side-effect of full-blown consciousness, though I can’t imagine why that should be; it might merely be that our perceptual systems are so good there’s generally no point in confusing us with anything but the top probability.

But I think this might be a clue that for reasons which remain obscure, Bayesian methods alone will turn out to be not quite enough, even for some fairly basic examples of cognition. I won’t be standing in the path of any of the early model robots, anyway.

2 Comments

  1. 1. Devine says:

    Consciousness arises from sensory input, but we are not born with a tabula rasa; inherited characteristics are hardwired into the DNA that constructs an organism, including the potential of a given consciousness for ordering sensory input. That is the beginning. Circuits get trained, redundancies are built in, and for most (possibly not so well in autistic consciousnesses) sensory filters enable consciousness to focus to varying degrees to given stimuli. Gender even plays a role in consciousness, or more specifically, the chemical differentials between male and female enable/disable the ability of the consciousness to filter/focus on given stimuli.

    What is most amazing about the human brain is how much processing does not require what we commonly call “conscious thought.” The brain is influenced by a complex and competing network of synapses, but some of the best brains seem quite adept at problem-solving without much concentration or focus being necessary. A good memory relay network can provide just the right word to me right when I need it, synchronizing my intellectual desire with the exact needed memory of a word, one that my more focused and analytical consciousness questions because while it takes note that the word seems to synchromesh with my needs, it cannot define the word exactly. But when I look up the definition, voila! It was exactly right, a neuronal syncopation, creating mental music out of abstemious solo processing. Our brain seems to have a symphony at its disposal, though we rarely require more than a quartet for most of what we do.
    When I encountered this website today–having read Edelman’s latest book last year–it stirred me to write the latter two paragraphs, and looking at them, it is easy to see that AI folks are up against it in trying to duplicate consciousness without any life. I hope to explore what looks to be an excellent and stimulating website–Thanks.

  2. 2. peter.hankins says:

    Thanks, Devine. I’m sure it’s true that a large amount of important stuff is hardwired – though that has the slightly scary implication that we might suffer from blind spots when dealing with novel situations.

    Our ability to pull out the right word (or relevant item in general) is indeed mind-boggling: it looks superficially like a hyper-efficient database system, but I stringly suspect it works on a different basis altogether.

Leave a Reply