Brittle

20 February 2005

A recent paper (pdf format) by Michael L Anderson and Donald R Perlis offers a new solution to the problem of brittleness.  Brittleness, in a robot or program, is an inability to cope with unexpected developments, and the paper, no doubt rightly, says it is arguably the most important problem in Artificial Intelligence, perhaps in computer systems generally. Anderson and Perlis quote the example of a contestant in the DARPA Grand Challenge, in which robots had to negotiate their way through a real-world journey. The hapless specimen in question drove into a fence it could not see, and then continued attempting to move forward for the rest of the contest. Perhaps better sensors would have helped, but sullenly persisting in a course which isn't getting anywhere is poor behaviour for a supposedly intelligent machine. The proposed solution is the Metacognitive Loop; roughly speaking, a mechanism whereby the robot or program can notice whether or not it is achieving its goals. If not, it can then try something else, whether a carefully considered Plan B, or more or less random variations in behaviour, until it gets better results. 

Clearly what we are dealing with here is our old friend (or enemy) the frame problem: in fact, although they never use that phrase, Anderson and Perlis cite the paper by McCarthy and Hayes which first clearly identified the problem, and in some ways their work is in harmony with McCarthy's view that the problem would probably be resolved by new (non-monotonic) approaches to logic. 

It must be acknowledged that on the face of it the AI fraternity frequently seem to have the more reasonable view. Some philosophers present such a gloomy case that we seem almost obliged to disbelieve in human consciousness, never mind the artificial variety; and approaches like that of Anderson and Perlis seem extremely reasonable and pragmatic.

They identify three distinct problems with 'common sense' reasoning. One is 'slippage' - the facts don't hold still over time, but are liable to change while you are reasoning about them, or clinging to outdated conclusions. The second is the 'KR mismatch' problem in which the representational conventions used impose unhelpful (but unavoidable) limitations on the ability to deal with circumstances which don't match them - especially those which give rise to novel expressions not catered for in the original scheme. The third issue is how to cope with contradictions. Contradictions in the system's data are inevitable, but making decisions in their prescence is obviously problematic, especially for a system which operates by classical rules of inference, since in classical logic there is a valid inference from a contradiction to anything at all.

There are two steps in the proposed solution. First, the paper advocates 'active logic'. In this approach, instead of logical inferences taking place instantly (or in an eternal Platonic realm), they follow a chronological sequence. The conclusion comes a moment after the premises, not at the same moment. This has two benign consequences. The system is not obliged to consider all the implications of every piece of data - surely an infinite task in principle anyway - and it can tolerate contradictory premises because it may not happen to be thinking about the two sides of the contradiction at any given moment. Where there is a real problem, the system will eventually derive a direct contradiction, which then serves as a useful warning that something is going wrong.

This part of the argument seems dubious. In the first place, if contradictions are signs of a problem, allowing them to remain unresolved in the heart of the system is surely dangerous; by the time a direct, explicit contradiction is noticed, the damage may have been done. Second, is it actually safe to assume that a direct contradiction is necessarily a problem? In ordinary thought that may be so, but in formal logic contradictions are a useful if not indispensable part of the process of reasoning: without them many proofs are going to be hard to derive.

The second step is the metacognitive loop, which seems unassailably sensible. The basic architecture is composed of two modules, one training the other, with a third overseeing both. This third module is what does the monitoring of performance against expectation, and steps in with new strategies and retraining where necessary.

The authors propose three potential projects which the metacognitive loop might enable: a robotic St Bernard, able to render basic assistance in unpredictable emergency situations; a Mars explorer, which might be able to cope well with finding things its designers had not expected; and finally a GePurS - General Purpose Scout, designed to operate in many different environments, with a large reservoir of experience, much of it redundant in any given situation, but lending a particular flexibility and generality to its cognitive abilities. Indeed, such a creature would surely be intelligent on an entirely new level, comparable with sophisticated animals or even human beings. They don't, of course, predict the imminent achievement of such high-grade performance.