Neural networks really seem to be going places recently. Last time I mentioned their use in sophisticated translation software, but they’re also steaming ahead with new successes in recognition of visual images. Recently there was a claim from MIT that the latest systems were catching up with primate brains at last. Also from MIT (also via MLU) though, has come an intriguing study into what we could call optical illusions for robots, which cause the systems to make mistakes which are incomprehensible to us primates. The graphics in the grid on the right apparently look like a selection of digits between one and six in the eyes of these recognition systems. Nobody really knows why, because of course neural networks are trained, not programmed, and develop their own inscrutable methods.

How then, if we don’t understand, could we ever create such illusions? Optical illusions for human beings exploit known methods of visual analysis used by the brain, but if we don’t know what method a neural network is using, we seem to be stymied. What the research team did is use one of their systems in reverse, getting it to create images instead of analysing them. These were then evaluated by a similar system and refined through several iterations until they were accepted with a very high level of certainty.

This seems quite peculiar and the first impression is that it rather seriously undermines our faith in the reliability of neural network systems. However, there’s one important caveat to take into account: the networks in question are ‘used to’ dealing with images in which the crucial part to be identified is small in relation to the whole. They are happy ignoring almost all of the image. So to achieve a fair comparison with human recognition we should perhaps think of the question being not ‘do these look like numbers to you?’ and more like ‘can you find one of the digits from one to six hidden somewhere in this image?’. On that basis the results seem easier to understand.

There still seem to be some interesting implications, though. The first is that, as with language, AI systems are achieving success with methods that do not much resemble those used by the human brain. There’s an irony in this happening with neural networks, because in the old dispute between GOFAI and networks it was the network people who were trying to follow a biological design, at least in outline.  The opposition wanted to treat cognition as a pure engineering problem; define what we need, identify the best way to deliver it, and don’t worry about copying the brain. This is the school of thought that likes to point out that we didn’t achieve flight be making machines with flapping, feathery wings. Early network theory, going right back to McCulloch and Pitts, held that we were better off designing something that looked at least broadly like the neurons in the brain. In fact, of course, the resemblance has never been that close, and the focus has generally been more on results than on replicating the structures and systems of biological brains; you could argue that modern neural networks are no more like the brain than fixed-wing aircraft are to birds (or bats).  At any rate, the prospect of equalling human performance without doing it the human way raises the same nightmare scenario I was talking about last time; robots that are not people but get treated as if they were (and perhaps people being treated like machines as a consequence.

A second issue is whether the deception which these systems fall into points to a general weakness. Could it be that these systems work very well when dealing with ‘ordinary’ images but continue go wildly off the rails when faced with certain kinds of unusual ones – even when being pout to practical use? It’s perhaps not very likely that  system is going to run into the kind of truly bizarre image we seem to be dealing with, but a more realistic concern might be the potential scope for sabotage or subversion on the part of some malefactor.  One safeguard against this possibility is that the images in question were designed by, as it were, sister systems, ones that worked pretty much the same way and presumably shared the same quirks. Without owning one of these systems yourself it might be difficult to devise illusions that worked – unless perhaps there are general illusions that all network systems are more or less equally likely to be fooled by? That doesn’t seem very likely, but it might be an interesting research project.  The other safeguard is that these systems are not likely to be used without some additional safeguards, perhaps even more contextual processing of broadly the kind that the human mind obviously brings to the task.

The third question is – what is it like to be an AI deceived by an illusion? There’s no reason to think that these machines have subjective experience – unless you’re one of those who is prepared to grant a dim glow of awareness to quite simple machines – but what if some cyborg with a human brain, or a future conscious robot, had systems like these as part of its processing apparatus rather than the ones provided by the human brain?  It’s not implausible that the immense plasticity of the human brain would allow the inputs to be translated into normal visual experience, or something like it.  On the whole I think this is the most likely result, although there might be quirks or deficits (or hey, enhancements, why not) in the visual experience.  The second possibility is that the experience would be completely weird and inexpressible and although the cyborg/robot would be able to negotiate the world just fine, its experience would be like nothing we’ve ever had, perhaps like nothing we can imagine.

The third possibility is that it would be like nothing. There would be no experience as such; the data and the knowledge about the surroundings would appear in the cyborg/human’s brain but there would be nothing it was like for that to happen.  This is the answer qualophile scpetice would expect for a pure robot brain, but the cyborg is more worrying. Human beings are supposed to experience qualia, but when do they arise? Is it only after all the visual processing has been done – when the data ariive in the ‘Cartesian Theatre’ which Dennett has often told us does not exist? Is it, instead, in the visual processing modules or at the visual processing stage? If so, then we were wrong to doubt that MIT’s systems are not having experiences. Perhaps the cyborg gets flawed or partial qualia – but what would that even mean..?