Explaining to humans

We’ve discussed a few times recently how deep learning systems are sometimes inscrutable. They work out how to do things for themselves, and there may be no way to tell what method they are using. I’ve suggested that it’s worse than that; they are likely to be using methods which, though sound, are in principle incomprehensible to human beings. These unknowable methods may well be a very valuable addition to our stock of tools, but without understanding them we can’t be sure they won’t fail; and the worrying thing about that is that unlike humans, who often make small or recoverable mistakes, these systems have a way of failing that is sudden, unforgiving, and catastrophic.

So I was interested to see this short interview with Been Kim, who works for Google Brain (is there a Google Eyes?) has the interesting job of building a system that can explain to humans what the neural networks are up to.

That ought to be impossible, you’d think. If you use the same deep learning techniques to generate the explainer, you won’t be able to understand how it works, and the same worries will recur on another level. The other approach is essentially the old-fashioned one of directly writing an algorithm to do the job; but to write an explicit algorithm to explain system X you surely need to understand system X to begin with?

I suspect there’s an interesting general problem here about the transmission of understanding. Perhaps data can be transferred, but understanding just has to happen, and sometimes, no matter how helpfully the relevant information is transmitted, understanding just fails to occur (I feel teachers may empathise with this).

Suppose the explainer is in place; how do we know that its explanations are correct? We can see over time whether it successfully picks out systems that work reliably and those that are going to fail, but that is only ever going to be a provisional answer. The risk of sudden unexpected failure remains. For real certainty, the only way is for us to validate it by understanding it, and that is off the table.

So is Been Kim wasting her time on a quixotic project – perhaps one that Google has cynically created to reassure the public and politicians while knowing the goal is unattainable? No; her goal is actually a more modest, negative one. Her interpreter is not meant to provide an assurance that a given system is definitely reliable; rather, it is supposed to pick out one’s that are definitely dodgy; and this is much more practical. After all, we may not always understand how a given system executes a particular complex task, but we do know in general how neural networks and deep learning work. We know that the output decisions come from factors in the input data, and the interpreter ought to be able to tell us what factors are being taken into account. Then, using the unique human capacity to identify relevance, we may be able to spot some duds – cases where the system is using a variable that tracks the relevant stuff only unreliable, or where there was some unnoticed problem with the corpus of examples the system learnt from.

Is that OK? Well, in principle there’s the further risk that the system is actually cleverer than we realise; that it is using features (perhaps very complex ones) that actually work fine, but which we’re too dim to grasp. Our best reassurance here is again understanding; if we can see how things seem to be working, we have to be very unlucky to hit a system which is actually superior but just happens, in all the examined cases, to look like a dodgy one. We may not always understand the system, but if we understand something that’s going wrong, we’re probably on firm ground.

Of course, weeding out egregiously unreliable systems does not solve the basic problem of efficient but inscrutable systems. Without accusing Google of a cunning sleight of hand after all, I can well imagine that the legislators and bureaucrats who are gearing up to make rules about this issue might mistake interpreter systems like Kim’s for a solution, require them in all cases, and assume that the job is done and dusted…

Entangled Consciousness

Entangled

Could the answer be quantum physics after all? Various people have suggested that the mystery of consciousness might turn out to be explicable in terms of quantum physics; most notably we have the theory championed ny Roger Penrose and Stuart Hameroff, which suggests that as-yet unknown quantum mechanics might be going on in the microtubules of neural cells. Atai Barkai has kindly shared with me his draft paper which sets out a different take on the matter.

Barkai’s target is subjective experience, or what he defines as the consciousness instance. We may be conscious of things in the world out there, but that kind of consciousness always involves an element of inference, explicit or not; the consciousness instance is free of those uncertainties, consisting only of the direct experience of which we can be certain. Some hardy sceptics would deny that there is anything in consciousness that we can be that sure of, but I think it is clear enough what distinction Barkai is making and that it is, as it were, first order experience he is aiming at.

The paper puts a complex case very carefully, but flinging caution to the winds I believe the gist goes like this. Typically, those who see a problem with qualia and subjective experience think it lies outside the account given by physics, which arguably therefore needs some extension. Barkai agrees that subjectivity is outside the classical account and that a more capacious concept of the universe is needed to accommodate it; but he thinks that quantum entanglement can already, properly read, provide what is needed.

It’s true that philosophical discussions generally either treat classical physics as being the whole subject or ignore any implications that might arise from the less intuitive aspects of quantum physics. Besides entanglement, Barkai discusses free will and its possible connection with the indeterminism of quantum physics. If there really is indeterminism in quantum physics; on this and other points I simply don’t have a clear enough grasp of the science to tackle the questions effectively (better informed comments would be welcome).

Entanglement, as I understand it, means that the states of two particles (or in principle, larger bodies) may be strongly connected in ways that do not depend on normal classical interaction and are not affected by distance. This means that information can in theory be transferred any distance, instantly and infallibly, which opens up the theoretical possibility of quantum computing, delivering the instant solution to computable problems of any finite size. Whether the brain might be doing this kind of thing is a question which Barkai leaves as an interesting speculation.

The main problem for me in the idea that entanglement explains subjectivity is understanding intuitively how that could be so. Entanglement seems to offer the possibility that there might be more information in our mental operations than classical physics could account for; that feels helpful, but does it explain the qualitative difference between mere dead data and the ‘something it is like’ of subjectivity? I don’t reject the idea, but I don’t think I fully grasp it either.

There is also a more practical objection. Quantum interactions are relevant at a microscopic level, but as we ascend to normal scales, it is generally assumed that wave forms collapse and we slip quickly back into a world where classical physics reigns. It is generally thought that neurons, let alone the whole brain, are just too big and hot for quantum interactions to have any relevance. This is why Penrose looks for new quantum mechanics in tiny microtubules rather than resting on what known quantum mechanics can provide.

As I say, I’m not really competent to assess these issues, but there is a neatness and a surprising novelty about Barkai’s take that suggests it merits further discussion.

Unconscious and Conscious

What if consciousness is just a product of our non-conscious brain, ask Peter Halligan and David A Oakley? But how could it be anything else? If we take consciousness to be a product of brain processes at all, it can only come from non-conscious ones. If consciousness had to come from processes that were themselves already conscious, we should have a bit of a problem on our hands. Granted, it is in one sense remarkable that the vivid light of conscious experience comes from stuff that is at some level essentially just simple physical matter – it might even be that incredulity over that is an important motivator for panpsychists, who find it easier to believe that everything is at least a bit conscious. But most of us take that gap to be the evident core fact that we’re mainly trying to explain when we debate consciousness.

Of course, Halligan and Oakley mean something more than that. Their real point is a sceptical, epiphenomenalist one; that is, they believe consciousness is ineffective. All the decisions are really made by unconscious processes; consciousness notices what is happening and, like an existentialist, claims the act as its own after the fact (though of course the existentialist does not do it automatically). There is of course a lot of evidence that our behaviour is frequently influenced by factors we are not consciously aware of, and that we happily make up reasons for what we have done which are not the true ones. But ‘frequently’ is not ‘invariably’ and in fact there seem to be plenty of cases where, for example, our conscious understanding of a written text really does change our behaviour and our emotional states. I would find it pretty hard to think that my understanding of an email with important news from a friend was somehow not conscious, or that my conscious comprehension was irrelevant to my subsequent behaviour. That is the kind of unlikely stuff that the behaviourists ultimately failed to sell us. Halligan and Oakley want to go quite a way down that same unpromising path, suggesting that it is actually unhelpful to draw a sharp distinction between conscious and unconscious. They will allow a kind of continuum, but to me it seems clear that there is a pretty sharp distinction between the conscious, detached plans of human beings and the instinct-driven, environment-controlled behaviour of animals, one it is unhelpful to blur or ignore.

One distinction that I think would be helpful here is between conscious and what I’ll call self-conscious states. If I make a self-conscious decision I’m aware of making it; but I can also just make the decision; in fact, I can just act. In my view, cases where I just make the decision in that unreflecting way may still be conscious; but I suspect that Halligan and Oakley (like Higher Order theorists) accept only self-conscious decisions as properly conscious ones.

Interesting to compare the case put by Peter Carruthers in Scientific American recently; he argues that the whole idea of conscious thought is an error. He introduces the useful idea of the Global Workspace proposed by Bernard Baars and others; a place where data from different senses can be juggled and combined. To be in the workspace is to be among the contents of consciousness, but Carruthers believes only sensory items get in there. He’s happy to allow bits of ‘inner speech’ or mental visualisation, deceptive items that mislead us about our own thoughts; but he won’t allow completely abstract stuff (again, you may see some resemblance to the ‘mentalistic’ entities disallowed by the behaviourists). I don’t really know why not; if abstractions never enter consciousness or the workspace, how is it that we’re even talking about them?

Carruthers thinks our ‘Theory Of Mind’ faculty misleads us; as a quick heuristic it’s best to assume that others know their own mental states accurately, and so, it’s natural for us to think that we do too: that we have direct access and cannot be wrong about whether we, for example, feel hungry. But he thinks we know much less than we suppose about our own motives and mental states. On this he seems a little more moderate than Halligan and Oakley, allowing that conscious reflection can sometimes affect what we do.

I think the truth is that our mental, and even our conscious processes, are in fact much more complex and multi-layered than these discussions suggest. Let’s consider the causal efficacy of conscious thought in simple arithmetic. When I add two to two in my head and get four, have the conscious thoughts about the sum caused the conscious thought of the answer, or was there an underlying non-conscious process which simply waved flags at a couple of points?

Well, I certainly can do the thing epiphenomenally. I can call up a mental picture of the written characters, for example, and step through them one by one. In that case the images do not directly cause each other. If I mentally visualise two balls and then another two balls and then mentally count them, perhaps that is somewhat different? Can I think of two abstractly and then notice conceptually that its reduplication is identical with the entity ‘four’? Carruthers would deny it, I think, but I’m not quite sure. If I can, what causal chain is operating? At this point it becomes clear that I really have little idea of how I normally do arithmetic, which I suppose scores a point for the sceptics. The case of two plus two being four is perhaps a bad example, because it is so thoroughly remembered I simply replay it, verbally, visually, implicitly, abstractly or however I do it. What if I were multiplying 364 by 5? The introspective truth seems to be that I do something akin to running an algorithm by hand. I split the calculation into separate multiplications, whose answer I mainly draw direct from memory, and then I try to remember results and add them, again usually relying on remembered results. Does my thinking of four times five and recalling that the answer is twenty mean there was a causal link between the conscious thought and the conscious result? I think there may be such a link, but frustratingly if I use my brain in a slightly different way there may not be a direct one, or there may be a direct one which is not of the appropriate kind (because, say, the causal link is direct but irrelevant to the mathematical truth of the conclusion).

Having done all that, I realise that since I’m multiplying by five, I could have simply multiplied by ten, which can be done by simply adding a zero (Is that done visually – is it necessarily done visually? Some people cannot conjure up mental images at all.) and halving the result. Where did that little tactic come from? Did I think of it consciously, and was its arrival in reportable condition causally derived from my wondering about how best to do the sum (in words, or not in words?) or was it thrust into a kind of mental in-tray (rather spookily) by an unconscious part of my brain which has been vaguely searching around for any good tricks for the last couple of minutes? Unfortunately I think it could have happened in any one of a dozen different ways, some of which probably involve causally effective conscious states while others may not.

In the end the biggest difference between me and the sceptics may come down to what we are prepared to call conscious; they only want the states I’m calling self-conscious. Suppose we take it that there is indeed a metaphorical or functional space where mental items become ‘available’ (in some sense I leave unclarified) to influence our behaviour. It could indeed be a Global Workspace, but need not commit us to all the details of that theory. Then they allow at most those items actually within the space to be conscious, while I would allow anything capable of entering it. My intuition here is that the true borderline falls, not between those mental items I’m aware of and those I merely have, but between those I could become aware of if my attention were suitably directed, and those I could never extract from the regions where they are processed, however I thought about it. When I play tennis, I may consciously plan a strategy, but I also consciously choose where to send the ball on the spur of the moment; that is not normally a self-conscious decision, but if I stopped to review it it easily could become one – whereas the murky Freudian reasons why I particularly want to win the game cannot be easily accessed (without lengthy therapy at any rate) and the processes my visual cortex used to work out the ball’s position are forever denied me.

My New Year resolution is to give up introspection before I plunge into some neo-Humean abyss of self-doubt.

Success with Consciousness

What would success look like, when it comes to the question of consciousness?

Of course it depends which of the many intersecting tribes who dispute or share the territory you belong to. Robot builders and AI designers have known since Turing that their goal is a machine whose responses cannot be distinguished from those of a human being. There’s a lot wrong with the Turing Test, but I still think it’s true that if we had a humanoid robot that could walk and talk and interact like a human being in a wide range of circumstances, most people wouldn’t question whether it was conscious or not. We’d like a theory to go with our robot, but the main thing is whether it works. Even if we knew it worked in ways that were totally unlike biological brains, it wouldn’t matter – planes don’t fly the birds do, but so what, it’s still flying. Of course we’re a million miles from such a perfectly human robot, but we sort of know where we’re going.

It’s a little harder for neurologists; they can’t rely quite so heavily on a practical demonstration, and reverse engineering consciousness is tough. Still, there are some feats that could be pulled off that would pretty much suggest the neurologists have got it. If we could reliably read off from some scanner the contents of anyone’s mind, and better yet, insert thoughts and images at will, it would be hard to deny that the veil of mystery had been drawn back quite a distance. It would have to be a general purpose scanner, though; one that worked straight away for all thoughts in any person’s brain. People have already demonstrated that they can record a pattern from one subject’s brain when that subject is thinking a known thought, and then, in the same session with the same subject, recognise that same pattern as a sign of the same thought.  That is a much lesser achievement, and I’m not sure it gets you a cigar, let alone the Nobel prize.

What about the poor philosophers? They have no way to mount a practical demonstration, and in fact no such demonstration can save them from their difficulties. The perfectly human robot does not settle things for them; they tell it they can see it appears to be able to perform a range of ‘easy’ cognitive tasks, but whether it really knows anything at all about what it’s doing is another matter. They doubt whether it really has subjective experience, even though it assures them that it’s own introspective evidence says it does. The engineer sitting with them points out that some of the philosophers probably doubt whether he has subjective experience.

“Oh, we do,” they admit, “in fact many of us are pretty sure we don’t have it ourselves. But somehow that doesn’t seem to make it any easier to wrap things up.”

Nor are the philosophers silenced by the neurologists’ scanner, which reveals that an apparently comatose patient is in fact fully aware and thinking of Christmas. The neurologists wake up the subject, who readily confirms that their report is exactly correct. But how do they know, ask the philosophers; you could be recording an analogue of experience which gets tipped into memory only at the point of waking, or your scanner could be conditioning memory directly without any actual experience. The subject could be having zomboid dreams, which convey neural data, but no actual experience.

“No, they really couldn’t,” protest the neurologists, but in vain.

So where do philosophers look for satisfaction? Of course, the best thing of all is to know the correct answer. But you can only believe that you know. If knowledge requires you to know that you know, you’re plummeting into an infinite regress; if knowing requires appropriate justification then you’re into a worm-can opening session about justification of which there is no end. Anyway, even the most self-sufficient of us would like others to agree, if not recognise the brilliance of our solution.

Unfortunately you cannot make people agree with you about philosophy. Physicists can set off a bomb to end the argument about whether e really equals mc squared; the best philosophers can do is derive melancholy satisfaction from the belief that in fifty years someone will probably be quoting their arguments as common sense, though they will not remember who invented them, or that anyone did. Some people will happen to agree with you already of course, which is nice, but your arguments will convert no-one; not only can you not get people to accept your case; you probably can’t even get them to read your paper. I sympathised recently with a tweet from Keith Frankish lamenting how he has to endlessly revisit bits of argument against his theory of illusionism, one’s he’s dealt with many times before (oh, but illusions require consciousness; oh, if it’s an illusion, who’s being deceived…). That must indeed be frustrating, but to be honest it’s probably worse than that; how many people, having had the counter-arguments laid out yet again, accept them or remember them accurately? The task resembles that of Sisyphus, whose punishment in Hades was to roll a boulder up a hill it invariably rolled down again. Camus told us we must imagine Sisyphus happy, but that itself is a mental task which I find undoes itself every time I stop concentrating…

I suppose you could say that if you have to bring out your counter-arguments regularly, that itself is some indicator of having achieved some recognition. Let’s be honest, attention is what everyone wants; moral philosophers all want a mention on The Good Place, and I suppose philosophers of mind would all want to be namechecked on Westworld if Julian Jaynes hadn’t unaccountably got that one sewn up.

Since no-one is going to agree with you, except that sterling band who reached similar conclusions independently, perhaps the best thing is to get your name associated with a colourful thought experiment that lots of people want to refute. Perhaps that’s why the subject of consciousness is so full of them, from the Chinese Room to Mary the Colour Scientist, and so on. Your name gets repeated and cited that way, although there is a slight danger that it ends up being connected forever with a point of view you have since moved on from, as I believe is the case with Frank Jackson himself, who no longer endorses the knowledge argument exemplified by the Mary story.

Honestly, though, being the author of a widely contested idea is second best to being the author of a universally accepted one. There’s a Borges story about a deposed prince thrown into a cell where all he can see is a caged jaguar. Gradually he realises that the secrets of the cosmos are encoded in the jaguar’s spots, which he learns to read; eventually he knows the words of magic which would cast down his rival’s palace and restore him to power; but in learning these secrets he has attained enlightenment and no longer cares about earthly matters. I bet every philosopher who reads this story feels a mild regret; yes, of course enlightenment is great, but if only my insights allowed me to throw down a couple of palaces? That bomb thing really kicked serious ass for the physicists; if I could make something go bang, I can’t help feeling people would be a little more attentive to my corpus of work on synthetic neo-dualism…

Actually, the philosophers are not the most hopeless tribe; arguably the novelists are also engaged in a long investigation of consciousness; but those people love the mystery and don’t even pretend to want a solution. I think they really enjoy making things more complicated and even see a kind of liberation in the indefinite exploration; what can you say for people like that!

AI turns the corner?

Is the latest version of AlphaZero showing signs of human style intuition and creativity?

The current AlphaZero is a more generalised version of the program, produced by Demis Hassabis and his team, that beat top human players of Go for the first time. The new version, presented briefly in Science magazine, is able to learn a range of different games; besides Go it learned chess and shogi, and apparently reached world-class play in all of them.

The Science article, by David Silver et al, modestly says this achievement is an important step towards a fully-general game-playing program, but press reports went further, claiming that in chess particularly AlphaZero showed more human traits than any previous system. It reinvented human strategies and sacrificed pieces for advantage fairly readily, the way human players do; chess commentators said that its play seemed to have new qualities of insight and intuition.

This is somewhat unexpected, because so far as I can tell the latest version is in fact not a radical change from its predecessors; in essence it uses the same clever combination of appropriate algorithms with a deep neural network, simply applying them more generally. It does appear that the approach has proved more widely powerful than we might have expected, but it is no more human in nature than the earlier versions and does not embody any new features copied from real brains. It learns its games from scratch, with only the rules to go on, playing out games against itself repeatedly in order to find what works. This is not much like the way humans learn chess; I think you would probably die of boredom after a few hundred games, and even if you survived, without some instruction and guidance you would probably never learn to be a good player, let alone a superlative one. However, running through possible games in one’s mind may be quite like what a good human player does when trying to devise new strategies.

The key point for me is that although the new program is far more general in application, it still only operates in the well-defined and simple worlds provided by rule-governed games. To be anything like human, it needs to display the ability to deal with the heterogenous and undefinable world of real life. That is still far distant (Hassabis himself has displayed an awareness of the scale of the problem, warning against releasing self-driving cars on to real roads prematurely), though I don’t altogether rule out the possibility that we are now moving perceptibly in the right direction.

Someone who might deny that is Gary Marcus, who in a recent Nautilus piece set out his view that deep learning is simply not enough. It needs, he says, to be supplemented by other tools, and in particular it needs symbol manipulation.

To me this is confusing, because I naturally interpret ‘symbol manipulation’ as being pretty much a synonym for Turing style computation. That’s the bedrock of any conventional computer, so it seems odd to say we need to add it. I suppose Marcus is using the word ‘symbol’ in a different sense. The ones and zeroes shuffled around by a Turing machine are meaningless to the machine. We assign a meaning to the input and so devise the manipulations that the output can be given an interpretation which makes it useful or interesting to us, but the machine itself knows nothing of that. Marcus perhaps means that we need a new generation of machines that can handle symbols according to their meanings.

If that’s it, then few would disagree that that is one of the ultimate aims. Those who espouse deep learning techniques merely think that those methods may in due course lead to a machine that handles meanings in the true sense; at some stage the system will come up with the unknown general strategy that enables it to get meaningful and use symbols the way a human would. Marcus presumably thinks that is hopeless optimism, on the scale of those people who think any system that is sufficiently complex will naturally attain self-awareness.

Since we don’t have much of an idea how the miracle of meaning might happen it is indeed optimistic to think we’re on the road towards it. How can a machine bootstrap itself into true ‘symbol manipulation’ without some kind of help? But we know that the human brain must have done just that at some stage in evolution – and indeed each of our brains seem to have done it again during our development. It has got to be possible. Optimistic yes, hopeless – maybe not.

New paths to AI disaster

I’ve never believed that robots dream of killing all humans. I don’t think paperclip maximisers are ever going to rule the world. And I don’t believe in the Singularity. But is AI heading in some dangerous directions? Oh yes.

In Forbes, Bernard Marr recently offered five predictions for the year ahead. They mostly strike me as pretty believable, though I’m less optimistic than he is about digital assistants and the likelihood of other impressive breakthroughs; he’s surely right that there will be more hype.

It’s his first two that prompted some apprehension on my part. He says…

  1. AI increasingly becomes a matter of international politics
  2. A Move Towards “Transparent AI”

Those are surely right; we’ve already seen serious discussion papers emerging from the EU and elsewhere, and one of the main concerns to have emerged recently is the matter of ‘transparency’ – the auditability of software. How is the computer making its decisions?

This is a legitimate, indeed a necessary concern. Once upon a time we could write out the algorithm embodied in any application and check how it worked. This is getting more difficult with software that learns for itself, and we’ve already seen disastrous cases where the AI picked up and amplified the biases of the organisation it was working for. Noticing that most top executives were white middle-aged men, it might decide to downgrade the evaluation of everyone else, for example. Cases like that need to be guarded against and managed; it ought to be feasible in such circumstances, by studying results even if it isn’t possible to look inside the ‘black box’.

But it starts to get difficult, because as machine learning moves on into more complex decision making, it increasingly becomes impossible to understand how the algorithms are playing out, and the desired outcomes may not be so clear. In fact it seems to me that full transparency may be impossible in principle, due to human limitations. How could that be? I’m not sure I can say – I’m no expert, and explaining something you don’t, by definition, understand, is a bit of a challenge anyway. In part the problem might be to do with how many items we can hold in mind, for example. It’s generally accepted that we can only hang on to about seven items (plus or minus a couple) in short-term memory. (There’s scope for discussion about such matters as what amounts to an item, and so on, but let’s not worry about the detail.) This means there is a definite limit to how many possible paths we can mentally follow at once, or to put it another way, how large a set of propositional disjunctions we can hang on to (‘either a or b, and if a, either c, d, or e, while if b, f or g… and there we go). Human brains can deal with this by structuring decisions to break them into smaller chunks, using a pencil and paper, and so on. Perhaps, though, there are things that you can only understand by grasping twenty alternatives simultaneously. Very likely there are other cognitive issues we simply can’t recognise; we just see a system doing a great job in ways we can’t fathom.

Still, I said we could monitor success by just looking at results, didn’t I? We know that our recruitment exercise ought to yield appointments whose ethnic composition is the same as that of the population (or at any rate, of the qualified candidates).  OK, sometimes it may be harder to know what the desired outcome is, exactly, and there may be issues about whether ongoing systems need to be able to yield sub-optimal results temporarily, but those are tractable issues.

Alas, we also have to worry about brittleness and how things break. It turns out that systems using advanced machine learning may be prone to sudden disastrous failure. A change of a few unimportant pixels in a graphic may make an image recognition system which usually performs reliably draw fantastic conclusions instead. In one particular set of circumstances a stock market system may suddenly go ape. This happens because however machine learning systems are doing what they do, they are doing something radically different from what we do, and we might suspect that like simpler computer systems, they take no true account of relevance, only its inadequate proxy correlation. Nobody, I think, has any good theoretical analysis of relevance, and it is strongly linked with Humean problems philosophers have never cracked.

That’s bad, but it could be made worse if legislative bodies either fail to understand why these risks arise, or decide that on a precautionary basis we must outlaw anything that cannot be fully audited and understood by human beings. Laws along those lines seem very likely to me, but they might throw away huge potential benefits – perhaps major economic advantage – or even suppress the further research and development which might ultimately lead to solutions and to further, as yet unforeseeable, gains.

That’s not all, either; laws constrain compliant citizens, but not necessarily everyone. Suppose we can build machine learning systems that retain a distinct risk of catastrophic failure, but outclass ordinary human or non-learning systems most of the time. Will anyone try to build and use such systems? Might there be a temptation for piratical types to try them out in projects that are criminal, financial, political or even military? Don’t the legitimate authorities have to develop the same systems pre-emptively in self-defence? Otherwise we’re left in a position where it’s not clear whether we should hope that the ‘pirate’ systems fail or work, because either way it’s catastrophe.

What on earth is the answer, what regulatory regime or other measures would be appropriate? I don’t know and I strongly doubt that any of the regulatory bodies who are casting a thoughtful eye over this territory know either.

Can we talk about this?

Can we even talk about qualia, the phenomenal parts of conscious experience? Whereof we cannot speak, thereof we must remain silent, as Wittgenstein advised, not lingering to resolve the paradoxical nature of the very phrase ‘whereof we cannot speak’ – it seems to do the very thing that it specifies cannot be done. We see what he meant, perhaps.

There is certainly a difficulty of principle in talking about qualia, to do with causality. Qualia have no causal effects – if they did, they would be an observable part of the world of physics, and it is part of their essential definition that they are outside, or over and above, the mere physical account. It follows, notoriously, that whatever we say or write about qualia cannot have been caused by them. At first glance this seems to demolish the whole discussion; no-one’s expressed belief in qualia can actually be the causal result of experiencing them.

But it is possible to talk sensibly of things that did not cause the talk. It takes a weirdly contorted argument to defend the idea that when I refer to Julius Caesar, the old Roman himself caused me to do it, but perhaps we can lash something together. It’s worse that I can talk of Nero and Zero the rollicking Romans, who existed only as heroes of a cartoon strip. If you’re still willing to grant them some causal role in physics, perhaps somehow through the material existence of the paper and ink in which they were realised, remember that I can even talk intelligibly about Baesar, who has no existence whatever, and in all likelihood was never spoken of before. He really cannot have caused me to write that last sentence.

So I would say that the absence of causal effects does not provide a knock-down reason why I cannot speak of qualia, though the fact that the other cases without causality involve entities that are fictions or delusions cannot be comfortable if I  want my qualia to be real. It seems as if there must be a sort of pre-established harmony effect going on, so that my words remain truthful on the matter even though they are not causally determined by it, which feels, as technical metaphysicians say, kind of weird.

But apart from the difficulty of principle, it seems awfully difficult to speak of qualia in practice too. How can we verbally pick out a particular quale? With real things, we choose one or more of their most salient attributes; with imaginary entities, we just specify similar properties. But qualia have no individual attributes of their own; the only way to pick them out is by mentioning the objective sensation they accompany. So, we typically get a quale of red, or a red quale. This is pretty unsatisfactory, because it means many interesting questions are excluded from consideration. We cannot really ask whether every sensation has a quale; we cannot ask how many qualia there are, because our way of referring to them just has baked into it the assumption that they exactly match up with the objective sensations. If green, as a matter of fact, was the only colour with no qualia, the fact would be occluded from us by the only language we can use to discuss the matter.

All of this might seem enough to justify our concluding that talk of qualia adds nothing to talk of objective sensations, so that even if, by some uncovenanted harmony, our talk of qualia proves to be metaphysically true, it has absolutely no informative value, and might as well be abandoned. What remains is the unconquerable conviction that there is something there, or to use the little phrase on which so much metaphysical weight has been rested, ‘there is something it is like’ to, for example, see red.

Can this phrase be explicated into something clearer? The first problem is the ‘it’; are we actually speaking of anything there? To me it seems that the ‘it’ in ‘something it is like’ is as merely grammatical as the ‘it’ in ‘it is raining’, which does not cause us to entertain the idea that there is something ineffable and non-physical about precipitation. The second problem is the ‘like’ which suggests we are making a comparison while leaving it quite unclear what is being compared. Is seeing red meant to be like seeing another colour? Is seeing red phenomenally meant to be like seeing red objectively (whatever that would mean)? In fact we seem obliged to conclude that no actual comparison is being made. Suppose we assert of hang-gliding or our first taste of champagne ‘there’s nothing like it!’  Are we managing here to assert after all that these experiences are unaccompanied by qualia? Surely not. If anything we’re saying that the relevant qualia are exceptionally powerful.

In the end, doing my honest best, I think ‘there is something it is like to see red’ simply asserts that the experience of seeing red really exists. I’m fine with that, and there are genuine mysteries attached; but there still seems to be nothing more we can say about qualia as a result. Haven’t we all been a bit too accepting for a bit too long of ‘there Is something it is like’?

Good vibrations?

Is resonance the answer? Tam Hunt thinks it might be.

Now the idea that synchronised neuron firing might have something to do with consciousness is not new. Veterans of consciousness will recall a time when 40 hertz was thought to be the special, almost magical frequency that generated consciousness; people like Francis Crick thought it might be the key to the unity of consciousness and a solution to the binding problem. I don’t know what the current state of neurology on this is, but it honestly seems most likely to me that 40 hertz, or a rate in that neighbourhood, is simply what the brain does when it’s thrumming along normally. People who thought it was important were making a mistake akin to taking a car’s engine noise for a functional component (hey, no noise, no move!).

Hunt has a bit more to offer than simply speculating that resonance is important somehow, though. He links resonance with panpsychism, suggesting that neurons have little sparks of consciousness and resonance is the way they get recruited into the larger forms of awareness we experience. While I can see the intuitive appeal of the idea, it seems to me there are a lot of essential explanatory pieces missing from the picture.

The most fundamental problem here is that I simply don’t see how resonance between neurons could ever explain subjective experience. Resonance is a physical phenomenon, and the problem is that physical stuff just doesn’t seem to supply the ‘what-it-is-like’ special quality of experience. Hard to see why co-ordinated firing is any better in that essential respect than unco-ordinated. In fact, in one respect resonance is especially unsuitable; resonance is by its nature stable. If it doesn’t continue for at least a short period, you haven’t really got resonance. Yet consciousness often seems fleeting and flowing, moving instantaneously and continuously between different states of awareness.

There’s also, I think, some work needed on the role of neurons. First, how come our panpsychist ascent starts with neurons? We either need an account of how we get from particles up to neurons, or an account of why consciousness only starts when we get up to neurons (pretty complex entities, as we kee finding out). Second, if resonating neurons are generating consciousness, how does that sit with their day job? We know that neurons transmit signals from the senses and to the muscles, and we know that they do various kinds of processing. Do they generate consciousness at the same time, or is that delegated to a set of neurons that don’t have to do processing?  If the resonance only makes content conscious, how is the content determined, and how are the resonance and the processing linked? How does resonance occur, anyway? Is it enough for neurons to be in sync, so that two groups in different hemispheres can support the same resonance? Can a group of neurons in my brain resonate with a group in yours? If there has to be some causal linkage or neuronal connection, isn’t that underlying mechanism the real seat of consciousness, with the resonance just a byproduct?

What about that panpsychist recruitment – how does it work? Hunt says an electron or an atom has a tiny amount of consciousness, but what does ‘tiny’ mean? Is it smaller in intensity, complexity, content, or what? If it were simply intensity, then it seems easy enough to see how a lot of tiny amounts could add up to something more powerful, just as a lot of small lights can achieve the effect of a single big one. But for human consciousness to be no more than the consciousness of an atom with the volume turned up doesn’t seem very satisfactory. If, on the other hand, we’re looking for more complexity and structure, how can resonance, which has the neurons all doing the same thing at the same time, possibly deliver that?

I don’t doubt that Hunt has answers to many of these questions, and perhaps it’s not reasonable to expect them all in a short article for a general readership. For me to suspend my disbelief, though, I do really need a credible hint as to the metaphysical core of the thinking. How does the purely physical phenomenon of resonance produce the phenomenal aspect of my conscious experience, the bit that goes beyond mere data registration and transmutes into the ineffable experience I am having?

A different Difference Engine

Consciousness as organised energy is the basis of a new theory offered by Robert Pepperell; the full paper is here, with a magazine article treatment here.

Pepperell suggests that we can see energy as difference, or more particularly actualised difference – that is to say, differences in the real world, not differences between abstract entities. We can imagine easily enough that the potential energy of a ball at the top of a slope is a matter of the difference between the top and bottom of the slope, and Pepperell contends that the same is equally true of the kinetic energy of the ball actually rolling down. I’m not sure that all actualised differences are energy, but that’s probably just a matter of tightening some definitions; we see what Pepperell is getting at. He says that the term ‘actualised difference’ is intended  to capture the active, antagonistic nature of energy.

He rejects the idea that the brain is essentially about information processing, suggesting instead that it processes energy. He rightly points to the differing ways in which the word ‘information’ is used, but if I understand correctly his chief objection is that information is abstract, whereas the processing of the brain deals in actuality; in the actualised difference of energy, in fact.

This is crucial because Pepperell wants us to agree that ‘there is something it is like’ to undergo actualised difference. He claims we can infer this by examining nature; I’m not sure everyone will readily agree, but the idea is that we can see that what it is like to be a rope under tension differs from what it is like to be the same rope when slack. It’s important to be clear that he’s not saying the rope is conscious; having a ‘what it is like’ is for him a more primitive level of experience, perhaps not totally unlike some of the elementary states of awareness that appear in panpsychist theories (but that’s my comparison, not his).

To get the intuition that Pepperell is calling on here, I think we need to pay attention to his exclusion of abstract entities. Information-based theories take us straight to the abstract level, whereas I think  Pepperell sees ‘something it is like’ as being a natural concomitant of actuality, or at any rate of actualised difference. To him this seems to be evident from simple examination, but again I think many will simply reject the idea as contrary to their own intuitions.

If we’re ready to grant that much, we can then move on to the second part of the theory, which takes consciousness to be a reflexive form of what-it-is-likeness. Pepperell cites the example of the feedback patterns which can be generated by pointing a video camera at its own output on a screen. I don’t think we are to take this analogy too literally, but it shows how a self-referential system can generate output that goes far beyond registration if the input. The proposal also plays into a relatively common intuition that consciousness, or at least some forms of it, are self-referring or second order, as in the family of HOT and HOP theories.

Taken all in all, we are of course a long way from a knock-down argument here; in fact it seems to me that Pepperell does not spend enough time adumbrating the parts of his theory that most need clarification and defence. I’m left not altogether seeing why we should think it is like anything to be a rope in any state, nor convinced that reflexive awareness of our awareness has any particular part to play in the generation of subjective consciousness (it obviously has something to do with self-awareness). But the idea that ‘something it is like’ is an inherent part of actuality does have some intuitive appeal for me, and the idea of using that as a base for the construction of more complex forms of consciousness is a tantalising start, at least.

 

Boltzmann Brains

Solipsism, the belief that you are the only thing that really exists (everything else is dreamed or imagined by you), is generally regarded as obviously false; indeed, I’ve got it on a list somewhere here as one of the conclusions that tell you pretty clearly that your reasoning went wrong somewhere along the way; a sort of reductio ad absurdum. There don’t seem to be any solipsists, (except perhaps those who believe in the metempsychotic version) – but perhaps there are really plenty of them and they just don’t bother telling the rest of us about their belief, since in their eyes we don’t exist.

Still, there are some arguments for solipsism, especially its splendid parsimony. William of Occam advised us to use as few angels as possible in our cosmology, or more generally not to multiply entities unnecessarily. Solipsism reduces our ontological demand to a single entity, so if parsimony is important it leads the field. Or does it? Apart from oneself, the rest of the cosmos, according to solipsists, is merely smoke and mirrors; but smoke takes some arranging and mirrors don’t come cheap. In order for oneself to imagine all this complex universe, one’s own mind must be pretty packed with stuff, so the reduction in the external world is paid for by an increase in the internal one, and it becomes a tricky metaphysical question as to whether deeming the entire cosmos to be mental in nature actually reduces one’s ontological commitment or not.

Curiously enough, there is a relatively new argument for solipsism which runs broadly parallel to this discussion, derived from physics and particularly from the statistics of entropy. The second law of thermodynamics tells us that entropy increases over time; given that, it’s arguably kind of odd that we have a universe with non-maximal entropy in the first place. One possibility, put forward with several other ideas by Ludwig Boltzmann in 1896, is that the second law is merely a matter of probability. While entropy generally tends to increase, it may at times go the other way simply by chance.  Although our observed galaxy is highly improbable, therefore, if you wait long enough it can occur just by chance as a pocket of low entropy arising from chance fluctuations in a vastly larger and older universe (really old; it would have to be hugely older than we currently believe the cosmos to be) whose normal state is close to maximal entropy.

One problem with this is that while the occurrence of our galaxy by chance is possible, it’s much more likely that a single brain in the state required for it to be having one’s current experiences should arise from random fluctuations. In a Boltzmannic universe, there will be many, many more ‘Boltzmann brains’ like this than there will be complete galaxies like ours. Such brains cannot tell whether the universe they seem to perceive is real or merely a function of their random brain states; statistically, therefore, it is overwhelmingly likely that one is in fact a Boltzmann brain.

To me the relatively low probability demands of the Boltzmann brain, compared with those of a full universe, interestingly resemble the claimed low ontological requirement of the solipsism hypothesis, and there is another parallel. Both hypotheses are mainly used as leverage in reductio arguments; because this conclusion is absurd, something must be wrong with the assumptions or the reasoning that got us here. So if your proposed cosmology gives rise to the kind of universe where Boltzmann brains crop up ‘regularly’, that’s a large hit against your theory.

Usually these arguments, both in relation to solipsism and Boltzmann brains, simply rest on incredulity. It’s held to be just obvious that these things are false. And indeed it is obvious, but at a philosophical level, that won’t really do; the fact that something seems nuts is not enough, because nutty ideas have proven true in the past. For the formal logical application of reductio, we actually require the absurd conclusion to be self-contradictory; not just silly, but logically untenable.

Last year, Sean Carroll came up with an argument designed to beef up the case against Boltzmann brains in just the kind of way that seems to be required; he contends that theories that produce them cannot simultaneously be true and justifiably believed. Do we really need such ‘fancy’ arguments? Some apparently think not. If mere common sense is not enough, we can appeal to observation. A Boltzmann brain is a local, temporary thing, so we ought to be able to discover whether we are one simply by observing very distant phenomena or simply waiting for the current brain states to fall apart and dissolve. Indeed, the fact that we can remember a long and complex history is in itself evidence against our being Boltzmanns.

But appeals to empirical evidence cannot really do the job; there are several ways around them. First, we need not restrict ourselves literally to a naked brain; even if we surround it with enough structured world to keep the illusion going for a bit, our setup is still vastly more likely than a whole galaxy or universe. Second, time is no help because all our minds can really access is the current moment; our memories might easily be false and we might only be here for a moment. Third, most people would agree that we don’t necessarily need a biological brain to support consciousness; we could be some kind of conscious machine supplied with a kind of recording of our ‘experiences’. The requirement for such a machine could easily be less than for the disconnected biological brain.

So what is Carroll’s argument? He maintains that the idea of Boltzmann brains is cognitively unstable. If we really are such a brain, or some similar entity, we have no reason to think that the external world is anything like what we think it is. But all our ideas about entropy and the universe come from the very observations that those ideas now apparently undermine. We don’t quite have a contradiction, but we have an idea that removes the reasons we had for believing in it. We may not strictly be able to prove such ideas wrong, but it seems reasonable, methodologically at least, to avoid them.

One problem is those pesky arguments about solipsism. We may no longer be able to rely on the arguments about entropy in the cosmos, but can’t we borrow Occam’s Razor and point out that a cosmos that contains a single Boltzmann brain is ontologically far less demanding than a whole universe? Perhaps the Boltzmann arguments provide a neat physics counterpart for a philosophical case that ultimately rests on parsimony?

In the end, we can’t exactly prove solipsism false; but we can perhaps do something loosely akin to Carroll’s manoeuvre by asking: so what if it’s true? Can we ignore the apparent world? If we are indeed the only entity, what should we do about it, either practically or in terms of our beliefs? If solipsism is true, we cannot learn anything about the external world because it’s not there, just as in Carroll’s scenario we can’t learn about the actual world because all our perceptions and memories are systematically false. We might as well get on with investigating what we can investigate, or what seems to be true.