Monthly Archives: January 2019

Explaining to humans

Posted on by
16

We’ve discussed a few times recently how deep learning systems are sometimes inscrutable. They work out how to do things for themselves, and there may be no way to tell what method they are using. I’ve suggested that it’s worse than that; they are likely to be using methods which, though sound, are in principle incomprehensible to human beings. These unknowable methods may well be a very valuable addition to our stock of tools, but without understanding them we can’t be sure they won’t fail; and the worrying thing about that is that unlike humans, who often make small or recoverable mistakes, these systems have a way of failing that is sudden, unforgiving, and catastrophic.

So I was interested to see this short interview with Been Kim, who works for Google Brain (is there a Google Eyes?) has the interesting job of building a system that can explain to humans what the neural networks are up to.

That ought to be impossible, you’d think. If you use the same deep learning techniques to generate the explainer, you won’t be able to understand how it works, and the same worries will recur on another level. The other approach is essentially the old-fashioned one of directly writing an algorithm to do the job; but to write an explicit algorithm to explain system X you surely need to understand system X to begin with?

I suspect there’s an interesting general problem here about the transmission of understanding. Perhaps data can be transferred, but understanding just has to happen, and sometimes, no matter how helpfully the relevant information is transmitted, understanding just fails to occur (I feel teachers may empathise with this).

Suppose the explainer is in place; how do we know that its explanations are correct? We can see over time whether it successfully picks out systems that work reliably and those that are going to fail, but that is only ever going to be a provisional answer. The risk of sudden unexpected failure remains. For real certainty, the only way is for us to validate it by understanding it, and that is off the table.

So is Been Kim wasting her time on a quixotic project – perhaps one that Google has cynically created to reassure the public and politicians while knowing the goal is unattainable? No; her goal is actually a more modest, negative one. Her interpreter is not meant to provide an assurance that a given system is definitely reliable; rather, it is supposed to pick out one’s that are definitely dodgy; and this is much more practical. After all, we may not always understand how a given system executes a particular complex task, but we do know in general how neural networks and deep learning work. We know that the output decisions come from factors in the input data, and the interpreter ought to be able to tell us what factors are being taken into account. Then, using the unique human capacity to identify relevance, we may be able to spot some duds – cases where the system is using a variable that tracks the relevant stuff only unreliable, or where there was some unnoticed problem with the corpus of examples the system learnt from.

Is that OK? Well, in principle there’s the further risk that the system is actually cleverer than we realise; that it is using features (perhaps very complex ones) that actually work fine, but which we’re too dim to grasp. Our best reassurance here is again understanding; if we can see how things seem to be working, we have to be very unlucky to hit a system which is actually superior but just happens, in all the examined cases, to look like a dodgy one. We may not always understand the system, but if we understand something that’s going wrong, we’re probably on firm ground.

Of course, weeding out egregiously unreliable systems does not solve the basic problem of efficient but inscrutable systems. Without accusing Google of a cunning sleight of hand after all, I can well imagine that the legislators and bureaucrats who are gearing up to make rules about this issue might mistake interpreter systems like Kim’s for a solution, require them in all cases, and assume that the job is done and dusted…

Entangled Consciousness

Posted on by
39
Entangled

Could the answer be quantum physics after all? Various people have suggested that the mystery of consciousness might turn out to be explicable in terms of quantum physics; most notably we have the theory championed ny Roger Penrose and Stuart Hameroff, which suggests that as-yet unknown quantum mechanics might be going on in the microtubules of neural cells. Atai Barkai has kindly shared with me his draft paper which sets out a different take on the matter.

Barkai’s target is subjective experience, or what he defines as the consciousness instance. We may be conscious of things in the world out there, but that kind of consciousness always involves an element of inference, explicit or not; the consciousness instance is free of those uncertainties, consisting only of the direct experience of which we can be certain. Some hardy sceptics would deny that there is anything in consciousness that we can be that sure of, but I think it is clear enough what distinction Barkai is making and that it is, as it were, first order experience he is aiming at.

The paper puts a complex case very carefully, but flinging caution to the winds I believe the gist goes like this. Typically, those who see a problem with qualia and subjective experience think it lies outside the account given by physics, which arguably therefore needs some extension. Barkai agrees that subjectivity is outside the classical account and that a more capacious concept of the universe is needed to accommodate it; but he thinks that quantum entanglement can already, properly read, provide what is needed.

It’s true that philosophical discussions generally either treat classical physics as being the whole subject or ignore any implications that might arise from the less intuitive aspects of quantum physics. Besides entanglement, Barkai discusses free will and its possible connection with the indeterminism of quantum physics. If there really is indeterminism in quantum physics; on this and other points I simply don’t have a clear enough grasp of the science to tackle the questions effectively (better informed comments would be welcome).

Entanglement, as I understand it, means that the states of two particles (or in principle, larger bodies) may be strongly connected in ways that do not depend on normal classical interaction and are not affected by distance. This means that information can in theory be transferred any distance, instantly and infallibly, which opens up the theoretical possibility of quantum computing, delivering the instant solution to computable problems of any finite size. Whether the brain might be doing this kind of thing is a question which Barkai leaves as an interesting speculation.

The main problem for me in the idea that entanglement explains subjectivity is understanding intuitively how that could be so. Entanglement seems to offer the possibility that there might be more information in our mental operations than classical physics could account for; that feels helpful, but does it explain the qualitative difference between mere dead data and the ‘something it is like’ of subjectivity? I don’t reject the idea, but I don’t think I fully grasp it either.

There is also a more practical objection. Quantum interactions are relevant at a microscopic level, but as we ascend to normal scales, it is generally assumed that wave forms collapse and we slip quickly back into a world where classical physics reigns. It is generally thought that neurons, let alone the whole brain, are just too big and hot for quantum interactions to have any relevance. This is why Penrose looks for new quantum mechanics in tiny microtubules rather than resting on what known quantum mechanics can provide.

As I say, I’m not really competent to assess these issues, but there is a neatness and a surprising novelty about Barkai’s take that suggests it merits further discussion.

Unconscious and Conscious

Posted on by
69

What if consciousness is just a product of our non-conscious brain, ask Peter Halligan and David A Oakley? But how could it be anything else? If we take consciousness to be a product of brain processes at all, it can only come from non-conscious ones. If consciousness had to come from processes that were themselves already conscious, we should have a bit of a problem on our hands. Granted, it is in one sense remarkable that the vivid light of conscious experience comes from stuff that is at some level essentially just simple physical matter – it might even be that incredulity over that is an important motivator for panpsychists, who find it easier to believe that everything is at least a bit conscious. But most of us take that gap to be the evident core fact that we’re mainly trying to explain when we debate consciousness.

Of course, Halligan and Oakley mean something more than that. Their real point is a sceptical, epiphenomenalist one; that is, they believe consciousness is ineffective. All the decisions are really made by unconscious processes; consciousness notices what is happening and, like an existentialist, claims the act as its own after the fact (though of course the existentialist does not do it automatically). There is of course a lot of evidence that our behaviour is frequently influenced by factors we are not consciously aware of, and that we happily make up reasons for what we have done which are not the true ones. But ‘frequently’ is not ‘invariably’ and in fact there seem to be plenty of cases where, for example, our conscious understanding of a written text really does change our behaviour and our emotional states. I would find it pretty hard to think that my understanding of an email with important news from a friend was somehow not conscious, or that my conscious comprehension was irrelevant to my subsequent behaviour. That is the kind of unlikely stuff that the behaviourists ultimately failed to sell us. Halligan and Oakley want to go quite a way down that same unpromising path, suggesting that it is actually unhelpful to draw a sharp distinction between conscious and unconscious. They will allow a kind of continuum, but to me it seems clear that there is a pretty sharp distinction between the conscious, detached plans of human beings and the instinct-driven, environment-controlled behaviour of animals, one it is unhelpful to blur or ignore.

One distinction that I think would be helpful here is between conscious and what I’ll call self-conscious states. If I make a self-conscious decision I’m aware of making it; but I can also just make the decision; in fact, I can just act. In my view, cases where I just make the decision in that unreflecting way may still be conscious; but I suspect that Halligan and Oakley (like Higher Order theorists) accept only self-conscious decisions as properly conscious ones.

Interesting to compare the case put by Peter Carruthers in Scientific American recently; he argues that the whole idea of conscious thought is an error. He introduces the useful idea of the Global Workspace proposed by Bernard Baars and others; a place where data from different senses can be juggled and combined. To be in the workspace is to be among the contents of consciousness, but Carruthers believes only sensory items get in there. He’s happy to allow bits of ‘inner speech’ or mental visualisation, deceptive items that mislead us about our own thoughts; but he won’t allow completely abstract stuff (again, you may see some resemblance to the ‘mentalistic’ entities disallowed by the behaviourists). I don’t really know why not; if abstractions never enter consciousness or the workspace, how is it that we’re even talking about them?

Carruthers thinks our ‘Theory Of Mind’ faculty misleads us; as a quick heuristic it’s best to assume that others know their own mental states accurately, and so, it’s natural for us to think that we do too: that we have direct access and cannot be wrong about whether we, for example, feel hungry. But he thinks we know much less than we suppose about our own motives and mental states. On this he seems a little more moderate than Halligan and Oakley, allowing that conscious reflection can sometimes affect what we do.

I think the truth is that our mental, and even our conscious processes, are in fact much more complex and multi-layered than these discussions suggest. Let’s consider the causal efficacy of conscious thought in simple arithmetic. When I add two to two in my head and get four, have the conscious thoughts about the sum caused the conscious thought of the answer, or was there an underlying non-conscious process which simply waved flags at a couple of points?

Well, I certainly can do the thing epiphenomenally. I can call up a mental picture of the written characters, for example, and step through them one by one. In that case the images do not directly cause each other. If I mentally visualise two balls and then another two balls and then mentally count them, perhaps that is somewhat different? Can I think of two abstractly and then notice conceptually that its reduplication is identical with the entity ‘four’? Carruthers would deny it, I think, but I’m not quite sure. If I can, what causal chain is operating? At this point it becomes clear that I really have little idea of how I normally do arithmetic, which I suppose scores a point for the sceptics. The case of two plus two being four is perhaps a bad example, because it is so thoroughly remembered I simply replay it, verbally, visually, implicitly, abstractly or however I do it. What if I were multiplying 364 by 5? The introspective truth seems to be that I do something akin to running an algorithm by hand. I split the calculation into separate multiplications, whose answer I mainly draw direct from memory, and then I try to remember results and add them, again usually relying on remembered results. Does my thinking of four times five and recalling that the answer is twenty mean there was a causal link between the conscious thought and the conscious result? I think there may be such a link, but frustratingly if I use my brain in a slightly different way there may not be a direct one, or there may be a direct one which is not of the appropriate kind (because, say, the causal link is direct but irrelevant to the mathematical truth of the conclusion).

Having done all that, I realise that since I’m multiplying by five, I could have simply multiplied by ten, which can be done by simply adding a zero (Is that done visually – is it necessarily done visually? Some people cannot conjure up mental images at all.) and halving the result. Where did that little tactic come from? Did I think of it consciously, and was its arrival in reportable condition causally derived from my wondering about how best to do the sum (in words, or not in words?) or was it thrust into a kind of mental in-tray (rather spookily) by an unconscious part of my brain which has been vaguely searching around for any good tricks for the last couple of minutes? Unfortunately I think it could have happened in any one of a dozen different ways, some of which probably involve causally effective conscious states while others may not.

In the end the biggest difference between me and the sceptics may come down to what we are prepared to call conscious; they only want the states I’m calling self-conscious. Suppose we take it that there is indeed a metaphorical or functional space where mental items become ‘available’ (in some sense I leave unclarified) to influence our behaviour. It could indeed be a Global Workspace, but need not commit us to all the details of that theory. Then they allow at most those items actually within the space to be conscious, while I would allow anything capable of entering it. My intuition here is that the true borderline falls, not between those mental items I’m aware of and those I merely have, but between those I could become aware of if my attention were suitably directed, and those I could never extract from the regions where they are processed, however I thought about it. When I play tennis, I may consciously plan a strategy, but I also consciously choose where to send the ball on the spur of the moment; that is not normally a self-conscious decision, but if I stopped to review it it easily could become one – whereas the murky Freudian reasons why I particularly want to win the game cannot be easily accessed (without lengthy therapy at any rate) and the processes my visual cortex used to work out the ball’s position are forever denied me.

My New Year resolution is to give up introspection before I plunge into some neo-Humean abyss of self-doubt.