A somewhat enigmatic report in the Daily Telegraph says that this problem has been devised by Roger Penrose, who says that chess programs can’t solve it but humans can get a draw or even a win.

I’m not a chess buff, but it looks trivial. Although Black has an immensely powerful collection of pieces, they are all completed bottled up and immobile, apart from three bishops. Since these are all on white squares, the White king is completely safe from them if he stays on black squares. Since the white pawns fencing in Black’s pieces are all on black squares, the bishops can’t do anything about them either. It looks like a drawn position already, in fact.

I suppose Penrose believes that chess computers can’t deal with this because it’s a very weird situation which will not be in any of their reference material. If they resort to combinatorial analysis the huge number of moves available to the bishops is supposed to render the problem intractable, while the computer cannot see the obvious consequences of the position the way a human can.

I don’t know whether it’s true that all chess programs are essentially that stupid, but it is meant to buttress Penrose’s case that computers lack some quality of insight or understanding that is an essential property of human consciousness.

This is all apparently connected with the launch of a new Penrose Institute, whose website is here, but appears to be incomplete. No doubt we’ll hear more soon.

60 Comments

  1. 1. Luís Ferreira says:

    I’m not fond of ambiguous sentences. “chess programs can’t solve it” (right now) shouldn’t be confused with “chess programs can’t solve it” (never, ever, due to their nature). While the former may be true, the latter is bound to become false, given enough time.

    If only people knew computers a bit better… most people are still back in the times of “computers can only do what you tell them to do”.

  2. 2. Jochen says:

    On the face of it, that’s a somewhat bizarre claim—at least, as usually understood: it’s clear that a sufficiently powerful computer will eventually find the solution, just by brute force; so there’s nothing in principle uncomputable about the puzzle (so it’s not some chess analogue of Penrose’s Gödelian argument).

    Instead, it’s argued that the brute-forcing strategy will quickly exhaust all available computational resources, which may well be a true claim—but then, this merely tells us that human brains don’t work by brute forcing through all options. Which of course nobody claims they do.

    In the end, if somebody finds a solution, then it’ll be something that can be written down, and proven, in a short series of steps. But then, at least in principle, a computer can likewise find that proof. Perhaps not a traditional chess program; but that then just tells us something about the way we write chess programs, not about the difference between human and artificial intelligence.

    (Of course, this ignores that no computer ever really ‘plays chess’; rather, there are some computational processes whose states we can interpret as chess moves. But that’s another matter.)

  3. 3. Hunt says:

    I think the computational remedy of conundrums like this is something like the distinction of higher order logics to predicate and first order logic, that is logics that account for statements about statement (about statement…etc.) You can’t simply reason (in the computational sense) about problems. A machine must have the capacity to reason about reasoning, or reason about reasoning about reasoning, etc. Only then can things like “hopeless outcome” or “it is not worth continuing” be formalized.

  4. 4. bob cousins says:

    > this merely tells us that human brains don’t work by brute forcing through all options.

    That is the only point of this puzzle.

    However, the real point of the puzzle is that it is a “publicity stunt” for the launch of the new Penrose Institute. In that respect it has some degree of success, people are talking about it.

    In a context of Dawkins and memes, the puzzle takes a different light. The real power of human insight is to apply new contexts.

  5. 5. Callan S. says:

    If it’s already in a draw state, then why would it be remarkable that humans can get a draw state?

    And how does he know humans can get a win? I presume a genuine win, not in a ‘this game sucks, I’m throwing the match’ way

  6. 6. David Bailey says:

    Assuming that existing chess programs do not recognise this particular bizarre situation, I think what this illustrates is that given any particular failing of an AI program, it is always possible to graft on a little extra code to eliminate that one deficiency.

    The problem is that a collection of ways of solving particular problems, is not the same as actual understanding – even if it can fool people into believing that the program understands what it is doing.

    Let’s imagine a car that can drive itself, and imagine for a moment that we mean the real thing – a car that would drive form a house in the suburbs, all the way to another location (i.e. not a car that would only drive on specially prepared highways). A moment’s thought will show that the task of writing such a program is totally open ended. For example, suppose that while driving, I see a lorry with a really badly secured load, I probably recognise that situation and pull back or try to overtake the vehicle. I once met a riderless horse loose on a motorway, etc etc. The number of such situations is essentially limitless. Driving, like so many other tasks, requires an ability to apply our understanding of the real world – not just an ability to follow particular algorithms – even though these may seem to approach the performance desired.

    This is what Penrose’s example illustrates – a program that doesn’t really understand (which maybe includes all computer programs) can always be fooled with carefully crafted examples.

  7. 7. Michael Murden says:

    To Jochen – I’ll bite. What’s the difference between what computers do when they ‘play chess’ and what humans do when they play chess?

    To David Bailey – Yes, but in fairness humans can also be fooled by carefully crafted examples.

    This reminds me of something from a few posts ago about Doug Lenat and the CYC project. The idea of teaching computers common sense seems to be based on the idea that we know to some extent how humans acquire common sense and the ability to apply it to their interaction with the world. We don’t know even a little bit (from a neuromechanical perspective) about how humans do this, so I would agree that attempts to incorporate the kind of common sense that would allow humans to understand chess positions that never occur in actual chess games are premature, but to claim that doing so is perpetually impossible is to ascribe supernatural characteristics to human mentation. I think that’s premature as well.

  8. 8. David Bailey says:

    To Michael Murden

    Agreed – we can be fooled, but it is worth remembering that that was a fairly simple chess problem – you didn’t need to be anywhere near the Grandmaster level at which these programs play, to understand it!

    This is an example of what is referred to as the brittleness of traditional AI (as opposed to artificial neural nets) and it is a common problem. Each time the AI program can be patched to fix a specific issue, but other problems turn up – rather reminiscent, I think, of the way the set of axioms in Gödel’s theorem can be expanded to deal with an undecidable statement, but that doesn’t stop other undecidable statements cropping up indefinitely.

    I think this does illustrate that just because a program can play chess, it does not mean that it understands chess – just as a program doing anything – solving quadratic equations, for example – does not understand what it is doing. For that reason, I don’t think it is reasonable to expect that a computer program playing brilliant chess tells us anything about human (or animal) consciousness.

  9. 9. Jochen says:

    What’s the difference between what computers do when they ‘play chess’ and what humans do when they play chess?

    Well, a computer program stands to a game of chess in the same relation as a text does to its content: it needs to be interpreted in the right way, while mental content doesn’t (on pain of circularity). Saying ‘this device computes x’, like saying ‘this text describes y’, always involves an act of interpretation that, in principle, could be different—we could imagine a language such that the text makes perfect sense in that language, but describes something else entirely; and likewise, we could imagine an interpretation such that the computer doesn’t play chess, but does something else entirely. The interpretation of the text or the machine states aren’t intrinsic, but at least to some degree arbitrary, in contradistinction to mental states, which at least appear to have definite content.

  10. 10. VicP says:

    So what is the goal for a computer to solve the chess problem? Human brains are evolved over thousands of years to solve the environmental problems of find the food supply, avoid the predator, produce offspring….or the goal directedness is a survival instinct shared by earlier organisms. It really is the underlying instinct for thought experiments like the trolley problem. Perhaps the problem involves not doing proper modeling of a computer program for the survival instinct and complex human visual system which can “see” the problem and get the answer more readily for an evolved human game like chess.

  11. 11. zarzuelazen says:

    Hi,

    The theories of Penrose are completely daft. The chess problem is trivial for humans *and* computers. You can see this easily simply by noting that any line of reasoning you can come up with, the *same* line of reasoning can be programmed into a computer.

    Penrose thinks that human intelligence can’t be computational because we seem to be able to bypass Godel’s theorem and make creative mathematical leaps of thought. Since computers are limited by Godel (says Penrose), we can do things computers’ can’t.

    But his arguments don’t hold any water. There’s a far simpler explanation as to how we can beat Godel that doesn’t involve any non-computability.

    The Godel theorems are only limitations on *formal* (deductive) systems. To beat these limitations, we simply use *non-deductive* methods, called induction and abduction. But these non-deductive methods are still computational, so there’s no reason why computers can’t deploy them as well.

    Induction is pattern recognition and probabilistic reasoning. We notice patterns and extrapolate them to conclusions that aren’t certain – instead we use probabilities.

    Abduction is about making inferences to the best explanation based on Occam’s razor and the notion of explanatory coherence – the idea here is that the answers most likely to be true are the simplest one that integrate concepts into a coherent theory. Again, these concluions aren’t certain though.

    So to sum up, the use of induction and abduction is what lets us beat Godel, but the conclusions reached by these methods aren’t certain and are still entirely computational.

    So there is no reason whatsoever for thinking that humans are doing anything non-computational.

  12. 12. Hunt says:

    I have to admit, I’m closer to zarzuelazen than John Searle, but for those who haven’t seen it, here Searle and Flordi making a long and particularly militant case:

    https://www.youtube.com/watch?v=KmEuKkV3Y8c

    Note: For anyone familiar with the argument, you’re not going to learn anything new, so save yourself the two hours.

  13. 13. vicp says:

    We know the answer is 6 or solve it as 1*6 2*3 3*2 6*1 or we know that we want to retrieve an item from the other side of the room without predicting all of the intermediate calculations. Better put nice deductive proofs and calculations are great for math books and chess programs but not how we naturally think.

    Speaking of survival instinct, there is no thinker more infatuated with the survival instinct than Sam Harris whether he admits it or not in his critique of religion. Fascinating listening if you are looking for the survival instinct in his thinking:

    https://youtu.be/p8TDbXO6dkk

  14. 14. zarzuelazen says:

    Exactly right, vicp, we aren’t using deduction most of the time, so Godel limitations don’t apply.

    Even for mathematical insight, deduction isn’t what mathematicians are usually doing, except at the end.

    The great mathematician Gauss had notebooks filled with *numerial* calculations(non-deductive methods). A lot of math is experimental,based on spotting patterns in the results of numerial calculations- so math isn’t really that much different from other sciences , it uses a mixtures of theoretical insights *and* experimental (numerial) data.

    Since the Godel limitation applies only to deduction, it’s not a real limitation on fully general reasoning methods.

  15. 15. Peter says:

    these non-deductive methods are still computational,

    I don’t think so. A formalisation of induction would surely refute Hume (and possibly Turing) and generally be a pretty big deal, wouldn’t it?

    Yes, you can get a human to spot the interesting patterns, or notice the unexpected problem, and then write in procedures to deal with them. But only after the fact.

  16. 16. Michael Murden says:

    Peter, maybe you and Zarzuelazen should get together and hash out what you mean by ‘computation.’ Just out of curiosity do you think what the neural nets you wrote about a few posts ago do is computation?

  17. 17. Jochen says:

    A formalisation of induction would surely refute Hume (and possibly Turing) and generally be a pretty big deal, wouldn’t it?

    In fact, induction can be formalized; and upon doing so, one sees that it’s not computable (so the formalization isn’t effective). This uses algorithmic information theory in order to arrive at a universal probability distribution, which associates a prior probability to any given object; however, that probability is uncomputable (even if one stipulates—as one must for the approach to work—that the world follows computable laws). Nevertheless, one can find approximations, and even prove that in the limit, agents based on this principle (such as Hutter’s AIXI) will perform asymptotically as well as special-purpose problem solvers on arbitrary problems.

    However, that humans take recourse to non-deductive reasoning methods is not an argument against Gödelian strategies, since the way humans derive the truth of the Gödel sentence is manifestly deductive—for a system F with Gödel sentence G, we have (roughly):

    1. F does not prove G (by Gödel’s argument)
    2. F is consistent
    3. G is ‘F does not prove G’

    From these, the conclusion that G is true follows deductively. However, this deduction can’t be carried out within F, since (by the second incompleteness theorem) F does not prove that F is consistent (but if F were not consistent, then it could both prove and not prove G, and hence, G would be wrong, so the premise is necessary).

    Hence, deducing the truth of G requires the ability to reason outside of F; but this violates a key premise of the argument, since it was stipulated that F describes the reasoning capacities of human beings. But if it, in fact, does, then we can’t derive the truth of the second premise, and consequently, can’t establish the truth of G. Hence, the argument is circular: we can derive the truth of a Gödel sentence if we can carry out reasoning outside of the system it’s the Gödel sentence of; but then, the argument simply doesn’t tell us anything about whether human reasoning capacities can be described by means of a formal system.

  18. 18. TonyK says:

    Any decent chess computer would refuse to touch this problem, on the grounds that the board is illegally placed: the bottom left square has to be (logical) black.

    [Probably my fault, the result of the way I reduced the original to monochrome… Peter :(]

  19. 19. zarzuelazen says:

    Penrose is about 20 years behind in his knowledge of the AI field apparently.

    The game of ‘Go’ is massively more complex than chess, and in fact ‘Go’ is the strongest possible test of pattern recognition (induction).

    The leading machine learning methods basically are completely equivalent to an approximation of induction (or probability theory). These entirely computational methods had no trouble at all beating the top Go player Lee Sedol in the famous match with Deepmind last year, and demonstrated super-human pattern recognition abilities.

    The field of mathematics doesn’t progress in the way Penrose thinks. As I mentioned, it’s actually a *combination* of theory AND numerical calculations. For instance, there is no deductive proof of the Goldbach conjecture- the evidence it’s true comes not from deduction, but from simple empirical calculation to test it.

  20. 20. Hunt says:

    Jochen,
    I always meant to ask in similar contexts:

    The interpretation of the text or the machine states aren’t intrinsic, but at least to some degree arbitrary, in contradistinction to mental states, which at least appear to have definite content.

    Apart from the fact that you don’t seem very sure about this, what makes them appear to have definite content?

  21. 21. Peter says:

    @Michael Murden: frankly not sure whether neural nets might involve computational processes subserving others that are not fully computational. What do you think?

  22. 22. Jochen says:

    Hunt:

    Apart from the fact that you don’t seem very sure about this, what makes them appear to have definite content?

    Well, to me, the simplest answer would be that they just *have* definite content—i.e. that my thinking about chess simply is about chess (although it doesn’t work quite that simply on my model). It certainly doesn’t seem to be the case that I could ‘interpret’ my thinking about chess differently; however, it’s the basic characteristic of anything information-based that it can be interpreted in arbitrary ways (a string of bits gives you no hint regarding what it’s about, whether it’s a game of chess or a letter to grandma).

    But then, there’s always someone who pipes up that this is just some sort of illusion, or metacognitive error, or what have you—hence the qualifier (which also serves as a reminder to the various stripes of eliminativists that this appearance is a bit of data their theories need to account for, which they typically don’t seem to do a very good job of).

  23. 23. Hunt says:

    Jochen, Thanks for the reply. I was afraid the question might be interpreted as dumb (well, maybe it is, but probably not much dumber than related questions). Since it seems the appearance of certain content is brushed over too quickly, I thought it might be useful to pause on it for a moment.

    It certainly doesn’t seem to be the case that I could ‘interpret’ my thinking about chess differently

    No, but you could be wrong about chess. For instance, you might not know that castling is a move. Would you then still be playing chess? You might be playing something you definitely intend, but what designates chess (the real one). A set of rules.

  24. 24. Jochen says:

    Hunt:

    No, but you could be wrong about chess. For instance, you might not know that castling is a move.

    Sure. But that doesn’t make the content of my thoughts any less definite; it simply makes me wrong when I call it ‘chess’, when it’s actually ‘chess-without-castling’. In fact, the possibility of such error implies that it is definite: there’s no objective way anyone of us would be wrong if I translate the string ‘01011001’ as ‘Y’, while you translate it as ’89’; but if I think chess doesn’t include the castling rule, then I’m objectively wrong about that.

  25. 25. Hunt says:

    Jochen,
    At least mistakes like this tell us we’re not dealing with something like ideal Platonic forms. The only reason you might be wrong is due to the concrete definition, or by a social convention. A person can live an entire life utterly misinformed about certain things, yet society still functions. Probably the case is true of all things. Perhaps “dog” means something slightly different to me than you. As long as there is enough semantic overlap, the world keeps turning.

    I know the “semantic network” idea has been rehashed before, but it continues to be the case that what something means must (I would say MUST) stand in relation to other things. Indeed, it’s hard to see how it could be otherwise. If my notion of “dog” isn’t determined by memory of dogs, first hand experience with dogs, knowledge of mammals, etc., then what could it be based upon? If I’m wrong about semantic relationships that determine dog, then I’m wrong about dog.

    When we hold in mind “dog” the brain is performing its amazing capacity to turn experience and knowledge into the integrated intention that points to dog. If any eliminativism should be invoked I think it should be here. We don’t hold everything in mind when we intend “dog”. That, I think, is an illusion. Therefore intention seems less mysterious to me than others, but still pretty mysterious.

    However,

    In this light it doesn’t seem so hard to imagine a computer, following the same relationships, formulating a similarly accurate depiction of “dog”, and prone to the same mistakes.

  26. 26. Jochen says:

    Hunt:

    At least mistakes like this tell us we’re not dealing with something like ideal Platonic forms.

    Not that I think we do, in any way, but I’m not sure what your argument is, here—the error is in the fit of mental content and world, not in the content itself: I could view a cylinder side-on and conclude it’s a square, and hence, um, ‘partake’ mentally of platonically ideal squareness, instead of the accurate cylindricality; but that wouldn’t make said squareness any less platonically perfect. (But all in all, I think Platonism falls to the same pitfalls as Cartesian dualism does.)

    However,
    In this light it doesn’t seem so hard to imagine a computer, following the same relationships, formulating a similarly accurate depiction of “dog”, and prone to the same mistakes.

    Yes, this is the reply that always comes up on this issue. I genuinely fail to see why, though: the impossibility of imbuing symbols with inherent meaning doesn’t become any less impossible, just because you have more symbols with whatever ‘relationships’ between them. The problem is exactly the same as trying to decipher the meaning of a page of coded text by adding further pages of coded text—it buys you precisely nothing, simply because the meaning isn’t contained in the coded text, but rather, in the combination of the coded text and its intended interpretation. Without it, there simply is no meaning there; and it’s exactly the same with computers.

    You can establish the meaning of some symbols via their relationships exactly if you can design a ‘universal language’ which is decipherable for anybody without any prior knowledge. But that’s clearly impossible.

    And mathematically speaking, the only information contained within a set of tokens and their relationships is just the number of tokens: the relationships between them simply tell you nothing about the meaning of the tokens.

  27. 27. Hunt says:

    Jochen,
    The symbols themselves don’t do anything. A program manipulating symbols does “do” something. You are going to respond that what the program does is open to interpretation. That becomes less and less tenable with complexity, but you’re going to say it becomes more and more tenable. This is a point of disagreement. (I just spared us two days of back and forth).

    I would say that the ambiguity of what a society of machines is up to is roughly comparable with the ambiguity of whether we’re playing real chess or chess w/o castling. At least there is a practical/nontrivial argument here.

  28. 28. Jochen says:

    Hunt:

    A program manipulating symbols does “do” something. You are going to respond that what the program does is open to interpretation.

    Well, it’s not quite right that it’s open to interpretation, rather than that it needs interpretation to actually do anything (besides just following the laws of physics in its evolution). So even if the interpretation were unique, that doesn’t mean no interpretation is needed; hence, even if additional complexity helped to narrow down possible interpretations (which is just false, mathematically), this wouldn’t touch on the basic problem.

    Even if the only possible interpretation of the symbols (physical machine states) ‘2’, ‘3’, ‘*’ and ‘6’ were as the corresponding numbers and the operation of multiplication, it would still have to be so interpreted in order for a machine to actually perform that operation; because without the interpretation, it simply follows a succession of physical states, none of which are the numbers 2, 3, or 6. Without the interpretation, what a computer does is just the same as what a stone does rolling down a hill; just as, without interpretation, symbols simply don’t refer to anything.

  29. 29. Hunt says:

    Even if the only possible interpretation of the symbols (physical machine states) ‘2’, ‘3’, ‘*’ and ‘6’ were as the corresponding numbers and the operation of multiplication, it would still have to be so interpreted in order for a machine to actually perform that operation;

    If that calculation were done in service to some other process, say figuring out somebody’s taxes, would you say the operation is as meaningless as if it’s done alone?

  30. 30. Jochen says:

    Hunt:

    If that calculation were done in service to some other process, say figuring out somebody’s taxes, would you say the operation is as meaningless as if it’s done alone?

    Yes, of course. But I think I see what you’re driving at: the elements of the world are not open to interpretation—a tax form is a tax form, a dog is a dog, and so on; so the computation going on inside a machine filling out tax forms, or in a robot walking the dog, can be rooted in this real-world definiteness, sort of ‘inheriting’ it for themselves. It doesn’t seem to make sense to describe a dog-walking robot as carrying out a computation corresponding to a chess program.

    But the problem with such stories is that they fallaciously mix two levels of description: the computation allegedly going on within the robot mind, and the already-interpreted elements of the real world. Here, I say ‘already interpreted’ because dogs, tax forms, trees, clouds and so on are not simply elements out there in the world; they are aspects of the way we conceptually coordinatize that world. The story of the robot walking the dog is told from the perspective of somebody to whom things like ‘dog’, ‘robot’, and so on, have conceptual meaning—to whom these things are already interpreted. It’s this interpreter’s perspective that grounds what apparent meaning there is within the computation performed by the robot.

    Eliminating all the conceptual access to the elements of the world from the tale makes it obvious that this does not provide a way to ground the meaning of the computation, since it already depends on grounded meaning. All of the meaning in the tale simply comes from the perspective of its narrator, a perspective which we can’t help but assume, without noticing we do so—which is what makes the problem so pernicious.

    Alternatively, to make a consistent tale, you could try simply describing it at a fully physical level: certain patterns of sensory data excite certain circuits, which excite other circuits, which drive motors and actuators; but at this level, you have no need to talk about computation or mind at all, and have nevertheless completely explained the robot’s performance.

    So, you can tell the tale at either of two levels: in intentional terms, with explicit recourse to concepts, reference, and meaning—in which case, it’s no wonder that the robot’s computation appears to mean something, but only because there is meaning in the narrative from the beginning—or in basic physical terms (note that I don’t intend here to make it seem as if intentional terms were non-physical; I’m talking about different modes of description, not different modes of being), in which case nothing like meaning etc. will be at all necessary for the tale.

    But what you can’t do—and what is the basic switcheroo behind such attempts to ground meaning in the alleged ‘real world’—is mix these two levels of description in order to make it seem as if the conceptual elements just ‘bleed over’ into the robot’s mind, because then, you’re simply engaging in circular reasoning. This is surprisingly hard to do, and the error is difficult to spot, simply because conceiving of things in terms of reference and meaning is just so natural to the human mind; but I think overcoming this error is essential to start and formulate a true theory of the mind.

    One thing that helps is to try and imagine the tale from the point of view of another robot, to whom the dog, the tax form, clouds and whatever else are themselves just particular patterns of bits; since those need to be interpreted in order to refer to anything themselves, they don’t help with the interpretation of the computation that the tax form filling robot performs (on pain of incurring a vicious regress).

  31. 31. Hunt says:

    This is surprisingly hard to do, and the error is difficult to spot, simply because conceiving of things in terms of reference and meaning is just so natural to the human mind; but I think overcoming this error is essential to start and formulate a true theory of the mind.

    The question is what do you have left after throwing out a generalized system of symbol processing. If we go digging into the brain we find neurons, synapses, neurotransmitters, etc. This is hand-waiving, but in general terms these are 21st century versions of Leibniz’s mill. There is no more semantic meaning in a neurotransmitter molecule than there is a string of bits, or a gear on an axle; yet the brain does seem to accomplish the task.

    I’m not going to touch the Hard Problem, and I would be quite satisfied with a mere simulation of how the brain integrates information. In fact, I would be satisfied with Google results that appeared to grasp the meaning of my queries. If these would still be meaningless, perhaps I can tempt you to consider that they may still capture, roughly, brain activity.

    If I say to you “A Disney mouse with large ears” and the name “Mickey Mouse” comes to your mind, the process by which the name is accessed might ultimately be rote symbol processing. Speculatively, if that process were replaced with an interface to a computer, the result would have no meaning until you interpreted it. However, elucidation of the process would be no less pertinent to the operation of the mind. It’s for reasons like this that I object to Searle’s objection to AI.

    And of course this is but one example. Most of the operations of our brains is unconscious, observer/interpretation independent. I hate to say, but Searle seems to obsess on the Cartesian Theater. Not that I think this is wrong. I think “the observer” is a profound mystery, as is the whole of phenomenology. It’s just not all we should be paying attention to.

  32. 32. Hunt says:

    I’m not sure if you slogged through the talk I linked in 12 (you probably don’t want to), but Searle would no doubt speak of “the brain’s biochemical processes,” as if there’s a bit more magic in that than talking about gears and cogs. There’s no magic in chemistry (barring a fundamental discovery, Cf. Penrose).

  33. 33. Jochen says:

    Hunt:

    There is no more semantic meaning in a neurotransmitter molecule than there is a string of bits, or a gear on an axle; yet the brain does seem to accomplish the task.

    Yes, because there is a crucial difference between neurotransmitters, gears and axles on the one hand, and bit strings on the other: the former don’t depend on interpretation; they simply are just what they are. Although it’s not really right to say that they are, e.g., gears and axles—gears and axles aren’t part of the world, but merely, part of how we coordinatize the world using concepts; different coordinatizations are possible where one wouldn’t find any talk of ‘gears’ and ‘axles’, but which nevertheless describe the same physical matters of fact.

    In contrast, a bit string is just a pattern of differences within some medium—say, a pattern of black and white marbles. The marbles, once again, just are what they are, but what the bit string they realize means is something dependent on the observer; consequently, bit strings can simply never suffice to underwrite an explanation of the observer themselves, since without an observer, there’s really just the pattern of black and white marbles. Information isn’t a fundamental quantity of the world, because it’s only something that arises due to the presence of minds; no mind, no information. Likewise, there is no computation out there in the world without a mind interpreting the behavior of a certain physical system as standing for something else.

    Basically, computers are simply something like universal modeling clay. Take, for instance, a small-scale model of the Venus de Milo: you can use this as a representation of the real thing, in order to, say, take measures of the relation between the distance of the eyes to the width of the mouth—if it is a faithful model (i.e. if the modelling relationship holds for the domain you’re interested in studying), this gives you information about that relationship for the real thing. But in order to do so, you need to interpret that small clay figurine as standing for the real Venus: in the absence of that interpretation—say, if you don’t know the original, and only have access to the ‘model’—it doesn’t tell you anything about the original; it simply is what it is, just as the original is what it is.

    This relationship obtains between patterns of information (or more accurately, patterns of differences) and what we choose them to represent. A computer is special in the regard that it can be interpreted as basically anything (as long as it has enough states to bear the required structure). That makes them useful; it also makes immediately clear that something like explaining the mind on the basis of computation, or ‘simulating’ the world, are just misunderstandings: just as the small clay model of the Venus de Milo isn’t itself the Venus de Milo, but just a different small clay figurine that may be interpreted as standing for the Venus de Milo, so is the pattern of electronic excitations within a computer not a world, or a mind, but something that may be interpreted as standing for a world, or a mind; but in itself, it’s simply a pattern of electronic excitations.

    Hence, it’s from such things—patterns of excitations, gears and axles, neurotransmitters, and so on, i.e. from things as they really are in themselves—that one must build minds; not from any alleged computation they might perform, since as soon as we say ‘x computes’, we say ‘x is being interpreted as (something)’, which is inevitably circular. A way must be found to make these things meaningful to themselves, such as to eliminate the dependence on outside observers; that’s what I try to do with the von Neumann construction.

    In fact, I would be satisfied with Google results that appeared to grasp the meaning of my queries.

    The problem is that there is simply no such thing as a ‘Google result’ out there in the world, without you interpreting a particular pattern of lit pixels on a screen as ‘Google results’. Computational entities are not part of the world; they’re part of the way we conceptualize the world. Unless this distinction is properly made and scrupulously upheld, no theory of the mind can ever get off the ground.

    There’s no magic in chemistry (barring a fundamental discovery, Cf. Penrose).

    Agreed: there is no magic in chemistry (or gears and cogs, or neurotransmitters). But there is mind-independent reality, there is definiteness, which is exactly what computation lacks. And that’s all that’s needed.

  34. 34. VicP says:

    If physics is true, there is tons of magic in chemistry which we do not understand for most chemical reactions. The forces of nature do not need to be thoroughly understood for most objective chemical processes. Perhaps for understanding the mental or first person we can objectively analyze to the point of normal science but the explanatory gap reveals the gap of understanding for the physical forces at the biochemical level, hence we stay stuck with computationalism and other ways of only looking at the problem through a mirror and a hall of mirrors.

  35. 35. Michael Murden says:

    To Peter (21)
    All computations are algorithms but not all algorithms are computations. To take symbolic inputs, manipulate them according to a set of rules and give symbolic outputs is to perform a computation. To take eggs, flour, chocolate etc as inputs and give pastries as outputs is not (at least as I see it) to perform a computation. In that sense I agree with Jochen’s assertion that symbols must be interpreted by a non-symbolic entity (such as a human being) in order to have meaning. The next logical question, whether neural networks can be non-symbolic entities, is perhaps premature.

  36. 36. Hunt says:

    Hence, it’s from such things—patterns of excitations, gears and axles, neurotransmitters, and so on, i.e. from things as they really are in themselves—that one must build minds; not from any alleged computation they might perform, since as soon as we say ‘x computes’, we say ‘x is being interpreted as (something)’, which is inevitably circular.

    Since processors, buffers, memory, etc. are things in themselves, it sounds like your proscription is against anything that might be modeled out of these base elements, in other words, anything formally computed. This amounts to a general rule for building a mind, even from “things in themselves” like gears, cogs and axles. If the operation of the gears, cogs and axles could be closely enough modeled (simulated) then that would indicate a fruitless step. In fact, if at each step a team were to pause and attempt to document and recap progress, it would seem to indicate they were on the wrong path. No formal description of what they were doing would ever be possible.

    The problem is that there is simply no such thing as a ‘Google result’ out there in the world, without you interpreting a particular pattern of lit pixels on a screen as ‘Google results’. Computational entities are not part of the world; they’re part of the way we conceptualize the world. Unless this distinction is properly made and scrupulously upheld, no theory of the mind can ever get off the ground.

    The result might be compiled by Larry Page himself (in his off hours). Saying a search result will never appear intelligent is a stronger assertion than saying it will never be intelligent. This was one of my objections to Floridi (in the talk I linked), who has sat on the Turing contest panel. Even if you and he and Searle are entirely correct, that doesn’t preclude the appearance of intelligence.

  37. 37. Jochen says:

    Hunt:

    Since processors, buffers, memory, etc. are things in themselves, it sounds like your proscription is against anything that might be modeled out of these base elements, in other words, anything formally computed.

    You can (conceivably) build a mind from processors, buffers and so on; but the mind won’t arise from the computation they perform, because without any mind already there, there is no computation. ‘Computation’ is just a mode of description of the operation of a certain system; without intentional minds, there are no such modes of description. Before the emergence of mind, there was no computation in the universe. Consequently, computation can’t be prior to mind. Describing a system as computing pi, e.g., is the same thing as taking the word ‘dog’ to mean a particular furry quadruped. And just as there are no words meaning certain things without interpretation, there is likewise no computation.

    The result might be compiled by Larry Page himself (in his off hours).

    Certainly. And the same thing would hold in this case: whatever Larry hands you is only a search result if you interpret it as such. For instance, if he were to compile them in Chinese, you wouldn’t be able to make use of them; it might as well be a recipe for Chicken Kung Pao. There’s two parts to all content: the symbols, code, syntactic rules, and so on, and their interpretation. The latter is incredibly easy to miss, again because it comes so natural to us; but just as fish can’t swim without the water they don’t notice, just as we can’t breathe without the air that’s invisible to us, there is no content-production without the interpretation (if all you have to start with are syntactic entities, that is).

    Even if you and he and Searle are entirely correct, that doesn’t preclude the appearance of intelligence.

    But said appearance of intelligence is only there if things are interpreted in the right way. Again, try judging the intelligence of a device that only produces Chinese output (presuming you don’t know Chinese). So there might be an appearance of intelligence if you interpret a certain state of the computer as ‘Hi, my name is Mycroft’; but upon a different interpretation, that state—that configuration of electronic excitations, lit pixels on a screen, ink marks on a printout, whatever—might be interpreted entirely differently. And of course, without interpretation, it would mean nothing at all; it would simply be a pattern of electronic excitations, or lit pixels, or ink marks. Without interpretation, there is not even an appearance of intelligence.

  38. 38. Hunt says:

    You can (conceivably) build a mind from processors, buffers and so on; but the mind won’t arise from the computation they perform, because without any mind already there, there is no computation.

    If this hardware mind could be built, then by the basic computational equivalence between hardware and software (hardware computation can be implemented by software, and vice versa), a program, written in some standard language, performing the same computation, might well, in fact probably would not, be a mind? From whatever process the mind arose, the computation would be merely epiphenomenal?

  39. 39. Hunt says:

    Actually, let me rephrase that a bit to fit the argument. Could anything be inferred about the operation of the mind from my interpretation of the computation?

  40. 40. Jochen says:

    Hunt:

    If this hardware mind could be built, then by the basic computational equivalence between hardware and software (hardware computation can be implemented by software, and vice versa), a program, written in some standard language, performing the same computation, might well, in fact probably would not, be a mind?

    The hardware mind does not compute. Software is really just another layer of abstraction, facilitating the interpretation of hardware as computing; but, once more, without interpretation, there’s no such thing as computation. The ‘hardware’ simply evolves according to the laws of physics, as any physical system does; one can then interpret this as a computation, but that’s only an additional gloss upon the system, something it’s being used to do, not something it inherently does.

    Could anything be inferred about the operation of the mind from my interpretation of the computation?

    No. You could interpret my brain as running a chess engine while I’m dreaming about eating ice cream in the sun. Computation is a gloss upon the evolution of a physical system, bestowed on it by an interpreting mind.

  41. 41. Jochen says:

    Perhaps it helps getting clear on this point to look at a specific example: a DNA-computer playing the game of Tic Tac Toe. For this purpose, nine bins are filled with different strands of DNA; the human player, wanting to make their mark in bin i, introduces DNA-strand i into all bins. This leads to chemical changes in every bin, with one becoming fluorescent—that’s the move the DNA-computer performs.

    We’ve got two clearly delineated levels here: first, the level of biochemistry, where chemical reactions occur (or fail to) based on which DNA-strands are introduced into the bins. Second, we have the level of the game of Tic Tac Toe being played—the computational level. Now, the key insight is that the computational level is just an interpretation of the basic biochemical level as something else—a game of Tic Tac Toe. Without this interpretation, there would be no computation, and no Tic Tac Toe-playing.

    However, what’s occurring independently of all interpretation is the biochemical reactions, or more broadly, the physical evolution of the system we’re considering. That’s the level on which we must look for mind, since that’s the only level there is, if we don’t surreptitiously introduce mind in the beginning, making the whole deal circular. Thus, mind is like the biochemical reactions going on in the bins; it’s not like the Tic Tac Toe game those reactions are mapped to, because anything necessitating such a mapping already requires a mind being present.

  42. 42. Jochen says:

    nine bins are filled with different strands of DNA

    For correctness, this ought to be ‘are filled with different DNA enzymes’.

  43. 43. Hunt says:

    The idea of a mind created from hardware components is interesting due to the strong functionalism inherent in the components themselves. Given that it’s hard to imagine what mental state might arise from a shift register, it’s doubly hard to imagine it wouldn’t in some way be related to “shift registering”. I don’t want to make more of the example than it’s worth, especially since you probably just interpreted it as a rhetorical device; however even the result that a mind could not be composed of such elements would be significant. What elements would or would not be permissible (a rhetorical question).

    A hardware mind could be translated to software. Since hardware/software translation is relatively unambiguous, it’s hard for me to imagine the program not being in some way relevant.

    On a slightly different tangent, I was reading the Quora page titled “What are some objections to Searle’s Chinese Room thought experiment?” (looking for help!). I know the Chinese Room argument isn’t quite the same as the interpretation argument. I think it took Searle another ten years to hit upon the IA, but it’s fair to say the two are related.

    https://www.quora.com/What-are-some-objections-to-Searles-Chinese-Room-thought-experiment

    There are many points about being wary of appeals to intuition, which I think are valid. Let me give one example from one of the top rated comments by Josh Siegle, a neuroscience Phd. A small excerpt:

    As a scientist, I’m fully aware that the world is filled with seemingly paradoxical scenarios that turn out to be true. But this claim seems over the line [Hunt: the equivalence of mental and computational states]. If instead of a string of Chinese characters, the man received a string of ones and zeros encoding a visual scene, would the room be having its own, separate visual experience while the man moves some paper around and reads the ink that adorns it? People make it sound like Searle was bonkers for claiming that such subjective experience wouldn’t arise. But what makes you so certain that it would?

    Now, this is an extraordinary thing for a neuroscientist to say, but I’m not blaming Josh. I have made and make as grave appeals to intuition. This is the reason we must be wary. If a page of Chinese characters isn’t enough to convince dubious readers, surely a string of bits could never conjure a visual image. But Josh has to realize that the optic nerve

  44. 44. Hunt says:

    Quote botch and premature submit. Here’s the whole thing over again…

    The idea of a mind created from hardware components is interesting due to the strong functionalism inherent in the components themselves. Given that it’s hard to imagine what mental state might arise from a shift register, it’s doubly hard to imagine it wouldn’t in some way be related to “shift registering”. I don’t want to make more of the example than it’s worth, especially since you probably just interpreted it as a rhetorical device; however even the result that a mind could not be composed of such elements would be significant. What elements would or would not be permissible (a rhetorical question).

    A hardware mind could be translated to software. Since hardware/software translation is relatively unambiguous, it’s hard for me to imagine the program not being in some way relevant.

    On a slightly different tangent, I was reading the Quora page titled “What are some objections to Searle’s Chinese Room thought experiment?” (looking for help!). I know the Chinese Room argument isn’t quite the same as the interpretation argument. I think it took Searle another ten years to hit upon the IA, but it’s fair to say the two are related.

    https://www.quora.com/What-are-some-objections-to-Searles-Chinese-Room-thought-experiment

    There are many points about being wary of appeals to intuition, which I think are valid. Let me give one example from one of the top rated comments by Josh Siegle, a neuroscience Phd. A small excerpt:

    As a scientist, I’m fully aware that the world is filled with seemingly paradoxical scenarios that turn out to be true. But this claim seems over the line [Hunt: the equivalence of mental and computational states]. If instead of a string of Chinese characters, the man received a string of ones and zeros encoding a visual scene, would the room be having its own, separate visual experience while the man moves some paper around and reads the ink that adorns it? People make it sound like Searle was bonkers for claiming that such subjective experience wouldn’t arise. But what makes you so certain that it would?

    Now, this is an extraordinary thing for a neuroscientist to say, but I’m not blaming Josh. I have made and make as grave appeals to intuition. This is the reason we must be wary. If a page of Chinese characters isn’t enough to convince dubious readers, surely a string of bits could never conjure a visual image. But Josh has to know that the optic nerve encodes images in a manner as abstracted from the actual scene as a string of bits, however intuitively this seems absurd.

  45. 45. Jochen says:

    Hunt:

    A hardware mind could be translated to software. Since hardware/software translation is relatively unambiguous, it’s hard for me to imagine the program not being in some way relevant.

    Hardware computation and software computation are equivalent—trivially so: both just involve interpreting certain states (machine states, program states) as symbolic entities. But both require a mind doing the interpreting. So there is no program without this interpretation; consequently, programs of any kind can’t possibly be relevant for the emergence of mind, since that would presuppose mind in order to yield mind.

    But Josh has to know that the optic nerve encodes images in a manner as abstracted from the actual scene as a string of bits, however intuitively this seems absurd.

    Again, that’s a level confusion, akin to claiming that what the DNA-system I described above does is playing Tic Tac Toe. The optic nerve doesn’t encode anything; encoding is a concept that only arises once you’ve introduced interpretation. Introducing it at the ground level immediately dooms any theory.

  46. 46. Hunt says:

    Let me just do this one more time, then I’ll drop it. You can of course ignore me at any time and get on with your life 🙂 So here goes:

    1. Posit a mind made of hardware. This doesn’t contradict your argument (even if it’s impossible).

    In some sense, we’re already done, since my interpretation of the computation done by the hardware can have no bearing on the operation of the mind, which I find very (ahem) counter-intuitive. But, we continue:

    2. The hardware can be directly translated to software, say a gigantic Java program.

    3. Even though this program would functionally duplicate the computational operation of the hardware in the rote sense that every hardware component would have a software routine, nothing about the mind could be determined by it.

    I’m just saying I find this interesting; I remain neutral (for now).

    The optic nerve doesn’t encode anything; encoding is a concept that only arises once you’ve introduced interpretation. Introducing it at the ground level immediately dooms any theory.

    But every optic nerve, provided it’s in a living person, is connected to the rest of the brain, which is doing the interpreting. One way or another, whether it’s frame based or not (it’s not, I’m sure), whatever gets visually perceived must pass down that channel. That encoded data, doubtless torrents of serial and parallel nerve impulses, are as removed from what we consider visual perception as any string of bits fed to a Chinese Room; so Josh’s appeal to non-intuition is merely an argument from personal incredulity. To him it seems absurd that a string of bits might be turned into visual perception, which is quite odd given that he’s a neuroscientist and presumably engaged in that very type of thing.

    I mean, it’s odd for more basic reasons as well, particularly that strings of bits are how most images are stored these days.

  47. 47. David Duffy says:

    “encoding is a concept that only arises once you’ve introduced interpretation” – so bee language is an encoding because other bees interpret it using their minds? And when we refer to the DNA encoding proteins?

  48. 48. Jochen says:

    Hunt:

    2. The hardware can be directly translated to software, say a gigantic Java program.

    No. You can use another physical system in such a way that you interpret it as a computational model of the ‘hardware mind’; but that isn’t in any way a ‘translation’ of it to software. It’s the same thing as taking an orrery, and interpreting it as the solar system; or taking a clay figurine, and interpreting it as the Venus de Milo.

    At the bottom, it’s like when a little kid claims her rag doll wants tea: the rag doll does not, thereby, actually acquire any desires. It’s exactly the same to interpret a given physical system as a computational simulation of a mind: this mere descriptive act does not imbue it with actual intentionality, because ‘being interpreted as a mind’ and ‘being a mind’ are two very different things. Most significantly, the former depends on pre-existing mentation; the latter doesn’t, and only in this way can we hope to create a basis for a theory of mind.

    But every optic nerve, provided it’s in a living person, is connected to the rest of the brain, which is doing the interpreting.

    Again, no: the brain doesn’t interpret its inputs—this is the homunculus fallacy. The optic nerve transmits electrochemical patterns, causing other electrochemical patterns within the brain; these don’t encode anything, they don’t refer to anything, they don’t compute anything, they simply are what they are. And if my speculations on the von Neumann replication process are on point, then it’s by a sort of ‘feedback’ process that one can then build notions such as ‘meaning’ from them.

    David:

    so bee language is an encoding because other bees interpret it using their minds? And when we refer to the DNA encoding proteins?

    We should distinguish here between different notions of information—typically, syntactic (or Shannon) information, pragmatic information, and semantic information. The latter is where we want to get: symbolic vehicles having some certain meaning.

    The bee’s waggle dance, however, carries only pragmatic information: the form of the dance causes other bees to do something. It’s a sort of instruction, or set of commands. It doesn’t have any more semantic information than a key has for a lock—it simply fits into certain expectations, pushes certain buttons, closes certain circuits, what have you. Of course, an intentional agent could look at the bee’s dance, interpret it, and garner semantic information about the whereabouts of flowers from it; but, assuming the bees do not themselves possess intentional agency, there’s nothing going on beyond one bee’s moves triggering another bee’s flight.

    The same goes, even more clearly, for DNA encoding proteins: it’s the form of the DNA that triggers, ultimately, amino acids being strung together to form proteins. Again, an intentional mind can look at a string of DNA and extract semantic information from it, by interpreting sets of three nucleobases as standing for certain amino acids; but no such interpretation is carried out on the level of gene transcription, nor does it need to.

    Really, a lot of this confusion originates with the term ‘information’ being indiscriminately applied to syntactic, pragmatic and semantic phenomena. Shannon ‘information’ is really nothing but the capacity to bear (semantic) information, while pragmatic ‘information’ is nothing but good, old physical causality. The question is how to construct semantic information out of these ingredients; but most answers to date (including everything mentioning ‘computation’, ‘encoding’, and so forth) really are just a bit of sleight of hand, equivocating on the different notions of ‘information’.

  49. 49. Hunt says:

    Again, no: the brain doesn’t interpret its inputs—this is the homunculus fallacy.

    Yes, I can see where it would be good to keep a strict definition of “interpret” as what a mind does. Even neuroscientist will use the language loosely, for instance “this area of the brain interprets” or even worse “the neuron interprets…” At the back of my mind I always think this is anthropomorphism. You don’t think anthropomorphism applies to elements of the brain, but I guess it’s as valid an objection as anthropomorphizing inanimate objects.

    Really, a lot of this confusion originates with the term ‘information’ being indiscriminately applied to syntactic, pragmatic and semantic phenomena.

    The field can use as much rigor as it can get.

  50. 50. David Duffy says:

    “semantic information…where we want to get: symbolic vehicles having some certain meaning.”

    Don’t you feel that you are somewhat begging the question, given that the computationalist viewpoint is that there is only physical causality? In the bee dance, there are meanings associated with each “expression”, even though we can’t get bees to explicate them for us. There is reference, and a certain amount of compositionality, in terms of the number of repetitions mapping onto distance. And there is flexibility and limited reasoning, in that other bees may not respond to your description of flowers (on a boat) out in the middle of the lake that presumably is also represented in their mental map of the area (I don’t know if that experiment has been repeated).

  51. 51. Jochen says:

    David:

    Don’t you feel that you are somewhat begging the question, given that the computationalist viewpoint is that there is only physical causality?

    Some may claim so, but in fact, such an eliminativist route isn’t really an option for the computationalist: if there’s nothing else, then there is also no computation. The claim that a given system computes something entails a claim that it (or its states) can be taken as symbolic for whatever it computes—i.e. that a computer computing a simulation of the solar system stands to the actual planets in the same relation as the word ‘dog’ stands to dogs, or the number of lights at the Old North Church stands to the British invasion strategy.

    If you deny that such relationships exist, you also deny that there is any computation—because then, the ‘computer’ you claim ‘simulates the solar system’ doesn’t do anything of that sort, but rather, merely follows its own physical evolution. Without its states, output, whatever standing for planetary positions, movements, and so on, there simply is no claim that it ‘simulates the solar system’. But this ‘standing for’ is just semantic information.

    So that’s indeed the foundational paradox of computationalism: if it appeals to some sort of representation, it can’t itself be used to explain how representation works; if it doesn’t appeal to representation, there simply is no computation.

    As for the bee dance, any appeal to ‘reference’ to explain the effect it has on other bees is simply superfluous, just as claiming that a strip of bimetal ‘refers to’ the temperature in a thermostat. The temperature causes the bimetal to bend, which causes current to flow (or not), which starts heating (or stops it). Again, an intentional mind could use the bendedness of the bimetal as referring to temperature; but without being so used, the strip of metal carries no semantic information.

    The same goes for the bees: the presence of flowers causes a behavior pattern, which causes behavior patterns in other bees. It’s not the sole cause of behavior, other causes may interfere to produce different behavior, but there’s nothing we need to add to the analysis here other than a web of cause-and-effect relationships.

  52. 52. David Duffy says:

    “the foundational paradox of computationalism: if it appeals to some sort of representation, it can’t itself be used to explain how representation works”. It is a brute fact that there is a (semantic) mapping from the codes emitted by the bee and the state of the world in terms of geographical distribution of useful calories – if it was not a correct mapping, the bees involved would die out. If I spend X calories on appropriate computation now, I can obtain 10X calories for tomorrow.

    Crist [2004]: “On the basis of criteria intuitively and deliberately abstracted, scientists have represented the honeybee dance as a bona fide linguistic system. I discuss how the dance is understood as rule-governed; both structurally stable and contextually flexible; symbolic in representing states of affairs distant in space and time; and performative, whether described as announcement, order, report, and so on, or translated into utterances that announce, order, report, and the like…`The honeybee dance language is a formal apparatus which itself is context-free, in such ways that it can, in local instances of its operation, be sensitive to and exhibit its
    sensitivity to various parameters of social reality in a local context’ [Sacks et al 1974].”

    https://www.researchgate.net/profile/Guenther_Witzany/publication/260420279_Communicative_Coordination_in_Bees/links/54958b450cf2ec13375b2967.pdf

    “Subsequent social interaction with bees of the same age is important to develop meaningful dances: carrying out linguistic behaviour and heeding the calls for specific action require some degree of practice and experience in participating in mutual interactions…Interestingly, bees recognise the sun as having a 24-h course, so that they can carry out their dance at the correct angle vis-à-vis the sun even in the dark…”

    Moving to another type of solar system simulation, there is an objectively estimable correlation (mutual information, whatever) between the movement of electrons in the computer running that program (or in the bee’s brain!) and the actual solar system. This is not so different from the correlation between photons of light hitting my retina and the Evening Star as I am looking at it, the correlation between those retinal events and neural events in higher visual centres, and between the neural events in my cortex and yours as you are reading this. How do I know this? There are various empirical tests we can do, such as measuring EEG synchrony in our brain regions, or setting ourselves to problem solving using the information imparted.

  53. 53. Jochen says:

    David:

    It is a brute fact that there is a (semantic) mapping from the codes emitted by the bee and the state of the world in terms of geographical distribution of useful calories – if it was not a correct mapping, the bees involved would die out.

    No. Just that something reliably covaries with (is correlated with, has nonzero mutual information with…) something else does not mean that it contains (semantic) information about it. Just because some vehicle is correlated with some state of affairs doesn’t make that vehicle into a representation of that state of affairs—such correlation is necessary, but not sufficient. The number of lights lit at the Old North Church reliably covaries with the British invasion strategy; but just knowing how many lights are lit does not tell you anything about that strategy if you don’t also know ‘one if by land, two if by sea’.

    That is, having the correlation isn’t enough: you must also know how to decode it. And this decoding is itself a piece of knowledge, represented to you in some way (by mere virtue of the fact that it is available to you). Consequently, attempting to cash out that knowledge again in terms of correlations is just going to run in circles.

    Again, this doesn’t mean that you can’t extract semantic information from the waggle dance, and in most circumstances, a little bit of anthropomorphism is perfectly harmless, just as in neuroscientific literature one often reads that ‘cortex’ interprets, wants, decides whatever. But in order to understand bee behavior, all one needs is covariance, without necessitating any mention of semantic information (that isn’t ultimately rooted in the confusion between syntactic and semantic information I noted earlier).

    Moving to another type of solar system simulation, there is an objectively estimable correlation (mutual information, whatever) between the movement of electrons in the computer running that program (or in the bee’s brain!) and the actual solar system.

    And of course, this mutual information is simply Shannon information, syntactic information, which may be used to carry semantic information, if interpreted or decoded correctly, but which does not inherently do so. Take again the number of lights at the Old North Church: you could equally validly decode its meaning as ‘Paul Revere hung two lights’.

    Moreover, if correlation were sufficient for reference, then pretty much everything would refer to everything else—the number of lanterns, for example, is also correlated with Paul Revere’s brain state, and likewise to the rider sent out to Lexington; but it would be absurd to say that said rider’s being sent out refers to Paul Revere’s brain state. Indeed, since effects always are correlated with their causes, one should expect them to refer the full set of such causes, ultimately going back to the initial conditions of the universe—but that’s not what we’re experiencing.

    Yet, with the right knowledge about physical laws—the right decoding mechanism—one can indeed gather information about the early universe from observing, say, a pencil: again, correlation is necessary—but far from sufficient.

  54. 54. Tom Clark says:

    Jochen and David, fwiw I found this quote from Roy Brassier (from a chapter called “Concepts and Objects” in the book The Speculative Turn: Continental Materialism and Realism) in a Facebook discussion, thought it was relevant to your conversation:

    “Kant underscored the difference between knowing, understood as the taking of something *as* something, classifying an object under a concept, and sensing, the registration of a somatic stimulus. Conception is answerable to normative standards of truth and falsity, correctness and incorrectness, which develop from but cannot be collapsed into the responsive dispositions through which one part of the world – whether parrot or thermostat – transduces information from another part of the world -sound waves or molecular kinetic energy. knowledge is not information: to know is to endorse a claim answerable to the norm of truth simpliciter, irrespective of ends. By way of contrast, the transmission and transduction of information requires no endorsement; it may be adequate or inadequate relative to certain ends, but never ‘true’ or ‘false’. The epistemological distinctiveness of the former is the obverse of the metaphysical ubiquity of the latter.”

  55. 55. David Duffy says:

    Hi Jochen. “correlation isn’t enough: you must also know how to decode it” – yes and no. You, I, Paul Revere and the bees need to have the appropriate informational and associated physical structures to take advantage of a signal or an environmental “tell”. But this is not mysterious.

    Here are Sclenker et al [2014] explaining why they feel safe performing “formal semantic analysis”
    of monkey alarm calls, which are significantly less complex than the bee language [my emphasis below, if it works on this website!]:

    http://link.springer.com/article/10.1007/s10988-014-9155-7

    In a strict sense, a semantic rule requires relatively little: a relation of denotation or satisfaction between primitive objects of a (usually discrete) system and parts of the world (e.g. objects or situations); and a way to extend this relation to a language obtained by combining these primitive objects into sequences. If the final relation is based on truth (yielding statements such as situation w satisfies sentence S if and only if ____), one can further posit that users of this system may have at their disposal a relation of entailment, with the condition that sentence S entails sentence S’ if and only if every situation that makes S true makes S’ true. Nothing in the definition of a semantic rule, or even of entailment, requires a high degree of rationality, let alone a theory of mind. In fact, even parts of pragmatics can be developed with relatively little machinery. For instance, in their simplest version scalar implicatures only require that subjects have at their disposal (i) a notion of satisfaction (to determine whether a sentence S – e.g. p or q – is compatible with the situation at hand); (ii) a notion of scalar alternatives (to determine whether the sentence S’ (e.g. p and q) competes with the sentence S); and (iii) a notion of entailment (to determine whether S’ is more informative than S). These three ingredients could suffice to yield the inference that if p or q was uttered, the more informative statement p and q is false.

    More generally, a prearranged code requires an earlier transmission of much more information between entities. In the bee case, the semantics is “hard coded” by particular gene products causing their nervous system to develop along a particular course.

    Hi Tom. Kant is a well known chauvinist regarding knowledge – personally I am happy to say that members of a bee hive know there is a good nesting site at a particular place, even though they can’t justify it to me. They wait until a number of scouts have visited the same spot, and return with a concurring report, then compare the different candidate sites until consensus is reached. There is a reason for this protocol, even though they cannot know what it is. Insert Wittgenstein quote about blind rule following here.

  56. 56. Jochen says:

    Hi Tom, thanks for the quote—although I have to confess I find it somewhat difficult to unpack. I think that, ultimately, the notion of knowledge articulated in the quote is yet something different than what I have in mind; the argument seems to be that knowing something is to stand in a special relation (that of endorsing a truth-evaluable claim) to a certain piece of information, so it’s similar to my claim that you need something more than (syntactic) information to get meaningful information in that regard.

    But in a sense, I’m not that interested in what it actually is that is needed; I’m content with pointing out that every attempt to paint syntactic information as meaningful (that I’ve encountered so far) smuggles in information that is already conceived of as meaningful at the ground level, thus merely kicking the can down the road.

    David:

    You, I, Paul Revere and the bees need to have the appropriate informational and associated physical structures to take advantage of a signal or an environmental “tell”. But this is not mysterious.

    Only if you gloss over it. But really, ‘having the appropriate structure’ is just once more the same question you attempted to answer in the first place—since that structure always includes having access to meaningful information. In order to decode the semantic information the number of lamps bears, we need to have access to meaningful information, connecting said number with the British invasion. So, pointing to the correlation helps nothing at all, as we’re still faced with the question of how we arrive at that semantic information to decode the correlation—which is exactly the original question.

    Consider, for instance, that the meaning of the signal would be reversed by merely changing the semantic information you, as an observer of the number of lights, has access to—if you (perhaps falsely) thought ‘two if by land, one if by sea’, then the same signal presented to you would switch meanings around. The logical conclusion from that is that the meaning is not in the signal (it can’t be, otherwise, you couldn’t invert it in this way), but rather, in the combination of the signal and the knowledge needed to decode it (represented to you as semantic information about precisely how the number of lamps and the British strategy are correlated).

    The quote you provide bears me out on this (my emphasis):

    In a strict sense, a semantic rule requires relatively little: a relation of denotation or satisfaction between primitive objects of a (usually discrete) system and parts of the world (e.g. objects or situations); and a way to extend this relation to a language obtained by combining these primitive objects into sequences.

    This relation of denotation is just the ‘interpretation’ or ‘semantic information’ or ‘decoding’ I’m harping on about: it associates symbols with their meanings. Once you have that in hand, of course you can do semantic analysis; and since humans are intentional agents, they’re capable of forming such relations. However, from the fact that we can form such relations, it doesn’t follow that the objects of our study can, too: otherwise, since I can associate the bendedness of a strip of bimetal with the temperature, it would follow that every thermostat is an intentional being. After all, one can certainly perform semantic analysis on thermostats: the denotation rule is given by the association of the bending angle of the bimetal with the temperature, and the sequence we order them in is simply their temporal succession. Thus, you get a sequence of temperatures as the meaning of the thermostat’s sequence of states.

    But of course, none of this means that the bending angle of the bimetal actually constitutes meaningful information about the temperature to the thermostat! All meaning here has been imported from the observer, i.e. you and me. The same happens when we read meaning into bee dances, ape vocalizations, or robot behavior (note that of course any and all of these may actually carry meaning to their subjects: it’s just that the mere fact that we can interpret them as meaningful carries no logical force towards a conclusion that they actually are).

    In the bee case, the semantics is “hard coded” by particular gene products causing their nervous system to develop along a particular course.

    Again, this is not semantics: it’s pragmatics. Like a lock fitting into a key, or indeed a bimetal bending a certain angle, the bee is constructed such that certain stimuli engender certain behavior. But this isn’t sufficient for semantic meaning—otherwise, the tap that causes a rock to roll down a slope would already carry semantic meaning to the rock.

    I realize that it is very hard to shed the habit of conceiving of the world in inherently semantic terms; but in order to make progress towards the foundations of a genuine theory of mind, we desperately must break it.

  57. 57. David Duffy says:

    Hi Jochen, as usual it seems to me that we have many points of agreement re mechanism in the real world. The difference seems to be about what it is that constitutes meaning and intentionality. You seem to hold that real intentionality only occurs in systems where there is the possibility of recursion or meta-level “processing” of a formal syntactic system. I see this as probing downwards from what you regard as the “typical” human being. I am more probing upwards. So, I would think that a 12 month old human who says “cup” in association with the appropriate act of ostension is exhibiting evidence of (low-level) intentionality (which actually precedes actual language use), but I think that you would regard this as some kind of aping of intentionality, as in the case of a chat-bot. At the other end of life, as I slowly lose cognitive function due to accumulation of neural plaques and tangles, again I get the impression that you would see a fairly early cut-off in this evolution once I am no longer able to do sophisticated things like, say, recursively embed phrases or understand complex circumlocutions.

  58. 58. David Duffy says:

    Hi again Tom. My comment re Kant was a little flippant, but came from the literature about animal cognition, as well as recently reading Jon Cogburn’s essay

    http://www.philpercs.com/2017/02/robert-brandom-and-richard-rortys-schocking-ableism.html

    starting from the amusing

    “a long history of Brandom talking about all mute animals as mere automatons (nearly any time he mentions parrots)”

  59. 59. Jochen says:

    David:

    You seem to hold that real intentionality only occurs in systems where there is the possibility of recursion or meta-level “processing” of a formal syntactic system.

    No, that’s not really what I’m getting at. In fact, I’ve so far refrained from giving a positive account of what I think is necessary for intentional cognition—my only argument is that computation, information processing, etc., are certainly not sufficient; that is, whatever systems may be intentional, they are not so due to carrying out some kind of computation.

    I don’t know exactly what systems are intentional—bees may well be, as may human infants. But it’s clear that just because we can conceive of a system in intentional terms, this doesn’t mean that it actually is intentional—anything showing a reliable correlation with some external factors can be talked about as if it was intentional, but ultimately, this does not give us any evidence regarding whether it actually is, or whether what intentionality is there, is mere projection.

    In this light, the computational theory of mind is in reality a kind of animism: because we are used to conceiving of entities in intentional terms, we project these terms on anything that seems vaguely human-like in behavior or appearance; but that doesn’t make computers minds anymore than a child claiming that her rag doll wants tea actually imbues said doll with any desires.

    As for my own beliefs, I think that if we want to come to a true understanding of intentionality, we must get around the need for the ‘relation of denotation’, since this relation always implies some external agency, making any account circular. Thus, I think in terms of symbols that supply their own meaning to themselves. Noting the structural equivalence between construction and representation, just as certain agents can be their own blueprints, certain patterns—in a cellular automaton or a neural network—can be their own representations. If you’re interested, a recent version of this account can be found here.

  60. 60. David Duffy says:

    Dear Jochen. I read your essay with pleasure. Invoking Von Neumann replicators and evolution seems pretty cybernetical to me. Once we have successful and unsuccessful replicators, then we have value, and the “internal model principle” then seems enough to me to give semantics, even if there is not necessarily a physical “partitioning into plant and controller components”.
    http://arxiv.org/pdf/q-bio/0309003.pdf

    I believe that there are several alternatives to denotational semantics. I quite like the little I understand on inferentialism and proof-theoretic semantics, though they are supposed to lead to anti-realism.

    You may have read Kirchhoff’s paper on the “life-mind continuity thesis”
    http://link.springer.com/article/10.1007/s11229-016-1100-6

    I brought up the examples of developing children etc to question the bright line view regarding intentionality – it can’t be that mysterious because even I can do it, using a brain that seems to use essentially the same architecture as other primates. In terms of energetics, my brain uses ~25 watts at rest and an additional 2 watts when cogitating. When asleep, this drops down to ~20 watts. In brain diseases, or in hypoxia, higher cognition falls away in parallel with drops in energy utilisation. Neanderthals probably ran on ~20 watts and H. erectus on 15 watts.
    http://rsos.royalsocietypublishing.org/content/3/8/160305
    This is linearly relatable to number of operations per second in the cortex (yes, I don’t know exactly what those operations are!), so maybe one needs just that much more. Speculations have been made on the intentional life of the Neanderthals based on some cultural evidence.

Leave a Reply