Picture: correspondent. Paul Almond’s Attempt to Generalize AI has reached Part 8:  Forgetting as Part of the Exploratory Relevance Process. (pdf)

Aspro Potamus tells me I should not have missed the Online Consciousness Conference.

Jesús Olmo recommends a look at the remarkable film ‘The Sea That Thinks’, and notes that the gut might be seen as our second brain.

An interesting piece from Robert Fortner contends that speech recognition software has hit a ceiling at about 80% efficiency and that hope of further progress has been tacitly abandoned. I think you’d be rash to assume that brute force approaches will never get any further here: but it could well be one of those areas where technology has to go backwards for a while and pursue a theoretically different approach which in the early stages yields poorer results, in order to take a real step forward.

A second issue of the JCER is online.

Alec wrote to share an interesting idea about dreaming:

“It seems that most people consider dreaming to be some sort of unimportant side-effect of consciousness. Yes, we know it is involved in assimilation of daily experiences, etc, but it seems that it is treated as not being very significant to conciousness itself. I have a conjecture that dreaming may be significant in an unusual way – could dreaming have been the evolutionary source of consciousness?
It is clear that “lower animals” dream. Any dog owner knows that. On that basis, I would conclude that dreaming almost certainly preceded the evolution of consciousness. My conjecture is this: Could consciousness possibly have evolved from dreaming?

Is it possible that some evolutionary time back, humans developed the ability to dream at the same time as being awake, and consciousness arises from the interaction of those somewhat parallel mental states? Presumably the hypothetical fusion of the dream state and the waking state took quite a while to iron out. It still may not be complete, witness “daydreams.” We can also speculate that dreaming has some desirable properties as a precursor to consciousness, especially its abstract nature and the feedback processes it involves.

Hmm.

Picture: Honeycomb series. Panpsychism, or perhaps it would be more accurate to say panexperientialism, has seemed to be quite a popular view in previous discussions here, but I’ve always found it problematic. Panexperientialism, as the name suggests, is the belief that experience is everywhere; that experience is the basis of reality, out of which everything else is built.  Objects which seem dead and inanimate ultimately consist of experience just as much as we do: it just doesn’t seem like that because the experiences which make them up are not our experiences.  The attractive feature of this view is that it removes some of the mystery from consciousness: instead of being a very rare phenomenon which only occurs in very specific circumstances, such as those which exist in our brains, consciousness of a sort is universal, and so it’ s not at all surprising that we ourselves are conscious.

One of the problems is the question of how many experiential loci we’re dealing with. Does the table have experiences? Does half the table? Do the table legs have four separate sets of experiences, and at the same time a sort of federal joint experience as a composite entity? There are ways to solve these problems, but they’re distinctly off-putting to me. More fundamentally, I’m inclined to doubt whether the theory is as helpful in explaining things as  it seems. OK, so my brain has experience just because it’s an object and all objects have experience; but surely that brain-as-an-object experience is the same kind of thing as the experiences rocks have, while the experience I’m interested in, the kind that influences my bodily behaviour, is something else; something which remains unexplained.

One of the sources of difficulty here, I think, as with many metaphysical theories, is that the philosophical point of view is not well integrated with any clear scientific conception.  When we need to pin down our loci of experience we’re left to rummage around and see what we can come up with – atoms? Too small.  Discrete physical entities? What exactly are they? (Shintoism, if I understand it correctly, has bitten this kind of bullet and given up on a sharp definition of what is animate: lots of things can have souls, but only if they’re salient or impressive. Mount Fuji definitely gets one, but some anonymous pile of dirt in your back garden is just a pile of dirt.)

But what about Finite Eventism? This theory (as expounded by Carey R. Carlson in Chapter 12 of Mind that abides: Panpsychism in the New Millennium) is a theory of physics first and foremost, one that happens to provide a neat basis for a solution to the mind-body problem taking a panpsychist/panexperientialist view. It is based on the late ideas of Russell and Whitehead, though one of its appealing features is that it dovetails well with quantum theory in a way Russell and Whitehead were not aware of.

The gist of the theory is a radical reduction of physics to a minimalist ontology consisting of events and the basic temporal relations of being earlier or later (there’s also cause and effect, which I take to be causally connected varieties of earlier/later). There are some rules which prevent inconsistency (events can’t be earlier/later than themselves) and that’s it. In particular, there is no initial concept of space or extension. We are allowed convergent and divergent causal paths, so we can construct complex multiple ‘honeycomb’ pathways like the one shown. The four consistent axes which appear in these diagrams are taken to make up a 4-D manifold of space-time, with neutrinos and electrons delineated by gaps, and the repetition of patterns in sequence representing persistence over time.  Quanta, in this theory, are represented by the steps between two events; the number of intermediate steps between two events corresponds with the relative frequency of the relevant path, and these relative frequency ratios provide relative energy ratios, following Planck’s E=hf.

I hope those brief,  inadequate remarks give a hint of how the basics of physics can be built up in a very elegant manner from the simple topology of these sequences: it looks impressive, though I must frankly admit that I’m not competent to explain the theory properly, never mind evaluate it. Readers may like to look at this short description (pdf).

The question for us is, what are these ‘events’? Carlson follows Whitehead in seeing them all as ‘occasions of experience’; in some ways they resemble the monads of Leibniz’s radically relativist ontology;  they are pure phenomenal moments. Carlson argues that the basis of all science is phenomenal experience; historically, in order to account for those experiences better through Newtonian style physics it became necessary to postulate unexperienced abstract entities; and then the phenomenal experience dropped out of the theory leaving us with a world made of fundamentally unaccountable entities.

The best part of it for me is that the theory provides relatively good and clear answers to the problems I mentioned above. It’s clear that the loci of experience are situated in the events: I take it that continuity of experience is guaranteed in exactly the same way as physical continuity by repeating patterns (though what the nature of those patterns amounts to is an interesting question).  If that’s so, then the relationship between the consciousness of my brain-as-object and the consciousness of my brain-as-brain is also helpfully clarified.

How far, though, is the actual nature of my consciousness clarified? The explanation of most of my mental characteristics is deferred upwards to be explained by the working of the brain. That is, no doubt, exactly as it should be: but it leaves me with no particular reason to adopt panpsychism. The one feature which the theory does explain is phenomenal experience (alright, a fairly important feature!); but it really only does so by telling us that things just do have phenomenal experience.  Why should we take it that the events are phenomenal in nature – doesn’t ontological parsimony suggest they should be featureless blips?

Still, I think this is the most viable and attractive formulation of panpsychism/panexperientialism I’ve seen.

Picture: Macaque. Can monkeys have blindsight? Sean Allen-Hermanson defends the idea in a recent JCS paper. Blindsight is one of those remarkable phenomena that ought to be a key to understanding conscious perception; but somehow we can never quite manage to agree how the key should be turned.  Blindsight, to put it very briefly, is where subjects with certain kinds of brain lesions deny seeing something, but can reliably point to it when prompted to try. It’s as though the speaking, self-conscious part of the brain is blind, but some other part, well capable of directing the hand when given a chance, can see as well as ever.

There are a number of ways we might account for blindsight. One of the simplest is to suppose that the visual system is degraded but not destroyed in these cases; the signals from the eye are still getting through, but in some way at reduced power. This reduced power level puts them below the limit required for entry into conscious awareness, but they are still sufficient to bias the subject towards the correct response when they are prompted to guess or have a random try. Another popular theory suggests that the effect arises because there are two separate visual channels, only one of which is knocked out in blindsight. There is a good neurological story which can be told in support of this theory, which weighs strongly in its favour;  against it, there have been reports of analogous phenomena in the case of other senses, where it is harder to sustain the idea of physically separate channels. Allen-Hermanson cites claims for touch, smell and hearing (I’ve wondered in the past whether the celebrated deaf percussionist Evelyn Glennie might be an example of “deafhearing”); and even suggestions that the case of alexithymia, in which things not consciously perceived nevertheless cause anxiety or fear, might be similar.  It’s possible, of course, that blindsight itself comes in more than one form with more than one kind of cause, and that there is something in both these theories – which unfortunately would make matters all the more difficult to elucidate.

For those of us whose main interest in is consciousness, blindsight holds out the tantalising possibility of an experimental route into the mystery of qualia, of what it is for there to be a way something looks.  It’s tempting to suppose that what is missing in blindsight patients is indeed phenomenal experience. Like the much-discussed zombies, they receive the data from their senses and are able to act on it, but have no actual experience.  So if we can work out how blindsight works, we’ve naturalised qualia and the hard problem is cracked…

Well, no, of course it isn’t really that easy. The point about qualia, strictly interpreted, is that they don’t cause actions;  qualia-free zombies behave just the same as normal people, and that includes speech behaviour. So the absence of qualia could have no effect on what you say;  since whatever blindsight patients are missing does affect what they say, it can’t be qualia.  Moreover we have no conclusive evidence that blindsight patients have no visual experience; it could be that they have the experience but are simply unable to report it. That might seem a strange state to be in, but patients with brain damage are known to assert or confabulate all sorts of things which are at odds with the evidence of their senses; in fact I believe there are subjects who claim with every sign of sincerity to see perfectly when in fact they are demonstrably blind, which is a nice reversal of the blindsight case.

Still, blindsight is a tantalising glimpse of something important about conscious experience, and has all sorts of implications. To pick out one at random, it casts an interesting light on split-brain patients.  In blindsight cases, we can have an apparent disconnect between the knowledge the patient expresses with the voice, and the knowledge expressed with the hand; that’s pretty much what we get in many experiments on split-brain patients (since normally only one hemisphere has use of the vocal apparatus and the other can only express itself by hand movements).  Any claims that split-brain patients are therefore shown to be two different people in a single skull are undercut unless we’re willing to take up the unlikely position that blindsight patients are also split people.

One interesting extension of blindsight research is the apparent discovery by Cowey and Stoerig of the same phenomenon in monkeys. There is an obvious difficulty here, since human blindsight experiments typically rely on the subject to report in words what they can see, something monkeys can’t do. Cowey and Stoerig devised two experiments; in the first the monkeys were trained to touch a screen where a stimulus appeared; all were able to do this without problems. In the second experiment, the stimulus did not always appear on cue; when it did not, the monkeys were required to press a separate button. Normal monkeys could do this without difficulty, but monkeys with lesions thought to be analogous to those causing blindsight now went wrong when the stimulus appeared in their blind spot, hitting the ‘no stimulus’  button. Taking the two experiments together, it was concluded that blindsight was effectively demonstrated; the damaged monkeys who could earlier touch the right part of the screen even when the stimulus was in their blind spot,  later ‘reported’ the same stimulus as absent.

(Readers may wonder about the ethical propriety of damaging the brains of living primates for these experiments; I haven’t read the original papers, but I suppose we must assume that at any rate the experiments had medical as well as merely philosophical value.)

Of course, these experiments differ significantly from those carried out on human subjects, and as Allen-Hermanson reports,  reasonable doubts were subsequently raised in a 2006 paper by Mole and Kelly, who pointed out that relying on two separate experiments, which made differing demands, made the results inconclusive.  In particular, the second task was more complex than the first, and it could plausibly be argued that the result of having to deal with this additional complexity was that the monkeys simply failed to notice in the second experiment the stimulus they had picked up successfully in the first.

Allen-Hermanson’s aim is to rescue Cowey and Stoerig’s conclusions, while acknowledging the validity of the criticisms. He proposes a new experiment: first the monkeys are trained to press a green button if there is a stimulus (no need to point to where it is any more), and a red one if there is none. Then we introduce two different stimuli: Xs and Os. Both the green and red buttons are now divided in two, one side labelled for X, the other for O. If there is a stimulus, the monkeys must now press either green X or green O depending on which appeared: if there is no stimulus, they can press either red button. Allen-Hermanson believes the blindsighted monkeys will consistently press red X correctly if the stimulus is X, even though they are effectively asserting that there is no stimulus.

Maybe. I can’t help feeling that all the monkeys will be puzzled by a task which effectively asks them to state whether a stimulus is present, and then, if not present, say whether it was an X or O. The experiment has not been carried out; but Allen-Hermanson goes on to suggest that Mole and Kelly’s alternative hypothesis is actually implausible on other grounds.  On their interpretation, for example, the blindsighted monkeys simply fail to notice a stimulus in their blind spot: yet it has been demonstrated that they cannot recognise objects as salient in monkey terms as ripe fruit when they are presented to the blind spot – so it seems unlikely that we’re dealing with something as simple as inattention.

What would it mean if monkeys did have blindsight? It would seem to show, at least, that monkeys are not automata; that they do have something which corresponds to at least one important variety of human consciousness. Allen-Hermanson proposes working further along the mammalian line, and he seems to expect that mammals and even some other vertebrates would yield similar results (he draws the line at toads).

At any rate, we’re left feeling that human consciousness is not as unique as it might have seemed.  I can’t help also feeling more strongly than before that the really unique feature of human awareness is the way it is shot through with language; we may not have the only form of consciousness, but we certainly seem to have the talkiest.

Picture: Martin Heidegger. This paper by Dotov, Nie, and Chemero describes experiments which it says have pulled off the remarkable feat of providing empirical, experimental evidence for Heidegger’s phenomenology, or part of it; the paper has been taken by some as providing new backing for the Extended Mind theory, notably expounded by Andy Clark in his 2008 book (‘Supersizing the Mind’).

Relating the research so strongly to Heidegger puts it into a complex historical context. Some of Heidegger’s views, particularly those which suggest there can be no theory of everyday life, have been taken up by critics of artificial intelligence. Hubert Dreyfus in particular, has offered a vigorous critique drawing mainly from Heidegger an idea of the limits of computation, one which strongly resembles those which arise from the broadly-conceived frame problem, as discussed here recently. The authors of the paper claim this heritage, accepting the Dreyfusard view of Heidegger as an early proto-enemy of GOFAI .

For it is GOFAI (Good Old Fashioned Artificial Intelligence) we’re dealing with. The authors of the current paper point out that the Heideggerian/Dreyfusard critique applies only to AI based on straightforward symbol manipulation (though I think a casual reader of Dreyfus  could well be forgiven for going away with the impression that he was a sceptic about all forms of AI), and that it points toward the need to give proper regard to the consequences of embodiment.

Hence their two experiments. These are designed to show objective signs of a state described by Heidegger, known in English as ‘ready-to-hand’. This seems a misleading translation, though I can’t think of a perfect alternative. If a hammer is ‘ready to hand’, I think that implies it’s laid out on the bench ready for me to pick it up when I want it;  the state Heidegger was talking about is the one when you’re using the hammer confidently and skilfully without even having to think about it. If something goes wrong with the hammering, you may be forced to start thinking about the hammer again – about exactly how it’s going to hit the nail, perhaps about how you’re holding it. You can also stop using the hammer altogether and contemplate it as a simple object. But when the hammer is ready-to-hand in the required sense, you naturally speak of your knocking in a few nails as though you were using your bare hands, or more accurately, as if the hammer had become part of you.

Both experiments were based on subjects using a mouse to play a simple game.  The idea was that once the subjects had settled, the mouse would become ready-to-hand; then the relationship between mouse movement and cursor movement would be temporarily messed up; this should cause the mouse to become unready-to-hand for a while. Two different techniques were used to detect readiness-to-hand. In the first experiment the movements of the hand and mouse were analysed for signs of 1/f? noise. Apparently earlier research has established that the appearance of 1/f? noise is a sign of a smoothly integrated system.  The second experiment used a less sophisticated method; subjects were required to perform a simple counting task at the same time as using the mouse; when their performance at this second task faltered, it was taken as a sign that attention was being transferred to cope with the onset of unreadiness to hand. Both experiments yielded the expected results.  (Regrettably some subjects were lost because of an unexpected problem – they weren’t good enough at the simple mouse game to keep it going for the duration of the experiment. Future experimenters should note the need to set up a game which cannot come to a sudden halt.)

I think the first question which comes to mind is: why were the experiments were even necessary?  It is a common experience that tools or vehicles become extensions of our personality; in fact it has often been pointed out that even our senses get relocated. If you use a whisk to beat eggs, you sense the consistency of the egg not by monitoring the movement of the whisk against your fingers, but as though you were feeling the egg with the whisk, as though there was a limited kind of sensation transferred into the whisk. Now of course, for any phenomenological observation, there will be some diehards who deny having had any such experience; but my impression is that this sort of thing is widely accepted, enough to feature as a proposition in a discussion without further support.  Nevertheless, it’s true that it this remains subjective, so it’s a fair claim that empirical results are something new.

Second, though, do the results actually prove anything? Phenomenologically, it seems possible to me to think of alternative explanations which fit the bill without invoking readiness-to-hand. Does it seem to the subject that the mouse has become part of them, part of a smoothly-integrated entity – or does the mouse just drop out of consciousness altogether? Even if we accept that the presence of 1/f? noise shows that integration has occurred, that doesn’t give us readiness-to-hand (or if it does, it seems the result was already achieved by the earlier research).

In the second experiment we’ve certainly got a transfer of attention – but isn’t that only natural? If a task suddenly becomes inexplicably harder, it’s not surprising that more attention is devoted to it – surely we can explain that without invoking Heidegger? The authors acknowledge this objection, and if I understand correctly suggest that the two tasks involved were easy enough to rule out problems of excessive cognitive load so that, I suppose, no significant switch of attention would have been necessary if not for the breakdown of readiness-to-hand.  I’m not altogether convinced.

I do like the chutzpah involved in an experimental attempt to validate Heidegger, though, and I wouldn’t rule out the possibility that bold and ingenious experiments along these lines might tell us something interesting.

Picture: Google chatbot. Bitbucket I was interested to see this Wired piece recently; specifically the points about how Google picks up contextual clues. I’ve heard before about how Google’s translation facilities basically use the huge database of the web: instead of applying grammatical rules or anything like that, they just find equivalents in parallel texts, or alternatives that people use when searching, and this allows them to do a surprisingly good – not perfect – job of picking up those contextual issues that are the bane of most translation software. At least, that’s my understanding of how it works.  Somehow it hadn’t quite occurred to me before, but a similar approach lends itself to the construction of a pretty good kind of chatbot - one that could finally pass the Turing Test unambiguously.

Blandula Ah, the oft-promised passing of the Turing Test. Wake me up when it happens – we’ve been round this course so many times in the past.

Bitbucket Strangely enough, this does remind me of one of the things we used to argue about a lot in the past.  You’ve always wanted to argue that computers couldn’t match human performance in certain respects in principle. As a last resort, I tried to get you to admit that in principle we could get a computer to hold a conversation with human-level responses just by the brutest of brute force solutions.  You just can a perfect response for every possible sentence. When you get that sentence as input, you send the canned response as output. The longest sentence ever spoken is not infinitely long, and the number of sentences of any finite length is finite; so in principle we can do it.

Blandula I remember: what you could never grasp was that the meaning of a sentence depends on the context, so you can’t devise a perfect response for every sentence without knowing what conversation it was part of.  What would the canned response be to;  ‘What do you mean?’  – to take just one simple example.

Bitbucket What you could never grasp was that in principle we can build in the context, too. Instead of just taking one sentence, we can have a canned response to sets of the last ten sentences if we like - or the last hundred sentences, or whatever it takes. Of course the resources required get absurd, but we’re talking about the principle, so we can assume whatever resources we want.  The point I wanted to make is that by using the contents of the Internet and search enquiries, Google could implement a real-world brute-force solution of broadly this kind.

Blandula I don’t think the Internet actually contains every set of a hundred sentences ever spoken during the history of the Universe.

Bitbucket No, granted; but it’s pretty good, and it’s growing rapidly, and it’s skewed towards the kind of thing that people actually say. I grant you that in practice there will always be unusual contextual clues that the Google chatbot won’t pick up, or will mishandle. But don’t forget that human beings miss the point sometimes, too.  It seems to me a realistic aspiration that the level of errors could fairly quickly be pushed down to human levels based on Internet content.

Blandula It would of course tell us nothing whatever about consciousness or the human mind; it would just be a trick. And a damaging one.  If Google could fake human conversation, many people would ascribe consciousness to it, however unjustifiably. You know that quite poor, unsophisticated chatbots have been treated by naive users as serious conversational partners ever since Eliza, the grandmother of them all. The internet connection makes it worse, because a surprising number of people seem to think that the Internet itself might one day accidentally attain consciousness. A mad idea: so all those people working on AI get nowhere, but some piece of kit which is carefully designed to do something quite different just accidentally hits on the solution? It’s as though Jethro Tull had been working on his machine and concluded it would never be a practical seed-drill; but then realised he had inadvertently built a viable flying machine. Not going to happen. Thing is, believing some machine is a person when it isn’t is not a trivial matter, because you then naturally start to think of people as being no more than machines.  It starts to seem natural to close people down when they cease to be useful, and to work them like slaves while they’re operative. I’m well aware that a trend in this direction is already established, but a successful chatbot would make things much, much, worse.

Bitbucket Well, that’s a nice exposition of the paranoia which lies behind so many of your attitudes. Look, you can talk to automated answering services as it is: nobody gets het up about it, or starts to lose their concept of humanity.

Of course you’re right that a Google chatbot in itself is not conscious. But isn’t it a good step forward?  You know that in the brain there are several areas that deal with speech;  Broca’s area seems to put coherent sentences together while Wernicke’s area provides the right words and sense. People whose Wernicke’s area has been destroyed, but who still have a sound Broca’s area apparently talk fluently and sort of convincingly, but without ever really making sense in terms of the world around them. I would claim that a working Google chatbot is in essence a Broca’s area for a future conscious AI. That’s all I’ll claim, just for the moment.

Picture: global workspace. Global Workspace theories have been popular ever since Bernard Baars put forward the idea back in the eighties; in ‘Applying global workspace theory to the frame problem’*,  Murray Shanahan and Baars suggest that among its other virtues, the global workspace provides a convenient solution to that old bugbear, the frame problem.

What is the frame problem, anyway? Initially, it was a problem that arose when early AI programs were attempting simple tasks like moving blocks around. It became clear that when they  moved a block, they not only had to update their database to correct the position of the block, they had to update every other piece of information to say it had not been changed. This led to unexpected demands on memory and processing. In the AI world, this problem never seemed too overwhelming, but philosophers got hold of it and gave it a new twist. Fodor, and in a memorable exposition, Dennett, suggested that there was a fundamental problem here. Humans had the ability to pick out what was relevant and ignore everything else, but there didn’t seem to be any way of giving computers the same capacity. Dennett’s version featured three robots: the first happily pulled a trolley out of a room to save it from a bomb, without noticing that the bomb was on the trolley, and came too; the second attempted to work out all the implications of pulling the trolley out of the room; but there were so many logical implications that it was stuck working through them when the bomb went off. The third was designed to ignore irrelevant implications, but it was still working on the task of identifying all the many irrelevant implications when again the bomb exploded.

Shanahan and Baars explain this background and rightly point out that the original frame problem arose in systems which used formal logic as their only means of drawing conclusions about things, no longer an approach that many people would expect to succeed. They don’t really believe that the case for the insolubility of the problem has been convincingly made. What exactly is the nature of the problem, they ask: is it combinatorial explosion? Or is it just that the number of propositions the AI has to sort through to find the relevant one is very large (and by the way, aren’t there better ways of finding it than searching every item in order?). Neither of those is really all that frightening; we have techniques to deal with them.

I think Shanahan and Baars, understandably enough, under-rate the task a bit here. The set of sentences we’re asking the AI to sort through is not just very large; it’s infinite. One of the absurd deductions Dennett assigns to his robots is that the number of revolutions the wheels of trolley will perform in being pulled out of the room is less than the number of walls in the room. This is clearly just one member of a set of valid deductions which goes on forever; the number of revolutions is also less than the number of walls plus one; it’s less than the number of walls plus two… It may be obvious that these deductions are uninteresting; but what is the algorithm that tells us so? More fundamentally, the superficial problems are proxies for a deeper concern; that the real world isn’t reducible to a set of propositions at all, that, as Borges put it

“it is clear that there is no classification of the Universe that is not arbitrary and full of conjectures. The reason for this is very simple: we do not know what thing the universe is.”

There’s no encyclopaedia which can contain all possible facts about any situation. You may have good heuristics and terrific search algorithms, but when you’re up against an uncategorisable domain of infinite extent, you’re surely still going to have problems.

However, the solution proposed by Shanahan and Baars is interesting. Instead of the mind having to search through a large set of sentences, it has a global workspace where things are decided and a series of specialised modules which compete to feed in information (there’s an issue here about how radically different inputs from different modules manage to talk to each other: Shanahan and Baars mention a couple of options and then say rather loftily that the details don’t matter for their current purposes. It’s true that in context we don’t need to know exactly what the solution is – but we do need to be left believing that there is one).

Anyway, the idea is that while the global workspace is going about its business each module is looking out for just one thing. When eventually the bomb-is-coming-too module gets stimulated, it begins sending very vigorously and that information gets into the workspace. Instead of having to identify relevant developments, the workspace is automatically fed with them.

That looks good on the face of it; instead of spending time endlessly sorting through propositions, we’ll just be alerted when it’s necessary. Notice, however, that instead of requiring an indefinitely large amount of time, we now need an indefinitely large number of specialised modules. Moreover, if we really cover all the bases, many of those modules are going to be firing off all the time. So when the bomb-is-coming-too module begins to signal frantically, it will be competing with the number-of-rotations-is-less-than-the-number-of-walls module and all the others, and will be drowned out. If we only want to have relevant modules, or only listen to relevant signals, we’re back with the original problem of determining just what is relevant.

Still, let’s not dismiss the whole thing too glibly. It reminded me to some degree of Edelman’s analogy with the immune system, which in a way really does work like that. The immune system cannot know in advance what antibodies it will need to produce, so instead it produces lots of random variations; then when one gets triggered it is quickly reproduced in large numbers. Perhaps we can imagine that if the global workspace were served by modules which were not pre-defined, but arose randomly out of chance neural linkages, it might work something like that. However, the immune system has the advantage of knowing that it has to react against anything foreign, whereas we need relevant responses for relevant stimuli. I don’t think we have the answer yet.

*Thanks to Lloyd for the reference.

Picture: correspondent. Aspro Potamos has drawn my attention to the emerging series of YouTube videos, somewhat polemical in tone, on the War against Neuroscience.   If you prefer something less forensic, you may like this series of podcasts on music and the brain from the Library of Congress.

Huping Hu, whose research has been mentioned here in the past, has started the Journal of Consciousness Exploration and Research: the inaugural issue is available online.

Ayad Gharbawi says:

I have been doing my own studies and research for thirty years now on three concepts: Mind, Vision and Reality. I felt it necessary to create novel notations, just as others, like Boole, did, because of the inadequacies of language as per this subject. I believe that I have come to an entirely new methodology when we seek to understand two fundamental issues when we study Mind, Vision and Reality and that is – both their Structure and the Function.

I have been deeply, fundamentally and existentially affected by Quantum Physics as a human being and in my own pattern of thinking and analyzing problems before me.  I obviously understand full well that my work is unorthodox because I am not just presenting a study in one particular niche in these studies – although, of course, I have studied specific issues – but, I am at this stage in my life, in a position to say that I have come to a general theory that comprises an understanding of Mind, Vision and Reality.

That is why, when I have presented one study/manuscript it is often difficult to make any sense from it, and the reason here is because, those single manuscripts that I have been submitting, do not, in and of themselves alone, explain fully my general theory on Mind, Vision and Reality.  I also know full well, that the history of science, shows far too many times, that when one researcher submits an entirely, unorthodox novel methodology in his thinking, he is quite likely to be rejected by the general established body of scientists and philosophers.

But, I still do try.

A sample is here; he particularly asks for views on two pieces here and here.

Picture: AI resurgent. Where has AI (or perhaps we should talk about AGI) got to now? h+ magazine reports remarkably buoyant optimism in the AI community about the achievement of Artificial General Intelligence (AGI) at a human level, and even beyond. A survey of opinion at a recent conference apparently showed that most believed that AGI would reach and surpass human levels during the current century, with the largest group picking out the 2020s as the most likely decade.  If that doesn’t seem optimistic enough, they thought this would occur without any additional fundingfor the field, and some even suggested that additional money would be a negative, distracting factor.

Of course those who have an interest in AI would tend to paint a rosy picture of its future, but the survey just might be a genuine sign of resurgent enthusiasm, a second wind for the field (‘second’ is perhaps understating matters, but still).  At the end of last year, MIT announced a large-scale new project to ‘re-think AI’. This Mind Machine Project involves some eminent names, including none other than Marvin Minsky himself. Unfortunately (following the viewpoint mentioned above) it has $5 million of funding.

The Project is said to involve going back and fixing some things that got stalled during the earlier history of AI, which seems a bit of an odd way of describing it, as though research programmes that didn’t succeed had to go back and relive their earlier phases. I hope it doesn’t mean that old hobby-horses are to be brought out and dusted off for one more ride.

The actual details don’t suggest anything like that. There are really four separate projects:

  • Mind: Develop a software model capable of understanding human social contexts- the signpost that establish these contexts, and the behaviors and conventions associated with them.
    Research areas: hierarchical and reflective common sense
    Lead researchers: Marvin Minsky, Patrick Winston
  • Body: Explore candidate physical systems as substrate for embodied intelligence
    Research areas: reconfigurable asynchronous logic automata, propagators
    Lead researchers: Neil Gershenfeld, Ben Recht, Gerry Sussman
  • Memory: Further the study of data storage and knowledge representation in the brain; generalize the concept of memory for applicability outside embodied local actor context
    Research areas: common sense
    Lead researcher: Henry Lieberman
  • Brain and Intent: Study the embodiment of intent in neural systems. It incorporates wet laboratory and clinical components, as well as a mathematical modeling and representation component. Develop functional brain and neuron interfacing abilities. Use intent-based models to facilitate representation and exchange of information.
    Research areas: wet computer, brain language, brain interfaces
    Lead researchers: Newton Howard, Sebastian Seung, Ed Boyden

This all looks very interesting.  The theory of reconfigurable asynchronous logic automata (RALA) represents a new approach to computation which instead of concealing the underlying physical operations behind high-level abstraction, makes the physical causality apparent: instead of physical units being represented in computer programs only as abstract symbols, RALA is based on a lattice of cells that asynchronously pass state tokens corresponding to physical resources. I’m not sure I really understand the implications of this – I’m accustomed to thinking that computation is computation whether done by electrons or fingers; but on the face of it there’s an interesting comparison with what some have said about consciousness requiring embodiment.

I imagine the work on Brain and Intent is to draw on earlier research into intention awareness. This seems to have been studied most extensively in a military context, but it bears on philosophical intentionality and theory of mind; in principle it seems to relate to some genuinely central and difficult issues.  Reading brief details I get the sense of something which might be another blind alley, but is at least another alley.

Both of these projects seem rather new to me, not at all a matter of revisiting old problems from the history of AI, except in the loosest of senses.

In recent times within AI I think there has been a tendency to back off a bit from the issue of consciousness, and spend time instead on lesser but more achievable targets. Although the Mind Machine Project could be seen as superficially conforming with this trend, it seems evident to me that the researchers see their projects as heading towards full human cognition with all that that implies (perhaps robots that run off with your wife?)

Meanwhile in another part of the forest Paul Almond is setting out a pattern-based approach to AI.  He’s only one man, compared with the might of MIT – but he does have the advantage of not having $5 million to delay his research…

Picture: Socrates looking at himself. Introspection, the direct examination of the contents of our own minds, seems itself to be in many minds at the moment.  The latest issue of the Journal of Consciousness Studies was devoted to papers on introspection, marking the tenth anniversary of the publication of The View from Within, by Francisco Varela and Jonathan Shear (which was itself a special edition of the JCS); and now Eric Schwitzgebel has produced a new entry for the Stanford Encyclopedia of Philosophy.

The two accounts are of course quite different in some respects. The encyclopaedia entry is a careful, scholarly account, neutral and comprehensive; the JCS issue is openly a rallying-cry in support of a programme flowing from Varela’s work.  This, it seems, called for an end to the ban on examination of lived experience;  the JCS gives the impression that it was something of a milestone, though Schwitzgebel’s piece does not mention it (he does cite an earlier paper by Varela, once again in the JCS).

What’s all this about a ban? Well, back in the nineteenth century, psychologists had no fears about using introspective evidence; it was thought that a proper scientific effort would lead to an objectively verifiable kind of phenomenology. We should be able to classify the elements of mental experience and clarify how they worked together, just by examining what went on in our own heads. A great deal of work was done on all this (It was a great disappointments for me to discover, on first opening Brentano’s Psychology from an Empirical Standpoint, that it consisted almost entirely of this kind of thing, and that the only passage about intentional inexistence, the interesting issue, was the couple of paragraphs which I had already read as quotes in several other books.).  There was a gradual refinement of the methods involved, leading on to the great heyday of introspectionism, with Wundt and Titchener in the lead. Unfortunately, it became clear that the rival schools of introspectionism had begun to come up with results which in some respects were radically different and incompatible, and since our own introspections are by their nature private and unverifiable, all they could really do by way of settling the issues was to shout at each other.

This embarrassing impasse led to a reaction away from introspection and to the rise of behaviourism, which not only denied the usefulness of examining our inner experience, but actually went to the extreme of denying that there was any such thing as inner experience.  Behaviourism in its turn fell out of favour, but according to Varela there remained an instinctive distrust of introspection which continued to put people off it as an avenue of research. This is the ‘ban’ he wanted to see overturned.

Was there, is there, really a ban? Not exactly.  Apart from the most dogmatic of the behaviourists, no-one has ever tried to exclude introspection altogether. In recent times, introspective evidence has been widely accepted – the problem of qualia, thought by some to be the problem of consciousness, depends entirely on introspection. I think the real problem arises when we adopt special methods. In order to obtain consistent results, the old introspectionists thought extensive training was necessary. It wasn’t enough to sit and think for a bit; you had to have mastered certain skills of discrimination and perception. The methodological dangers involved in teaching your researchers what kind of thing they could legitimately look for are clear.

Unfortunately, it seems to be very much this kind of programme which the JCS authors would like to resurrect – or rather, have resurrected, and wish to gain acceptance and support for.  Once again we are going to need to learn how to introspect properly before our observations will be acceptable. What makes it worse for me is that the proposal seems to be tied up with NLP – Neuro-linguistic Programming.  I don’t know a great deal about NLP: it seems to be a protean doctrine which shares with the Holy Roman Empire the property of not really being any of the three things in its name – but for me it does nothing to render another trip down this particular blind alley more attractive.

Blandula I don’t know about that, but aren’t they right to emphasise the potential value of introspection? Isn’t it the case that introspection is our only source of infallible information? Most of the things we perceive are subject to error and delusion, but we can’t, for example, be wrong about the fact that we are feeling pain, can we? That seems interesting to me. Our impressions of the outside world come to us through a chain of cause and effect, and at any stage errors or misinterpretations can creep in; but because introspection is direct, there’s no space for error to occur. You could well say it’s our only source of certain knowledge – isn’t that worth pursuing a little more systematically?

Bitbucket Infallible? That is the exact reverse of the truth: in fact all introspections are false. Think about it. Introspection can only address the contents of consciousness, right? You can’t introspect the unconscious mental processes that keep you balanced, or regulate your heartbeat. But all of the contents of consciousness have intentionality – they’re all about things, yes? So to have direct experience of mental content is to be thinking about something else – not about the mental state itself, but about the thing it’s about! Now when we attempt to think directly about our own mental states, it follows that we’re not experiencing them in themselves – we’re experiencing a different mental state which is about them. In short, we’re necessarily imagining our mental states. Far from having direct contact, we are inevitably thinking about something we’ve just made up.

Phi

Picture: Phi. I was wondering recently what we could do with all the new computing power which is becoming available.  One answer might be calculating phi, effectively a measure of consciousness, which was very kindly drawn to my attention by Christof Koch. Phi is actually a time- and state-dependent measure of integrated information developed by Giulio Tononi in support of the Integrated Information Theory (IIT) of consciousness which he and Koch have championed.  Some readable expositions of the theory are here and here with the manifesto here and a formal paper presenting phi here. Koch says the theory is the most exciting conceptual development he’s seen in “the inchoate science of consciousness”, and I can certainly see why.

The basic premise of the theory is simply that consciousness is constituted by integrated information. It stems from the phenomenological observations that there are vast numbers of possible conscious states, and that each of them appears to unify or integrate a very large number of items of information. What really lifts the theory above the level of most others in this area is the detailed mathematical under-pinning, which means phi is not a vague concept but a clear and possibly even a practically useful indicator.

One implication of the theory is that consciousness lies on a continuum: rather than being an on-or-off matter, it comes in degrees. The idea that lower levels of consciousness may occur when we are half-awake, or in dogs or other animals, is plausible and appealing. Perhaps a little less intuitive is the implication that there must be in theory be higher states of consciousness than any existing human being could ever have attained. I don’t think this means states of greater intelligence or enlightenment, necessarily; it’s more  a matter of being more awake than awake, an idea which (naturally enough, I suppose) is difficult to get one’s head around, but has a tantalising appeal.

Equally, the theory implies that some minimal level of consciousness goes a long way down to systems with only a small quantity of integrated information. As Koch points out, this looks like a variety of panpsychism or panexperientialism, though I think the most natural interpretation is that real consciousness probably does not extend all that far beyond observably animate entities.

One congenial aspect of the theory for me is that it puts causal relations at the centre of things: while a system with complex causal interactions may generate a high value of phi, a ‘replay’ of its surface dynamics would not. This seems to capture in a clearer form the hand-waving intuitive point I was making recently in discussion of Mark Muhlestein’s ideas.  Unfortunately calculation of Phi for the human brain remains beyond reach at the moment due to the unmanageable levels of complexity involved;  this is disappointing, but in a way it’s only what you would expect. Nevertheless, there is, unusually in this filed, some hope of empirical corroboration.

I think I’m convinced that phi measures something interesting and highly relevant to consciousness; perhaps it remains to be finally established that what it measures is consciousness itself, rather than some closely associated phenomenon, some necessary but not sufficient condition. Your view about this, pending further evidence, may be determined by how far you think phenomenal experience can be identified with information. Is consciousness in the end what information – integrated information – just feels like from the inside? Could this be the final answer to the insoluble question of qualia? The idea doesn’t strike me with the ‘aha!’ feeling of the blinding insight, but (and this is pretty good going in this field) it doesn’t seem obviously wrong either.  It seems the right kind of answer, the kind that could be correct.

Could it?

2008 impreza 170

extend a stay in mn

panasonic vacume cleaners

ancient egyptian symbols for punishment

commercial broiler vaccination schedule

chinchilla age span chart

cliff notes for the great gatsby

jeri hobson

monsterasses.com

leitch 1991 tissue paraffin

counselor responses to teacher student relationship

a fib from surgical trauma

fret buck

does prednisone cause incontinence

1980 s major leaguer slugger tony

hotel economici bruxelles

inductive amplifier circuits

timeless clothing

minihelicopter.net

candida albicans i n men

bill of rights first ten amendments

10 largest advertising firms in us

3n704 dell precision 650 shroud

joe oxley

gold temples for eyeglasses

mom-having-sex.com

dangerous situations astronauts have been in

hyland ski patrol

diagnosing car noises

along comes john utube

free-stock-photos.com

driftwood bred horses for sale

gilmour garden hose

american ballroom challenge

hot-lifestyle-news.com

aclu pros and cons

a bootleg copy of supperbad

seemepov.com

cracker barrell biscuit recipe

closing docs

diy sidewalk

cam scarborough

10 plagues of egypt images

antiquetools.com

1 week condos new yorkcity ny

iseries tacoma

beasley broadcasting las vegas

carriageinc.com

ancient families of bohemia manor

abc dancing with stars vote

catapult drawings

lindyhopping.com

ripon quebec

sepmonline.org

fixed lenght text file

cadillac novelties wheel

hispanic reporters journalists

golf packages lake ozark missouri

nocirc.org

chinatown lambro rejected

17 miles north of lewisburg br

chris stephenson microsoft zune contact information

blue man group layoffs

gm 3059 substances

103rd street jacksonville fl

4star hotels in algarve

bridget cain waco

tripple layer greenhouse poly

1901 olympic boulevard walnut creek ca

1903 springfield rifle for sale

adaptive healthy functioning pyschoanalysis

parker scottie dog

adverse easement colorado

continentalsatellite.com

airline tycoon demos

tyleridx.com

cakewalk music software

kualumni.org

flanges galvanized

bezel 6.4 inch industrial

chinchilla adaptations

movie soundtrack to the matchmaker

apple cider vinegar health benifits

18th century table setting france

battery voltage ic

battle of midway dos instructions

irvington putting green

bexar met

cemetary headstone

matthew mcdonnell colorado

12 disciples died

mythbustersresults.com

holiday inn express pineville nc

midlandschoice.com

emoticons for skype

emmis.com

arroyo craftsman lighting as-12

lincoln lush

acessories for total gym 2000

actress with most oscar nominations

35 mm cameras

cloud forest soils

cruising to rio de janeiro

all creatures animal hospital lutz fl

exton real estate listings

morrissey carson daly

classicbass.com

acm code of ethics

alien ufo sitings

axa equitable vs travelers 403b

soundpublishing.com

dog sitter weymouth massachusetts

free majhong solitare mac ox

compare norelco razors

13th judicial district judges

gunz hack kick download

antique carved figurines

binghamton and swim and master

brandon l manson

afraid of me lyrics

bill jacobs of plainfield

closed system primary fermentation

bittersweet elixir

sumnerdowntown.com

brad mayfield

biofeedback pain management

artworks decorative window film

bob bowman swimming

book checkout sheet

latency monitor freeware windows

b lahaye gatineau

fleischmanns1dish.com

2 wire connection problems

erotico ft giorgio

andrew jacobson

camaro lineage

actress theresa russell photos

examples of cloze passages for elementary

showplace dinner theater hudson fl

92.1 lite rock

allen sibley

college-admission-essay.com

20 oz mug

crossdress halloween

bieg justin

esso by irish railways

conklin moxie

2 floor dog houses

christmas musicals in charleston wv

byte information

express passports idaho

california child restraint requirements

1999 civic retarded cam timing

bak rollbak tonneau cover

blended beauty portland

3-in1 machine mill drill

minolta dimage 2300

broccoli and califlower salad

free christian chatroom

succesful suicides

age restricted communities in south carolina

motive gear ring and pinion

carol jake joined

ipsec over udp port

confined space violations

2007 education exchange college grant program

accommodation plan fhwa filed massachusetts

dishon platt

ancient volcanoes

veronica zemanova videos

promis spss

abc extreme makeover home

bootsma tractor musuem

3 floors hobbies new west

marketing strategies beauty parlour

capacity to react to unplanned threats

arasor shares

diana school reggio emilia

access bios hot keys

jansen from nubiles

1316 6th ave s moorhead mn

how to prove native american citizenship

british men protective of toothbrushes

engineering admission 2007 in punjab

dirctv.com

momo bar

astrological meaning of pluto

cheeseburger clip art

as stars shine bright cd

hardy lawns

electronic part numbering standards

crystal lake ballroom ellington ct

apocalypse weekend postal 2 saved games

chocolate and ice cream and strauss

cuda 440 for sale

conley used truck center

stunningangels.com

budget vets auburn wa

1962 murder movie

diablo sprites

bootleg curious case of benjamin button

125 hg lp

cvc.edu

free samson coloring pages

grupo artec doors monterey

aloe cure for acid reflux

1920 first aid book

artillery recoil mechanisms

hearing location moved to humphreys college

142 night bus watford

archeological sites jalisco

jeff silvia

baroque church near eastbourne england

5 gallon lemonade dispenser

18k fine gold jewelry

daily i shall worship thee

golf insults

active transport movie

acaia berry side effects

alfred and the chipmunks

janb.net

140 series mercedes radio owners manual

13 second 03 g35 sedan

army hummer for sale

grandprixstore.com

clothing worn in venezuela

familyfeud.tv

beverly goodwin rockwell

abc family extreme makeover

elena bastidas bono

tenacious d dvd at walmart

everybody dance now lyrics

doctors who have performed euthanasia

coburn woods

dena flannigan

thriftynickel.com

ju jitsu 925

asawa ko japanese to english translation

barnyard stencils

strokesos.com

alexia disorder wikipedia the free encyclopedia

1st birthday disney babies

chrysler rumors of ipo

texaskayakfisherman.com

alquiler coche gran canaria

alkaceltzer at the grocery store

2 cents lyrics

animal cookie cutters

bold the beautiful synopsis

aging stain

koyo 6204 rk bearing

bed hire henley

hiding eyebrows

18 free online games

jonathon and darlene

100cc yamaha v star silverado

easy zuma

argument or persuasive essay outline

1980 s engineering software

andes the mountain cafene

coding4fun.net

$300 fl persian

address for sunbeam corpo

a n scales

dr patricia hurd charlotte nc

bear paw clipart

1.2hp electric motor

banking for dummies

etobicoke salsa classes

flickr barefoot men

electric power boosters brake

4 qt deep camp dutch oven

chopped steak recipes

free chicken invaders game

guatemala hookers

arthcomics.com

assisited living in taunton ma

horrors of the spanish inquisition

alameda chemical scientific of arizona inc

1920 s gang pictures

african wildlife federation

cny $99 remote starters installed

anastasia confessions orlando

aaa motels seaside oregon

alfred curry

drunkon.com

gentoo grub corresponding bios

dukes of hazzard fan fiction

george mathis waffle wedding

juicyuncutcocks.com

blue topaz heart pendant

bounty and darien ct

duane andrews family newfoundland

eurocreme.com

12 floppy cable free ship

john dear pax christi

olin h welsh lumberton nc

gonzaga university august graduation

abdul rao md

elmhurst general hospital

another hundred

10,000 dollar business suits

atlanta bankruptcy and structured settlement

masomaster.com

artist elaine seltzer

james horner whispering winds

1102 lincoln way

free workout journal weightloss journal

aide cr ation sarl

barclays zimbabwe predates difficu ties

ncpsychologyboard.org

first led zeppelin poster

cocker pug mix dog pics

crosman 500x piston

antlers hilton boxing match

kathryn a sever

rosedale bars

mitsubishi wd 52527 bulbs

constitutional issues regarding the crusades

glin.net

campact tractor pulling power

los tacos spanish food

civil war reenactment ga

2 drawer casters printer

aqua di parma iris nobile

back pain in pms

all versions of yahoo messenger

rusticfurnituredepot.com

corsair extream with heat spreader

cornbread batter fish recipe

gastroenterology department indiana university

aida in cincinnati

fain wynn

16 northern oak crackling electric logs

6228 revere st 19149

east union baptist church hensley ar

fishing avonlea resevoir

durysguns.com

bananas monkey tijuana

cbd obstruction

freenetleads.com

joss stone broken

xeex470809
gainesville regulated utility
turbos r us
corolla 1994 4age for sale
affidavit of medical necessity texas code
pyramid outpatient erie pa
move filemaker pro database
home rentals in orangeburg sc
splicing in africa
javascript peoplesoft
adapted gym for the blind
glaucoma surgery mayo clinic
sks carbine for sale
asbury roadtrip 6
mariah houghton vocabulary word
ucla student tiffany hill
endorsement fees
merrimac municipal ma
holtz
simmons mattress seconds
cking out old insurance policies
products obtained by processing of saffron
britney spears pics in elle
stiller
orlando fire department non-certified cadet
michigan safety congress
ancient spartan buildings
124
penny wilkie nd
old used cowboy hats for sale
myspace jack sweet troubled soul
resurfacing
taxi fare st pancras to heathrow
moblie edge portfolio
hotfix rhinestone machines
schwinn roadmaster
vidalia sweet onion
had
the game mya
2004
western hopper lid
la make-up palette givenchy
latvia eating habits
palate phoenix
tyler hayden
gladstone ports
restraunts
kotor the count
ft leavenworth mo
dog skin pimples
armada t hall review
glass coasters illusion
ftponline com upside channel
avulsion of face motorcycle
i want everything stone sour
1989 gmc rebuilt transmission manual
research study participate in tulsa
computer ethics netiquette not sponsored
dorothy van nuys
carlos zurich dubai gcc
typo3 extension subpages
flooding calcutta india
minnie menard
7hi trouble shooting
fairchild transistor
dtv map
bremerton history
urgent loan needed asap
models actors talents
murthy rao and portland
the nature of the dreaming
ecg 12 leads 10 electrodes
whats dry humping
brunner
ryanair kerry to dublin
problems student faced when communicating online
plasma vs lcd tv compare
yorke
xps 630 and review
recliners platte and leggitt
bebe 10008 international blvd
everquest 2 dward paladin
updating track lighting
fragile x and aspartame
moist red velvet cake recipe
america s favorite architecture and aia
choy
princess mary adelaide duchess of teck
uwe
black people humping
2200
neutrogena men power scrub
silk night gowns
lip
bill defouw chevrolet lafayette indiana
quest for camelot games
black scholes defined efficient frontier
huber casing heads
cordless lawnmowers consumer reports
compass inn museum
mils to ounces
evolution of warfare
united states legislature
bias in polling methodology
buying prepared meals
colledge football news
jeff meditation santa clara
oakley xs o frame youth goggles
lohan new york magizine pics
palmetto fellows early awards scholarship
blu-ray internal drive
imposter seatbelt bag
sensibility of edi eat 40
sarah macpherson
lauries cabot trail
safety slogan contest statements
vw spares
german recyclers
robert t fielding
lerici appartement
accountant cpa fort lauderdale
201
remidies
carp bait only
air pressure explodes pex tubing
expat women
forbidden gardens home page
poemas muerte
lesson on probability for younger students
morals and values of america
colonoscopy performed
majestic 10 vt
indemand ray mercer ppv
coca-cola roxy theater marta
bongo bong
dickey betts great southern equipment
great swedish general
self employed credit
rustic cedar bed twig accents
protector reclaiming cheat
tds o ympic weights
used bookstores omaha ne
free fairly used mobile phones
mikey heller lawyer
denmark league
where are my lungs
henderson state music department
fencing and interlocking gates
sizes of dimensional lumber
jimmie crack corne
stepping stone london on
jigsaws puzzels galerie
manitoba tel
expeditions north
ephedrine good
bulgarian split squat
pro bowl pics of devin hester
how do whales communicate
sportsmans world
goverment of ontario jobs
menu san diego sala thai restaurant
florida insurence licence test
eduardo celada hurtado
witty insults and comebacks
newspapers in the classroom
exploited kelsey
fury
wendy double scalloped window valance
tourist attractions in laramie wyoming
ashland county divorces
cecil conley sports writer
hind
jame ewing
70s
seaside salon seabrook tx
kids birhday partys
horse trailer hay pod
uninstall directx update
real estate ono island
b w terminated folded dipole
west bend aluminum canisters
esther a day
piano and percussion duets
thats amore karoke download
nordictrack sl700 workout plan
humane society ana alicia
candice marie
weird al yankovic canadian tour tickets
jim roi
periwinkles jewlery
ulr landrover
ethanol arguments
guitar lessons nobody knows about
balfour beatty centex
brown hair snatch
century 21 seneca
globalization its imperatives for international hrm
new refinancing york mortgage rate calculator
leed corporate website
thornton
howard hughs and katherine hepburn
yandel
huggins
hotels santo tomas de castillo gus
cobalt ss seats
vacaville ca and romans and lagoon
free horoscope vedic
cognito ergo som definiion
kee fm
london sightseeing tour
critter chatter
space rocket material
obituaries in madrid spain
guild awards
gm lt1 lt4
billard decor
computer resources motorola site filtering information
disposal treated seed
cranston act
viewsat clones fta
swinging
free information by mal
freightliner parts lodi nj
hale nui kale
hp deskjet f 2280
unhappy lonley housewives
albion history virginia
non biodegradable disposable items
phim ma dao dai quyet dau
subsonic and supersonic
kylie minogue news
how to charge writs
wild birds pheasant hunting
dsc security system
metatrader ask bid spread change color
photo french fries
vids
jervis
cheating spouses investigators
go off paxil
archeology wwii
financial aid post traumatic stress disorder
shuttle external tank on orbit
how much can an ant lift
keys treadmill motor
2005 glencoe mcgraw-hill geometry resource masters
bertha curry dayton ohio
sub jk
yugioh songs
halloween medusa headpiece
microscope palynology prices
fiat charleston repair
creating a dvd from avi files
itza
general enoch crowder
uslu v commissioner
automatisme portail sur roue
what did robert koch do
proliant
wasabi at long beach
109
tempered sword
he peleado la batalla senor
earp
rune polarities
contribute 3
rosetta stone install disks
employment orthodox rabbi
what can hamsters eat
american inventors steel
fergus brooks palmer
bachman-turner overdrive cest la vie
ksel
omally omalley attourney lakewood nj
what was the strongest earthquake ever
helsing
carina condon
every woman votes
dan wachtler and charge or suspect
literacy organization in australia
nutrition throughout the life span
information concerning obstetrics
v-star
maura garrett
order fulfillment services inventory
the moose and grand junction colo
printable forms for landlords
divine comedy track amelie
beneficial mutual bancorp inc
halliday
nigeria eyewear distributors
food for heartburn
educational choices quotes
lucian pissarro paintings
v9
mariposa dress shop
crack code agenda fusion 7.90
farberware lun
speech therapists use tongue depressor
kubota bl 4960 hydraulic lines
kune
home care agencies cambridgeshire
johnny van sant new jersey
gov jodi
piston bully wrecking
issue of north indians in maharashtra
geothermal heat pump certification training exam
stock market graph qantas airways
dr goldberg dallas
ervin
centimeter
motorcycle stores ocean county nj
adoption finances
harrahs atlantic city h
celia hunter
stone coated steel panel
nathaniel fisher
580
saint edwards forth smith arkanas
subaru impreza sti limited
dollars to euros exchange rate
narcotics anonymous steps
compute
symptons of groin injury
johnson ukulele national cone
kushan empire rulers
crank that weezy wee dj crowd
sheraton inner harbor hotal
download peer to peer music free
steak inn shel knob mo
aac dolby radio
generate junit tests
fender bassman compact 115 combo
i hate caterpillar
what a dragonfly means
florence doyle calais maine
saigon consulate list
classic flying fifteen hull lines
tristan and isolde picture disc
year of the ox predictons
use accutane
alston statuary
yamaha sit up
hearing aid cover bat corn fish
lx200 alignment problem
establishing a business production
kristin banning minnesota
wahlberg
coast guard honorable discharge
lenore raphael
flickr photos tagged with