Background

Other than the volumes I speak indirectly, I don't say much about myself on this blog. Readers get to know one corner of my mind, but not too much about my life. Perhaps that's fitting, as my mind is apparently atypical, but my life is probably not, or at least not in a way which makes for an interesting movie.

But one cogent fact you might want to know at this point, which I don't think I've revealed previously, is that I have a Master's Degree in Computer Science, concentrated (for a while) in Artificial Intelligence (AI).

Yes, that's true: yours truly was, for a while, a specialist in AI.

To invoke the riff about laws and sausage, it's one of the reasons I'm less impressed with the whole idea than the average SciFi geek, I suspect.

When I was studying AI back in the late 1980s, many of my academic betters were convinced that true AI was coming soon -- predictions often mentioned the year 2000. Earlier, people were convinced we'd have real AI by the 1980s or 1990s. I'm sure we go back far enough, we can find people expecting intelligent robot servants in the 1960s, 50s, or even earlier in the twentieth century.

So far, no robots. Nor flying cars. Nor truly intelligent machines.

The Singularity

The "technological singularity", to boil it down badly but quickly, is a hypothetical point at which machines get better at doing "human" tasks than humans do, including designing ever-smarter machines. At that point, the future "won't need us" anymore, according to Bill Joy, the man who designed my favorite text editor.

Like most academic speculations (both valid and drivel) this idea has already percolated into popular culture. In particular, think SkyNet, from the Terminator series -- a groundbreaking science fiction film which seriously changed our dominant paradigm by opening us to the idea that a pro-choice Austrian bodybuilder might make a good Republican governor of California. But most viewers apparently dismissed the series' other implication -- that the machines might be taking over soon -- and went back to their fair-trade lattes.

Yet in some circles, "the singularity" is taken quite seriously, nearly rising to the level of religious dogma. This hit me in the face when I was reading a recent article about a proposed technique for creating invisibility, and noticed the interviewee inserted the idea into his response (underline added):

"These materials would comprise a complete -- almost magical -- mastery over light," said David Schurig, an assistant professor of electrical and computer engineering at North Carolina State University, who did not participate in the new research. "They would enable not just invisibility cloaking, but arbitrary control over the richest information channel humans employ. One thing I know for sure, it will happen sometime between now and the technological singularity."

The "technological singularity" isn't just a theory for this guy. It's an inevitable future event, as sure as the coming Communist Revolution was for Marxists and as the return of Jesus is to Christians. (Yours truly included.) But the difference, to again jam on one of my favorite riffs, is that I recognize my beliefs are religious, whereas almost all Marxists and Singularitarians do not.

And he's certainly not alone: I once worked with a reasonably bright programmer who kept his head shaved, and tended to be quite emaciated. He lived on a calorie-restricted diet because, he explained, he hoped to extend his life long enough to see the day that his consciousness could be downloaded (uploaded?) onto a machine, and thus achieve immorality.

Certainly, a nice extreme case, but also a colorful illustration of how some humans can look to science with a fervor which outdoes that of many religious folk, and for many of the same reasons.

Whither the Singularity?

('Whence' and 'whither' are a pair of useful yet obsolete words in the English language. Their disuse testifies indirectly to our contemporary illiteracy. The former means "from where something comes" and the latter means "to where something goes.")

There are some Singulitarian heretics, also known as 'anticognitivists' who don't think we'll be facing a machine-initiated takeover any time soon. One of them, David Gelernter, argues (and I certainly agree with his title) that AI is currently "Lost in the Woods".

I believe it is hugely unlikely, though not impossible, that a conscious mind will ever be built out of software. Even if it could be, the result (I will argue) would be fairly useless in itself. But an unconscious simulated intelligence certainly could be built out of software--and might be useful. Unfortunately, AI, cognitive science, and philosophy of mind are nowhere near knowing how to build one. They are missing the most important fact about thought: the "cognitive continuum" that connects the seemingly unconnected puzzle pieces of thinking (for example analytical thought, common sense, analogical thought, free association, creativity, hallucination). The cognitive continuum explains how all these reflect different values of one quantity or parameter that I will call "mental focus" or "concentration"--which changes over the course of a day and a lifetime.

Without this cognitive continuum, AI has no comprehensive view of thought: it tends to ignore some thought modes (such as free association and dreaming), is uncertain how to integrate emotion and thought, and has made strikingly little progress in understanding analogies--which seem to underlie creativity.

My case for the near-impossibility of conscious software minds resembles what others have said. But these are minority views. Most AI researchers and philosophers believe that conscious software minds are just around the corner. To use the standard term, most are "cognitivists." Only a few are "anticognitivists." I am one. In fact, I believe that the cognitivists are even wronger than their opponents usually say.

In his objection, Gelernter provides what is probably the most succinct summary of the deficiency of current "AI" projects, by pithily describing the typical "AI" program in this way:

In conscious thinking, you experience your thoughts... A machine that is capable only of unconscious intelligence "reads" its thoughts as if they were on cue cards. One card might say, "There's a beautiful rose in front of you; it smells sweet." If someone then asks this machine, "Seen any good roses lately?" it can answer, "Yes, there's a fine specimen right in front of me." But it has no sensation of beauty or color or fragrance. It has no experiences to back up the currency of its words. It has no inner mental life and therefore no "I," no sense of self.

Indeed, that's precisely what passed for "AI" when I was studying it. (And no doubt, still does today.) A computer program "feels" something or "likes" something only because a "card" (instruction) is inside of it, telling it to say it feels that way, or assert it believes this or that fact. "It has no inner mental life", it's just reading from cue cards which attempt to simulate the verbal output of something (a human) which does. It can reasonably be said to be "intelligent" -- it certainly can "reason" in a simple way -- but there's no conscious "I" present.

Part of the reason AI was lured into this particular blind alley (useful for human interface, but not useful, I believe, for creating actual intelligence) was the way the field was defined from the start. A now-famous mathematician named Alan Turing proposed, echoing Descartes, that a computer could be considered "intelligent" if you could have a conversation with it (Turning proposes it sits in another room, and we pass slips of paper back and forth), and be unable to tell if you were talking to a computer or human.

If it looks like a duck, walks like a duck, and quacks like a duck, it's a duck.

Seems simple enough, no?

The Chinese Room

Well, not so fast. Gelernter trots out a counterargument, proposed by John Searle (a professor of philosophy) back in 1980, which parodies Turing's own -- the "Chinese Room" argument.

Searle asks us to take the other role: We're in a room, and there are observers outside who must gauge, not if we're intelligent, but if we're capable of understanding Chinese. As in Turing's test, slips of paper are passed back and forth.

To help us answer, we have the instructions from a Chinese-speaking software program written for us, which we execute by hand. We look up the incoming symbols in one dictionary, which helps us match some common Chinese phrases in a second dictionary, which then refers us to a third dictionary of popular responses, which we copy down and push back under the door.

To those outside, it would surely seem like we understood Chinese, but the truth is that we do not: We are simply following instructions to simulate an understanding of the Chinese language.

Gelernter boils this kind of argument down even further, for us laymen:

So what is it like to be a computer running a complex AI program? Exactly like being a computer running any other kind of program.

Computers don't know or care what instructions they are executing. They deal with outward forms, not meanings. Switching applications changes the output, but those changes have meaning only to humans. Consciousness, however, doesn't depend on how anyone else interprets your actions; it depends on what you yourself are aware of. And the computer is merely a machine doing what it's supposed to do--like a clock ticking, an electric motor spinning, an oven baking. The oven doesn't care what it's baking, or the computer what it's computing.

The Chinese Gymnasium

One subsequent objection and response is the so-called "The Connectionist Reply", which complains that Searle misrepresents the situation: the brain is not like a computer, with a single CPU executing a linear set of instructions -- it's a massively parallel system with many agents operating independently.

It must sound like the height of arrogance for me dismiss this as incredibly stupid response, given that it probably comes from people with much more education and expertise that I possess -- but I do. As any third-year computer science student knows, parallel systems can always be simulated (slowly) using a single processor which simply switches between tasks. Other than speed, a single processor is not fundamentally different than having many processors.

Analogously, we'd have the guy in the room follow a more complicated set of instructions, switching hats between many roles. He still wouldn't understand Chinese.

Searle cuts to the chase with a slightly different reply:

Imagine, if you will, a Chinese gymnasium, with many monolingual English speakers working in parallel, producing output indistinguishable from that of native Chinese speakers: each follows their own (more limited) set of instructions in English. Still, Searle insists, obviously, none of these individuals understands; and neither does the whole company of them collectively.

Zero understanding, times a hundred, is still zero.

And Gelernter responds to the many-processors-run-faster argument like this:

Yet it seems arbitrary to the point of absurdity to insist that doing many primitive operations very fast could produce consciousness. Why should it? Why would it? How could it? What makes such a prediction even remotely plausible?

A complete lack of understanding, occurring a thousand times faster, is still a complete lack of understanding.

Emergent Systems

Yet I think both arguments above fail because they're subtle straw men. The mind is an emergent system; intelligence or consciousness is not found in each of the elements, but only in their aggregate.

The "intelligence" or "Chinese fluency" exhibited from the "Chinese Room" does not reside in the man's mind, yet it does reside in the software he manually executes. The proof is that same result (an appearance of fluency in Cantonese or Mandarin -- which, we're never told) occurs whether a computer or the man-in-the-room executes the algorithm. The software is the thing which contains whatever limited "understanding" is being demonstrated. The same thing holds for the Chinese Gymnasium.

Similarly, Gelernter's argument that "the computer doesn't care about what it's computing" -- so it's no more intelligent when running an AI program than Quicken -- also misses the mark. Any alleged intelligence or consciousness wouldn't reside in the computer hardware, but rather, in the software it was executing. Back up the software and it's current state, you back up the "mind"; copy the software and state, and you copy the "mind".

To give Gelernter credit where due, he seems to realize this; he (or his editor) calls that argument the "The Cognitivists' Best Argument", and seems to admit he doesn't have a fully fleshed-out, knock-down, drag-out rebuttal.

Searle also anticipated this objection, and responded like this:

The systems reply grants that "the individual who is locked in the room does not understand the story" but maintains that "he is merely part of a whole system, and the system does understand the story"..... Searle... insists the systems reply would have the absurd consequence that "mind is everywhere." For instance, "there is a level of description at which my stomach does information processing" there being "nothing to prevent [describers] from treating the input and output of my digestive organs as information if they so desire." Besides, Searle contends, it's just ridiculous to say "that while [the] person doesn't understand Chinese, somehow the conjunction of that person and bits of paper might."

Yet the question, in the "Chinese Room" problem, doesn't ask whether "mind" is everywhere, but whether understanding of Chinese can exist in scraps of paper, encoding and describing and algorithm. Certainly, it can.

"Intelligence" (though not "mind") certainly does exist everywhere: the rippling of a piece of fabric performs more mathematical and logical operations than the most sophisticated computer graphics program used today. (Something akin to this observation is, in fact, the premise behind quantum computing.)

This "intelligence" or "computation" (if we can call it that) does not appear to us as such because it is often small-scale, disorganized, and chaotic. When it is organized into larger-scale systems, it usually does non-conscious things like governing the way waves ripple or crystals form. Conversely, the larger-scale, human-like intelligence in the "Chinese Room" only exists at that level of organization and specificity because an extremely intelligent group of people worked hard to encode part of their own intelligence into its configuration.

(Why we should also find sophisticated mathematics and logic operating all around us, independent of us, and why our own minds are apparently built up from such, are profound and shocking questions indeed.)

Advantage: Gelernter!?

So far, I've disagreed with key arguments posed by Gelernter and the anticognitivists, and come down, on each, on the side of the cognitivists. I think both Gelernter and Searle's arguments about the essence of AI and the Chinese Room are invalid.

If you're a regular reader, you recognize that this may be a cue that I'm about to again induce philosophical whiplash by agreeing, in the end, with Gelernter. I think the anticognitivists have, by far, the stronger case. The killer argument is right in front of us, and has been for years.

The truth is that we're nowhere near close to solving the problem of consciousness, nor of creating a machine-based consciousness.

Sometimes, inductive reasoning is more powerful than deductive reasoning: if your mental model fails over and over, it's trying to tell you that there's something wrong with it; there something important you've neglected, something more you need to learn. The devil, as they say, is often in the details.

Despite their alleged education and intelligence (or, perhaps because of it), the AI community has been wrong, time after time, about when we will see reasonable-looking machine intelligence, or when we will see conscious machines. Each time, it turned out that the problem was more complex than we had envisioned: we had, yet again, neglected key questions and challenges. We predicted each remaining layer of problems would be easy to solve, and each time, we were wrong.

So, if we are to learn from history, the key question for this community should be: Are we again oversimplifying the problem?

Gelernter argues they are. In fact, as noted above, he argues that we haven't even yet discovered or defined what consciousness is yet. We just hope it will happen ("magically", almost) if we glue a lot of simple non-conscious parts together and start 'em up. (As in the SciFi short story Press Enter, for example.)

Similarly, we haven't even clearly identified and quantified many other crucial aspects of the mind:

Consciousness has been a "mystery" (as Turing called it) for thousands of years, but the mind holds other mysteries, too. Creativity is one of the most important; it's a brick wall that psychology and philosophy have been banging their heads against for a long time. Why should two people who seem roughly equal in competence and intelligence differ dramatically in creativity? It's widely agreed that discovering new analogies is the root (or one root) of creativity. But how are new analogies discovered? We don't know. In his 1983 classic The Modularity of Mind, Jerry Fodor wrote, "It is striking that, while everybody thinks analogical reasoning is an important ingredient in all sorts of cognitive achievements that we prize, nobody knows anything about how it works."

Until we at least admit these blind spots, Gelernter argues, we don't even have a chance of moving forward.

Religion and AI

We don't yet fully understand how a single neuron works. We don't yet have a technique for detecting a single neuron's state in the brain, much less it's entire currently-encoded range of responses, and adjacent synaptic connections. Much less the ability to do so for each of the billions of neurons in the brain. Much less the computing power to run the required simulation (should we be able to create one) in anything remotely approaching real time.

And, as Gelernter points out, we're not even working some of the other key problems involved in creating the kind of machine consciousness required for the prophesied "technological singularity".

True, its possible we could overcome all these barriers within a few decades, but, other than blind faith, there's no particular reason to think there won't be more layers of undiscovered complexity to describe and wrestle with, as there have been in the past.

So, for better or worse, there appears to be no evidence for my friend's conviction he will be able to download his brain into a machine in his lifetime. Nor for the prediction of Vernor Vinge, the man who first coined the term "technological singularity", that "I'll be surprised if this event occurs before 2005 or after 2030."

Why are futurists so often wrong, particularly in the area of machine intelligence? Why did my associate, an otherwise intelligent person, structure his life and hopes around a conviction which is, frankly, not sustainable given a look at the state of the art? And why do we just believe that if we throw enough non-intelligent small parts together, without solving the bigger problem of the nature of consciousness, that we'll just magically get something which is conscious?

Controversially, I submit to you that this is, in fact, an ancient human impulse, the idea if we just make something which looks like us -- seems to be structured the way we are -- we can create something which not only "alive", but more powerful and wiser than we are. I believe the Jewish prophet Habakkuk noticed this impulse operating in the people around him, when he wrote, scathingly:

Of what value is an idol, since a man has carved it? ...
For he who makes it trusts in his own creation...

Woe to him who says to wood, 'Come to life!'
Or to lifeless stone, 'Wake up!'
Can it give guidance?
It is covered with gold and silver;
there is no breath in it. (Habakkuk 2:18-19)

(Interestingly, people's propensity to be impressed with the idea of a human-seeming machine also plays into the world-domination scenario spelled out in the last book of the bible.)

AI as a Cargo Cult

After US armed forces had withdrawn from primitive people-groups in the South Pacific, at the end of WWII, some tribes began to make runways, straw statues of US airplanes, radio huts and wooden radar towers. They would stage elaborate rituals which imitated the way planes took off or came in for landings, believing that if they imitated the visible outer form, they would also somehow recreate the underlying mechanism and consequential result.

This was, I will remind you again, a religious ceremony, although they undoubtedly didn't see it that way.

From their point of view, it was possible that if they just imitated the airport workers, and the "landing procedure" precisely enough, the airmen would return, and bringing Western goods, stimulating their economy. There was nothing in their knowledge or experience that proved it couldn't be so, after all.

Similarly, the enduring pagan impulse has been to believe that if we just threw together something which seemed like us in one way, but "bigger" than us in another -- be it a new kind of powerful government, an elaborate statue encrusted with gold or precious gems, or a silicon-based machine -- that the whole would superior to the parts, and that it would somehow "wake up" and guide us into utopia.

To quote Donald Fagan, of Steely Dan fame:

Just machines, to make big decisions,
Programmed by fellows with compassion and vision,
We'll be clean when their work is done
We'll be eternally free, and eternally young.

We have an impulse to trust in such things, Habakkuk insists, because we believe they are a more powerful, a more godlike extension of ourselves. We made them, and now they will make (or break) us. Don't take my word for it -- that's also exactly what Sun Microsystems' Chief Scientist Bill Joy wrote about the underlying motivations of top AI scientists:

The dream of robotics is, first, that intelligent machines can do our work for us, allowing us lives of leisure, restoring us to Eden.... A second dream of robotics is that we will gradually replace ourselves with our robotic technology, achieving near immortality by downloading our consciousnesses; it is this process that Danny Hillis thinks we will gradually get used to and that Ray Kurzweil elegantly details in The Age of Spiritual Machines.

Joy and others cannot describe the project without phrases like "Eden", "immorality", and "spiritual". Our environment is certainly more technologically sophisticated that the idol-worshipers Habakkuk and Isaiah criticized, but there's no reason to believe those same underlying religio-psychological impulses will have suddenly disappeared from human nature. Once glance at the lingo, and you'll see they haven't.

Nothing wrong with a religious faith, by which I mean a philosophical belief or hope which can't exactly be justified from the available evidence (or sometimes even runs counter to it), but let's at least be clear about it when we embrace one. "Spiritual Machines"? Sorry, not this year.

Conclusion

I'm no Luddite. I enjoy using computers, earn a comfortable living by making them smarter, and think that they have tremendous potential to benefit humanity -- though I'd be remiss to fail to admit they also pose risks to privacy and freedom. As you might guess from my own postgraduate specialization, I'm certainly not opposed to AI research. Far from it.

Yet we're a long way from producing anything truly intelligent, much less conscious. As Gelernter and others point out, computers can't yet even master some very simple common-sense tasks that normal humans (or even dogs) can perform. We have "smart" chess-playing computers, but they're still not very good at "Go".

And our collective (again, religiously-based, I feel) unwillingness to even admit these blind spots will probably slow down research for many decades. To overcome a problem, you must be willing to admit you have it, not just keep hoping that enough computing power will magically deliver cognition, much as a gambling addict hope maybe a few more quarters will put him on easy street this time.

It is possible someday we will surmount all these barriers -- perhaps in a century or so. But perhaps not: I suspect there may be things about the brain which will elude us for a long time. But the later is just a hunch, whereas the former criticisms arise from a cold, hard look at the real problems facing the field of AI, and our dismal record in predicting our rate of progress there.

Are you worried about the technological singularity? Don't be: there are far bigger problems facing humanity here and now.