I for one welcome our AI overboard

11 Jul 2024

I’ve been very reticent to comment on the current AI hype train, mostly because it’s very easy to be wrong in a bull market. But you will be relieved to know I am now ready to put my hat in the ring.

First I would like to say that, like everyone else, I’m deeply impressed by recent advances in AI, particularly large language models (LLMs). They can give you flawless explanations of topological quantum field theory in one breath, then seamlesly switch to a nuanced discussion of the history of cartography. Tailor-made answers to your burning questions on heraldry, hermeneutics, object-oriented programming, epistemology, and just about any other domain of expert human knowledge your heart desires. They’re really quite amazing. I would be extraordinarily proud if I had been part of the teams that brought LLMs to the world.

But in the wake of the current wave of exuberance, I feel compelled to point out the manifold ways in which LLMs are extremely dumb. Here is but a partial list of tasks that state-of-the-art LLMs have failed at spectacularly:

• Basic arithmetic, e.g. multiplying together two 3-digit numbers.
• Understanding relations, e.g. if X is Y, then Y is X. If you ask ‘Who is Olaf Shhulz?’, you can correctly be told that he is the 9th Chancellor of Germany, and then immediately be given a different answer to the question ‘Who is the 9th Chancellor of Germany?’.
• Creating a 3x3 word search containing specified 3-letter words.
• Name three famous people who all share the exact same birth date and year.
• Counting e.g. the numbers of letters in a string, number of paragraphs in a document, etc.
• Simple logical problems, with uncommon variations. For example, river-crossing problems, but with no restrictions on which which items can be taken or left together.
• Spatial reasoning. For example, ‘Four children - Alex, Bella, Charlie, and Dana - are sitting around a picnic table. Alex is facing Bella. Charlie is sitting to the right of Bella. Who is sitting to the left of Alex?’.

You can find a more in depth exploration of various LLM fails in the appendix of this paper.

What these examples indicate is that LLMs don’t have any understanding of the world that is in any way similar to what you or I have. For example, if you give me the word ‘chair’, it brings to my mind a lifetime’s worth of relevant physical, sensory information. I know what they look and feel like. I know the precise purpose they serve, having experienced the discomfort of having to stand for prolonged periods of time. I know that chairs can be used in all sorts of ways that have little or nothing to do with their intended purpose, because I have actually done it before (wink wink). LLMs have none of this. What they have is, loosely speaking, a bunch of words that they associate with chairs from being exposed to internet text in which the word chair appears.

What this means is that I understand the concept of a chair and its manifestation in the real world in a way that guarantees I’m not going to make obvious, stupid errors that LLMs sometimes make. I can also exhibit chair-related creativity, like finding novel uses that you won’t find written about anywhere on the internet (nudge nudge), because I have an understanding of the physical reality of a chair that is more than just a cloud of associated words.

In short, LLMs may very well be stochastic parrots. To the extent they give good answers, it is only because human experts have provided good answers to a similar question somewhere on the internet that the LLMs have cobbled together. In a very real sense, you might think of LLMs as souped-up versions of search engines that actually construct a tailor-made, friendly prose answer to your particular query instead of making you spend hours piecing it all together from a collection of top hits from stack exchange, academic papers and obscure blog posts. Or, as Noam Chomsky bluntly put it, ‘high-tech plagiarism’.

Another way to think of an LLM is a smart 10 year old that has read and memorised a textbook on quantum mechanics. If you ask them typical questions about the subject, they may be able to give you impressive-sounding answers. Hell, they might even be able to convince you they are a child prodigy. But they have zero understanding of the subject, a fact which will manifest itself as soon as you venture even a little bit outside of what can be constructed by placing roughly the right words in roughly the right order in a superficial act of mimicry.

In light of this, I find it very difficult to take seriously any notion that we are blindly speeding into an AI-driven apocalypse. I recall seeing an online exchange where one person was taking as a sure harbringer of our impending doom the idea that LLMs were on the brink of being able to consistently perform arithmetic. To which they received the rather acerbic reply ‘So can my fu**ing calculator’. Well, quite. And the calculator doesn’t need billions of parameters, hundreds of talented young minds, the entire text of the internet and the power supply of a small country to create.

The most popular version of the AI apocalypse scenario goes something like: we will soon create artificial general intelligence (AGI) that exceeds human capabilities across a wide range of cognitive tasks. This will kick-off a chain of recursively improving AIs, ultimately leading to a superintelligence that will be able to outwit and outperform all of humanity across the board, leaving us utterly at its mercy. If this superintelligence happens to have an objective that is not carefully aligned with the interests of humans, it could all go seriously tits up. Take the so-called paperclip maximiser, dreamt up by philosopher Nick Bostrom,

Suppose we have an AI whose only goal is to make as many paper clips as possible. The AI will realize quickly that it would be much better if there were no humans because humans might decide to switch it off. Because if humans do so, there would be fewer paper clips. Also, human bodies contain a lot of atoms that could be made into paper clips. The future that the AI would be trying to gear towards would be one in which there were a lot of paper clips but no humans.

Are you scared yet folks?

Well no, not really. To me, this stuff just sounds like the fevered, masturbatory fantasies of sex-starved sci-fi fanboi man-children. In fact, that description might be eerily close to the truth. Exhibit A: Bostrom himself. Exhibit B: Elizier Yudkowsky, leading AI-doomer, and a man who thinks it’s a positively spiffing idea for superpowers to conduct pre-emptive military strikes on each others’ AI labs, running the risk of precipitating a nuclear holocaust, in order to prevent the development of AGI. I suppose the hundreds of millions of lives that would be lost is a small sacrifice at the altar of his very serious sci-fi fantasies. Incidentally, he’s the author of Harry Potter and the Methods of Rationality, which is basically a sci-fi reimagining of Harry Potter. These people are clearly drawing heavily on a literary tradition that has enthralled us for many decades to weave their apocalyptic tales.

Given that cutting edge LLMs are recommending people eat rocks as part of a healthy diet and glue cheese to pizzas to prevent slippage, the latter based purely on the shitposting of a presumed teenage internet edgelord, I think we can confidently say we are safe for the time being. More generally though, the AI technology singularity scenario requires a long chain of reasoning, each step of which is, to put it gently, extremely speculative. Maybe it will take 50 more quantum leaps, each of a similar magnitude to that which brought us LLMs, to get to AGI. And there is nothing inevitable about each step of the putative self-improving AI explosion. Who knows if any of this is even possible, or if it is, whether it will happen. Maybe there are hard limits to what can be done, not least of which are the tremendous amount of energy and data that appear to be required to train these AIs.

The whole thing sounds like a big conjunction fallacy to me. That’s when a specific proposition is thought to be more likely than a more general proposition of which it is a special case. For example, consider the following experiment carried out by Amos Tversky and Daniel Kahnemann where they asked a group of policy experts to rate the probability of the following events:

1. Russia invades Poland
2. Russia invades Poland and the US breaks diplomatic relations with them the following year.

They collectively assigned a 1% chance to option 1, and a 4% chance to option 2. This, of course, cannot be the case since 2 is a special case of 1. What might be going on here is that the participants find it difficult to imagine a scenario in which Russia invades Poland and thus assign it lower probability than a more detailed scenario that is more representative of what a real scenario would look like.

And so it is with AI-doomer fantasies. Most people didn’t spend the requisite amount of time in their parents basement during adolescence to be able to magick up the paperclip maximiser from the depths of their imagination. But given a fantastical sci-fi inspired story explaining one way it could happen they start to wildly overestimate its probability.

I find this fixation all particularly hard to swallow given that at this particular moment in history, we may well be on the brink of global nuclear war. Nuclear weapons pose a far bigger and proven threat to humanity that we have basically been ignoring for decades. Anyone who thinks otherwise should read the story of Vasily Arkhipov, the razor-thin, one-man margin that stood between us and nuclear annihilation back in 1962. There have been several more close calls since then too. Let’s be clear about this: The only reason the nuclear armageddon hasn’t happened yet is pure luck. The USA may be about to lose its position as global hegemon, and shows no indication of going quietly or peacefully. Countries around the world are increasing military spending and reintroducing the draft. Sorry folks, but I have bigger fish to fry than a damn paperclip maximiser.

My prediction for AI is that it will lead to disruption in some industries, and some handsome payoffs in increased efficiency. The AI doomers will continue their fearmongering, and for them AGI will be like fruit above Tantalus - always within reach, never in our hands.