Dennis Schubert’s superb LLM analogy, which you need to read! (also available in a post on Schubert’s blog, the part that begins Let’s imagine you’re colorblind) inspired me to write down my own way of understanding LLMs. I imagine it’s much less likely to drive the point home than Schubert’s, but it works for me and may work for somebody else.
It took me about a year from when the concept of an LLM sailed in over the horizon until I found a way of thinking about it that made it understandable and predictable to me: it’s a database. Nobody seems to understand what I mean when I say this, probably because people think of a database system as something much more specific than I do, but for me the analogy works, so let me try to explain what I mean.
You build a database by feeding data into a system that then lets you pose queries which give you some of the data back, combined in intricate ways to produce information. (Information is not what’s stored in the database, it’s what happens to you when you interpret data.) An LLM differs from most databases in that its data is in a smeared-out format, a many-dimensional sludge where everything is mixed together. You usually can’t read out the exact data that were fed into it, and everything it was fed with isn’t exactly all kept in storage. When you ask it something, it goes to the place in the sludge that best matches your query and picks up stuff to form something that looks like an answer. If what you ask actually has strongly supported answers in the data, you probably get a good reply. If the data doesn’t contain any answer, or conflicting answers, the reply is likely to be nonsense, but presented by the LLM in a tone of utmost confidence.
The database analogy saves me from Elizaishly anthropomorphizing the LLM. Nothing that I’ve seen since I came up with it has contradicted it. The LLM doesn’t reason. It doesn’t creatively extrapolate from data like humans do, it merely interpolates.
Note that I’m not saying that computers can’t reason. In fact, computers are much better than humans at reasoning from specific premises. In that aspect, even the Casio calculator I got when I started secondary school is much more intelligent than I am. But an LLM doesn’t reason.
Could it learn to reason, with enough training? After all, humans have learned to reason although our hardware wasn’t exactly designed for, so…? Well, I can’t prove that it won’t happen, just like I can’t prove that that running practice won’t make your body develop wheels, but I don’t think it’s going to happen.
And furthermore, creatively extrapolating from data is something we still don’t know how to make machines do. The capability of coming up with a new solution, a new fruitful way of seeing something, from just a few examples, without being trained with massive amounts of data like an artificial neural network needs. The only way we know of emulating this is to make the machine randomly or exhaustively generate suggestions and then test each of them, which is only useful in search space limited by strict rules (like a game for instance).
Is it even possible for a machine to have this capability? Well, yes, my computer scientist’s answer is that obviously it’s possible: since humans can do it, machines can do it. It’s as impossible for a computer scientist to find a useful definition of “machine” that doesn’t include humans as it is for a biologist to find a suitable definition of “animal” that doesn’t include humans. But there is the possibility that it’s beyond our human capability to create other kinds of machines with a power which we ourselves have.
Before we can come up with a way of getting computers to do what we do, we need an understanding of how we do it ourselves. No, I don’t think it’s viable to train and fine-tune LLMs or other “AI” until they spontaneously develop capabilities that are beyond our comprehension. The fact that some people think – or worry – that it is viable, I attribute to a combination two flaws in their understanding. The first is anthropomorphizing, thinking that LLMs or other “AI” are getting to be like humans. The second is romanticising or overestimating what it is to be human: thinking that humans have a “general intelligence” that is the ultimate all-encompassing capability of the most powerful beings in the universe, and that any creature advanced enough is going to go down the same path as us.
I don’t see LLM development spiraling into a singularity-like scenario where a system creates other systems that are “smarter” or more imaginative than itself. The essence of a current artificial neural network doesn’t allow such a capability. It’s a database! The idea of AI technology either exterminating or lifting humanity to a celestial existence (or both) remains a stimulating fantasy.
Does that mean we’re safe? That AI technology won’t hurt us? Certainly not, technological development is always dangerous, AI is hurting us already, and accidents with catastrophic consequences are quite likely. It’s possible for AI systems, or for humans employing AI systems, to cause human extinction somehow, without the systems developing “consciousness” or “superintelligence” or whatever. But I consider it vastly more likely that human extinction comes from either ordinary biological disease (which could of course be due to a virus developed using AI tools) or from an astronomical event like an asteroid crashing into the earth.
In any case, human extinction is going to happen sooner or later, and the thought of that is an immense source of comfort to me.
