I blame Star Trek.

For the last 60-odd years, we’ve grown used to seeing people talking to computers and getting factual answers back. And then we look at LLMs, and their chat interfaces and assume that we’ve hit Star Trek technology. But we haven’t because LLMs don’t do what we think they do. They are, on the whole, guessing machines. They guess what the first letter, the first word, the first sentence in any response should be. And they build from that.

Modern LLMs are trained on enough stuff that they’re superb at guessing — but they still regularly get things wrong. Even the name we give these errors — “hallucinations” — is misleading. It assumes that the rest of their output is not a hallucination. It is. It’s just one that happens to align with reality better. Really, LLMs just produce guesses on a scale from “spot on” to “wildly and dangerously inaccurate”.

And that means that we’re making mistakes. We’re misunderstanding what LLMs are, and then using them for tasks they’re not well suited to. I’ve had enough laughably wrong answers in the AI summaries at the top of Google Search that, for now, I’m ignoring them.

Figuring out what AI is for

In fact, we might be going through a process of eliminating the tasks LLMs aren’t good at, knowing that what remains will be their true purpose. (With apologies to Sherlock Holmes.)

For example, the sudden arrival of DeepSeek on the scene has upended some of our assumptions about what LLMs are. The general idea has been that they are, essentially, server-based tools. This is the end of that. The sheer compute needed to run them and, especially, train them, made them look like an infrastructure business. Those big VC valuations are, in part, being based on the idea that the one big moat to entry into the field is access to the compute to make it work.

DeepSeek, with its much smaller compute and energy footprint, for similar results, changes the game. Could more models be trained and run on consumer devices? Probably. Suddenly, Apple’s bet on on-device AI, while starting out rough, is beginning to look like a better one than many people predicted. Maybe AI will end up being less of a product and more of a feature.

What does the true AI interface look like?

We’ve argued before that today’s chat-interface with LLMs has too many similarities with the old command line interfaces of pre-GUI computers to be the final or only form of an LLM interface. Back at NEXT23, Matt Webb gave us a very different vision of what AI could look like: a series of agents that assisted us with specific tasks, and especially those where guessing could be useful.

We don’t want our tools guessing at facts. AI has proved to be pretty bad at summarising news, for example, from Apple’s pulled summary notifications to the latest research by the BBC:

The findings are concerning, and show:

• 51% of all AI answers to questions about the news were judged to have significant issues of some form
• 19% of AI answers which cited BBC content introduced factual errors – incorrect factual statements, numbers, and dates
• 13% of the quotes sourced from BBC articles were either altered or didn’t actually exist in that article.

We have enough of a misinformation problem with humans, so maybe let’s not add AI to the equation? But we do want them to help out where creativity is needed. For example, guessing what the best headlines for this piece might be is where AI can help it. And, indeed, I fed ChatGPT an early draft of this article, and tweaked one of its headline suggestions. Microsoft might have one of its biggest branding wins by calling its AI CoPilot. Because that’s precisely what AI seems to be best at: helping out on knowledge-informed creativity tasks.

Exploring AI’s future role

We are in the early days of this new technology. And, providing it does prove to be a new internet, rather than a new metaverse or new blockchain, it might be years before we finally understand what it is. One could certainly make an argument that the internet only found its true consumer expression with the launch of the iPhone, decades after the first computers were connected to a TCP/IP network. We probably have no more idea of what the LLMs of the future will look like than I could envisage the iPhone when I bought my first dial-up modem and hooked it up to my Mac.

There have certainly been attempts to take AI and make it into a physical product. But so far, the track record isn’t good. Humane, the company that made the AI Pin — a wearable, AI-infused badge, in essence — got sold at a knock-down price to HP. The actual pins themselves will be bricked. Not a great outcome if you invested $700 in one less than a year ago.

One could even suggest that, right now, AI is a powerful tool in search of a solution. And we’re applying it in places that it’s not suited for, and which might run the risk of damaging its reputation. When you see an AI summary of an idea that you’re familiar with, and see that it’s wrong, it undermines your faith in AI summaries of things you don’t know about.

The great guessing machine

Yet, when we apply AI to tasks that require a degree of guessing, they’re capable of guessing faster than humans, and sometimes better than us. To take a couple of examples, AI is helping with both chip design and medical science.

It’s coming up with — guessing at — wireless chip designs that are both more powerful than human-designed ones, but which also take an alternative design path:

Not only did the chip designs prove more efficient, the AI took a radically different approach — one that a human circuit designer would have been highly unlikely to devise. The researchers outlined their findings in a study published Dec. 30 2024 in the journal Nature Communications.

Meanwhile, Google’s AI Co-Scientist is allowing research teams at Imperial College London to explore hypotheses in a fraction of the time traditional methods take:

“This effectively meant that the algorithm was able to look at the available evidence, analyze the possibilities, ask questions, design experiments and propose the very same hypothesis that we arrived at through years of painstaking scientific research, but in a fraction of the time.” A matter of days, specifically.

Again, guessing. This is where AI really shows its strength: highly educated guesses. When they are delivered in a way that makes them useful to the user, we have something valuable. We’ve developed something that’s new: an informed guessing technology, rather than an information technology. But we’re still treating it as if it were like the old tool. Once we let go of our old assumptions, and let it loose to do what it’s really good at, we’ll start really making the most of it as a technology.

Agentic AI: guessing on our behalf

And so, let’s stick with our mental model of the current chatbot-style AI as the command line of AI. If we deploy it where automated guesswork saves human labour — or accelerates it — then we’re playing to its strengths. If computers were, in Steve Jobs’ words, bicycles for the mind, perhaps LLMs are bicycles for the intuition.

Build me an AI into my writing tools that suggests alternative phrasing or headlines, then I’m interested. Build me an AI agent, that goes out and culls reading material for me from the internet based on what it knows about me and what it knows about the subjects I’m interested in: I’m interested. But try to summarise all that reading for me? No, thanks. The stakes are low, if it guesses wrong about what I’m interested in. The stakes are very much higher if it presents me with inaccurate guesses about what the information actually is.

One day, we’ll have that Star Trek computer. And maybe AI will even help us build it. But, for now, we’ll just have to settle for a tool that helps us make better guesses — or makes them for us — and we’ll just have to boldly go forward into the future that opens up for us, where no one has been before…

Image by Placidplace / Pixabay