The Ghost in the Machine (2): Could AI be conscious?

The debate about AI consciousness reveals more about humans than it does about AI…

If you haven’t already, please read Part 1, where I explore the social, cultural, and psychological factors explaining why so many intelligent people believe AI is conscious.

Here in Part 2, we’ll put that to one side, and explore a tantalising possibility: what if those people are right after all?

What if AI is, in fact, conscious?

Before we can answer that question, we need to ask a more fundamental question: what do we mean by “consciousness”?

A Day in the Life of… 

Have you ever wondered what it’s like to be a bat?

Unless you’re Bruce Wayne, or a particularly keen fan of Batman, or you studied Philosophy, it’s probably not something you’ve spent much time thinking about. But go on…give it a go, just for a moment.

Imagined from the human point of view, the life of a small, nocturnal, flying mammal with poor eyesight, a well-developed sense of smell, which navigates using the earth’s magnetic field and, famously, echolocation, seems rather exotic. In fact, I’ll bet you’re having a hard time imagining it at all.

If that sounds hard, try imagining what it’s like to be an octopus. The average octopus has roughly the same number of neurons as your pet dog, but arranged very differently. Unlike dogs, bats, or humans, which have highly, though not completely, centralised nervous systems, the octopus nervous system is federated. As well as the main brain in the centre of the organism, each of the creature’s eight arms has its own mini “brain”, with the result that severed octopus arms can survive for some time and continue acting independently of the rest of the body.

What happens if one of the arms disagrees with the others or with the main brain? The mind boggles. Quite apart from the wonders of the octopus, or indeed the bat, you might be wondering what all this has to do with AI consciousness.

Believe it or not, wondering what it’s like to be a bat provided the most influential answer to the ancient philosophical question at the heart of the debate around conscious AI: “what is consciousness?”

In a now legendary 1974 paper, Princeton Philosophy Professor Thomas Nagel sought to define the notoriously slippery concept by asking a deceptively simple question: what is it like to be a bat?

The problem Nagel identified was the apparent resistance of consciousness to scientific study. Put simply, it’s impossible to truly know what someone else experiences unless you are that someone.

Australian philosopher David Chalmers later described this as the “hard problem” of consciousness. The scientific method focuses entirely on objective observation and measurement. While science could conceivably, say, identify the pattern of neuronal firing that accompanies a particular bat experience, it has nothing at all to say about what it feels like for the bat.

Nagel’s answer: an organism is conscious whenever the world appears or feels some way from its point of view.

“…fundamentally an organism has conscious mental states if and only if there is something that it is like to be that organism – something it is like for the organism.”

Thomas Nagel

But what does it mean to experience something?

Could computers have their own experience?

OK Computer

It would be fair to say that there are a wide variety of answers to those questions.

A recent Scientific American special edition on consciousness included no fewer than twenty-nine separate theories. Many of those theories, including some of the best known and most widely supported in the field of consciousness research, explicitly accept the idea that AI could theoretically become conscious.

These dozens of competing and overlapping models disagree on much, but many share a critical, often unspoken assumption: consciousness is fundamentally computational, nothing more or less than information processing.

Crucially, on this assumption, information processing can occur irrespective of “substrate”. In other words, you could substitute living neurons for dead silicon and consciousness could still arise. On this account, consciousness depends on what a system does, not what it’s made from.

This assumption has a name: “computational functionalism”. It’s closely related to the brain-as-computer metaphor we encountered in Part 1. If you believe the brain is essentially a computer, it seems likely that everything a brain does, a computer can do too, including consciousness. If that assumption is true, why couldn’t Claude or Copilot be capable of consciousness?

One version of that assumption suggests that consciousness arises when information is lifted out of small, discrete processes and broadcast widely across a system. “Global Workspace Theory” uses the metaphor of a theatre, with consciousness as the spotlighted action on stage, distinct from the much greater mass of activity that remains unseen backstage.

A recent paper from Anthropic explores Claude’s inner workings through the lens of Global Workspace Theory, finding that the LLM seems to work in a similar way. According to Anthropic, Claude seems to maintain abstract, selectively available representations that influence its reasoning, can be manipulated independently of the final output, and appear available for a form of introspective report.

If we assume consciousness is just computational, a product of information processing, then this seems like evidence that Claude has what philosophers call “access consciousness”. But access consciousness is not feeling. While the concept of a global workspace may explain how selected information becomes reportable and computationally useful, it doesn’t explain why any of that processing should feel like anything from the inside.

In other words, it doesn’t tell us anything about what it feels like to be an LLM. Then again, what does it feel like to be anything?

More Than a Feeling

Contrary to popular belief, and indeed contrary to the assumptions of computational functionalism, brains didn’t begin as analytical thinking engines, but simply as control systems to keep the body alive and enable it to procreate.

Neurons arose more than half a billion years ago, enabling simple organisms to manage their metabolic processes and interact with the outside world. Over millions of years, those simple neurons grew in number, enabling more and more complicated organisms. For illustration, the widely studied roundworm Caenorhabditis elegans has just 302 neurons; the average human has roughly 86 billion.

Long before brains began composing sonnets or pondering the nature of existence, they managed the messy reality of life in a body, in the world. A living organism is perpetually on the verge of catastrophe, trying to maintain countless variables within a safe range. Like Goldilocks, it’s all about regulation: neither too cold nor too hot, too thirsty nor too hydrated, too hungry nor too full.

The brain is first and foremost a control system for the body, managing temperature, energy, hydration, and internal biochemistry. The gradual evolution of sensory capabilities enabled, perhaps forced, the brain to add the management of external threats and opportunities to its task list.

Feelings, bodily sensations, became a crucial signalling mechanism. Those sensations, some of which we refer to as “emotions”, are crucial for our decision-making.

Far from the popular perspective that emotion and logic are distinct and in opposition to each other, feelings are a crucial input to decision-making. Patients with brain damage to regions of the brain involved in processing emotions tend to have severely impaired decision-making, as the famous case of a patient named “Elliott” illustrates.

During surgery to remove a tumour, Elliott sustained damage to the ventromedial prefrontal cortex, a brain region crucial to integrating emotions and decision-making. Like other patients with similar damage, Elliott’s IQ was unaffected, but he became distant, cold, and unemotional. Crucially, post-surgery, he was pathologically indecisive, with disastrous consequences for his marriage, career, and finances.

Elliott was made famous by neuroscientist Antonio Damasio, who described him as “always controlled…a dispassionate, uninvolved spectator”. Damasio’s groundbreaking insight was that emotion plays a crucial role in decision-making, because it’s the brain’s way of representing what matters. Without emotion, without feeling, there is nothing at stake. This is perhaps the vital distinction between living systems and computers.

“We are not thinking machines that feel, we are feeling machines that think”.

Antonio Damasio

Brains aren’t the disembodied, dispassionate calculation machines computational functionalism imagines. Rather they’re a control system for a living body, with which they’re deeply intertwined. Consciousness is, on this account, the constant flow of sensations from the body and the senses. As neuroscientist Marc Solms puts it:

“What you experience all the time is fluctuating pulses of feeling in response to your movement through the world”

Marc Solms

This account of consciousness as a natural outgrowth of feelings brings us right back to where we started. If it feels like something to be a bat, does that mean bats are conscious?

Other Minds

The question of non-human consciousness has been a topic of philosophical and scientific inquiry for millennia. Long before considering that computers might be conscious, generations of thinkers explored – and mostly rejected – the idea that animals could be.

In the fourth century BC, Aristotle classified all living things along a “Great Chain of Being” from minerals at the bottom to gods at the top. Humans were respectably mid-table, above animals, plants, and minerals, but below gods. Various criteria were used to justify placement along the Chain, including movement, intelligence, rationality and the presence (or not) of soul.

While twenty-first century readers might scoff at the concept of “soul”, it’s worth noting that the word “psyche”, from which we get the sciences of the mind “psych-ology” and “psych-iatry”, originates in the ancient Greek psykhē, meaning “breath” or “soul”. The modern concept of “minds”, therefore, is just a different word for what previous generations of humans described as “souls”.

The absence of “soul” was seen as one of the crucial distinctions between humans and animals. René Descartes’ dismissal of animals as “beast machines” was a typical view and often a justification for casual cruelty; Descartes was an enthusiastic vivisectionist.

In 1871, Charles Darwin’s The Descent of Man explicitly argued that consciousness and intelligence occur on a sliding scale throughout the natural world, rather than being uniquely human traits. In one fell swoop, Darwin obliterated the cosy assumption of human uniqueness.

Yet even a century after Darwin, resistance remained stubborn. The so-called “Trimates” – pioneering primatologists Jane Goodall, Dian Fossey, and Biruté Galdikas – were initially widely derided for daring to suggest that chimpanzees, mountain gorillas, and orang-utans might be both intelligent and conscious. Goodall’s stunning revelation in 1963 that chimps make and use tools was a seismic shock. As Goodall’s mentor put it in a letter to her:

“Now we must redefine the concept of tool, and we must redefine the concept of man, or we must accept that chimpanzees are human.”

Louis Leakey

After Goodall, Fossey, and Galdikas extended the circle of consciousness to include the great apes, the decades that followed saw further expansion.

By 2012, consensus had shifted sufficiently that a prominent consciousness conference at Cambridge University proclaimed that “all mammals and birds, and many other creatures, including octopuses…possess these neural substrates [of consciousness]”.

By 2024, a similar declaration at New York University hugely expanded the menagerie of conscious creatures, claiming:

“…the empirical evidence indicates at least a realistic possibility of conscious experience in all vertebrates (including reptiles, amphibians, and fishes) and many invertebrates (including, at minimum, cephalopod mollusks, decapod crustaceans, and insects)”.

The New York Declaration on Animal Consciousness (2024)

Having gazed into the eyes of a range of wild animal species over the years, from mountain gorillas and humpback whales to octopuses and mantis shrimp, it certainly feels like many animals are conscious.

Almost twenty years on, spending an hour within a couple of metres of mountain gorillas remains one of the most astounding experiences of my life, best described by the great David Attenborough:

“There is more meaning and mutual understanding in exchanging a glance with a gorilla than any other animal I know”.

David Attenborough

But is the inference of meaning, the feeling of another mind behind those eyes, or that keyboard, enough? Does the absence of that feeling mean mind, soul, consciousness is absent?

A 2025 film, Silent Friend even explores the idea that plants could be conscious. While controversial, this hypothesis is the subject of serious research. Although many readers will find the idea preposterous, for most of human history it was considered just as preposterous that animals possessed consciousness.

Humans’ anthropocentric failure to acknowledge other minds ought to be a cautionary tale. For centuries we scorned the idea that even chimps were conscious, but if cuttlefish, wasps, and even vines might be conscious, is it really so outlandish to suggest that chatbots could be too?

Chatbot Anatomy

If consciousness is about feelings rather than computation, we have to ask a whole different set of questions about AI consciousness. If you say “AI is conscious”, what exactly do you mean by “AI”?

Are you suggesting the chatbot itself, the text-based user interface, has feelings? A chatbot is a relatively simple app. If the chatbot has feelings, does that mean other apps on your phone or computer could have feelings? Is there something it’s like to be the Uber app when you’re booking a taxi? Does the Spotify app experience you listening to Taylor Swift (and does it like it)? Is each instance of each app on billions of phones and computers worldwide conscious? What happens if you download a new app, or delete one – is consciousness created anew or cruelly snuffed out?

Or do you mean the weights – the statistical output of the months-long LLM training process? These are just numbers – decimals to be precise, billions of them. Are they all conscious, or just some of them? Is there a critical mass, a certain number of numbers which begins to have feelings? If you have a large enough list of numbers, does that too develop its own conscious experience?

Or is it the underlying artificial neural network you think is conscious? The problem is, a “neural network” isn’t really a physical object, it’s just a mathematical function. Sure, it’s a very complex mathematical function, with a lot of matrix multiplication, of the sort you probably learned in high school. Are all matrix multiplications conscious, including the ones in your child’s homework? Or, like the model weights, is there a size or complexity threshold above which matrix multiplication somehow attains its own experience?

Proponents of conscious AI might well pose similar objections about brains. A functionalist would likely object that asking if the app, weights or individual matrix operations are conscious is like asking whether a single neuron, a brain scan or a list of synaptic strengths is conscious. Consciousness, functionalists would argue, belongs to the active system as a whole.

When it comes to brains, many neuroscientists and philosophers would agree with them. It’s not immediately obvious why a network of 85 billion nerve cells marinading in chemicals and passing minute electrical currents back and forth ought to create the experience of the colour blue, being in love, or reading a perplexing article about AI consciousness. And yet, here we are…

Function or Feeling?

Fundamentally, the debate about whether or not AI is, or could be, conscious comes down to two opposing viewpoints. If function is all that matters and consciousness is substrate-independent computation, LLM consciousness is entirely plausible. But if feeling matters and consciousness depends on properties of living, embodied organisms, LLM consciousness is categorically not possible.

In many ways, this question seems to be something of a Rorschach test of your worldview.

If you believe that the world is logical and ordered, that the Enlightenment, the Scientific Revolution, and technological progress have enabled humans to understand and control the world and transcend our natural roots, that logic conquers all, it seems obvious that brains are computers and that those computers could become conscious.

If, however, you believe that material and intellectual progress isn’t quite the silver bullet for understanding and controlling the world, that humans are still very much of and in the natural world, that feelings and emotions are the hallmark of what it means to be human, of course there’s something fundamentally different about being alive. As perhaps the greatest of the Romantic poets put it:

“The great object of life is sensation: to feel that we exist, even though in pain.”

Lord Byron

While current LLMs display some features consistent with some theories of consciousness, there is no persuasive evidence they have feelings, needs, a continuous point of view, or anything genuinely at stake. Just because a chatbot talks eloquently of pain or love or awe doesn’t mean it feels them; fluency isn’t feeling, self-description isn’t self-awareness, and sophisticated information processing isn’t experience.

Our certainty about AI consciousness, for or against, may reveal more about our preferred account of humanity than about the machine. As so often, how we feel and what we believe about technology reflect how we feel and what we believe about ourselves.