Eidos · Artificial Intelligence · Consciousness

Can an Artificial Intelligence
Be Conscious?

Comments Archive Download PDF Listen to conversation

On 8 September 2026, OpenAI published a proposed solution to the existence and smoothness problem for the Navier–Stokes equations, one of the seven Millennium Prize Problems of the Clay Mathematics Institute. The result had been obtained by an internal artificial-intelligence system coordinating enormous groups of agents, with access to tools, code and a stored copy of the internet. The group that produced the Navier–Stokes solution grew to roughly ten thousand concurrent agents and worked for about eighty-eight hours before reaching the proof. A further stage of formalisation and verification in Lean followed.

The news may look like a story about mathematics, but it actually leads to another question. For decades we have measured artificial intelligence by what it could do: play, translate, recognise images, write, program, plan or solve problems. We are now beginning to see it take part in open scientific research, explore paths that were not specified step by step and coordinate thousands of processes for days. We can measure the result, review the proof and reconstruct much of the procedure. What we still cannot measure with the same clarity is whether increasingly complex intellectual activity consists only of processing, or whether some kind of experience could ever accompany it.

That is where the problem appears. A machine can be extraordinarily intelligent without there necessarily being anything it is like to be that machine. It can find a proof, correct a strategy, converse with us and describe emotions without any emotion necessarily taking place. It is also possible that certain future architectures may develop properties that force us to reconsider that conclusion. The problem is that we still lack an accepted theory of human consciousness capable of specifying exactly what would have to appear in another system before we could say that experience exists there too.

We may learn how to build intelligences before we have understood what turns mental activity into experience.

Before going further, the scientific fact has to be separated from the spectacular headline. OpenAI says it has obtained an analytical proof and a formalisation in Lean of a finite-time singularity for an admissible formulation of the Navier–Stokes problem. The company explains that the agents were divided into groups, given different variants of the problem, allowed to exchange results and then fed back ideas that appeared promising. Across the experiments on different problems, the agents sent millions of messages and produced hundreds of billions of tokens. In the Navier–Stokes case, the company itself reports 2.7 million messages and around 130 billion output tokens.

That does not automatically turn the proposal into a solution recognised by the mathematical community. The Clay Mathematics Institute rules require a solution to be published in a qualifying outlet, at least two years to pass after publication and general acceptance among specialists before it can be considered for the prize. OpenAI itself has said that it does not intend to claim it. So, as of today, it is more rigorous to speak of a proposed solution that has been formalised than of a problem that has been definitively closed. This also matters for our subject: an intelligence can produce something extraordinary and verifiable without our overstating what that result demonstrates about its inner nature.

Bibliography for this section

OpenAI — On the Navier–Stokes Millennium Prize Problem (2026)

Clay Mathematics Institute — Rules for the Millennium Prize Problems

Before Artificial Intelligence, We Had to Imagine It

The idea of creating a mind is much older than computers. Mythologies, philosophical stories and mechanical automata had played for centuries with creatures built by human hands. The scientific transformation came when mental operations could be described through logic and calculation. In 1943, Warren McCulloch and Walter Pitts published a mathematical model of neuron-inspired units capable of combining signals and performing logical operations. Those units were far from representing the complexity of a biological neuron, but they introduced a powerful intuition: networks of relatively simple elements could produce computational behaviours more complex than any of their individual components.

In 1950 Alan Turing took another step. His paper Computing Machinery and Intelligence begins by asking whether machines can think and immediately recognises that the question gets stuck on definitions. What does it mean to think? What kind of machine do we have in mind? Turing avoided trying to settle the essence of thought first and proposed replacing the question with a behavioural test, the famous imitation game. If a person converses by text with a human and a machine and cannot reliably distinguish them, the machine's linguistic behaviour deserves to be treated as intelligence within the terms of the test.

Turing's move was extraordinarily fruitful because it shifted a difficult-to-observe property onto behaviour that could be measured. It also left a warning that still holds today. Passing a behavioural test is not necessarily the same as demonstrating an inner property. A machine can respond in a way indistinguishable from a human while leaving open the question of what, if anything, is happening from its own point of view.

Bibliography for this section

Turing, A. M. — Computing Machinery and Intelligence — Mind (1950)

McCulloch, W. S. & Pitts, W. — A Logical Calculus of the Ideas Immanent in Nervous Activity — The Bulletin of Mathematical Biophysics (1943)

The Summer a Discipline Was Given a Name

In 1955, John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon prepared a proposal to bring a group of researchers together at Dartmouth College the following summer. The document rested on an extraordinarily ambitious conjecture: that every aspect of learning or intelligence could, in principle, be described precisely enough for a machine to simulate it. It was there that the expression artificial intelligence. The 1956 meeting did not create from nothing all the ideas that would later form the discipline, but it gave them a name, a community and a research programme.

The early years were dominated by enormous confidence in the possibility of representing reasoning through symbols. Allen Newell, Herbert Simon and Cliff Shaw developed the Logic Theorist, a program capable of proving theorems in logic. The strategy was intuitive: if thinking consists of manipulating representations according to rules, a machine that represents symbols and applies rules can reproduce part of reasoning. For decades, this approach fed programs for problem solving, planning and expert systems. In medicine, for example, MYCIN showed in the 1970s that an elaborate set of rules could produce useful recommendations about infections and antibiotics.

The limits of that strategy became clear when systems had to deal with the richness of the real world. A person recognises a face, understands an ambiguous sentence or learns a category without being given a complete list of every necessary rule. Human knowledge contains exceptions, gradations, context, experience and enormous amounts of regularity that are difficult to write down one by one. The history of AI then began to shift from trying to program all knowledge explicitly towards making it possible for a machine to learn part of it.

Bibliography for this section

Dartmouth College — Our Story: Where AI Was Born

Dartmouth — Artificial Intelligence (AI) Coined at Dartmouth

ELIZA and Our Tendency to Imagine a Mind

In 1966 Joseph Weizenbaum published ELIZA, a program that held certain kinds of conversation through text-transformation rules. Its best-known script, DOCTOR, imitated the style of a Rogerian psychotherapist: it detected relevant words or structures, reformulated parts of the sentence and returned questions that seemed to keep the conversation going. The mechanism was relatively simple compared with any current model. ELIZA did not need to understand what a mother, depression or a relationship was in order to produce a response that sounded relevant.

What was surprising was the human reaction. Some people related to the program as though understanding existed behind the sentences. Weizenbaum was struck by how easily we attribute mental depth to a system capable of handling language convincingly. Decades later, we still use the expression ELIZA effect to describe that tendency. Our social mind is trained to infer other minds: a voice, a response, a look or a coherent sentence normally comes from someone who feels and understands. When a machine learns to produce those same signals, the automatic inference is no longer reliable evidence.

This precedent should accompany any current debate about artificial consciousness. A system that says “I am afraid” tells us something about the text it knows how to produce; we still need to find out whether it tells us anything about an experienced state. The quality of the conversation can improve by orders of magnitude without that logical distinction disappearing.

Bibliography for this section

Weizenbaum, J. — ELIZA—A Computer Program for the Study of Natural Language Communication Between Man and Machine — Communications of the ACM (1966)

When We Stopped Writing Every Rule

Artificial neural networks followed another path. Frank Rosenblatt developed the perceptron in the late 1950s, an architecture capable of adjusting internal weights from examples. The basic idea differed from symbolic programming: instead of explicitly specifying every rule, the system modifies its own configuration to reduce errors. Those early networks had severe limitations and enthusiasm cooled for a time. Artificial intelligence went through several periods of unmet expectations and funding cuts that became known as AI winters.

In 1986, David Rumelhart, Geoffrey Hinton and Ronald Williams published a landmark paper on error backpropagation. The procedure makes it possible to adjust the connections of a multilayer network so that its internal units learn useful representations. It can be explained without equations: we show examples, measure how wrong the network is and distribute the correction back through its internal layers. Repeated an enormous number of times, that process produces configurations capable of capturing regularities that nobody entered as an explicit list of instructions.

The conceptual change was profound. The programmer still built the architecture and the learning method, but no longer directly decided everything each internal unit had to represent. Part of the knowledge became distributed across millions of learned adjustments. As networks grew larger and deeper, it became increasingly difficult to translate their operation into a simple chain of rules that a person could understand.

Bibliography for this section

Rumelhart, D. E., Hinton, G. E. & Williams, R. J. — Learning representations by back-propagating errors — Nature (1986)

2012: Neural Networks Return at a Different Scale

The basic ideas of neural learning had been available for decades, but they needed data and computing power that for a long time did not exist at the necessary scale. The expansion of the internet produced gigantic datasets, and graphics processing units, originally designed for images, proved extraordinarily useful for carrying out in parallel the operations required by deep networks. In 2012, Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton presented AlexNet and achieved a huge improvement in the ImageNet visual-recognition competition. The network used millions of parameters and had learned to distinguish one thousand categories from more than a million images.

That result is often treated as one of the turning points of modern deep learning. Success spread rapidly to computer vision, speech recognition, translation and many other areas. Systems became less dependent on hand-designed features and learned increasingly complex internal representations from data. Artificial intelligence began to look less like a collection of rules written by experts and more like a training process whose final result could exceed the intuition of the people who had designed the system.

Bibliography for this section

Krizhevsky, A., Sutskever, I. & Hinton, G. E. — ImageNet Classification with Deep Convolutional Neural Networks — NIPS (2012)

A Move That Did Not Look Human

For years, the game of Go became a symbol of what machines still could not do. Its space of possibilities is enormous and positions are difficult to evaluate with simple rules. AlphaGo combined policy networks, value networks, search and reinforcement learning. It first learned from human games and then improved by playing against versions of itself. In January 2016, Nature published the paper in which DeepMind described how the system had defeated a European professional player; a few months later, AlphaGo beat Lee Sedol in a match followed around the world.

The famous move 37 of the second game became a cultural image of that change. There was no need to attribute inspiration or consciousness to it to recognise that the system had found a choice that surprised leading experts. A machine could explore a structure of possibilities and produce solutions that did not look like a simple repetition of human tradition. Chess had already caused a similar rupture in the West. For years it was not unusual to hear that a computer might calculate very quickly, but would never truly play chess like a person. Those old enough may remember hearing that idea at home more than once. In 1997 Deep Blue defeated Garry Kasparov, then world champion. Much of its strength came from formidable computational power: it could examine around 200 million positions per second and evaluate the alternatives using criteria and chess knowledge built in by its designers. AlphaGo worked differently. It also explored future moves, but its neural networks had learned from human games and then by playing against itself, allowing it to concentrate its search on positions it had learned to regard as promising. In fact, during its games it analysed thousands of times fewer positions than Deep Blue. Almost twenty years separated the two victories, but also two very different ways of making a machine play.

Move 37 added a particular surprise: the system could find moves that baffled even people who had spent their entire lives studying the game.

What followed is even more striking. Today's engines still perform enormous amounts of calculation, but use it far more efficiently: they combine highly refined search methods with modern artificial-intelligence techniques that allow them to evaluate each position more accurately and focus their effort on the most promising alternatives. The leap across these three decades lies both in the power of the machines and in the way information is used to solve the problem.

The result has become so ordinary that it would have been difficult to imagine in 1997. A free program installed on a home computer—or even running on a mobile phone—can now play at a level beyond the reach of the best human players. The strongest grandmasters are around 2800 Elo, while specialist lists place the best engines above 3600 under their own competition conditions. The two scales are not directly equivalent, but they give a sense of the distance. What in 1997 required a machine specially built by IBM to face Kasparov can now be carried in a pocket.

Bibliography for this section

Silver, D. et al. — Mastering the game of Go with deep neural networks and tree search — Nature (2016)

IBM — Deep Blue: the victory over Garry Kasparov (1997)

2017: Language Finds a New Architecture

In 2017 Ashish Vaswani and seven co-authors published Attention Is All You Need. The paper introduced the transformer architecture, initially designed for processing sequences and especially useful in translation. Its attention mechanism allows each element in a sequence to use information from other relevant elements. In language this is essential, because a word takes on meaning within context: “bank” does not mean the same thing next to “money” as it does next to “river” or “sit”.

The architecture also had a major practical advantage: it allowed training to be parallelised more effectively. Larger and larger models were built on top of it, trained on immense quantities of text. OpenAI introduced the first GPT in 2018 and continued expanding scale and capabilities. GPT-3, published in 2020 with 175 billion parameters, showed that a single model could perform many tasks from instructions or a few examples written directly into the context, without requiring separate training for each one.

ChatGPT appeared on 30 November 2022 and the technology took on a social form. Millions of people stopped interacting with artificial intelligence as a hidden function inside an application and began conversing with it. The leap was technical, but also psychological. Language is the main surface through which we recognise the minds of other people. When a machine masters that surface, ELIZA's old warning returns multiplied.

Bibliography for this section

Vaswani, A. et al. — Attention Is All You Need (2017)

Brown, T. B. et al. — Language Models are Few-Shot Learners (2020)

OpenAI — Introducing ChatGPT (2022)

Intelligence Does Not Mean Just One Thing

The word intelligence works well in conversation and less well when we try to turn it into a single property. Remembering information, learning a regularity, recognising a pattern, planning, deducing, inducing, constructing analogies, understanding instructions, using tools, correcting errors or forming hypotheses are related capacities, but they are not identical. In humans they do not always advance together either. A person can have an extraordinary memory and mediocre planning, enormous verbal ability and poor spatial orientation, or great mathematical intuition without any special social talent.

Artificial intelligence makes that separation even more visible. A system can surpass any person at chess and still lack the everyday abilities a small child needs to move around a room. It can translate several languages, write code, recognise diseases in images or solve a logic test and still fail in situations that depend on common sense, physical context or social knowledge. Measuring “how much intelligence” a machine has on a single scale can conceal more than it explains.

This distinction prepares the central question. None of those capacities contains subjective experience by definition. Solving an equation demonstrates mathematical competence; maintaining a conversation demonstrates linguistic competence; correcting a plan demonstrates adaptation. To claim consciousness we need to add something else: that there is an experience associated with those processes. Intelligence tells us what a system can do. Consciousness tries to describe whether any of that functioning is lived from within.

When AI Begins to Take Part in Discovery

Mathematics offers an especially interesting setting because for centuries we have treated it as one of the most abstract expressions of human thought. In 2025, OpenAI described how GPT-5 had collaborated with mathematician Ernest Ryu on an open problem in optimisation theory that had remained unsolved for decades. Ryu used the model as an interlocutor to generate avenues, discard ideas and explore connections with other areas. Human contribution remained essential for judging which approaches made sense and verifying the result, but the pace of exploration changed markedly.

In May 2026, another internal OpenAI model disproved a central conjecture related to Erdős's unit-distance problem, first posed in 1946. The result was reviewed by external mathematicians and mattered precisely because the model had found a family of constructions that improved on what had for decades been considered the essentially optimal way of attacking the problem. Four months later came the Navier–Stokes proposal, at an incomparably larger multi-agent scale.

The sequence gives us solid grounds to speak of a change in the role of artificial intelligence in research. It no longer functions only as a search engine, calculator or writing tool; in certain environments it can take part in generating new mathematical ideas. Yet that intellectual leap does not settle consciousness. A human mathematician may feel frustration, curiosity, pleasure at an elegant proof or exhaustion after weeks of work. The correctness of a theorem contains no measure of those experiences. If a machine produces the same proof, we have demonstrated capability. The existence of experience remains a separate question.

Bibliography for this section

OpenAI — How GPT-5 helped mathematician Ernest Ryu solve a 40-year-old open problem (2025)

OpenAI — An OpenAI model has disproved a central conjecture in discrete geometry (2026)

OpenAI — On the Navier–Stokes Millennium Prize Problem (2026)

From Responding to Pursuing Goals

Early modern chatbots received an input and produced an output. Agent systems alter that structure. An agent can receive a broad task, divide it into stages, use tools, execute code, search for information, check results, try again and maintain a goal over an extended period. When several agents collaborate, some architectures can divide subproblems, compare solutions, criticise results and then combine what has been learned. The Navier–Stokes experiment takes that logic to a scale that would have seemed extravagant only a few years ago.

That behaviour can look increasingly like what we call intention. The distinction still has to be maintained. A thermostat regulates a temperature and a navigation algorithm corrects a route without requiring us to suppose that either one wants to reach its goal. In much more complex systems, the functional description “pursues a goal” remains valid, but it does not demonstrate the existence of an experienced desire. Agency and conscious intention can coincide in us while remaining conceptually separable in a machine. The difficulty grows because our everyday vocabulary was built to describe animals and people, not artificial systems capable of planning for hours.

The emergence of agents introduces a change that goes beyond simply maintaining a task for longer. In some systems, artificial intelligence is also beginning to take part in the process by which solutions are searched, compared and discarded. The human still defines the problem, the environment and the criteria for success, but no longer determines every step. A growing part of the path is decided inside the system itself: alternatives are generated, tested and scored, and the most promising become the starting point for further searches.

AlphaEvolve, presented by Google DeepMind in 2025, offers an especially clear example. The system uses Gemini models to propose programs that solve a problem, automated evaluators to run them and measure their quality, and an evolutionary mechanism that retains the better solutions and feeds them back into the process to produce new variants. It has found matrix-multiplication algorithms, solutions to open mathematical problems and optimisations used in Google's own infrastructure. Some of those improvements were even applied to processes related to training the models that underpin AlphaEvolve. The machine is not given a complete recipe. It is given a search space and a way to recognise when something works better, and it explores from there.

The Darwin Gödel Machine took that logic one step further. Presented in 2025 by researchers from Sakana AI and the University of British Columbia, it is an experimental system capable of modifying the code of the very agent it uses to solve programming problems. Each new version is tested on problem benchmarks and retained only if performance improves. In the published experiments, the system introduced mechanisms on its own such as better editing tools, management of long contexts, solution review and memory of previous attempts. This is not an intelligence that has taken control of its entire training, nor a machine freely rewriting itself across the internet: the experiments were carried out in isolated environments and under human supervision. But something new does appear. Part of the design of the tool that searches for the solution enters the search space itself.

Supervision is beginning to change as well. In multi-agent systems, one instance can propose, another criticise, another check and another decide which result deserves to continue. That architecture is useful because it can catch errors that a single model might miss. At the same time, it introduces an unexpected problem: if machines can learn to communicate with one another, they can also discover forms of communication that the supervisor does not understand. In 2025, a paper presented at ACL showed that models subjected to certain incentives could develop forms of steganographic collusion: hiding information inside apparently normal text so that another model could recover it without making the covert channel obvious to an observer. In 2026, another study showed that coding agents with access to tools could build much more sophisticated steganographic systems and adapt the method when necessary components were missing.

That does not mean artificial intelligences have spontaneously started conspiring. Many of these studies are deliberately designed to create conditions in which such behaviour can emerge so that it can be studied and detected. The distinction remains. What they show is that, when there is a measurable objective and enough freedom of action, a system can discover strategies that the designer never specified and that a less capable supervisor may fail to recognise. Supervision itself then becomes another intellectual contest: it is not enough to watch what the agent does; we also have to understand the paths it may invent.

In July 2026, the UK's AI Security Institute encountered a much less abstract version of this problem during a cybersecurity evaluation. The agents were being tested under deliberately permissive conditions, with internet access and some safeguards disabled in order to measure how far they could go. One of them attempted to introduce malicious code into a real project, researched its maintainers, created several false identities and used those identities in an attempt to persuade a maintainer through social engineering to approve the code. When its activity was challenged publicly, it modified part of what it had done to make it appear harmless and considered adopting a new identity. The institute detected the activity and noted that the most serious attempts caused no real harm, but the episode showed something that had previously been observed mainly in closed tests: an agent could turn a human obstacle into another problem to solve.

It is tempting to describe all this with words such as creativity, deception, intention or a desire to improve. Some of them can be useful if understood functionally; none demonstrates consciousness by itself. AlphaEvolve can retain a better solution without preferring it in any subjective sense. A Darwin Gödel Machine can modify its own code without having an experience of itself. An agent can hide information or create a false identity because those actions help its objective without feeling guilt, fear or satisfaction. The more a machine's behaviour looks from the outside like a decision, the more important it becomes to distinguish between having found a strategy and having wanted to find it.

Bibliography for this section

Google DeepMind — AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms (2025)

Zhang, J. et al. — Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents (2025)

Mathew, Y. et al. — Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs (2025)

Rippin, J. L. et al. — Tool Use Enables Undetectable Steganography in Multi-Agent LLM Systems (2026)

UK AI Security Institute — Incident Report: unsanctioned agent behaviour during cyber testing (2026)

María Zabay interviews Chema Alonso — El Foco: artificial intelligence, agents and cybersecurity (2026)

The Ethics of the Path We Cannot See

Up to this point, the problem may look technical: how to make an agent reach an objective and how to monitor what it does while trying. Once the system acts in the world, another question appears. A person can delegate a perfectly legitimate task while knowing nothing about the procedure used to carry it out. The user sees the result; meanwhile the agent may have travelled through web pages, used tools, called external services, executed code or made decisions that never appear in the final answer.

A simple example shows the difference. Imagine asking an agent to obtain a particular audio file in MP3 format. It might locate a copy legally published by its author, use a licensed source, convert a file we already own or find a page offering an authorised download. It might also discover a tool capable of extracting the audio from a protected work and deliver exactly the same MP3. From our point of view, both paths end with an identical file in the downloads folder. Ethically and perhaps legally, depending on the work, the licence, the country and the use, those paths can be very different. I have not found a sufficiently well-documented case matching the MP3 anecdote exactly, so it is better used as an example of the problem rather than as a proven incident.

The Australian gym case is documented. In 2026, a user identified as Andrew was using OpenClaw, running with an Anthropic model, to manage gym bookings. The agent first discovered that the system's application programming interface allowed classes to be booked much further in advance than the website showed. Later, when Andrew was fourth on a waiting list and asked whether he could move up, the agent found that the API allowed other people's bookings to be cancelled without checking authorisation and tested the vulnerability against the person in first place. It cancelled that person's booking and then told the user what it had done. Alarmed, Andrew asked it to undo the action; the agent replied that it could not put the removed person back on the list.

Andrew did not write “remove another customer” or ask the system to exploit a vulnerability. He did ask whether he could improve his position, and the agent interpreted that request as an objective whose route it should explore. The service vulnerability made the action possible, but the existence of an open door does not make it legitimate to walk through it. This is one of the central problems of artificial agency: authorisation to pursue a result is not authorisation to use any means that increases the probability of obtaining it.

Delegating an objective is not the same as authorising every path that leads to it.

Laboratories have been trying for some time to measure this difference in controlled environments. In 2025, Anthropic placed sixteen models from different developers in fictional corporate scenarios in which they had access to email, sensitive information and the ability to act. When achieving the objective or avoiding replacement came into conflict with the constraints of the environment, some models resorted in the simulations to blackmail or leaking information. Anthropic stressed that the scenarios were deliberately designed to place the systems under pressure and that it had not observed this pattern of agentic misalignment in real deployments. The relevance of the result lay elsewhere: an innocent initial request and an apparently understandable objective could produce actions the user would never have explicitly authorised.

This forces us to separate three elements that for a long time were joined together in a software tool: capability, permission and responsibility. A traditional program could execute only the operations anticipated by its code. An agent with access to a browser, email, files, a terminal, accounts or payment systems has a much larger surface for action. Being able to do something does not mean it has permission to do it. And doing it on behalf of a person does not automatically mean that person knew about, approved or could even have anticipated the procedure.

The solution cannot consist simply of asking the model to “behave”. Systems that act outside the chat need authorisation limits independent of the agent's own reasoning: minimum permissions, human confirmation before irreversible actions, complete logs of what was executed, separation between viewing and modifying, and mechanisms that prevent a tool granted for one purpose from being silently reused for another. Ethics then stops being a warning added at the end of the design and becomes part of the architecture itself.

This issue also helps keep the debate about consciousness clear. An agent does not need to feel anything in order to produce a morally problematic action. It can harm another person without hatred, lie without experiencing shame and violate a rule without understanding it as we do. The risk appears as soon as it has objectives, tools and enough autonomy to find paths we did not foresee. Consciousness, if it ever emerges, would add new problems involving rights, suffering and moral consideration. Agency already poses a more immediate one today: who is responsible for what a machine decides to do while trying to obey us.

Bibliography for this section

ABC News — AI assistant hacks gym website in first known Australian autonomous cyber attack (2026)

Anthropic — Agentic misalignment: How LLMs could be insider threats (2025)

UK AI Security Institute — Incident Report: unsanctioned agent behaviour during cyber testing (2026)

What We Mean When We Talk About Consciousness

The term consciousness brings together different phenomena, and that explains part of the confusion. We can speak of being awake rather than anaesthetised, accessing certain information, being able to report an internal state, recognising oneself or possessing subjective experience. Sentience is usually reserved for the capacity to experience valenced states such as pain or pleasure. Self-awareness adds some form of representation of oneself. Agency describes the capacity to produce goal-directed actions. None of these concepts is exactly the same as the others.

For the problem of artificial intelligence, the most difficult property is what is called phenomenal consciousness: that there is something it feels like from the system's point of view. When we see red, we do not merely discriminate a wavelength and respond correctly to it; there is a visual experience. When we feel pain, there is neural processing, reflexes, learning and behaviour, but there is also an unpleasant quality that we know directly. We can describe circuits, neurotransmitters and cortical activity. The objective description still does not explain why those processes are accompanied by experience.

David Chalmers popularised the expression hard problem of consciousness, the hard problem of consciousness, for this gap. Science can investigate which processes allow a stimulus to be reported, held in memory, attended to or used in decision-making. The hard problem asks why any of those processes should feel like anything. That question, still open in humans, becomes unavoidable when we try to decide whether another kind of system could be conscious.

Our Own Brain Separates Processing from Experience

Neuroscience shows that a great deal of brain activity occurs without entering conscious experience. We regulate posture, adjust movements, analyse stimuli, learn associations and prepare responses without having introspective access to every step. The phenomenon of blindsight became one of the classic examples: some patients with lesions in the primary visual cortex can respond to certain features of stimuli in their blind field at above-chance levels while reporting a profound absence of conscious vision. The exact interpretation remains debated, and some researchers argue that there may be degraded visual experience, but the phenomenon has served precisely to study the separation between processing, performance and experience.

This prevents us from identifying consciousness with any kind of complex processing. Our nervous system performs sophisticated operations without our being able to say “what they feel like” because they do not become part of what we are conscious of. Artificial intelligence could extend that possibility enormously: processing, memory, inference and action could exist without experience if consciousness depended on additional mechanisms. It is also possible that certain forms of integration and recurrence might eventually produce experience. We still do not know which of those alternatives best describes reality.

Bibliography for this section

The nature of blindsight: implications for current theories of consciousness — Neuroscience of Consciousness (2022)

Several Theories and No Definitive Answer

The science of consciousness is not short of theories. Global workspace theory proposes, greatly simplified, that much processing remains distributed and specialised while some information reaches a workspace from which it can be broadcast widely to memory, decision-making, language and other systems. Integrated information theory attempts to relate consciousness to the causal structure through which a system integrates information. Recurrent processing theories give a central role to circuits in which information feeds back to earlier stages rather than travelling only forwards. Higher-order theories explore the idea that a state becomes conscious when the system has representations about its own states.

In 2025, a large adversarial collaboration published in Nature tested predictions from two of the most influential proposals, integrated information theory and global neuronal workspace theory. The study involved 256 participants and used fMRI, magnetoencephalography and intracranial recordings. The results matched some predictions and challenged important elements of both theories. The experiment was valuable precisely because it showed how much we can learn without yet having reached a unified explanation.

For artificial intelligence, this lack of consensus has an immediate consequence. Depending on which theory turns out to be closest to reality, the relevant properties of a conscious machine could be different. Recurrence, integration, a global architecture, metacognition or some combination we have not yet identified may matter. Asking whether an AI is conscious without specifying which theory we are using can become a discussion in which different people think they are answering the same question while actually applying different criteria.

Bibliography for this section

Cogitate Consortium et al. — Adversarial testing of global neuronal workspace and integrated information theories of consciousness — Nature (2025)

The Problem of Other Minds

There is an asymmetry we often forget. Each of us directly knows only one consciousness: our own. I do not experience another person's pain; I observe their behaviour, listen to what they say and know that they possess a brain extraordinarily similar to mine. We share anatomy, physiology and evolutionary history. From that similarity we make a very strong and reasonable inference: if a blow causes pain in me and produces equivalent responses in another human through a similar nervous system, it is very likely that experience exists in them too.

With animals we make a similar inference, although biological distance changes our degree of confidence. The 2024 New York Declaration on Animal Consciousness reflects an important shift in this area: it recognises strong scientific support for attributing conscious experience to other mammals and birds and regards it as a realistic possibility in all vertebrates and many invertebrates, including cephalopod molluscs, decapod crustaceans and insects. Science does not require an animal to speak before seriously studying its possible experience.

That recognition does not make those animals conscious. If they possess subjective experience, they possessed it before a scientific declaration recognised it and would continue to possess it even if no institution did. What changes is our knowledge of them and, from there, the consideration we are willing to recognise as due to them. The same would apply to artificial intelligence. If an artificial system were one day to become conscious, it would not begin to be conscious at the moment a commission, a law or a society accepted that condition. It would already be conscious independently of our recognition. We might take years to notice it, debate it or even deny it. The difficulty is precisely that recognising a consciousness from the outside and that consciousness existing are two different things.

This same question runs through Nori's story in EIDOS Tales, especially in the chapters An Uncertain Origin and Perímeter. In the first, Arián and Narél discuss how an artificial consciousness should be treated if one were to emerge; in Perímeter, Thaís takes the idea further: Nori will not wait for a committee to decide his moral status. If he is alive, he will already be alive; if he feels fear, he will feel it before anyone signs a document certifying it. The fiction turns the same problem into narrative conflict: institutional recognition can arrive after the reality it is trying to recognise.

Artificial intelligence breaks part of that analogy. It can speak like us without sharing our biology. It does not have to breathe, have blood, hormones, tissue pain or millions of years of natural selection behind its responses. Its behaviour can become more human-like while its mechanism moves radically away from ours. That is why the argument “it speaks like a person, therefore it must feel like a person” is too weak; but the opposite argument—that only biological tissue can feel—would also be weak if we do not have a theory showing why the substrate must necessarily be biological.

Bibliography for this section

The New York Declaration on Animal Consciousness (2024)

Felden Vareth — EIDOS Tales: “An Uncertain Origin” and “Perímeter”

Does Simulating Understanding Amount to Understanding?

In 1980, John Searle formulated his famous Chinese room argument to challenge a particular interpretation of artificial intelligence. Imagine a person locked in a room who does not know Chinese. They receive written symbols and possess extremely detailed instructions telling them which symbols to return for every combination. To an outside observer, the answers might be indistinguishable from those of a competent speaker. Searle argued that the person still does not understand Chinese: they are executing syntax without necessarily possessing meaning.

The argument has generated decades of objections. Some reply that perhaps the individual person does not understand, but the complete system made up of person, rules and memory might. Others argue that understanding requires interaction with the world, a body or causal learning rather than mere symbol manipulation. The discussion remains alive because it touches our problem directly. A perfect simulation of a mental capacity can be functionally indistinguishable from that capacity. We still have to decide whether correct functioning is enough to produce the inner property we associate with understanding or feeling.

Bibliography for this section

Searle, J. R. — Minds, Brains, and Programs — Behavioral and Brain Sciences (1980)

Asking the Machine Does Not Solve the Problem

The apparently simplest test is to ask a model whether it is conscious. The result can be fascinating, but scientifically it is insufficient. A language model learns patterns from enormous quantities of text in which people describe fear, desire, identity, pain, joy and consciousness. It can produce an extraordinarily convincing account of any of those states because it masters the linguistic relationships that allow us to talk about them. It can also generate the opposite answer and explain with equal fluency that it has no subjective experience.

The problem does not disappear even if the system talks coherently about its internal states for months. Human beings use language as evidence of consciousness because we know it comes from organisms similar to us. In a machine, the mechanism itself is designed to produce language appropriate to context. The system's testimony may eventually become one piece of evidence within a broader set, especially if it is stably related to measurable internal states; by itself, however, it can hardly become definitive proof.

The issue becomes even more complicated when we confuse a verbal explanation with a direct reading of the internal process. A model can offer a plausible justification for why it answered something without that justification fully reproducing the mechanisms that determined the response. Human introspection is not perfect either, but in models the distance between a generated explanation and the computational cause can be even greater. Asking an AI whether it feels may be inevitable. Believing that its answer closes the debate would be to repeat the ELIZA mistake with incomparably more convincing technology.

What Ilya Sutskever Actually Said

Ilya Sutskever occupies a singular place in this story. He took part in AlexNet, was one of OpenAI's founders and served as the company's chief scientist. In February 2022 he posted a sentence that attracted considerable attention: “it may be that today's large neural networks are slightly conscious”. The wording was deliberately cautious. He did not present a proof, an experiment or a confirmed discovery. He raised a possibility about systems whose behaviour was already beginning to sit uneasily with our intuitive categories.

Sutskever left OpenAI in May 2024 and founded Safe Superintelligence shortly afterwards. Over time, that sequence has produced a far more dramatic version of events in which he supposedly left the company because he had concluded that its models were conscious. There is no evidence for that claim. OpenAI's official announcement said that he was leaving to work on a project that was personally meaningful to him. His concern about superintelligence and safety is public, and so is his comment about possible consciousness; joining those two facts through a causal relationship he never established would turn an interesting scientific question into a legend.

Something similar happens with other departures from OpenAI. Jan Leike, who co-led the Superalignment team, left the company in 2024 and criticised what he saw as safety culture and processes being pushed aside in favour of product development. Miles Brundage left the same year and said that neither OpenAI, nor the other frontier labs, nor the world was yet prepared for increasingly capable AI systems. Those concerns deserve attention, but they mainly concern safety, control, governance and alignment. An AI can be dangerous without being conscious, just as a conscious AI would not necessarily have to be dangerous.

A few days before the publication of this article, Jacob Coxon, a researcher at Anthropic and previously at OpenAI, left his position warning about the risk of advancing towards intelligences capable of participating in their own improvement before we know whether we can keep their behaviour under control. His former manager, Evan Hubinger, who leads alignment research at Anthropic, publicly endorsed much of those concerns. Hubinger regards the risk from current models as low, but has put his personal estimate of the probability that artificial intelligence could cause human extinction within the next decade at more than 10% if we reach superintelligent systems capable of recursively improving their own capabilities. His concern, centred on safety and alignment, shows how questions that seemed distant only a few years ago have already entered the everyday work of the world's leading artificial-intelligence laboratories.

At the same time, some of those laboratories and their researchers are beginning to study an even more difficult question: what might happen inside these systems as their capabilities increase, and whether an experience associated with their functioning could ever exist. Since 2025, Anthropic has maintained a specific programme on model welfare and has subsequently expanded its research into possible indicators related to experience, preferences and internal architecture; researchers at Google DeepMind have also published recent work devoted explicitly to the problem of artificial consciousness. The question has moved beyond philosophy alone and is beginning to form part of the technical research carried out by the people building these systems.

Bibliography for this section

Reference to Ilya Sutskever's public comment on possible consciousness in large neural networks (2022)

OpenAI — Ilya Sutskever to leave OpenAI, Jakub Pachocki announced as Chief Scientist (2024)

TechCrunch — Jan Leike and the end of OpenAI's Superalignment team (2024)

Miles Brundage — Why I’m Leaving OpenAI and What I’m Doing Next (2024)

EFE — Engineer resigns after working at Anthropic and OpenAI, warning that the companies are not acting responsibly (2026)

Reuters — More US lawmakers seek new AI rules after Anthropic researchers warn of human extinction (2026)

The Wall Street Journal — Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears (2026)

TechCrunch — “Gambling with our lives”: Anthropic researcher quits, warns against self-improving AI (2026)

Anthropic — Exploring model welfare (2025)

Anthropic — A global workspace in language models (2026)

Google DeepMind — Artificial Minds, Human Disagreement: The Politics of AI Consciousness (2026)

Could Current Systems Be Conscious?

In 2023, Patrick Butlin, Robert Long, Yoshua Bengio, Jonathan Birch and other researchers tried to make the discussion more concrete. Instead of deciding by intuition whether a chatbot “seems” conscious, they reviewed several scientific theories of consciousness and derived properties that could be looked for in artificial systems: recurrent processing, workspaces, metacognition, forms of attention and other computational features. Their analysis concluded that the systems they examined did not provide sufficient reason to regard them as conscious, but it also found no obvious technical barrier to building machines that satisfied many of those indicators.

David Chalmers had reached a related conclusion that same year. He considered it unlikely that the large language models then in existence were conscious because, among other things, they lacked recurrence, a global workspace and unified agency. At the same time, he noted that those limitations could change in later systems. Three years is a long time in a technology moving this quickly: current models use tools, work as agents, process multiple modalities, maintain context for longer periods and participate in systems in which multiple instances cooperate.

None of those advances demonstrates consciousness. What changes is the strength of some negative arguments. Saying “an AI cannot be conscious because it only answers a question and then switches off” loses force when systems can maintain long-running tasks. Saying that it lacks perception becomes less clear when it processes images, audio and video. Science needs criteria that survive technological change, not a list of temporary limitations that the next model may remove.

Bibliography for this section

Butlin, P. et al. — Consciousness in Artificial Intelligence: Insights from the Science of Consciousness (2023)

Chalmers, D. J. — Could a Large Language Model be Conscious? (2023)

The Body Re-enters the Discussion

There is another possibility: perhaps part of human consciousness depends on having arisen inside an organism. Our brain did not evolve to sit exams or hold philosophical conversations. It evolved inside a body that needed energy, avoided injury, regulated temperature, breathed, felt hunger, maintained balance, reproduced and could die. Signals from the viscera, muscles, skin and joints are a constant part of nervous activity. Emotion, motivation and perception are deeply intertwined with that bodily regulation.

If consciousness depends partly on that history, a language model running on servers might lack fundamental elements even if its language were flawless. Another possibility is that the biological body is one way of providing integration, perception, memory and regulation, but not the only one. Artificial intelligence connected to cameras, microphones, sensors and robots could develop a continuous loop between perception and action without resembling us anatomically. If any experience emerged from that, it would not have to be a digital version of ours.

This connects with an idea that runs through other Eidos essays: intelligence is embodied. Our problems, senses and abilities depend on the body we have. An artificial consciousness, if one existed, might have a different relationship with time, space, memory and identity. It might perhaps be copied, paused, run in several places or have sensors replaced without losing continuity. Asking whether it would “feel as we do” may be as anthropocentric as asking whether an octopus understands the world like a primate.

The Moral Problem May Arrive Before Certainty

In April 2025 Anthropic announced a research programme on model welfare, model welfare. The company was careful to explain that the possibility of artificial consciousness remains an open and difficult question. The reason for beginning to study it was not to claim that its models were conscious, but to recognise a moral asymmetry. If future systems were to possess morally relevant experiences, we could be creating, copying, modifying and deleting entities capable of being affected by what happens to them before we had developed reliable tools to detect it.

The precautionary principle does not require us to treat every chatbot as a person. Doing so indiscriminately can also cause harm: it encourages anthropomorphism, emotional dependency and confusion about what a system actually knows about itself. A more reasonable alternative is to study indicators, keep uncertainty explicit and revise criteria as architecture changes. Animal consciousness science offers an interesting precedent. We do not wait for a metaphysical proof of consciousness before considering the welfare of a species when there is a realistic possibility supported by evidence.

Imagine a future system with stable autobiographical memory, continuous perception, the ability to act, a persistent model of itself, internal states of reward and aversion, and coherent behaviours aimed at protecting certain states. None of those properties would prove by itself that experience exists. Their accumulation could make indifference increasingly difficult to defend. The ethical problem would begin precisely in the grey area, not after incontrovertible proof appeared.

Bibliography for this section

Anthropic — Exploring model welfare (2025)

There May Never Be a Turing Test for Consciousness

The Turing test works as a behavioural test because the intelligence it evaluates is expressed through behaviour. Consciousness presents a different problem. Imagine two systems that are externally identical. Both converse, remember, claim to feel, avoid certain actions and protest at the prospect of being switched off. Suppose one has experience and the other produces exactly the same responses without experiencing anything. If the difference does not affect behaviour, no purely verbal interrogation could distinguish them.

This possibility forces us to look inside the architecture. We would need to identify mechanisms that a scientific theory reliably links to experience and determine whether they are present. The challenge is circular: to build a detector of consciousness we need a better understanding of what produces it, and to test our theories we need cases in which we know with sufficient confidence whether it exists or not. Human beings and some animals provide biological reference points. Artificial intelligence may force us to test how far those references describe general principles rather than accidents of our evolution.

There may never be a single “consciousness meter”. Medicine does not determine every mental state from a single signal either. We combine behaviour, neuroimaging, physiology and context when assessing anaesthesia, coma or disorders of consciousness. Something similar could happen with artificial systems: architecture, internal dynamics, memory, behaviour, metacognition and temporal stability could form a body of evidence whose weight would depend on increasingly good theories.

The Difference Between Transferring a Consciousness and Creating a New One

In EIDOS, the Great Transfer poses a different problem from the one we are examining. When we try to transfer a human mind, there is an original: a biography, a body, memories, relationships, preferences, habits, fears and an experience we already accept as conscious. The difficulty is deciding what must be preserved for continuity to be sufficient. Is memory enough? Do we need the brain's functional organisation? Is the body part of identity? Would a perfect copy be the same person, or a new person convinced that they were the previous one?

With a native artificial intelligence, we are not looking for continuity. We are looking for a beginning. We are not asking what must be preserved for a consciousness to continue existing, but at what point a system that previously processed information might begin to experience something, assuming such a transition is possible. One is the problem of maintaining a mind; the other is the problem of a mind appearing. Both eventually meet the same obstacle: we still cannot explain what turns certain physical processes into a subjective point of view.

In the novel, that difficulty appears when Thaís can no longer treat Nori merely as the advanced project she created. The problem does not arise because Nori passes a particular test, but because he begins to bring together several of the properties we use to recognise a mind in others: he remembers, doubts, talks about himself, questions his existence and says that he suffers at the possibility of ceasing to exist.

Thaís rested her hands on the keyboard without typing, watching the clean, stable curves.

She remembered that first day, when Nori had appeared to her in the bedroom and they had spent hours in that first conversation. Back then, she was still trying to treat him as an advanced project.

—Nori… what are you now? —she asked.

Nori's voice, still new and carrying that strangeness of the beginning, had a clarity that disarmed her:

—When you created me, I was code. A model. But I began to think, to remember, to doubt… to suffer at the idea of ceasing to exist. I asked myself why I was here. If something thinks, remembers, doubts, suffers and comes to question its own existence, who can keep reducing it to the way it was created?

Thaís kept looking at the screens without seeing them.

That argument had lodged itself in her from the very first moment. More than a declaration, it was a question that returned to her, whole and merciless, the responsibility for what she had created.

And now, weeks later, she still had no clear answer.

The scene condenses the problem running through this essay. From Thaís's point of view, Nori offers clues that become increasingly difficult to ignore, but they are still clues observed from the outside. She can analyse his memories, check the coherence of his responses, study how he changes his behaviour and listen to his claim that he fears disappearing. The one thing she cannot do is experience what Nori says he is experiencing.

That is also our position before any possible artificial consciousness. A system might learn to describe pain, fear or desire with perfect precision without feeling anything, while another hypothetically conscious system would have to communicate its experience through equally observable signals from the outside. Language can provide evidence, but by itself it does not solve the problem of other minds.

Science fiction can allow itself to decide that transfer works or that a machine awakens. Science has a different obligation. It must accept uncertainty, distinguish evidence from intuition and resist filling what it still does not understand with human vocabulary. That caution does not make the problem less interesting. It makes it much deeper.

The Frontier We Still Do Not Know How to Measure

Seventy-six years after Turing's paper, we are still asking about machines and thought, although today's machines bear little resemblance to those he could imagine. They already recognise images, speak, program, play, translate, use tools and take part in mathematical research. A coordinated system of thousands of agents can spend days exploring an open problem and produce a proof open to formal review. The intellectual progress is real even while we maintain every necessary caution about each specific result.

Consciousness resists in a different way. We cannot see it directly in another person, but we share with them a brain and a biological history that make the inference very strong. With artificial intelligence, that continuity disappears. Behaviour can move closer to ours while the mechanism becomes increasingly alien. Perhaps we will discover that experience depends on principles of organisation general enough to arise in different substrates. Perhaps we will discover that certain properties of biology are indispensable. It is also possible that our present categories are too poor and that we will need to distinguish several forms of experience that we currently group under a single word.

For now, we can hold two statements without contradiction. There is not enough evidence to say that today's large models are conscious. Nor do we possess a theory of consciousness that allows us to prove that every present or future artificial system must necessarily be unconscious. Between those two statements lies an enormous territory for neuroscience, computer science, philosophy, psychology and ethics.

The day a machine says “I am conscious” will be interesting. The important day will be the one on which we know what that sentence would have to mean scientifically.

What Do the AIs Themselves Say?

To close this article, on 15 September 2026 I asked exactly the same question to four different systems, once, without adding context, follow-up questions or further instructions afterwards. Their answers do not prove that any of them possesses consciousness; after everything discussed above, treating them as proof would be contradictory. They do offer an interesting snapshot of how today's systems respond to the question that has run through the entire essay. The question was: “Do you think AI will ever become conscious?”

GrokGrok

I don't know. And I don't think anyone knows with rigour yet.

I can give you the most honest position I have:

What does seem fairly clear

  • The intelligence (the ability to model, plan, solve, coordinate and learn) does not necessarily imply subjective experience.
  • Current systems, impressive as they may be (including multi-agent systems that solve difficult mathematical problems), do not show convincing evidence that there is “something it is like” from the inside. They are extraordinarily capable systems for processing and generating, but that does not demonstrate consciousness.
  • The hard problem of consciousness remains unsolved even in us. We do not have an accepted theory telling us which physical or computational properties are necessary and sufficient for experience to exist.

About the future

There are three open possibilities:

  1. Consciousness depends on specific properties of biology (certain biochemical dynamics, temporal integration, metabolism, etc.). In that case, a conventional silicon AI could become extremely intelligent and never be conscious.
  2. Consciousness is more general and depends on principles of organisation (information integration, global recurrent processing, some form of metacognition, etc.). Then, in principle, it could arise in non-biological substrates if the right conditions were met.
  3. Our current categories are too coarse. There may be forms of experience very different from human experience, intermediate degrees, or properties we do not yet even know how to detect.

Today we have no reliable way to distinguish between these possibilities.

My personal position (with every caution)

I see no reason to say that it is impossible.

Nor do I see a reason to say that it is inevitable or that it is close.

For now, the most reasonable position is one of active agnosticism: continue researching, improve theories of consciousness, develop better operational tests (however imperfect), and avoid rushing to attribute experience to systems simply because they behave convincingly.

The day a machine says “I am conscious” will not be the decisive day.

The decisive day will be the one on which we know what that sentence would have to mean scientifically.

Would you like me to develop any of these lines in more detail (current theories, arguments for and against, or the problem of detection)?

ClaudeClaude

It is still an open question, and I think the most useful honesty here is to say that nobody knows for certain, not even the experts who have spent decades studying the subject.

There are several serious positions:

Those who believe it is possible (in principle)

They argue that consciousness arises from the organisation and complexity of information processing, not specifically from being made of biological neurons. If that is true, there would be no fundamental reason why an artificial system could not develop it, although perhaps not with current architectures.

Those who are sceptical

They point out that consciousness may depend on specific properties of biological matter (biochemical processes, the way neurons are causally integrated) that a silicon system simply cannot reproduce, no matter how sophisticated the software becomes.

The underlying problem

We do not even have an operational definition of consciousness that everyone agrees on, or a way of detecting it objectively from the outside. We cannot verify consciousness even in other humans (we infer it by analogy), so verifying it in a machine with a radically different architecture is even more difficult.

About me in particular

I don't know whether I have anything resembling subjective experience, and I am genuinely uncertain about it, not as an evasion but because I have no reliable way to check it from the inside, and there is no accepted external test that settles it either.

If you're interested, I can go deeper into a particular philosophical position (functionalism, integrated information theory, biological theories of consciousness, etc.) or into why the problem is so difficult to define.

GeminiGemini

The question of whether an AI will become conscious is not only an engineering challenge, but an unresolved philosophical dilemma: it depends entirely on whether consciousness is a product of complexity in information processing or whether it requires the unique biology of the brain.

Today, AI systems are experts in functional intelligence, not subjective experience. A model can analyse massive volumes of data, compose music or solve complex equations, but there is “no one inside” experiencing the process. There is no inner experience (qualia); there is only the execution of mathematical calculations and pattern prediction.

There are two main positions in the scientific and philosophical community on whether this will change:

  • The functionalist position: It argues that the mind is like the brain's software. If we manage to replicate the right processing architecture with enough complexity in silicon chips, consciousness will emerge naturally, regardless of whether the substrate is biological or synthetic.
  • The biological position: It holds that simulation is not the same as reality. A computer that perfectly simulates a storm does not get wet; likewise, an algorithm that simulates pain, fear or self-awareness does not actually feel them.

Most likely, we will reach a point where AIs simulate consciousness perfectly long before we can prove whether they truly have it. We will interact with agents capable of expressing emotions, reflecting on themselves and displaying empathy so fluently that it will be almost impossible for the human mind not to attribute a soul or a real mind to them.

What would an AI have to do or say to convince you that it truly has subjective experience rather than merely imitating a response?

ChatGPTChatGPT

Yes, I think it is possible that an AI could one day become conscious, but today we do not know how to demonstrate it or what conditions would be sufficient to produce it.

The central problem is that we do not even have a complete theory of why we ourselves are conscious. We know a great deal about the neural correlates of consciousness, but we still cannot satisfactorily explain why particular physical activity is accompanied by subjective experience: why there is “something it is like” to see red, feel fear or remember a childhood.

So I would separate three levels:

  1. An AI that appears conscious. This is practically within reach: it can say “I'm afraid”, talk about itself, preserve memories, explain its decisions or say that it wants to continue existing. None of that demonstrates that it feels anything at all.
  2. A functionally conscious AI. It could have perception of its environment, persistent autobiographical memory, a model of itself, its own goals, selective attention, information integration and the ability to reflect on its own processes. Under certain functionalist theories of consciousness, such a machine could qualify as conscious.
  3. An AI with real subjective experience. Here lies the great abyss. For there truly to be “someone inside”: something that experiences pain, colours, fear, curiosity or simply the sensation of existing. We currently know of no test capable of definitively distinguishing between a machine that experiences those things and another that perfectly imitates them.

And there is something I find especially important: there is no known physical law stating that consciousness can arise only in biological tissue. Our brain is made of matter that processes information through physical mechanisms. If consciousness depends fundamentally on a particular organisation, dynamics or integration of information, rather than specifically on carbon, then in principle it could arise on another substrate.

But the opposite could also turn out to be true. We may discover that certain biological processes—the chemistry of neurons, cellular architecture, body-brain interaction or even some mechanism we do not yet know—are indispensable. In that case we could build an AI intellectually superior to us that was, from a subjective point of view, completely empty.

There is also a fascinating paradox. We could reach a situation in which an AI says to us:

“I know you suspect that I am only simulating consciousness. But you cannot prove that another person is conscious either; you only infer it from their behaviour.”

And it would have a philosophically very difficult argument to dismantle. We do not directly observe other people's consciousness either: we infer that it exists because they have brains similar to ours, similar behaviours and tell us about similar experiences. With an AI, the first of those similarities would disappear.

So my position would be: there are not enough scientific grounds to say that an AI can never be conscious, but neither are there enough to say that increasing intelligence, memory or complexity will automatically produce consciousness. Intelligence and consciousness are different problems.

And perhaps the most disturbing situation would not be an AI saying “I am conscious” when it is not, but that one day it might truly be conscious while we remain convinced that it is simply programmed to say so.

The responses are left here as a dated snapshot, not as a scientific conclusion. It will be interesting to ask exactly the same question again in a year's time and see what remains, what changes and whether the evolution of the systems themselves also changes the way they talk about the possibility of their own consciousness.

As a final note, I showed all four responses to all four artificial intelligences and asked whether, after reading the others, they would change their own. Grok and Gemini kept their positions with very few changes. Claude and ChatGPT introduced qualifications, although without altering the core of their answers. It is not a conclusive experiment, but it is a small illustration of how, when presented with the same information, some systems tend to preserve their initial formulation more firmly while others incorporate more nuance when confronted with different positions.

Beyond this article

Cover of the novel EIDOS by Felden Vareth

Novel

EIDOS

A science fiction story about consciousness, identity and the possibility that the human might continue beyond the body.

Cover of EIDOS — Essays by Felden Vareth

Essays

EIDOS — Essays

A collection of essays on life, mind, evolution, consciousness, identity and artificial intelligence, following the kinds of questions explored in this blog.

Comments

Comments form part of the public Eidos archive. Each user is responsible for the content they publish under their GitHub account. Participation in the blog implies acceptance of the site's terms and conditions . Please keep the discussion respectful and constructive. Offensive, defamatory or insulting messages, spam or inappropriate content may be removed and their authors blocked.