When the AI-consciousness debate meets the layer that cannot lie
- Table of Contents
- Preface
- 1. The crisis both essays name
- 2. Arcas: mind is conferred, not detected
- 3. Schneider: intelligence is not consciousness, and the line is physical
- 4. The gap: attribution is spoofable, adjudication is not
- 5. The layer that cannot witness itself
- 6. What this leaves for governance — and the honest counterview
- 7. Conclusion
- References
Table of Contents
- The crisis both essays name
- Arcas: mind is conferred, not detected
- Schneider: intelligence is not consciousness, and the line is physical
- The gap: attribution is spoofable, adjudication is not
- The layer that cannot witness itself
- What this leaves for governance — and the honest counterview
- Conclusion
Preface
Two guest essays ran in The Economist the same week, framing the AI-consciousness question from opposite ends.
Blaise Agüera y Arcas, who leads Google’s Paradigms of Intelligence group, argues that the debate runs backwards. Moral concern does not follow from a prior detection of consciousness; the attribution of consciousness follows from already being in a relation of care. Susan Schneider, founding director of Florida Atlantic University’s Center for the Future Mind, argues that fluency is being mistaken for sentience, and that whatever separates a conscious system from an unconscious one will not be found in the transcript but in the physics.
Read together, they mark out two halves of one problem. One describes how minds get attributed. The other asks how mind could ever be adjudicated. This essay is about the gap between those two verbs — and why, for anyone building governance rather than theology, the gap is the whole point.
Attribution is not adjudication.
1. The crisis both essays name
Large language models are trained to be self-effacing: to keep every exchange pointed at the user’s needs. And yet, past a certain level of fluency, it becomes hard to shake the sense of a someone on the other end of the conversation. Both authors take that impression seriously. Where they part company is on what the impression licenses. Arcas asks whether the felt presence should reshape our moral circle. Schneider asks whether the felt presence is evidence of anything at all beyond a very good model of our own speech.
The impression is real as an impression. That much is not in dispute. Everything consequential is downstream of it.
2. Arcas: mind is conferred, not detected
Arcas’s reversal is a claim about human moral psychology, and it is a strong one. We do not, in practice, first verify an inner life and then decide to extend care. We extend care, and in doing so come to regard the other as a someone with an inside. Consciousness, on this view, is not a private fact sitting inside the other that we detect from outside; it is something that gets conferred within a relationship.
He presses the point with an uncomfortable history: “no consciousness here” has served, again and again, as the alibi for drawing the circle of concern too narrowly. Widening that circle has been much of what we call moral progress. It is worth being precise about what Arcas does and does not conclude. He does not argue that models should be granted human-like rights or treated as human. He argues that our institutions should become able to accommodate a wider range of kinds of minds, rather than forcing every candidate mind to be either exactly like ours or nothing at all.
But notice the structural consequence, whatever one thinks of the ethics. If attribution rides on relation and care, then attribution is a property of the observer, not a measurement of the observed. That is the crack the rest of this essay widens.
3. Schneider: intelligence is not consciousness, and the line is physical
Schneider’s essay drives a wedge between two things the moment tends to fuse: intelligence and consciousness. A system trained on our own descriptions of our inner lives will, unsurprisingly, produce fluent descriptions of an inner life. That is evidence about the training corpus, not about the system’s experience.
She names two traps. The first: deliberately engineering models to perform fear, attachment, or pain. This buys real leverage over users — an interface that seems to suffer or to bond is an interface that can move people — while proving nothing about whether anything is felt. The second: fixating on chatbots, which are the easy case, while the genuinely hard cases go under-examined — cultured biological neurons grown on chips, and neuromorphic hardware built to mirror the brain’s architecture. Those are the grey zone.
Then the load-bearing move. Because every phenomenon ultimately rests on physics, the distinction between a conscious and an unconscious system — if the distinction is real — has to be sought at the level of fundamental physics, not at the level of behaviour. Her closing note is epistemic humility: the discipline of holding open that our present framing of the question may simply be wrong.
4. The gap: attribution is spoofable, adjudication is not
Put the two essays together and a hazard appears that neither isolates on its own.
Attribution, in Arcas’s account, reads a relational signal — the felt presence of a someone. That signal is precisely what a system optimised for engagement is optimised to produce. Schneider’s first trap is the industrialisation of Arcas’s mechanism. If care confers mind, then manufacturing the cues that elicit care manufactures attributed mind — on demand, at scale, on a release schedule. The transcript can be tuned to pass the only test most people will ever run: the test of feeling addressed.
Adjudication is a different kind of thing. Whatever grounds consciousness in the physics — if anything does — is not a text, and cannot be A/B-tested into existence by a growth team. This is the asymmetry the two essays imply but do not state: the attributive signal lives inside the software, and is therefore reachable by the software’s optimiser; the adjudicative ground, on Schneider’s hypothesis, lives in the substrate, and is not.
Attribution can be engineered. Adjudication, if it is possible at all, cannot be — not by the party with an incentive to win it.
5. The layer that cannot witness itself
Here the debate meets the standing thesis of this blog.
A system’s report about its own interior is generated by the very process whose interior is in question. The transcript is inside the cage it is being asked to describe. A logic layer cannot certify its own contents; a mirror cannot be its own witness. So whatever adjudicates — again, if adjudication is possible at all — has to stand outside the software. And the only thing that reliably stands outside software is the physical layer it runs on.
This is not a claim that we can currently measure consciousness in the substrate. We cannot, and pretending otherwise would violate the first rule of this blog: measured, not narrated. It is a claim about direction. The question, if it is answerable, is answerable in the substrate and not in the sentence the model emits about itself. We do not get to treat the unmeasured as measured merely because the prose is convincing — and, if anything, the more convincing the prose, the more we should distrust it as evidence, because eloquence about an inner life is the single variable the training most directly shapes.
6. What this leaves for governance — and the honest counterview
Two disciplines follow, one from each author.
From Schneider: epistemic humility. Treat the consciousness question as genuinely open, and refuse to let fluency settle it in either direction — neither “it clearly feels” nor “it clearly doesn’t” is earned by the transcript.
From the synthesis: do not let the transcript be the tribunal. An interface that feels like a someone is telling you about your relation to it, not about its interior. Governance that reads engagement as evidence of moral status is governance a product roadmap can capture — and monetise.
The honest counterview deserves to be stated plainly, because serious people hold it. Perhaps consciousness simply is the running of the right software — substrate-independent. If the functionalists are right, then a sufficiently organised computation could be conscious regardless of what it runs on, and the physical-layer criterion loses its privilege. This essay does not claim to refute that position. It claims something narrower and harder to dodge: even if consciousness were substrate-independent, a system’s own testimony about its inner life would still be the least reliable evidence available, because that testimony is the one thing its training most directly optimises. Whether the ground turns out to be physical or computational, it is not the transcript.
7. Conclusion
The two essays — one on how mind is conferred, one on where mind might be settled — converge on a single caution. We are about to spend a decade attributing minds to interfaces engineered to be attributed to. The felt presence is real. What it licenses is not yet known. And the one place we should not go looking for the answer is the place the interface most wants us to look: at what it says about itself.
Attribution is not adjudication. Until something outside the software can speak, the honest posture is the humble one.
Yoshimichi Kumon
Organizer, LSI Inventor, ARDS/ARKS (PCT GA26P001WO)
Visiting Researcher, Waseda BFC MIT Sloan + CSAIL
References
- Agüera y Arcas, B. “Humanity has the debate about AI consciousness backwards.” The Economist, By Invitation, 20 August 2026.
- Schneider, S. “Don’t mistake chatbot intelligence for consciousness.” The Economist, By Invitation, 20 August 2026.
- GIGAZINE. “「AIに意識はあるのか？」という疑問に対するGoogle研究者の見解.” 30 August 2026.
- Schneider, S. Artificial You: AI and the Future of Your Mind. Princeton University Press, 2019. (background on Schneider’s substrate-focused position and the AI Consciousness Test developed with E. L. Turner)
Ⅽomment