Artificial-intelligence systems such as large language models (LLMs) can exhibit behaviour that seems strikingly intelligent. This has led many people, including some experts, to wonder whether these systems could have some form of consciousness — the capacity for subjective experience. However, we think that existing AI systems are not conscious, and that, along current trajectories of development, they are not likely to become conscious. Greater clarity about whether AI could become consciousness will be crucial as society grapples with the challenges posed by this increasingly powerful technology.


I think we’re conflating two different things here: establishing what phenomenon we’re referring to, and establishing the properties that reliably identify or explain that phenomenon.
I’m referring specifically to subjective experience: what it’s like to feel pain, perceive color, experience emotions, dream, etc. We have a meaningful starting point for investigation, even if we disagree about the underlying nature of those experiences.
I agree with your banana example, at least to a point. We can identify bananas without understanding their molecular composition. But notice what you’ve actually demonstrated: we can recognize instances of a phenomenon without fully understanding what constitutes it…
You can describe the observable characteristics of a banana and use them to identify other bananas. But what happens when you encounter something that shares those characteristics without being a banana, or a banana that doesn’t exhibit the characteristics you’ve chosen? You need to investigate further, potentially refining your classification criteria.
Now consider consciousness. Unlike a banana, subjective experience isn’t directly observable by third parties. I can recognize my own experiences, but determining whether another entity has comparable experiences is a different problem entirely. That’s why establishing reliable criteria matters.
Your Florgleblorp analogy would apply if I were referring to something for which we had no identifiable phenomenon whatsoever. But that’s not what I’m doing. I’m referring to subjective experience, and asking what constitutes it and how we can determine whether it exists in other systems.
I absolutely agree that we need provisional definitions and operational criteria to conduct research. What I reject is treating those criteria as though their validity were established simply because we’ve agreed to define consciousness in those terms.
Here’s my own example for you: we could define consciousness as the ability to verbally report internal states. That’s a perfectly testable definition. But whether it actually identifies subjective experience, rather than merely the ability to produce convincing reports, remains a question. There’s nothing there to differentiate between correlation and causation.
So I’m not arguing that we should investigate something without knowing what we’re talking about. I’m arguing that our initial characterization of a phenomenon shouldn’t be confused with a validated scientific account of what constitutes it or how to detect it.
If you want to prove consciousness, I stand my ground on the argument that you first need to qualify consciousness as a thing. What is it?
You may find that consciousness is actually something which was deeply misunderstood and unexpected. Such a discovery would allow you to draw genuinely meaningful conclusions about whether or not LLMs can ever truly be conscious. That’s how you form a theory.
You suggested, earlier, that someone may be able to detect consciousness by measuring whether the agent’s observation can collapse a waveform. But quantum mechanics does not establish that consciousness is required to collapse a wavefunction. In physics, an observer is not necessarily a conscious entity. A measurement apparatus can register a quantum event without any human being present. The measurement can occur whether someone watches the results live, reviews them tomorrow, or never examines them. By the same logic, an LLM could pass your hypothesized test.
Simply deciding that LLMs shouldn’t count as conscious, without even truly understanding what qualifies consciousness, is just moving the goalpost. Or bikeshedding, perhaps.
Experience is not subjective. It is contextual, but contextuality is a property of physical reality, not of subjectivity.
Yes, Chalmerites like to prepend “what it is like” to things, as if it somehow proves anything. You can tell when a phrase means literally nothing when everyone just repeats it verbatim without ever rewording it in different ways, because the phrase carries no meaning, and only has the appearance of meaning by being repeated a lot.
Pain, color, emotions, and dreams are all real occurrences in the real world that really happen and all have real physical explanations for them. Prepending “what it is like” to them doesn’t mean anything. “Like” means “akin to” or “similar to,” and “what it is” regards being. And so the phrase, if taken literally, is about “similar being,” which I don’t see how that even makes linguistic sense in this context.
Of course, as Chalermites always do, you will probably get frustrated with me for taking the words literally, because I’m supposed to interpret them abstractly or something, but then there is zero clarity on what this abstract interpretation is supposed to be.
It’s literally physically impossible for something to share all observable characteristics of a banana without being a banana.
Of course, if we have a very loose definition of a “banana” that is so broad it would even include an apple, then either the word itself “banana” really does just refer to banana+apple, or we would add on to the definition to separate the two. Words are ultimately a social construct, what is real is what they refer to when used in practice, and not the specific definition in the dictionary. I’m sure you know what a “dog” is, but few people have ever intentionally looked up the word “dog” in a dictionary. You can just know what it means by its social use.
You are being so absurdly vague that I have no choice but to just try and guess at what you even mean by this, so I may get my guessing wrong, but that’s on you for being so extremely abstract. It seems that there are two halves to what you are trying to say, so I am going to attempt to parse them.
First, again, there is nothing “subjective” about experience. Experience is just when X happens to Y, i.e. a rock experiences erosion in the rain. It seems to me that one half of how you are you are using “subjective” here to refer to “my own experiences,” i.e. “X happens to myself,” which is a strange way to use the word “subjective,” because it would be like saying, if I eat a sandwich, I’m engaging in “subjective eating,” or that my house is a “subjective house” because it’s owned by a subject. But, whatever, words are words, but I would refer to it more meaningfully as “experience of a subject.” Second, you say that it “isn’t directly observable by third parties.” Of course, the experiences of a subject are obviously observable by third-parties, so you must be introducing some second criterion here, likely you are trying to say, it is the experience of a subject, as it is realized from the subject’s own vantage point. That is what I presume you mean by “isn’t directly observable by third parties.”
Actually, the first half seems irrelevant, because any experience, whether it happens to a subject or not, as it is realized from a subject’s vantage point specifically, is not realized by the vantage points of other subjects. If I see a rock experience erosion in the rain, no one else on earth can perceive it exactly the way I did. (The term “accessible” does not really make much sense, because what is realized is not something to be “accessed.”)
But, there is no mystery about this at all. Obviously, if they were to perceive what I perceived exactly as how I perceived it, they would have to be standing in the same place as I am, they would have to be of the same height as me, they would have to have the same nose in front of their face as I do, they would have to have the same memories such that they contemplate upon it the same way, so on and so forth.
That is to say, they would have to be me.
Indeed, talking about accessing another person’s vantage point does not even make linguistic sense. You could imagine getting closer and closer to my vantage point, by gradually taking atoms from my body and replacing them with your own, until you become identical to I, but the moment that identity occurs, then you would just be me, and you would cease to exist, so you would have failed to achieve my vantage point, you would have just stopped existing. Indeed, this is because how we define objects in the first place inherently entails that object’s vantage point, and so they are basically two ways of describing the same thing.
If you think the “mystery” is that we cannot access each other vantage points… that is not mysterious at all. So what is the mystery? Even after this whole aside, I am not sure what you think is so mysterious.
You have not identified any phenomenon at all. You are acting as if X experiencing Y suddenly evolves into some new phenomenon if X is a subject, but there is no explanation as to why this is the case. It would be like saying that houses are banal, but houses owned by people, those are “subjective houses,” and require a special explanation… why? You kind of skipped that step.
Notice how your definition, then, suddenly entails nothing about so-called “subjective experience.”
The moment you start to talk about anything real, you suddenly have to move away from the meaningless, Chalermite language.
Yeah, you can see the essence of Chalmers in here as well. On one hand, you want to make it look like you are being reasonable by giving a definition that has some concrete meaning, but then you immediately backpedal to Chalmerland and start talking about how we could never actually connect that concrete definition to the abstract thing you were supposed to be defining in the first place… and so you would’ve just wasted our time.
Why would I want to prove something that isn’t even defined in the first place?
Well, I can’t misunderstand anything because I have no understanding of what this “consciousness” supposedly is to begin with, so if someone did discover something unexpected and labeled it as “consciousness,” it would just be a new understanding.
That’s more akin to taxonomy than theory building, just looking for ways to categorize things into this or that group.
the fuck
I literally said the opposite
I wrote a giant rant about how that is insanely batshit idiotic and that quantum mechanics has nothing to do with consciousness at all.
No, there is no “hypothesized test,” what on earth are you talking about
I don’t decide that LLMs shouldn’t count as “conscious,” because I don’t have a reason to even take the concept of “consciousness” seriously to begin with!
You insist that words derive their meaning from social usage, then declare “what it’s like” meaningless by dissecting its individual words into “similar being.” That’s not a serious linguistic argument. You’re contradicting your own account of how language works.
Likewise, a rock “experiencing erosion” and a person experiencing pain involve two different meanings of “experience.” One means undergoing a physical change; the other involves feeling something. You’re equivocating between them. And a camera has a vantage point, too. That doesn’t mean it has subjective experience.
Nothing I’ve argued requires consciousness to be nonphysical or inexplicable. A physical explanation of pain doesn’t negate the existence of felt pain. The question is what produces that experience and how we can reliably identify it in other systems.
I incorrectly attributed someone else’s quantum-collapse proposal to you, I retract that. I see that was from the original commenter, not you.
But here’s the fundamental issue: Do you actually deny there’s a distinction between a nervous system detecting harmful stimuli and someone consciously feeling pain?
Because if you recognize that distinction, you already understand what phenomenon we’re investigating. Disputing the terminology doesn’t make it disappear.