Artificial-intelligence systems such as large language models (LLMs) can exhibit behaviour that seems strikingly intelligent. This has led many people, including some experts, to wonder whether these systems could have some form of consciousness — the capacity for subjective experience. However, we think that existing AI systems are not conscious, and that, along current trajectories of development, they are not likely to become conscious. Greater clarity about whether AI could become consciousness will be crucial as society grapples with the challenges posed by this increasingly powerful technology.
Almost none of this is relevant to the discussion here – I’m not interested in philosophical discussions of consciousness when it comes to the question of whether an LLM is conscious. Though I’m already aware of most of what you said here, and the philosophy of consciousness is interesting in general to me!
Quick nitpick: I never said it’s impossible for a machine to be conscious, I said it’s impossible for an LLM to be conscious. We may well invent something new in the future that qualifies as both “machine” and “conscious,” but an LLM is not that thing.
When people ask “is an LLM conscious?” I assume they’re asking from a practical, scientific perspective – does the LLM experience the world in a similar way to the way I (and presumably, others) do? If that is the question, the various definitions I already posted are what we should be using to answer. Since an LLM is a deterministic algorithm whose only purpose is to generate a plausible response to input based on its training data, it cannot possibly think, or have feelings. It does not even exist continuously in time, and it is fixed and unchanging, unable to incorporate any new information at all. We don’t actually need to measure or test anything here; that an LLM lacks all of these things demonstrates that it’s not possible for it to be conscious.
(I apologize in advance for the mountain of text. My main point is about how I don’t think you can cleanly separate the philosophical and scientific approaches to consciousness, and how the line between humans and simpler processes is a lot blurrier than your list of differences makes it seem.)
Hmm. I see what you’re saying, although I will admit this threw me for quite the loop:
Because, IMO, consciousness is almost entirely within the realm of philosophy. I’m not sure how much you can do with a scientific approach to it; I cannot think of many hypotheses to test that could give useful (ie. unambiguous and certain) results about the nature of consciousness. But perhaps you’re right that that’s giving up too easily; we can (and often must) make assumptions and reason by analogy to figure out what the most likely state of reality is, even if technically we cannot be entirely sure.
To avoid the pitfalls of unknowability, you could assume that all humans are conscious (reasonable imo, obviously) and that having certain humanlike properties is a good indicator that something might be conscious (which properties these are is of course a subject of great debate). I do think it’s a mistake to say that lacking these properties means it’s “not possible” for something to be conscious-- just that if we want to have any reasonable chance of accurately guessing what’s conscious, these properties (along with an understanding of said something’s inner workings) can be a useful guide. So, let’s evaluate the similarities and differences.
Humans share some of these properties with LLMs. Humans are only conscious for a period of time (while we are awake (or while dreaming?)) where we are processing information. We too lack full continuity of consciousness. The parallel of this for a neural network of any kind might be during the time they are generating output or training-- in that period of time, they are also processing information. So I think you have narrowed the window in which it is reasonably possible for an LLM to be conscious (“awake”) to just the period of time in which it is running. (Also worth noting is that human brains don’t fully shut down while we are unconscious, not sure how to fit that in here though).
You also mention randomness-- true, maybe randomness plays a part in consciousness (true randomness, not pseudorandomness), and LLMs are deterministic. I think you could relatively easily construct a neural network that does use true randomness, but 1. true randomness is “expensive” to use in terms of computers, usually only being used to seed PRNGs, so it would take FOREVER to run, and 2. it would probably require new training techniques, and at that point maybe you could just skip to the experiment after the end of the next paragraph.
The last difference is that they don’t take input, produce output, and update themselves (train) all at the same time like we do. We also do this continuously, rather than in discrete steps. Therefore you can say that LLMs (or any current neural networks) lack an integrated and locally continuous experience. This is probably the biggest blocker to LLM consciousness that I see being taken seriously, and I’m inclined to agree, because it is the biggest difference between humans and neural networks.
So, what if we construct a machine using an LLM as its core that has these properties? One that could:
Anyways, sorry for getting sidetracked in hypothetical scenario land. I agree that there are enough differences that it is unlikely for an LLM to be conscious in the same way that a human is. But I am curious what makes you so confident that those particular properties are necessary for consciousness to exist at all on some level-- could consciousness not simply be a phenomenon that arises whenever something is being computed, and human brains are just a continuous and very complicated computer with the ability to learn and act simultaneously?
I suppose that line of thinking could quickly be expanded to the “basically everything has a tiny bit of consciousness” philosophy, and then you would have to come up with a more specific term for human-like consciousness; but that only delays the moral dilemmas until we create machines that have more human properties (like the one I proposed above)-- or humans that have more machine ones (brain organoids or whatever they’re called are fucking terrifying philosophically).