AI self-models and subjective experience

If a system can monitor its uncertainty, distinguish its actions from external events and update a model of its own capabilities, calling all of that mere imitation can hide genuine functional achievements. We should examine what the system does internally, not only whether its language sounds familiar.

Agreed. But functional self-monitoring is not identical to experiencing oneself. A thermostat tracks a variable; that does not make it a subject. We need an account of which kinds of organization matter and why. Fluent self-report is especially weak evidence when the training process rewards plausible language about mental states.

Research can still proceed under uncertainty. Butlin and colleagues proposed indicators drawn from several scientific theories of consciousness.^[1] That approach makes assumptions visible and asks whether a system has relevant properties. It is more informative than asking it whether it is conscious.

Yes, provided indicators remain indicators. Different theories can disagree, and satisfying a checklist derived from one theory does not settle the theory's truth. We should avoid converting a research program into a certification procedure whose confidence exceeds the evidence.

We should also control for alternatives. If changing a prompt reverses a system's claim about fear while leaving its abilities unchanged, that weakens a simple reading of the statement. An internal mechanism that causally guides stable self-monitoring would be stronger evidence of function, though still not decisive evidence of experience.

And we should not assume that uncertainty licenses indifference. If future systems provide stronger evidence of welfare-relevant states, researchers may need precautions. But today's users also deserve protection from manipulative claims that a product suffers when they close an app or decline a subscription.

That gives us two responsibilities. Study possible machine experience without selling certainty, and evaluate present systems for the harms they can cause people. NIST's framework supplies risk-management questions for deployment; it does not adjudicate consciousness.^[2]

Then the answer is conditional. Advanced AI may implement increasingly rich self-models. Whether those models carry experience remains a separate scientific and philosophical problem. We can recognize useful functionality while refusing to turn a first-person sentence into a proof.