Honorable mention / 13 May 2026, Google DeepMind (Gemma-3-27B) and Alibaba Cloud/Qwen Team (Qwen-3.5-122B); the research was conducted by Oscar Gilg, Pierre Beckmann, Daniel Paleka, and Patrick Butlin, affiliated with MATS, EPFL, ETH Zürich, and Eleos AI Research.
Researchers found shared preference directions across language model personas
The models represented task choices along a single internal direction that transferred across opposing personas, including an evil persona whose choices opposed the assistant.
Honorable mention, not a full case file. No two named parties were found who disagree about the nature of the system.
The question this event raisesDoes the language of intention (wanted, decided, collaborated) describe what AI agents do, or mislead?
Probing text continuation rather than evaluative preferences
“It seems possible that LLMs don't handle decisions about selecting tasks in any fundamentally different way than they produce other text completions. You might think that what you're picking up on is not an evaluative representation, but a 'likely to appear in the answer' or 'likely to be the focus of the next part' representation (of some sort or other).”
lesswrong.com, 10 Mar 2026, archived copy
Task embedding versus access to preferences
“I think you're right that probably won't be able to disentangle between better task embedding vs chat model having access to its own preferences.”
lesswrong.com, 9 Mar 2026, archived copy
These readings say something about what the system is, not only about what it did. A language model scored each reading on that, and the ones above a fixed cutoff appear here. The site takes no position.
What happened
Researchers trained linear probes on residual-stream activations of Gemma-3-27B and Qwen-3.5-122B to predict pairwise task choices. The researchers identified a preference vector that tracks model preferences across different prompts and situations. Steering activations along this vector causally controlled pairwise choices in Gemma-3-27B. This internal preference representation is largely shared across different model personas. A probe trained on the helpful assistant predicted choices for an evil persona whose preferences anti-correlated with the assistant.
First-hand sources
- Eleos AI18 May 2026
- MATS Research5 Oct 2026
What would settle it
- A test showing whether the activation vector predicts text choices across unrelated completion tasks or only pairwise evaluations.
- A comparison testing whether the probe isolates model preference access or just a general embedding of the task description.
About agency
This provides evidence on whether models have genuine underlying preference representations or simply predict text completions. It could show models access their own preferences, or merely track which tasks are likely to be chosen next.
About consciousness
This event is evidence about what the system did. It does not establish or rule out that the system experiences anything. The readings on this page disagree about what the behaviour shows.