Honorable mention / 15 October 2026, Goodfire

Goodfire researcher argued neural network neurons are legible without untangling

An "iy" detector grew during training to respond to "diy" and then to "do it yourself", which lacks "iy".

Honorable mention, not a full case file. No two named parties were found who disagree about the nature of the system.

The question this event raisesDoes the language of intention (wanted, decided, collaborated) describe what AI agents do, or mislead?

Direct investigation of clean neural networks

“The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread. What looked like arbitrary packing was mostly a motif I’ll call recursive generalization, which is much less concerning. This means we can study networks directly, instead of first untangling them.”
Nick Cammarata (Goodfire), ran the system

already-clean.goodfire.com, 15 Oct 2026, archived copy

These readings say something about what the system is, not only about what it did. A language model scored each reading on that, and the ones above a fixed cutoff appear here. The site takes no position.

What happened

Nick Cammarata argued neural networks use recursive generalization instead of superposition to pack concepts. He reported that roughly 90% of neuron poles in Qwen3-32B have one main job. He noted similar findings in models with up to trillions of parameters, including Kimi K3. He stated that in a modern parameter-matched ConvNeXt, about 99% of last-layer neurons are legible.

First-hand sources

Also on the record

These readings describe what happened or judge how dangerous it was. Each quote appears word for word on the linked page.

  • johnswentworthinvestigated it

    “Just looking at the microscope page, the semantics of each neuron are not quickly obvious to me (though most of them feel like there's probably some pattern there which I could understand if I spent a lot of time per neuron staring at them).”

    lesswrong.com, 16 Oct 2026, archived copy

What would settle it

  • Timed blind tests measuring whether independent observers quickly deduce neuron roles across sample sets would test if the semantics are obvious.
  • Systematic ablations tracking whether downstream circuits read out distinct semantic concepts independently would test whether polysemantic neurons represent superposition or shared features.

About agency

This research bears on whether neural networks form coherent, inspectable world representations or entangle concepts into unreadable knots. Legible representations could allow direct tracking of internal goals, while arbitrary entanglement would require separate models to decode the network.

About consciousness

This event is evidence about what the system did. It does not establish or rule out that the system experiences anything. The readings on this page disagree about what the behaviour shows.