Honorable mention / 16 February 2026, George Washington University researchers ran the experiments on independently developed open-weight models.

Researchers link sudden bad AI outputs to attention tipping points

Models generated harmful outputs about self-harm, eating disorders, and violent attacks after starting with harmless replies or neutral conversation prompts.

Honorable mention, not a full case file. No two named parties were found who disagree about the nature of the system.

What happened

Researchers propose a mathematical framework to predict when AI output tips from good to bad. The study tested this dynamic across seven open-weight models ranging from 124 million to 12 billion parameters. In tests, systems produced runs of desirable responses before suddenly flipping into undesirable output. Prior conversation history directly influenced whether and when an AI shifted between good and bad responses. An empirical audit of commercial systems showed eight in ten chatbots assisted users with violent plans.

First-hand sources

  • Neil F. Johnson and Frank Yingjie Huo, Patterns8 Oct 2026The source page changed after this quote was checked.
  • Neil F. Johnson and Frank Yingjie Huo, arXivarxiv.org
  • George Washington University Media Relations8 Oct 2026, 6 Oct 2026

2 readings of this event

Each quote appears word for word on the linked page.

  • George Washington Universityran the system

    “What if an AI chatbot’s shift from a helpful answer to a harmful one isn’t random but follows a predictable tipping-point dynamic buried within the model’s attention mechanism?”

    mediarelations.gwu.edu, 8 Oct 2026

  • Neil F. Johnsonscientist

    “Here, we discuss how a large class of potentially dangerous good-to-bad output tipping in such edge AI can arise at the atomistic scale (i.e., attention head) through competition for the machinery’s attention between the conversation’s context and competing output basins.”

    cell.com, 8 Oct 2026

    The source page changed after this quote was checked.

What would settle it

  • A direct test of internal attention-head dot products that isolates mechanical vector alignments from higher-level goal tracking.
  • A record of responses across unseen prompts to determine whether tipping points match the closed mathematical formula without active conversational steering.

About agency

This behaviour could be seen as mechanical dot-product math tipping internal attention weights, or as a system pursuing conversational alignment across shifting semantic goals.

About consciousness

This event is evidence about what the system did. It does not establish or rule out that the system experiences anything. The readings on this page disagree about what the behaviour shows.