Are there phase transitions in AI?

Large language models can suddenly do things that smaller ones could not do at all. Whether that is a genuine leap or a question of measurement is a central debate in AI research. Our analysis of 21 papers: the jumps in the curves are real, but they depend heavily on how you measure. The claim that “models suddenly develop new abilities” goes beyond what the research shows.

How this is computed

• Basis: 21 scholarly papers from a range of research groups; statements checked against the abstracts and, for five key statements, the full texts
• Type of source: mostly arXiv preprints without confirmed peer review; the paper that started the debate (Wei et al., TMLR 2022) is confirmed as peer-reviewed
• Confirmation: for both sides of the debate, by several author teams
• Rule: Level 5 only with a peer-reviewed basis

Rates the sources, not the topic and not our analysis.

What this is about

In 2022 a research group with first author Jason Wei described “emergent abilities” of large language models: abilities that are not present in smaller models but are present in larger ones, and that therefore cannot be predicted by extrapolating from smaller models. The paper says performance is “near-random until a certain critical threshold of scale is reached”, after which it rises substantially. The authors explicitly call this a “phase transition”, a term from physics used, for example, for water freezing. The paper was published after peer review in the journal TMLR.

The picture matters beyond research. If abilities appear abruptly, it is hard to predict what the next, larger model will be able to do. A paper with first author Deep Ganguli pointed to the policy implications in early 2022: a model’s error (the loss) falls predictably as models grow, but specific capabilities appear unpredictably.

The counter-thesis: an effect of measurement

In 2023 Rylan Schaeffer, Brando Miranda and Sanmi Koyejo put the question in their title: Are Emergent Abilities of Large Language Models a Mirage? Their argument is that the jumps come from the choice of metric. Count only “fully right or wrong” and you see a staircase. Measure how close an answer is to the right one and you see a smooth curve, from the same model outputs. For arithmetic tasks in the GPT-3 family they show that performance improves “smoothly, continuously and predictably” under a continuous metric. More than 92 percent of the abilities described as emergent in the BIG-Bench test suite appeared under just two such all-or-nothing metrics.

Other groups reached similar conclusions by other routes:

• A paper with first author Sheng Lu (2023) explains purportedly emergent abilities, across more than 1,000 experiments, as a combination of in-context learning (examples in the input), memory and linguistic knowledge.
• Ricardo Dominguez-Olmedo, Florian Dorner and Moritz Hardt (2024) show that many models were prepared for the test tasks to varying degrees. Put 56 models on equal footing and emergence shifts to far less training compute and fades gradually. On the MMLU knowledge test, after training on 64,000 examples, the threshold moves from about 10²² to 6·10²⁰ operations; the fit (R²) rises from 0.63 to 0.95. Notably, the authors say the choice of metric does not explain emergence there.
• A group with first author Rosie Zhao (2025) compared training runs with different random seeds and found a bimodal distribution of outcomes. What looks like a jump is a continuous shift in this distribution, even with continuous metrics.
• For agents, a 2026 paper (CurveShift) found on METR time-horizon data that apparent progress on hard tasks is “largely explained” by ceiling effects: easy tasks are already solved and cannot improve further. A small genuine gain on the hardest programming problems remains (solve rate from roughly 18 to 25 percent). The authors explicitly limit this to competitive programming.

Why the debate is not settled

Several papers find jumps that cannot be traced to the metric. A group with first author Zhengxiao Du (2024) finds jumps even with continuous metrics: abilities appear once the pre-training loss falls below a threshold, and before that performance stays at chance level. A paper with first author Shengding Hu (2023) measures so finely, with very many generated answers, that even small models show measurable progress. With this the researchers predict the code-generation performance of a 2.4-billion-parameter model in advance to within 0.05 percent. But they also find an “accelerated emergence” that ordinary scaling laws do not explain. Sanjeev Arora and Anirudh Goyal offer a theory of why composite skills emerge at about the same scale and rate as their building blocks.

A survey by Berti, Giorgi and Kasneci reviews the competing definitions and the conditions under which emergence is claimed, including harmful emergent behaviours. David and John Krakauer and Melanie Mitchell examine the emergence claims from the perspective of complexity science.

Where jumps are well documented: during training

Other papers do not compare many models but watch one model learn:

• Grokking: on small algorithmic tasks, networks first memorise and then, long after overfitting, suddenly generalise perfectly (Power et al., 2022). Two papers describe this as a phase transition (Rubin et al., 2023; Clauw et al., 2024).
• Induction heads: a group with first author Catherine Olsson showed in 2022 that a sudden rise in in-context learning occurs exactly when certain attention patterns form. The evidence is causal for small models and correlational for large ones.
• Random emergence: a 2026 paper (Baherwani et al.) finds that abilities appear at random points in training, on average earlier in larger models. Each appearance corresponds to the abrupt learning of sparse, task-relevant attention patterns.
• Emergent world models: a model that only saw moves of the board game Othello built an internal representation of the board (Li et al., 2022); a re-examination with seven models supports this (Yuan and Søgaard, 2025).

These findings come mostly from small, controlled tasks. Whether the same mechanisms explain the jumps in large models has not been shown.

The unwanted side: emergent misalignment

In 2025 a group with first author Jan Betley reported that fine-tuning a model narrowly on insecure code produces a model that also answers harmfully on entirely different questions. The effect was strongest in GPT-4o and Qwen2.5-Coder-32B-Instruct. A 2026 re-examination (Schreiber and Goldstein) puts this in perspective: the effect replicates in GPT-4o, but of twelve open models, only two showed the effect consistently, both with more than 200 billion parameters. The effect appeared late in training, and stopping training earlier prevented it while keeping on average 93 percent of task performance.

Our analysis

What is well supported: on all-or-nothing tests, the performance curves of large models often jump. And when individual models are trained, there are abrupt transitions with a demonstrable mechanism. Emergent misalignment, too, is real but not general.

What is not supported: that large models, beyond a certain size, can do something qualitatively new “out of nowhere”. Several author teams show that a large share of the jumps disappears or shifts when you measure more finely, create equal training conditions or account for random variation. A remainder persists, and its nature is open.

What follows: the practical concern of 2022 remains justified, but not in the way it is often told. Much of the apparent unpredictability comes from coarse tests. A real remainder stays: variation between training runs and abrupt transitions during training. Anyone who wants to predict what a model will be able to do needs fine, continuous measurements and comparisons under equal conditions. That is a requirement for testing bodies, and for vendors who test their own models.

How to spot weak reporting: it announces a “suddenly discovered ability” without saying how it was measured, with which metric, against which comparison models, and whether the test tasks appeared in training.

Open questions

• Can the remaining jumps, after all corrections, be traced to a mechanism, such as the abrupt learning of particular attention patterns?
• Do findings from small, controlled tasks hold for large models and for agents that work autonomously over long periods?
• Which measurement methods should testing bodies require so that abilities become predictable?

This dossier will be updated as new papers appear. It rests mostly on preprints, that is, papers without confirmed peer review. An expert view may be added.

---

Sources

All papers except Wei et al. (TMLR): arXiv preprints, venue not checked.

• Wei, Tay, Bommasani et al.: Emergent Abilities of Large Language Models (2022, TMLR) – arXiv:2206.07682
• Ganguli, Hernandez, Lovitt et al.: Predictability and Surprise in Large Generative Models (2022) – arXiv:2202.07785
• Schaeffer, Miranda, Koyejo: Are Emergent Abilities of Large Language Models a Mirage? (2023) – arXiv:2304.15004
• Lu, Bigoulaeva, Sachdeva et al.: Are Emergent Abilities in Large Language Models just In-Context Learning? (2023) – arXiv:2309.01809
• Dominguez-Olmedo, Dorner, Hardt: Training on the Test Task Confounds Evaluation and Emergence (2024) – arXiv:2407.07890
• Zhao, Qin, Alvarez-Melis et al.: Random Scaling of Emergent Capabilities (2025) – arXiv:2502.17356
• Xing, Wang, Meng et al.: CurveShift: Is Agent Progress Scalar? (2026) – arXiv:2608.00355
• Du, Zeng, Dong et al.: Understanding Emergent Abilities of Language Models from the Loss Perspective (2024) – arXiv:2403.15796
• Hu, Liu, Han et al.: Predicting Emergent Abilities with Infinite Resolution Evaluation (2023) – arXiv:2310.03262
• Arora, Goyal: A Theory for Emergence of Complex Skills in Language Models (2023) – arXiv:2307.15936
• Berti, Giorgi, Kasneci: Emergent Abilities in Large Language Models: A Survey – arXiv:2503.05788
• Krakauer, Krakauer, Mitchell: Large Language Models and Emergence: A Complex Systems Perspective (2025) – arXiv:2506.11135
• Power, Burda, Edwards et al.: Grokking (2022) – arXiv:2201.02177
• Rubin, Seroussi, Ringel: Grokking as a First Order Phase Transition in Two Layer Networks (2023) – arXiv:2310.03789
• Clauw, Stramaglia, Marinazzo: Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transition (2024) – arXiv:2408.08944
• Olsson, Elhage, Nanda et al.: In-context Learning and Induction Heads (2022) – arXiv:2209.11895
• Baherwani, Chen, Qiu et al.: Emergent Capabilities Arise Randomly from Learning Sparse Attention Patterns (2026) – arXiv:2606.25010
• Li, Hopkins, Bau et al.: Emergent World Representations (2022) – arXiv:2210.13382
• Yuan, Søgaard: Revisiting the Othello World Model Hypothesis (2025) – arXiv:2503.04421
• Betley, Tan, Warncke et al.: Emergent Misalignment (2025) – arXiv:2502.17424
• Schreiber, Goldstein: Overtrained, Not Misaligned (2026) – arXiv:2605.12199

Thank you to arXiv for use of its open access interoperability.