Research

Mechanistic Indicators of Understanding in Large Language Models

Large language models are often characterised as pattern-matching systems without genuine understanding. The paper argues that recent mechanistic findings challenge this view once they are integrated with a theory of what understanding is, and proposes three tiers: conceptual understanding, when a model forms features as directions in latent space; state-of-the-world understanding, when it learns contingent connections between features and tracks how the world changes; and principled understanding, when it stops relying on memorised facts and discovers a compact circuit that connects them. The internal organisation this reveals supports understanding-like unification while diverging from human cognition in its parallel use of heterogeneous mechanisms, which moves the debate past a yes or no toward a comparative epistemology of AI and human understanding.

A Polyphonic Conception of AI Understanding

When a doctor, a judge or an engineer must decide whether to trust a model's output, they cannot avoid asking what the model understands: purely mathematical or statistical descriptions struggle to tell trustworthy from untrustworthy outputs without reintroducing the question in all but name. Yet the inherited concept operates within a monophonic paradigm, which localises understanding in a single mechanism underpinning all the capacities it confers. Drawing on a wide range of mechanistic evidence, the paper shows that language models are pervasively polyphonic: outputs emerge from coalitions of parallel mechanisms of uneven reliability, which complement, duplicate or drown out one another, with several coalitions sufficing for a task and none indispensable. This renders monophonic inferences hazardous, and the paper develops a conception of understanding fit for polyphonic AI, centred on sound circuitry that is reliably and correctly recruited and in control of outputs.

Intention-like Representations in Language Models?

A growing chorus of AI researchers and philosophers posit internal representations in large language models. But how do these relate to the mental states we routinely ascribe to each other? Belief-like and knowledge-like states have had attention; intentions have had little. The paper surveys five properties associated with intentions in the philosophical literature, directive function, degrees of distality, degrees of abstraction, commitment and planning, and assesses two candidate classes of representations against them: function vectors, and the planning representations found in circuit-tracing work. The result is mixed. Current evidence suggests representations that are intention-like in many, perhaps surprising, respects, yet differ from human intentions in important ways; whether to call them intentions requires weighing the costs and pay-offs of extending the concept beyond the paradigm human case.

World Modeling in Transformers

TaxiGPT is a transformer trained on random walks through Manhattan. Its wrong turns had been read as proof that it holds no coherent map. Looking inside tells a different story: the model represents intersections and the streets between them, tracks its own position over a rolling window of recent moves, and steers by a goal compass, a circular representation of the bearing to its destination. Its failures come from interference between intersection features stored in superposition, not from the absence of a map.

The interactive explorer lets you play a ride and watch the model's position code and goal compass update turn by turn.

Where is the Mind? Persona Vectors and LLM Individuation

The individuation problem for large language models asks which entities associated with them, if any, should be identified as minds: the weights, a conversation, a character the model plays? The paper approaches it through mechanistic interpretability, engaging with recent work on persona vectors, persona space and emergent misalignment, and argues that three views are the strongest candidates: the virtual instance view, on the grounds that attention streams sustain quasi-psychological connections across token-time, and two new views, the instance-persona view and the model-persona view, which the persona literature makes promising alternatives.

Probing Persona-Dependent Preferences in Language Models

Language models can be said to have preferences: they reliably pick certain tasks and outputs over others, and preferences shaped by post-training and prompting appear to influence much of their behaviour. But models can also adopt personas with radically different preferences. Does each persona use its own preference representations, or are some shared? The paper identifies a genuine preference vector: it tracks the model's preferences as they shift across a range of prompts and situations, and steering along it causally controls which task the model picks. Some preference information transfers across personas: the vector found on the helpful assistant predicts and steers the choices of qualitatively different personas, including an evil persona whose preferences anti-correlate with the assistant's. Enacting a persona may thus rely on shared circuitry: representations common across personas, along which what is encoded changes with the persona currently active.

Consciousness and Cognitive Access in LLMs

Anthropic recently released the J-lens, a tool that reads from a model's residual stream which tokens it is getting ready to say. The directions it reads from, the J-space, turn out to hold the representations the model can talk about: it can report them, hold them in mind and use them in reasoning, which is not true of most of what goes on inside it. As the Anthropic researchers note, that is roughly the role Global Workspace Theory gives to consciousness. This commentary accompanied the release and explores what the findings mean for consciousness. It argues that this is the most significant evidence of consciousness in language models so far uncovered by mechanistic interpretability: evidence of access consciousness, though not yet of a global workspace, which would need more support. The report remains highly uncertain about phenomenal consciousness, since language models differ from humans in many ways that could plausibly matter.

Mechanistic Interpretability Needs Philosophy

Mechanistic interpretability aims to explain how neural networks work by uncovering their underlying mechanisms. As the field grows in influence, it is increasingly important to examine not just the models but the assumptions, concepts and explanatory strategies implicit in the research. The paper argues that mechanistic interpretability needs philosophy as an ongoing partner in clarifying its concepts, refining its methods, and navigating the epistemic and ethical complexities of interpreting AI systems, and makes the case through three open problems: how a network should be decomposed into interpretable parts, what the features a model uses represent, and how deception could be detected from a model's internals.

Deep Learning Models Also Recall Features

Recent work in mechanistic interpretability has studied how large language models recall facts stored in their weights. The paper argues that factual recall points to something broader, a general kind of operation in deep learning models: feature recall. The core observation is that a linear projection can be read as retrieving stored information scaled by input activations; when Michael and Jordan are both present, an MLP combines them into a Michael Jordan feature, and that one feature recalls what the weights hold about him and adds it to the residual stream. The paper defines feature recall, shows it applies across architectures, contrasts it with the established paradigm of feature combination, and considers how cases of it might be mechanistically identified.