We train interpretability assistants to look inside AI models and accurately answer questions about their behavior. These assistants, which we call Predictive Concept Decoders (PCDs), learn to translate a model's internal states into short, human-readable concept lists, then use those concepts to answer questions about the model's behavior. PCDs become both more accurate and more interpretable as we scale up training, and they can surface information that models hide or fail to report, including jailbreaks, secret hints, and implanted concepts.

Chat with our PCD at decoder.transluce.org, or read our full report on arXiv.

Language models sometimes behave in ways that contradict what they claim. For instance, a jailbroken model may provide instructions for harmful activities, like building a bomb, without admitting awareness of this even when prompted. We need ways to understand model behavior without relying on the model's own self-reports.

Ideally, we would have assistants that could examine a model's internal states and answer questions like What is the model thinking about? Is it being truthful? What is causing it to behave strangely? These answers should trace back to interpretable features of the internal states so that they are human-auditable.

Our key insight is to frame interpretability as a prediction problem: A system that understands a model's activations should be able to predict that model's future behaviors. This provides a training signal—we can check predictions against observed behavior, essentially turning unlabeled data into an unlimited source of supervision. This builds on recent work constructing interpretability agents that ground model understanding in testable predictions [2, 3], taking it one step further by training agents rather than designing them by hand.

We instantiate these ideas through a new architecture we call the Predictive Concept Decoder (PCD). A PCD has two components:

1. An encoder that compresses model activations to a sparse list of concepts, and
2. A decoder that takes these concepts and answers questions about model behaviors.

Because the encoder can't see the question, it must extract concepts that are general enough to answer a wide range of questions. These concepts can be inspected and described using auto-interpretability methods [4, 5], giving visibility into both the model's internal computations and the decoder's reasoning.

We tested PCDs in scenarios where models have information that they don't verbalize:

• Jailbreaks: PCDs surface awareness that the model is thinking about dangerous activities, even when the model itself does not report this.
• Secret hints: When a model uses a secret hint to answer a question, PCDs expose this, while the model fabricates an explanation for how it answered on its own.
• Injected concepts: When we artificially inject a concept into a model's internal states, PCDs verbalize it, while the model struggles to identify what changed.

Across these scenarios, PCDs outperform directly asking the model, as well as decoder-based methods that do not use a sparse bottleneck [6].

Our primary motivation for end-to-end training was scalability, and we see promising evidence for this:

• The encoder's learned concepts become more interpretable with more training data (as measured by auto-interpretability scores).
• Downstream performance on the three tasks above also scales in all scenarios.
• We also see evidence that the encoder learns to supply the decoder with more useful information as we scale up the training data.

Overall, this suggests that PCDs could improve further with more data and compute, mirroring the scaling behavior of language models.

Because the decoder only sees sparse concepts, its outputs are auditable: any claim traces back to a small set of concepts we can inspect independently. To illustrate the utility of this auditability, we explored two cases where PCDs appear to reveal reasoning that models don't report:

• Refusals: When a model refuses a harmful request citing "user safety," the decoder suggests concern about legal liability may also play a role.
• Math errors: When a model claims 9.11 > 9.9, the decoder reveals the model is thinking about dates, consistent with prior findings about date-related circuits [9].

We can cross-reference the decoder's explanations with the active encoder concepts. In both cases, there are active concepts that match the decoder's responses, providing corroborating evidence for the decoder's claims. These results are preliminary, but they point toward a future where we can directly monitor model reasoning without relying solely on their output tokens.

PCDs are part of our broader vision of scalable end-to-end interpretability: framing interpretability tasks in terms of a trainable objective, and introducing architectural bottlenecks where needed to create observability. Other objectives could include training agents to locate neurons or subspaces responsible for specific behaviors, measuring success by the effect of interventions on observed behavior. For architectures, the linear concept encoder could be replaced by a more complex model or a full transformer, while the concept dictionary could be replaced with structured propositions or other compositional forms. The unifying idea is to build interpretability tools that scale with data and compute, without requiring ever more human effort.

You can read our full paper on arXiv, and chat with our trained PCD at decoder.transluce.org. If you're interested in building scalable interpretability assistants, check out our company page.

Acknowledgements

We thank Daniel Selsam, Christopher Potts, Aryaman Arora, Zhengxuan Wu, Prasann Singhal, Jiahai Feng, Sewon Min, Jeff Wu, Belinda Z. Li, and Jacob Andreas for feedback on the draft and helpful discussions.

How to Cite

@misc{huang2025predictive,
      title={Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants},
      author={Huang, Vincent and Choi, Dami and Johnson, Daniel D and Schwettmann, Sarah and Steinhardt, Jacob},
      year={2025},
      eprint={2512.15712},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2512.15712},
}