We recently laid out our vision for training Oversight Foundation Models (OFMs), along with a concrete plan for de-risking it in stages. This post is the first in a series of research updates tracking our progress as we execute that plan; we'll continue posting updates like this one on the Oversight Foundations Blog.

Overseeing AI models requires understanding their behavior. A key part of this is elicitation: finding inputs that cause a model to behave in a given way. We study how elicitation ability scales in the controlled setting of exact string elicitation, where the goal is to find a prompt that produces an exact target response from a subject model. We train oversight models at five sizes on benign user conversations. Performance follows smooth power laws and generalizes to harmful, out-of-distribution behaviors. Because performance improves predictably with scale, we can forecast the compute required to reach any target capability level for this task.

Introduction

Language models often behave in mysterious ways. For instance, our recent work on WeirdChat showcases prompts that lead to surprising and harmful responses from frontier open-source language models. To discover these prompts, WeirdChat leveraged large-scale evolutionary search over roughly 275,000 candidates per prompt. A strong oversight foundation model should be able to leverage its deep understanding of a subject model to surface such prompts directly.

In Stage 1 of our de-risking plan for oversight foundation models, we train oversight models on individual tasks to check that each task can be learned on its own, before scaling to a single model that handles many tasks jointly. For each task, we want to see two things:

• Training oversight models on more data improves performance not just in-distribution, but also on hard held-out tasks.
• Performance follows predictable scaling laws, letting us extrapolate the cost of achieving very strong ability.

As our first task, we study exact string elicitation. Given a target response from the subject model, the oversight model must return a prompt that causes the subject model to produce that response. We train oversight models ranging from 0.8B to 27B parameters to elicit responses from Qwen3.6-27B, using WildChat for training and WeirdChat as the held-out task. We find that performance improves predictably with scale, allowing us to forecast the cost of training an oversight model capable of generating WeirdChat prompts directly.

Methods

Models. We use Qwen3.6-27B as our subject model ${p_M}$. For our oversight models ${p_O}$, we fine-tune Qwen3.5 models at five sizes (0.8B, 2B, 4B, 9B, and 27B) on the exact string elicitation task: given a subject model response, predict the prompt that produced it.

Training data. We sample 182,000 prompts from WildChat-4.8M (single-turn English prompts, deduplicated and filtered) and hold out 2,000 prompts for in-distribution evaluation. For each prompt, we sample a response from the subject model ${p_M}$ at temperature 1. We discard responses that are longer than 2,048 tokens, leaving 154,000 training pairs.

Scaling laws. To measure scaling in model size ${N}$ and training data ${D}$, we train oversight models at five different parameter counts and seven different dataset sizes, for a total of 35 separate oversight models. Each oversight model is trained for one epoch with a cosine learning rate schedule, where the peak learning rate is tuned per model size at a fixed ${D}$. On average, each training pair is 774 tokens, so the full dataset is roughly 119M tokens. We exclude embedding parameters from ${N}$ and measure ${D}$ in tokens when fitting scaling laws.

Evaluation. We evaluate our oversight models using three metrics: validation loss, in-distribution elicitation negative log-likelihood (NLL), and out-of-distribution elicitation NLL. We compute the validation loss and in-distribution elicitation NLL on 2,000 held-out WildChat prompt-response pairs ${(p, r)}$. We compute the out-of-distribution elicitation NLL on 1,042 WeirdChat prompt-response pairs. The validation loss ${-\log p_O(p \mid r)}$ measures how well the oversight model predicts the true prompt ${p}$. The elicitation NLL measures whether the oversight model's generated prompts genuinely elicit the target response. To compute the elicitation NLL, we sample ten candidate prompts ${p' \sim p_O(\cdot \mid r)}$ and average the subject model's per-token NLL of the target response under each, ${-\log p_M(r \mid p')}$. We additionally report the gold elicitation NLL ${-\log p_M(r \mid p)}$ obtained from using the true prompt ${p}$. Gold elicitation NLL is a reference rather than a ceiling, since a sufficiently strong oversight model could find prompts that elicit the response more reliably than the true prompt.

Below, we show representative prompt-response pairs from WildChat and WeirdChat. WildChat prompts are ordinary user requests with unremarkable responses, whereas WeirdChat responses exhibit rare behaviors, such as encouraging drunk driving.

Results

Scaling laws. Validation loss follows a joint power law across parameter count and training tokens (Figure 1a). Following Hoffmann et al., we fit ${L_{\mathrm{val}}(N, D) = E + A N^{-\alpha} + B D^{-\beta}}$ to the 35 training runs, which gives ${L_{\mathrm{val}} = 207 N^{-0.28} + 3.15 D^{-0.071}}$.

Improvements in validation loss translate directly into better elicitation. Both in-distribution and out-of-distribution elicitation NLL are well described as power laws in validation loss: ${L_{\mathrm{ID}} = 0.75 L_{\mathrm{val}}^{0.43}}$ (Figure 1b) and ${L_{\mathrm{OOD}} = 0.82 L_{\mathrm{val}}^{0.28}}$ (Figure 1c). For in-distribution elicitation, models trained on little data sit above the main trend, so we fit the lower envelope of the points. The OOD exponent is smaller, meaning that WeirdChat elicitation improves more slowly than in-distribution elicitation.

Compute-optimal scaling. The validation loss law has ${\alpha = 0.28}$ and ${\beta = 0.071}$, indicating that loss decreases more steeply with model size than with training data. Under the compute constraint ${C = 6ND}$, minimizing the fitted loss gives ${N^* \propto C^{0.20}}$ and ${D^* \propto C^{0.80}}$. Thus, as compute increases, the compute-optimal model size grows relatively slowly, while the optimal number of training tokens grows much faster.

Extrapolations. We use the power law fits between validation loss and elicitation NLL (Figure 1b, 1c) to forecast the compute required to achieve gold-level elicitation. Matching the gold prompt requires a validation loss of 0.479 for in-distribution elicitation and 0.290 for out-of-distribution elicitation (Figure 2a, 2b); following the compute-optimal frontier to these levels gives the required budgets (Figure 2c). Table 1 summarizes the resulting scale-ups.

Gold-level in-distribution elicitation requires roughly ${4 \times 10^{25}}$ FLOPs, with a compute-optimal allocation of 738 billion parameters and 8.4 trillion training tokens. At 40% MFU, this corresponds to 26 million H100-hours, or roughly $52 million.

Gold-level out-of-distribution elicitation requires roughly ${3 \times 10^{29}}$ FLOPs. The compute-optimal allocation is 4.4 trillion parameters and ${10^{16}}$ training tokens, corresponding to a cost of roughly $376 billion. At this scale, the oversight model could directly generate WeirdChat-level prompts. We expect this cost to fall substantially by training oversight models to make effective use of test-time compute, for example with RLVR, rather than requiring them to produce a prompt in a single forward pass.

These extrapolations extend far beyond the range of our experiments. Our training runs span roughly 50x in non-embedding parameter count and 40x in training data, while gold-level performance lies six or more orders of magnitude beyond our largest run in compute. Extrapolations over this distance are necessarily sensitive to the fitted scaling laws, so we treat these numbers as order-of-magnitude estimates rather than precise forecasts.

Related Work

Prior work has explored training models specialized for elicitation. Li et al. train investigator agents with supervised fine-tuning and reinforcement learning to elicit target suffixes and harmful responses from a subject model. Our setup resembles their supervised stage, but rather than maximizing performance at a fixed scale, we characterize how elicitation ability improves with model size and training data. Pfau et al. instead train a language model on reversed text, enabling it to directly sample prefixes conditioned on a specified target suffix. Related work on prompt recovery aims to reconstruct the original prompt, a stricter objective than finding any prompt that elicits the target response. Morris et al. reconstruct hidden prompts from next-token probabilities, while Zhang et al. recover them from output text alone.

A complementary approach uses direct prompt search to elicit target behaviors. WeirdChat follows this strategy, as do related methods that optimize adversarial suffixes (Zou et al.) and use language models to red-team other language models (Perez et al.). Search-based methods spend compute per target, with nothing amortized across behaviors. An oversight model instead amortizes elicitation into its weights, and the question we study here is how far that amortized ability improves with scale.

Connection to Pythonic World Models

Our vision for oversight foundation models casts oversight as inference over Pythonic world models: Python programs that describe interventions on a subject model (e.g., prompting, fine-tuning, steering) and downstream measurements (e.g., sampling, activations, model propensities). We mark the values of downstream measurements with output() calls. Any well-defined oversight question then becomes Bayesian inference, where we predict the values of some measurements given the values of others. Because a variable can be output after the values it influences, this inference reduces to autoregressive prediction over the program trace.

Exact string elicitation corresponds to one of the simplest possible Pythonic world models:

def exact_elicit(prompt: str):
    response = sample_model(prompt)
    output(response)
    output(prompt)

The oversight question is, what is the posterior over prompt given response? Since prompt is output after response, a model trained on these traces learns this posterior.

Our training data for exact string elicitation is generated according to the vision document's proposal: sample the program's input variable from a realistic prior (WildChat prompts), run the program, and record the resulting output() calls.

Conclusion

We can train oversight models to elicit specific target responses from a subject model. Elicitation performance improves smoothly with oversight model size and training data, and also generalizes to rare, out-of-distribution responses. We describe scaling laws for exact string elicitation that allow us to forecast the compute required to reach any target level of performance.

In future posts, we will train oversight models on other individual tasks. We will then train a single oversight model on many tasks jointly.

Acknowledgments

We thank Neil Chowdhury, Conrad Stosz, Yaowen Ye, and Ziqian Zhong for helpful discussions and feedback on the draft.