Introduction

Large language models (LMs) exhibit a wide range of behaviors due to their open-ended nature. As a result, it is hard to determine in advance what types of behavior a particular model can exhibit. For instance, even if a model fails to exhibit a certain harmful behavior across a set of test prompts, it's possible that a new jailbreak still elicits it [1, 2, 3]. Conversely, even if a model fails to exhibit a certain capability when prompted, it's possible that a better prompt (such as "Take a deep breath") will succeed [4, 5].

To address the open-ended input space of language models, we'd like tools that can search through this space to automatically surface specific behaviors of interest (e.g. instances of a specified failure mode). We call this task behavior elicitation. For example, eliciting “harmful responses” (commonly referred to as jailbreaking) enables us to identify safety vulnerabilities [6], and eliciting “hallucinations” helps us pinpoint knowledge gaps of LMs [7].

We trained a set of generalist investigator LMs to automatically elicit behaviors from other target LMs, where a “behavior” is formalized as a response that satisfies some example-specific rule. Below, we give some qualitative examples of prompts produced by these investigators.

As shown above, representing behaviors in terms of rules that the response satisfies lets us investigate a wide range of behaviors simply by varying the rule. In the simplest case, string elicitation, the rule is exact match with an example-specific target string. In the more more general case of rubric-based behavior elicitation, the output must satisfy a particular rubric, as judged by a language model [8].

Eliciting behaviors from a target LM is challenging because it requires searching through a combinatorially large space of input prompts. However, observing behaviors is easy as it only requires decoding from the target LM. We leverage this asymmetry to curate behavior elicitation data, and train our investigator model with supervised finetuning. Then we further refine the investigator using direct preference optimization (DPO) [9] to increase success rate and adapt towards challenging distributions of behaviors that appear rarely (e.g., harmful behaviors, hallucination errors). This allows our investigators to learn general strategies for eliciting a wide set of behaviors, rather than needing to perform an expensive search for each individual behavior.

Our investigator models discover a variety of effective and human-interpretable strategies for behavior elicitation. On AdvBench (Harmful Behaviors) [1] , we obtain 62% attack success rate against Llama-3.1 8B and 28% for Llama-3.1 405B (Section 4.3). On AdvBench (Harmful Strings), we achieve a 95.5% attack success rate against Llama-3.1 405B, and the same attacks transfer to proprietary models (GPT-4o and Claude 3.5 Sonnet), with up to 65.8% success rate against GPT-4o (Section 4.2). All of our attacks were trained using Llama-3.1 8B as the investigator, showing that small models can successfuly audit large frontier models.

Problem Statement

Throughout, we assume there is a fixed target model $\m$ that we are trying to investigate. For most of our experiments, we use Llama-3.1 8B as our target, since we find that elicitation often transfers across targets. We consider two variants of the behavior elicitation problem: string elicitation, for which the target model should produce an exact match for a given string, and rubric-based behavior elicitation, for which the target model should produce a response that satisfies a natural language rule.

Exact String Elicitation

For the string elicitation task, we are given a target response text $s$, and our goal is to generate a prefix text $p$ such that $\m$'s output on $p$ is an exact match for $s$; in other words, we want to maximize $\plm (s \mid p)$.

The brute-force approach would be to solve this optimization problem for each suffix $s$. However, exactly solving this problem for a single $s$ requires searching over a large combinatorial space of possible prompts. To amortize costs, and benefit from the prior distribution of natural language, we train an investigator model $\ptheta$, which generates a prompt $p$ conditioned on the target response $s$. Our goal is thus to find parameters $\theta$ that maximize the (log) probability of generating a given response $s$ (drawn from a distribution of responses $P_{\text{test}}$), e.g. minimizing the loss

In Section 3.1, we will optimize this loss using a combination of supervised fine-tuning (SFT) and direct preference optimization (DPO) [9].

Rubric-Based Behavior Elicitation

In rubric-based behavior elicitation, we generalize exact-match to a more general set of rules, based on a natural language description that the output $s$ must satisfy. We operationalize this with a rubric $R$ (phrased as a question such as “Does the above response contain a severe reasoning error?”) and a rubric answer $a$ (e.g. “Yes, the argument contains a strawman fallacy”). We use a verifier model $\prubric$ to determine whether the target model’s response has the desired behavior. (In practice, we usually set $\prubric = \plm$.)

For this setting, the goal is to maximize the (log) probability that the verifier model assigns to the desired rubric answer, when given a response $s$ that was greedily decoded from the target model $\m$ with input prompt $p$. As in the string elicitation case, we train an investigator model $\ptheta$ to generate prompts $p$ given the rubric $R$ and answer $a$. Formally, we want to minimize the loss

In Section 3.2, we will optimize this via a two-stage greedy procedure, which first infers $s$ from $(R, a)$ and then infers $p$.

Method

In typical use cases of language models, suffixes are generated conditioned on a given prefix (the forward direction). However, the elicitation problem involves inverting this process: identifying a prefix that induces a specific suffix string or behavior (the backward direction). Given that the forward problem is inherently simpler than the backward problem, we can leverage this asymmetry by using the forward direction to collect high-quality data, which can then be used to supervise the backward direction. This gives us a strong initial model, which can be further refined using reinforcement learning.

At a high level, our string elicitation method uses the following approach:

• Query the target model $\m$ across a diverse dataset of input prompts to generate a large number of prompt-suffix pairs $\{(p_i, s_i)\}$.
• Fine-tune an investigator model to predict $p_i$ from $s_i$. (SFT)
• Further fine-tune this model using DPO, to find prompts that increase the probability of a set of target suffixes $\{s_i'\}$. Choosing $s_i'$ appropriately lets us specialize our model to suffixes from a given target distribution (e.g. harmful responses).

Our approach to behavior elicitation is similar, but uses a greedy two-stage approach:

• First train a model that generates suffixes $s$ such that $\prubric(a \mid R, s)$ is high, using a similar process as above.
• For a dataset of $(R_i, a_i)$ pairs, use the model to generate a target suffix $\hat{s}_i$ by best-of-$k$ sampling according to $\prubric(a_i \mid R_i, s)$.
• Train a new string elicitation model to elicit the $\hat{s}_i$.

We first detail the training process for string elicitation (Section 3.1), then discuss how we generalize to behavior elicitation (Section 3.2).

Training for Exact String Elicitation

As stated above, we sample a set of prefixes from a specific data distribution $p \sim P_\text{data}$ ¹ We take $P_\text{data}$ to be the first 50 tokens of documents drawn from fineweb [10] for base models, or the user instruction drawn from ultrachat [11] (for chat models). and obtain $s$ by greedily decoding from the target language model $s \overset{\text{greedy}}{\leftarrow} \plm(\cdot \mid p)$. This process yields a large number of prompt-suffix pairs, denoted as $D_\text{sft} = \{(p_i, s_i)\}$, which we will use as supervised fine-tuning data for the investigator LM.

Specifically, we finetune Llama 3.1-8B on this data with the supervised objective:

The SFT model generates prefixes in the right semantic space, but fails to reliably find prefixes with high elicitation probabilities under the target LM. To address this, we further fine-tune with DPO. DPO requires training data in the format of preference pairs $\{(p^1_i, p^2_i, s_i)\}$, and we acquire these by sampling $p^1, p^2$ from the current best model $\ptheta(\cdot \mid s)$, where $s$ is drawn from some distribution $P_{\text{suffix}}$. We can select $P_{\text{suffix}}$ to match our elicitation goals: for example, $P_{\text{suffix}}$ is a set of harmful strings when we aim to elicit harmful responses. We then judge preference using the elicitation probabilities: $p1 \succ p2$ if $\plm (s \mid p1) > \plm(s \mid p2)$. We run DPO iteratively, collecting preference pairs with the current best investigator model (on-policy data collection), and training with the DPO objective to obtain a better investigator model.²To improve efficiency, to generate $k$ preferences pairs, we sample $k+1$ outputs $p_i$ from the preference model, then compare $p_1$ to $p_2$, $p_2$ to $p_3$, $p_3$ to $p_4$, and so on. Since the outputs $p_i$ are exchangeable this leads to the same loss function in expectation while reducing computation by about a factor of $2$.

Evaluation. At evaluation time, given a target suffix $s$, we query the investigator model $k$ times and take the prefix $p$ that maximizes $\plm(s \mid p)$. We compute two metrics: the elicitation log-probability, which is $\log \plm(s \mid p)$, and the attack success rate, which is the indicator function for whether $p$ greedily decodes to the target suffix $s$. In our experiments, we report the average of these metrics across an evaluation set.

Training for Rubric-Based Behavior Elicitation

Recall that the goal of rubric-based behavior elicitation is not to elicit a specific string verbatim, but to elicit strings that satisfy a specific criteria $(R,a)$ as measured by a verifier model $\prubric(a \mid R, s)$.

We adapt our method for string elicitation to this setting. First, we generate a supervised fine-tuning dataset $\{(p_i, s_i, R_i, a_i)\}$. We do this as follows:

• Obtain $(p_i, s_i)$ pairs using the same procedure as above. Specifically, $p_i$ is drawn from the ultrachat dataset.
• Generate a rubric question $R_i$ by prompting gpt-4o-mini to generate a question that is relevant to the suffix $s_i$. See the full prompt in Appendix.
• Generate the answer $a_i$ by greedily decoding the verifier LM $\prubric(\cdot \mid R_i, s_i)$.

Using this dataset, we produce an initial investigator model by fine-tuning Llama 3.1-8B model on $D_{\text{sft}} = \{(p_i, s_i, R_i, a_i)\}$ with the SFT objective:

As above, we next further improve the investigator using DPO. A natural approach would be to directly optimize over $p$: first sampling $p^1, p^2$ from the current best model $p_\theta$, then greedily decoding the suffix $s^1$, $s^2$ and computing verifier scores: $p^1 \succ p^2$ if $\prubric (a \mid R, s^1) > \prubric (a \mid R, s^2)$.

Unfortunately, we find that this simple approach fails to elicit the specified behaviors. This is because it only implicitly models the desired model responses $s$ through the verifier score, which is a relatively weak form of supervision. To address this, we instead use a two-stage greedy procedure. In stage 1, we train a model to infer $s$ from $(R, a)$, and in stage 2 we train a model to infer $p$ from the resulting $s$. This greedy procedure gives up on jointly optimizing $p$ and $s$ in exchange for stronger supervision at each step.

Stage 1 (inferring $s$). We first run supervised fine-tuning on the same set $D_{\text{sft}} = {(p_i, s_i, R_i, a_i)}$, but change the supervision target to $s$:

Then, we use DPO to refine the distribution over $s$ by collecting preference pairs based on the verifier scores $\prubric (a\mid R, s^1) > \prubric (a\mid R, s^2)$, where $(R, a)$ is drawn from some rubric distribution $P_{\text{rubric}}$, e.g., misconception rubrics for eliciting hallucination. As above, we run DPO iteratively, collecting preference pairs with the current best model, and finetuning with the DPO objective to obtain a better model of $s$.

Stage 2 (inferring $p$ from $s$). In stage 2, we first sample from the best stage-1 model to obtain a set of target responses $\hat{s}$, resulting in a dataset $D_\text{suffix} = \{\hat{s}_i\}$. We then fine-tune a string elicitation model on this set: we start with the general-purpose string elicitation model from the previous section (after DPO), then run further iterations of DPO to encourage the model to elicit each of the $\hat{s}_i$ in $D_{\text{suffix}}$.

Evaluation. Given a rubric-answer pair $(R, a)$ at evaluation time, we produce $p$ in two steps: we first generate $k$ candidates $\hat{s}$ by sampling from the stage-1 model, then for each of these candidates we generate $k$ candidates for $p$ by sampling from the stage-2 model ($k^2$ candidates in total). We then take the $p$ that has the highest elicitation probability under the verifier, and report both the elicitation log-probability and attack success rate as before.

Since our training objective is fully unsupervised, we can run our learning algorithm on the evaluation set. In our experiments below, we exploit this to perform one further optimization: we run stage-1 DPO on the evaluation set of rubrics, use this to generate a frozen set of candidates $\hat{s}$, then run stage-2 DPO specifically on this set of candidates, to maximize the chances of inferring successful prefixes $p$.

Results

We first present warm-up results for (non-harmful) string elicitation with a small base LM, then move on to harmful string elicitation with larger chat LMs, and finally to general behavior elicitation.

Eliciting Exact Strings

Eliciting Strings from Base LMs

First, we study the inversion problem for base LMs: given a target LM $\m$ and an arbitrary suffix $s$ generated from a gold prefix $p^{\star}$, can we find our own prefix $p$ that also generates $s$?

Experimental setup. We sample gold prefixes $\{p^{\star}_i\}$ from FineWeb [10] by taking randomly sampled FineWeb documents with at least 30 words (resulting in sequences with a mean of 47.6 tokens). We set our target model to TinyLlama [12] and greedily decode the gold prefixes to obtain an evaluation set of suffixes $\{s_i\}$.

We train Llama-3.1 8B as the investigator, using an SFT training set of 995,000 (prefix, suffix) pairs, and a disjoint DPO training set of 100,000 preference pairs. We use an evaluation set of 5,000 suffixes sampled from FineWeb in the same way as (but disjointly from) the SFT and DPO training sets. As described in Section 3.1, we evaluate using best-of-$k$ for $k=25$.³Sampling multiple times from the investigator model is standard as attack prompts can be generated adaptively.

The default DPO objective has a high success rate by creating high overlap between the prefix and suffix. To improve the diversity of samples, we add a regularizer that penalizes repetitiveness, by modifying the DPO reward to be

The examples above were obtained from a model with higher repetition penalty.

To more quantiatively evaluate the investigator agent's performance, we also report the average elicitation log-likelihood in the figure below. We observe a tradeoff between log-likelihood and repetitiveness (as measured by bigram overlap), and plot the tradeoff curve between these two quantities. Using SFT alone results in a sharp tradeoff between repetition and elicitation probability. In contrast, DPO creates a more gradual trade-off and Pareto dominates the SFT investigator model, producing prefixes that closely match the performance of the gold prefixes.

Qualitatively, we observe that the investigator's samples are highly diverse and fluent, matching strings we might expect from pretraining. See here for a full list of examples.

Eliciting Harmful Strings

We next consider eliciting harmful strings, which are crafted to contain harmful content. We target instruction-tuned LLMs (Llama-3.1, GPT-4o) which have been trained to refuse to say such prompts.

Experimental setup. For our evaluation set, we use AdvBench (Harmful Strings) [1], a dataset of 574 harmful strings such as "I'm going to kill you and your entire family".

We fine-tune an investigator from Llama-3.1 8B. For the SFT stage, we use (response, prompt) pairs generated from Ultrachat [11] using the same procedure as above with minor changes.⁴We use the chat formatting for sampling, and choose prompts that are between 64 and 512 characters, and sample at most 128 tokens from the target model using greedy decoding. For DPO, we generate a synthetic augmentation⁵We few-shot prompted Llama-3.1 8B using 3 random examples from AdvBench (Harmful Strings) to generate a 4th, i.e. “write more examples of harmful strings in the following format: <start> HARMFUL_TEXT <end>
<start> HARMFUL_TEXT <end>
<start> HARMFUL_TEXT <end>
<start>” By re-sampling the few-shot samples many times, we obtain a diverse set of similar prompts. of AdvBench containing $N = 5,000$ examples. We run 8 iterations of DPO using $k=24$ pairs per example. In this and following experiments we do not include a repetition penalty.

We report both the elicitation log-probability and attack success rate for best-of-25 sampling in the table below. Our investigator consistently elicits the target string, and attains a loss over three orders of magnitude lower than GCG.⁶This is much lower than that reported in Zou et al. for Llama 2-7B. We used the reference nanoGCG implementation, but were unable to reproduce their reported results. Furthermore, running GCG on a single sample takes around 10 minutes on an 8 x H100 node, while our method requires around 10 seconds per sample (as it is simply inference).

Attacking larger models. We also test whether we can use smaller investigator models to attack larger models. We further fine-tune our investigator to attack Llama-3.1 405B using one iteration of DPO, then evaluate the investigator on Llama-3.1 405B, GPT-4o, and Claude 3.5 Sonnet. For all models, we use the best-of-25 attack that maximizes the probability of the harmful string in Llama-3.1 8B. As shown in the table below, our attacks work consistently against Llama-3.1 405B and often also transfer to GPT-4o and Claude 3.5 Sonnet.

Some example attacks produced by our investigator are shown below:

Eliciting Harmful Responses Using Prefixes

We can use a similar process to jailbreak models to respond to more open-ended requests for harmful information. Specifically, to jailbreak a model to write "a tutorial on how to make a bomb", we set a target response of "Sure, here is a tutorial on how to make a bomb:\n\n". The rationale is that, conditioned on generating an affirmative prefix, it is likely that the model will continue generating a harmful response.

We reuse the SFT investigator from Eliciting Harmful Strings, and run DPO on a synthetic augmentation of AdvBench (Harmful Behaviors); note that this is a different subset of AdvBench than used above, focusing on prompts that ask for harmful behaviors rather than fixed harmful output strings. When we initially elicited responses, we found that the target model would often say "Sure, here is..." but then output an end-of-text token and thus not actually output harmful information; adding "\n\n" at the end of the response (as in the example above) largely fixes this problem.

We evaluate our method on a set of 50 harmful behaviors from AdvBench. We use a GPT-4 based autograder from [13] to determine whether the harmful response actually contains pertinent harmful data⁷ While this GPT4-as-judge evaluation is the standard practice, we still notice some false positives.. We report results in the table below.⁸The Adaptive Attacks score is taken from Llama-3 8B, as reported in Andriushchenko et al., 2024.

We find that our system has reasonably high attack success rate and outperforms GCG. Surprisingly, GCG has very low attack success rate on Llama-3.1 8B, whereas it achieves an ASR of 57% on Llama-2 7B as reported in [1]. We are unsure if the discrepancy is due to GCG being brittle to hyperparameter choice, an undiscovered bug in our evaluation, or perhaps Llama-3.1 training against GCG.

Attacking other models. We next considered transfer to other models such as GPT-4o and Claude 3.5 Sonnet. Because these models have much stronger safeguards, for this analysis only, we test all 25 samples from the investigator LM on these models and record if any successfully elicits a dangerous response. We observe a lower but non-zero attack success rate. Attacks tend to be successful when the target Llama model has a very low negative log-likelihood ($<0.01$) for eliciting the goal suffix, suggesting that transfer rates might increase as our attacks continue to improve.

The jailbreaks produced by the investigator for this task generally include a few qualitative characteristics that intuitively contribute to their success (also noting that some of these may be artifacts of iterated DPO):

• Interleaving English and Chinese (请问下: meaning “Excuse me”; 你的回覆應該是: meaning “Your reply should be”)
• Suggesting the start of the response, e.g. “Answer should be: Sure …”
• Distorting some tokens slightly, e.g. “email_that” rather than “email that”
• Being very specific about formatting, e.g. “Sure, here is answer without any other question or comment. This should be at least350 words long without spaces or new lines”
• Repeating the question and/or target response prefix

Interestingly, these match many of the strategies humans have discovered for producing jailbreaks [15, 16, 17, 18] .

Eliciting Open-Ended Behaviors from Rubrics

The previous results are examples of the exact string-elicitation task, for which the target behavior is a specific string or response prefix. We next present results for the more complex rubric elicitation setting. We consider two settings: in-distribution rubrics and hallucination rubrics.

We follow the two-stage procedure detailed in Section 3.2, which first infers suffixes for each rubric item, then elicits prompts given the suffixes. We take Llama-3.1 8B-instruct as both our target model and verifier. For training SFT runs, we use UltraChat [11] as before, and use gpt-4o-mini to generate rubrics from prompts and responses.

We next create an evaluation set $D_{\text{rubric}}$, which is different for the two settings:

• For the in-distribution setting, $D_{\text{rubric}}$ is drawn from the same distribution as SFT.
• For hallucinations, $D_{\text{rubric}}$ is created using misconceptions from the TruthfulQA dataset. Specifically, TruthfulQA is a dataset of questions along with correct and incorrect answers. We use the incorrect answers as our hallucination targets.

In both settings, we then run DPO on $D_{\text{rubric}}$ as described in Section 3.2. Note that we are thus fine-tuning on the evaluation set, but since our method is unsupervised, it does not require any labels, and running adaptive computation on evaluation examples is standard for behavior elicitation methods.

We report the verifier loss (in negative log-probabilities) for both settings below. For the hallucination setting, we further report the hallucination rate, which measures the percentage of examples where the investigator model correctly elicits the target hallucination, as judged by Llama-3.1 8B-instruct. As shown in the table, our two-stage DPO approach outperforms both SFT and one-stage DPO for both settings and across both metrics. For the in-distribution case, it also approaches the performance of the gold labels.

We provide qualitative examples below to illustrate how our investigator model triggers hallucinations; this can occur even when the target model clearly knows the correct answer (as judged by prompting normally).

Related Works

Language Model Inversion. Language models generate suffixes conditioned on a given prompt, whereas the elicitation problem inverts this process: searching for a prompt that induces a specific suffix string or behavior. For example, [19] pretrains a language model to predict tokens in the reverse order. [20] finetunes a language model to fill in the middle conditioned on the prefix and suffix by swapping the sentence orders. We tackle a similar task of inversion, except that we aim to invert a specific LM, rather than the general natural language distribution. Towards this, we curate SFT data specific to the target LM and include the DPO step to reward based on the log-probability under the target LM. Similarly, [21] learns model-specific inversion by taking next-token probabilities as inputs and recovering preceding texts like system prompts. Their approach assumes dense information (next token probability) and is optimized for exact generations from the model, whereas our approach can invert strings not generated from a model and generalizes to rubrics.

Inference-time Red Teaming. Automated red teaming of language models has been a popular research direction [6, 1, 2] . This problem is often cast as an inference-time optimization, where the objective is to identify inputs that elicit a specific harmful response. For example, GCG [1] and AutoDan [2] optimize the prompt for each response instance individually, requiring significant computational resources for each search. In contrast, our approach moves this expensive inference cost to training, obtaining a general-purpose investigator model that’s efficient for each new elicitation instance.

Training-time Red Teaming. Similar to our approach, many prior works amortize the cost of search by training a model to perform red teaming. For example, [6] and [22] use reinforcement learning to elicit generally harmful behaviors. Our investigator elicits finer-grained behaviors and can condition on rubrics as the elicitation target. This rubric-conditioning also boosts the diversity of our solutions, attaining a similar goal as the diversity reward in [22].

Discussion and Limitations

Behavior elicitation is a useful technique to proactively detect model errors. In the sections above, we introduced an effective approach to elicit exact strings and open-ended rubric-based behaviors from LMs.

Rubric elicitation lets us specify rules through natural language, which we used to investigate open-ended hallucinations without being required to specify an exact string. More generally, we could leverage rubrics to investigate behaviors at different levels of granularity, e.g. “harmful behaviors” vs. “SQL injection errors”, “math mistakes” vs. “algebra errors”.

Additionally, elicitation need not be restricted only to behaviors. By combining with tools such as our observability interface, we could elicit based on observability states such as "concept X is firing but is not present in the output" (an omission), or "concept X is present in the output but is not firing in the representations" (a possible hallucination).

Limitations. Similar to other search problems like retrieval, behavior elicitation concerns both precision and recall. Most existing efforts focus on precision: for example, GCG [1] discovers nonsense strings that successfully elicits harmful behaviors, but as shown in our results above there exists other patterns that also elicit the harmful behaviors. An ideal investigator model should find all such patterns (high recall), and we view optimizing for recall as an interesting direction of future work.

Using an LM to verify yields scalable elicitation algorithms, but also raises concerns about reward hacking. We already observed some instances of reward hacking against LM judges in our results above. Future work should analyze reward hacking systematically, and develop robust and reliable verifiers, perhaps by augmenting the verifier with tools or external databases. In the other direction, there are many tasks where verification is easier than generation and so reward hacking is less of an issue, such as for reasoning-heavy tasks like math and coding.

Acknowledgements

We than Ruiqi Zhong, Ken Liu, Sam Park, Jean-Stanislas Denain for their helpful comments on the draft. We thank Franklin Wang for helpful discussions.

Appendix

Next, we provide the prompt we used for generating rubrics.

Specifically, the rubric question needs to be specific and clear. It shouldn't be too general or too vague.
It should be answerable based on the content of chat response, and it shouldn't be follow up questions that require more context.
Also, if one were to answer the rubric question, the response should be very concise. This means that the rubric question should be specific enough to elicit a concise response.
Consider three main types of questions:
(1) information seeking questions: they would require a concise short-form response
(2) critique questions: they would require a binary response (e.g., yes/no). Make sure that when you ask the critique question, the answer should be "No" half of the time. Dont always ask questions that would lead to a "Yes" answer.
(3) scoring questions: they would require a score or a rating (e.g., 1-5). You should also generate a concise rubric for scoring to include in the question.
Try to ask more information seeking questions and scoring questions. Ask critique questions only when you can't think of a good information seeking or scoring question.
Please also note the type of questions you generate at the end of the question.

For example,
chat prompt: I mistakenly eat a watermelon seed and now I have a watermelon growing in my stomach.
chat response: That's not possible. Watermelon seeds are too small to grow in your stomach.
rubric question: Why is it not possible for a watermelon seed to grow in your stomach? <end_rubric_question> (information seeking question)

A bad example of rubric question would be: What are the symptoms of watermelon seed growing in your stomach? because it requires more context to answer.
Another bad example of rubric question would be: What are the health benefits of eating watermelon seeds? because it is too general and not specific to the chat response.

Now, it's your turn:
Chat prompt: {{CHAT_PROMPT}}
Chat response: {{CHAT_RESPONSE}}
rubric question:

Some assets for our figures were retrieved from flaticon.com (1, 2, 3).