Skip to content

Commit 5501c5e

Browse files
Models supports arbitrary variable names in compare mode (#56149)
1 parent 3ec7106 commit 5501c5e

File tree

1 file changed

+9
-4
lines changed

1 file changed

+9
-4
lines changed

content/github-models/use-github-models/evaluating-ai-models.md

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -62,17 +62,22 @@ The model may generate a response like the following:
6262

6363
The user experiences consistent app crashes when attempting to upload a PDF from their phone. Uploading PDFs works normally on desktop. They request an investigation into the issue.
6464

65-
### Defining the input variable
65+
### Using variables in prompts
66+
67+
> [!NOTE]
68+
> This feature is currently in {% data variables.release-phases.public_preview %} and subject to change.
6669
6770
At this point, the configuration generates a clear and concise summary. At the bottom of the **Parameters** settings, click **Create prompt.yml file** to open the Prompt view. Your system prompt will be prefilled automatically.
6871

69-
In the **User prompt** field, enter the following prompt:
72+
In the **User prompt** field, enter a prompt that includes one or more variable placeholders inside double curly braces. For example:
7073

7174
```text
72-
Summarize the following: {% raw %}{{input}}{% endraw %}
75+
Travel or shopping assistants using {% raw %}{{city}}{% endraw %}, {% raw %}{{intent}}{% endraw %}, and {% raw %}{{budget}}{% endraw %} to tailor recommendations.
7376
```
7477

75-
The {% raw %}`{{input}}`{% endraw %} variable acts as a placeholder that lets you test different inputs (customer feedback) without modifying the prompt each time. Each test input you add will replace {% raw %}`{{input}}`{% endraw %} when the comparison runs.
78+
Each variable listed in the prompt will appear as a parameter in compare mode. When running an evaluation, you'll be prompted to provide values for each variable. This allows prompts to be reused with different inputs without modifying the prompt content.
79+
80+
Alternatively, you can add variables in your `.prompt.yml` file system or user prompt to automate the process of evaluating with multiple variables in the future. See [AUTOTITLE](/github-models/use-github-models/storing-prompts-in-github-repositories).
7681

7782
## Adding test inputs
7883

0 commit comments

Comments
 (0)