Skip to content

Commit b90cdce

Browse files
committed
docs: rewording
Signed-off-by: Ettore Di Giacinto <[email protected]>
1 parent fc3502b commit b90cdce

File tree

1 file changed

+11
-8
lines changed

1 file changed

+11
-8
lines changed
Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,27 @@
1-
21
+++
32
disableToc = false
4-
title = "✍️ Constrained grammars"
3+
title = "✍️ Constrained Grammars"
54
weight = 15
65
url = "/features/constrained_grammars/"
76
+++
87

9-
The chat endpoint accepts an additional `grammar` parameter which takes a [BNF defined grammar](https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_form).
8+
## Overview
109

11-
This allows the LLM to constrain the output to a user-defined schema, allowing to generate `JSON`, `YAML`, and everything that can be defined with a BNF grammar.
10+
The `chat` endpoint supports the `grammar` parameter, which allows users to specify a grammar in Backus-Naur Form (BNF). This feature enables the Large Language Model (LLM) to generate outputs adhering to a user-defined schema, such as `JSON`, `YAML`, or any other format that can be defined using BNF. For more details about BNF, see [Backus-Naur Form on Wikipedia](https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_form).
1211

1312
{{% alert note %}}
14-
This feature works only with models compatible with the [llama.cpp](https://github.com/ggerganov/llama.cpp) backend (see also [Model compatibility]({{%relref "docs/reference/compatibility-table" %}})). For details on how it works, see the upstream PRs: https://github.com/ggerganov/llama.cpp/pull/1773, https://github.com/ggerganov/llama.cpp/pull/1887
13+
**Compatibility Notice:** This feature is only supported by models that use the [llama.cpp](https://github.com/ggerganov/llama.cpp) backend. For a complete list of compatible models, refer to the [Model Compatibility](docs/reference/compatibility-table) page. For technical details, see the related pull requests: [PR #1773](https://github.com/ggerganov/llama.cpp/pull/1773) and [PR #1887](https://github.com/ggerganov/llama.cpp/pull/1887).
1514
{{% /alert %}}
1615

1716
## Setup
1817

19-
Follow the setup instructions from the [LocalAI functions]({{%relref "docs/features/openai-functions" %}}) page.
18+
To use this feature, follow the installation and setup instructions on the [LocalAI Functions](docs/features/openai-functions) page. Ensure that your local setup meets all the prerequisites specified for the llama.cpp backend.
19+
20+
## 💡 Usage Example
2021

21-
## 💡 Usage example
22+
The following example demonstrates how to use the `grammar` parameter to constrain the model's output to either "yes" or "no". This can be particularly useful in scenarios where the response format needs to be strictly controlled.
2223

23-
For example, to constrain the output to either `yes`, `no`:
24+
### Example: Binary Response Constraint
2425

2526
```bash
2627
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
@@ -29,3 +30,5 @@ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/jso
2930
"grammar": "root ::= (\"yes\" | \"no\")"
3031
}'
3132
```
33+
34+
In this example, the `grammar` parameter is set to a simple choice between "yes" and "no", ensuring that the model's response adheres strictly to one of these options regardless of the context.

0 commit comments

Comments
 (0)