Skip to content

Commit c1cab7a

Browse files
Update main README for 0.9.0 release (#1660)
### Description This PR updates the repo's main README about the new way to set inputs for non-LLMs. ### Motivation and Context This will make users aware of an upcoming breaking change for the next release (0.9.0).
1 parent bfa9e11 commit c1cab7a

File tree

1 file changed

+1
-7
lines changed

1 file changed

+1
-7
lines changed

README.md

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,6 @@
11
# ONNX Runtime GenAI
22

3-
Note: between release candidate 0.7.0-rc2 and release 0.7.0 there is a breaking Python API change in `tokenizer.encode(prompt)`. Previously this method returned a Python list and now returns a numpy array. When concatenating the tokens generated by two prompts to pass to `append_tokens` e.g a system prompt and a user prompt, you must use the following instead of `system_prompt + input_tokens`:
4-
5-
```python
6-
system_tokens = tokenizer.encode(system_prompt)
7-
input_tokens = tokenizer.encode(prompt)
8-
generator.append_tokens(np.concatenate([system_tokens, input_tokens]))
9-
```
3+
Note: between 0.9.0 and the 0.8.3 release, there is a breaking API change. Previously, the inputs for non-LLMs would be set with `params.SetInputs(inputs)`. Now, inputs for non-LLMs are set with `generator.SetInputs(inputs)`. With this change, inputs for all models and their modalities are set on the `generator` instead of the `generatorParams`. The inputs for LLMs are set with `generator.append_tokens(tokens)` and the inputs for non-LLMs are set with `generator.SetInputs(inputs)`.
104

115
[![Latest version](https://img.shields.io/nuget/vpre/Microsoft.ML.OnnxRuntimeGenAI.Managed?label=latest)](https://www.nuget.org/packages/Microsoft.ML.OnnxRuntimeGenAI.Managed/absoluteLatest)
126

0 commit comments

Comments
 (0)