Ch 11 - 3.5 (SML Hugging Face) - Deprecated method used in the onnxruntime-genai library

[issambhd]

Error #1:

Need to create the generator first, then use generator.append_tokens() to feed in the prompt.
The old way of setting params.input_ids is deprecated and no longer supported.

New updated code :
params = og.GeneratorParams(model)
params.set_search_options(**search_options)

generator = og.Generator(model, params)

prompt = "<|system|>You are a famous chess player.<|end|><|user|>Could you make a comparison between chess and football ? Imagine a storyline in french :)<|end|><|assistant|>"
input_tokens = tokenizer.encode(prompt)

generator.append_tokens(input_tokens)

Error #2:
The method compute_logits() has been removed in recent versions (0.6.0+), and the generation flow has been simplified.

New updated code :

while not generator.is_done():
generator.generate_next_token()
new_token = generator.get_next_tokens()[0]
print(tokenizer_stream.decode(new_token), end='', flush=True)