You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Description
Currently the user can limit the prompt input size and model output size only when using emulated data type.
Please separate the input & output sizes configuration from the type of the data.
Suggested Implementation
Add --data-max-input-tokens and --max-output-tokens parameters to the CLI.
If the data-type is file or transformers and the input size is larger than data_max_input_tokens it should cut-off the input at the limit.