Can the fine-tuned model be used for inference in a Python 3.10 environment？

[wangwenmingaa]

When using the axolotl framework for model fine-tuning, a Python 3.11 environment is required. Is it necessary to use Python 3.11 for inference with the fine-tuned model as well? Can I use a Python 3.10 environment instead for inference?

[NanoCode012]

@wangwenmingaa , you can use other tools like vllm for inference. They're more open to other py versions. Our inference is just for sanity testing. Py310 should work, last I tested.

The reason we limit to Py311 was to reduce our test scope and make it simpler for us to focus on other features.