How to load a model in bf16 format

[georgel2004]

In turbomind.py the data_type is hard coded to fp16.
If I change the value to bf16 I could see increase in performance speed.
How could I load a model in bf16 format using the pipeline?

[lvhan028]

turbomind doesn't hard code data_type fp16. It supports bf16 if torch_dtype in model's config.json is bf16 and the target device supports bf16 as well

lmdeploy/lmdeploy/turbomind/turbomind.py

Line 271 in 7c4e75b

data_type = output_format