Support bfloat16 dtype in model builder

[BowenBao]

Hi team, is there plan to support building bfloat16 models from model builder? cc @kunal-vaishnavi

[kunal-vaishnavi]

Yes, we want to support building bfloat16 models from the model builder. We will need to revisit how the weights are saved because they are currently converted to NumPy arrays and NumPy doesn't support bfloat16 precision currently.

onnxruntime-genai/src/python/py/models/builder.py

Lines 406 to 417 in e41fb2c

	def make_external_tensor(self, np_data, name, **kwargs):
	tensor = numpy_helper.from_array(np_data)
	tensor.name = name
	filename = f"{name}.bin"
	external_data_helper.set_external_data(tensor, location=filename)
	with open(os.path.join(self.cache_dir, filename), "wb") as f:
	f.write(tensor.raw_data)
	tensor.ClearField("raw_data")
	tensor.data_location = TensorProto.EXTERNAL
	self.initializers.append(tensor)

[BowenBao]

Thanks Kunal, I'm happy to contribute to this, will be sending in a pr soon.

[kunal-vaishnavi]

Here is the PR for bfloat16 support in the model builder.