Is it necessary to support new types of tensor?

[loloxwg]

The input and output of reasoning frameworks are typically tensor types, but I didn't find this type in SurrealDB.

[maxwellflitton]

@loloxwg hey thanks for looking into this package. We are coming close to a big release with updates. In terms of the return types, we need the package to gel with the SQL language and other languages such as Python. It's possible that we can build an option to return tensors. What is your use case for surrealml? I can see if I can accommodate it.

[loloxwg]

I plan to start by using it on the MNIST dataset for digit recognition. @maxwellflitton

[loloxwg]

I want the input type for this model to be a tensor with the shape [1, 1, 28, 28].

[slygren]

@loloxwg hey thanks for looking into this package. We are coming close to a big release with updates. In terms of the return types, we need the package to gel with the SQL language and other languages such as Python. It's possible that we can build an option to return tensors. What is your use case for surrealml? I can see if I can accommodate it.

@maxwellflitton - do you have any timeline on the update?

[slygren]

I have a somewhat similar request - allowing for input to be of different types than pure scalars. For my use case where I'm doing state recognition of timeseries data stored in SurrealDB, I perform GAF conversion of a time series section to create images (shape (1, 3, 64, 64)) and want to pass them through a pytorch/onnx/surml model.

Unfortunately, surml does not allow inputs of required type & shape

This works

import onnxruntime as ort

model = torch.load('./cnn_model.pth', map_location=torch.device('cpu'))
model.eval()

test_input = torch.rand(64, 64, 3, dtype=torch.float32) * 2 - 1
test_input = test_input.permute(2, 0, 1).unsqueeze(0)
onnx_file_path = './cnn_model.onnx'

torch.onnx.export(
    model,
    test_input,
    onnx_file_path,
    export_params=True,
    opset_version=18,
    do_constant_folding=True,
    input_names=["input"],
    output_names=["output"],
    dynamic_axes={
        "input": {0: "batch_size"},
        "output": {0: "batch_size"}
    }
)

res = await con.query('SELECT VALUE value FROM timeserie_data:["ts_1", time::now()-3h]..=["ts_1", time::now()-2h]')
downscaled = downScale(np.array(res))
img = getAndMakeData(downscaled)
shaped_img = np.expand_dims(np.transpose(img, (2, 0, 1)), axis=0)

ort_session = ort.InferenceSession(onnx_file_path)
input_data = shaped_img.astype(np.float32)
outputs = ort_session.run(None, {"input": input_data})
print("Model output:", outputs)

while this fails

file = SurMlFile(model=onnx_model, name="tsPeriodState", engine=Engine.ONNX)
file.raw_compute(shaped_img.astype(np.float32))

with the error

TypeError: argument 'input_vector': only length-1 arrays can be converted to Python scalars

[maxwellflitton]

@slygren I think the conversion is down to PyO3 and we're ripping this out and going for raw C bindings. @loloxwg I'm just focusing on the Python client this week and then will get back on the surrealML and work out what the estimate is.

[maxwellflitton]

Hey @slygren and @loloxwg thanks for being patient. I have now finished the major update for surrealML so we now have a dynamic C lib for the core of the ML engine. The python client is now working with this build approach so we can also link other clients to the dynamic C lib such as JavaScript etc. I'll now look into this. I've done a simple test and it's breaking. In the rust code I inspect the ONNX model and try to reshape based off this but it looks like it literally tries to reshape the tensor exactly like the input dims as opposed to multiples of this. Thanks for raising it and I should be able to solve this over this week