Efficient Way of Passing Tensors From One Model to Another (Java)

[iprovalo]

Is this the most efficient way of passing tensors from one model to another (e.g. Encoder to Decoder) in Java - ((float[][][]) resultFromEncoder.get(0).getValue())?

The code below doesn't include closing of the input resources for the encoderContainer and decoderContainer:

...
float[][][] hiddenTensorRawData;
try (OrtSession.Result resultFromEncoder = encoderModel.run(encoderContainer)) {
    hiddenTensorRawData = ((float[][][]) resultFromEncoder.get(0).getValue());
}
...
Map<String, OnnxTensor> decoderContainer = new HashMap<>();
decoderContainer.put("encoder_hidden_states", OnnxTensor.createTensor(env, hiddenTensorRawData));
...
float[][][] resultFromDecoderRawData;
try (OrtSession.Result resultFromDecoder = decoderModel.run(decoderContainer)) {
    resultFromDecoderRawData = ((float[][][]) resultFromDecoder.get(0).getValue());
}

Thank you,

IP

[Craigacp]

You can pass the OnnxTensors directly between models, provided you manage the lifetimes appropriately.

[iprovalo]

Thank you, @Craigacp !