Replies: 1 comment
-
Iris_ONNX_Inference_Sample.zip Code snippet attached here. ONNX model inferred from Python as well as from C++ module. Please help me understand why results from these 2 Inferences are not matching. Thanks and Regards, |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
My ONNX Model has been built on float pointing numbers, where number accuracy is of high importance upto 10 decimal precision .
ONNX model input is array of float numbers and output is label.
When I am inferring this ONNX model in C++ using 'Microsoft.ML.OnnxRuntime\1.13.1\runtimes', for same set of input, output label keeps differing. But when I infer same ONNX model from python, label output is consistent.
C++ code snippet:
int64_t* pShape = mInputDims.data();
size_t shape_len = mInputDims.size();
std::vector inputTensorValues(inputTensorSize);
inputTensorValues.assign(inputValues.begin(), inputValues.end()); //inputValues means std::vector
// Create input tensors of ORT::Value, which is a tensor format used by ONNX Runtime
Ort::MemoryInfo memoryInfo = Ort::MemoryInfo::CreateCpu(OrtAllocatorType::OrtArenaAllocator,
OrtMemType::OrtMemTypeDefault);
inputTensors.push_back(Ort::Value::CreateTensor(memoryInfo, inputTensorValues.data(),
inputTensorSize, pShape, shape_len));
// Create output tensor buffer
mOutputDims[0] = 1;
size_t outputTensorSize = vectorProduct(mOutputDims);
std::vector<int64_t> outputTensorValues(outputTensorSize);
// Create output tensors of ORT::Value
outputTensors.push_back(Ort::Value::CreateTensor<int64_t>(memoryInfo, outputTensorValues.data(), outputTensorSize,
mOutputDims.data(), mOutputDims.size()));
session->Run(Ort::RunOptions{ nullptr }, inputNames.data(), inputTensors.data(), 1,
outputNames.data(), outputTensors.data(), 1);
// Get the inference result from Labels list; count = LabelList's size
int64_t* outArr = outputTensors.front().GetTensorMutableData<int64_t>();
int64_t cls_idx = std::max_element(outArr, outArr + count) - outArr;
m_label = labels[cls_idx];
what can be the reason for this inconsistency of prediction?
In python float value is of 64 bit and in C++ float value is 8-bit. Can this data loss be the reason of inconsistency? How to deal with it?
Thanks in advance,
Best Regards,
Veena
Beta Was this translation helpful? Give feedback.
All reactions