Building onnxruntime on AIX

[tvkai]

Hi,
Actually i attempted the build of onnxruntime on AIX which is BIG endian, the build process was failing because of a static assert ..

./build.sh --config RelWithDebInfo --build_shared_lib
[ 22%] Building CXX object CMakeFiles/onnxruntime_session.dir/ONNX/onnxruntime/onnxruntime/core/session/inference_session.cc.o
In file included from /ONNX/onnxruntime/cmake/external/flatbuffers/include/flatbuffers/flatbuffers.h:20,
from /ONNX/onnxruntime/onnxruntime/core/flatbuffers/schema/ort.fbs.h:7,
from /ONNX/onnxruntime/onnxruntime/core/framework/session_state_flatbuffers_utils.h:9, |
from /ONNX/onnxruntime/onnxruntime/core/session/inference_session.cc:28: |
/ONNX/onnxruntime/onnxruntime/core/session/inference_session.cc: In member function 'onnxruntime::common::Status onnxruntime::InferenceSession::LoadOrtModel(std::function<onnxruntime::|
common::Status()>)': |
/ONNX/onnxruntime/cmake/external/flatbuffers/include/flatbuffers/base.h:127:40: error: static assertion failed: ORT format only supports little-endian machines |
#define FLATBUFFERS_LITTLEENDIAN 0 |
^ |
/ONNX/onnxruntime/onnxruntime/core/session/inference_session.cc:1014:17: note: in expansion of macro 'FLATBUFFERS_LITTLEENDIAN' |
static_assert(FLATBUFFERS_LITTLEENDIAN, "ORT format only supports little-endian machines"); <=== |
^~~~~~~~~~~~~~~~~~~~~~~~ |
make: 1254-004 The error code from the last command is 1. |

Stop.

looking at the code it is for storing the models in 'ort' format, can some one help me understand more on this check,
it seems FLATBUFFERS already supported on Big Endian. Also is there any way in the build where i can exclude this
'ort' format support ?.

make: 1254-004 The error code from the last command is 2.

[RajalakshmiSR]

@snnn Do you have any suggestions on big-endian support ?

[tvkai]

@snn

Actually i try to test below changes here, and I observed many tests are passing.

The file is :
onnxruntime/test/providers/provider_test_utils.cc

void OpTester::AddInitializers(onnxruntime::Graph& graph) {
for (auto index : initializer_index_) {
auto& data = input_data_[index];
auto& tensor = data.data_.Get();
ONNX_NAMESPACE::TensorProto tensor_proto;
// 1. set dimension
auto& shape = tensor.Shape();
for (auto& dim : shape.GetDims()) {
tensor_proto.add_dims(dim);
}
// 2. set type
tensor_proto.set_data_type(
data.def_.TypeAsProto()->tensor_type().elem_type());
// 3. data
if (data.def_.TypeAsProto()->tensor_type().elem_type() ==
ONNX_NAMESPACE::TensorProto_DataType_STRING) {
const std::string* string_data = tensor.Datastd::string();
for (auto i = 0; i < shape.Size(); i++) {
tensor_proto.add_string_data(string_data[i]);
}
} else {
auto buffer_size = tensor.DataType()->Size() * shape.Size();
char bytes = (char)tensor.DataRaw();
/onnx is little endian serialized always-tweak byte order if needed/
if (1) { // Todo: need to introduce flag check which checks whether the machine is BIG endian or LE.
std::cout<<"Doing byte swapping in Add Initializer provider_test_utils.cc"<<std::endl;
const size_t element_size = tensor.DataType()->Size();
const size_t num_elements = shape.Size();
for (size_t i = 0; i < num_elements; ++i) {
char start_byte = bytes + i * element_size;
char end_byte = start_byte + element_size - 1;
/* keep swapping */
for (size_t count = 0; count < element_size / 2; ++count) {
char temp = start_byte;
start_byte = end_byte;
end_byte = temp;
++start_byte;
--end_byte;
}
}
}
tensor_proto.set_raw_data(tensor.DataRaw(), buffer_size);
}
// 4. name
tensor_proto.set_name(data.def_.Name());
graph.AddInitializedTensor(tensor_proto);
}
}

With this change many tests are succeeding. But i see still some tests are failing, i think it is due to libonnxruntime internally creating initializers .. any pointers on this ? .

[tvkai]

Any idea on this ??. @snnn

[tvkai]

@snnn Basically i see the code is trying to keep the raw data in the tensorproto as Little Endian. and I see we are using UnpackTensor() everywhere whereever we are trying to refer to the tensorproto, Instead of that can we convert to big
endian these during the load from the model file.

[madurais]

I see from the code when doing UnpackTensor with Raw Data, (onnxruntime/core/framework/tensorprotoutils.cc) UnpackTensorWithRawDataImpl(), ReadLittleEndian()->CopyLittleEndian() is called where endianness is changed for Tensor with Raw Data on Big Endian systems. This is applicable for Initialized Tensors loaded from File. But when Tensor data is added from code from test framework, or from onnxruntime core path, the raw data is expected to be in Little Endian. This requires all testcase or applications to modify code to have input data in LittleEndian.

For ONNX models the graph is loaded in onnxruntime/core/graph/graph.cc, and for ORT models from onnxruntime/core/session/inference_session.cc. Initialize of the session happens in inference_session.cc for either case. If that code is updated to convert all Little Endian Raw data to big endian on bid endian system, then UnpackTensor need not work on changing Endianness. This avoids changing the testcases based on Endianness of the system.

Please let us know if I miss something. Thanks

[tvkai]

@snnn any suggestions and what do you think on this ?

[tvkai]

@tianleiwu @snnn @skottmckay @edgchen1 any thoughts on this discussion ?

[skottmckay]

If there a use case where you'd want/need to run a minimal build (reduced binary size) on AIX? That's the only setup where an ORT format model is expected to be used.

If not, making the static assert a runtime check may be all that's needed.