[circle-mlir] Upgrade ONNX-MLIR and other externals

[seanshpark]

Let's upgrade ONNX-MLIR and other externals projects to latest available.

Steps to do

• check with latest version (or latest release version)
• prepare full working change
• post a PR (this will fail with current CI Docker image)
• upgrade Docker image (PR should pass with this)
• land PR

[seanshpark]

ONNX-MLIR, 0.5.0.0 branch is available, release is not yet. let's check with this branch.

• onnx-mlir is 660bd8efa664bd2d05801a7a793fe29925313b4b
• in onnx-mlir/utils/clone-mlir.sh file , we can get commit hash of `llvm-project
• it's b270525f730be6e7196667925f5a9bfa153262e9

onnx-mlir

$ git submodule update --init --recursive
Submodule 'third_party/stablehlo' (https://github.com/openxla/stablehlo.git) registered for path 'third_party/stablehlo'
Cloning into '/home/.../circle-mlir/externals/onnx-mlir/third_party/stablehlo'...
Submodule path 'third_party/benchmark': checked out 'a4cf155615c63e019ae549e31703bf367df5b471'
warning: unable to rmdir 'third_party/benchmark': Directory not empty
Submodule path 'third_party/onnx': checked out 'b8baa8446686496da4cc8fda09f2b6fe65c2a02c'
Submodule path 'third_party/onnx/third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b'
Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b'
Submodule path 'third_party/rapidcheck': checked out 'ff6af6fc683159deb51c543b065eba14dfcf329b'
Submodule path 'third_party/stablehlo': checked out '459e481b77f8537aae3f8e8e8ad9550721afe202'

• third_party/mlir-hlo is no more
• third_party/onnx/third_pary/benchmark is also no more
• from third_party/onnx/README.md file, protobuf is v21.12, same as before

[seanshpark]

MLIR code changes

CIR_Op has hasOptions2
-> these are StableHLO
-> ignore for now

changes from value.func<TYPE>() to mlir::func<TYPE>(value)

• cast
• dyn_cast
• dyn_cast_or_null
• isa

use of deleted function ‘mlir::OpOperand::OpOperand(const mlir::OpOperand&)’
from auto pad_mutable = mirrorpad_op.getPadMutable();

• --> auto &pad_mutable = resize_op.getPadMutable();

[seanshpark]

Conversion failures

Conv2d segment fault in shape inference

• Conv2DOpAdaptor op(operands, attr);
• --> Conv2DOpAdaptor op(operands, attr, properties);

[seanshpark]

Conversion failures

ConvTranspose2d_F32_R4

loc("/op/ConvTranspose"): error: failed to legalize operation 'onnx.ReverseSequence'
// -----// IR Dump After mlir::Circle::ConvertONNXToCirclePass Failed (onnx-to-circle) //----- //
func.func @main_graph(%arg0: tensor<1x2x3x3xf32> {onnx.name = "onnx::ConvTranspose_0"}) -> (tensor<1x2x3x3xf32> {onnx.name = "3"}) {
  %0 = "onnx.NoValue"() {value} : () -> none
  %1 = onnx.Constant dense<0> : tensor<8xi64>
  %2 = onnx.Constant dense<1> : tensor<1xi64>
  %3 = onnx.Constant dense<[[[[0.507964373]], [[5.765850e-01]]], [[[0.457271814]], [[-0.339164436]]]]> : tensor<2x2x1x1xf32>
  %4 = onnx.Constant dense<[0.446791559, 0.0622497983]> : tensor<2xf32>
  %5 = "onnx.Transpose"(%3) {perm = [2, 3, 0, 1]} : (tensor<2x2x1x1xf32>) -> tensor<1x1x2x2xf32>
  %6 = "onnx.ReverseSequence"(%5, %2) {batch_axis = 1 : si64, time_axis = 0 : si64} : (tensor<1x1x2x2xf32>, tensor<1xi64>) -> tensor<1x1x2x2xf32>
  %7 = "onnx.ReverseSequence"(%6, %2) {batch_axis = 0 : si64, time_axis = 1 : si64} : (tensor<1x1x2x2xf32>, tensor<1xi64>) -> tensor<1x1x2x2xf32>
  %8 = "onnx.Transpose"(%7) {perm = [2, 3, 0, 1]} : (tensor<1x1x2x2xf32>) -> tensor<2x2x1x1xf32>
  %9 = "onnx.Transpose"(%8) {perm = [1, 0, 2, 3]} : (tensor<2x2x1x1xf32>) -> tensor<2x2x1x1xf32>
  %10 = "onnx.Conv"(%arg0, %9, %4) {auto_pad = "NOTSET", dilations = [1, 1], group = 1 : si64, kernel_shape = [1, 1], pads = [0, 0, 0, 0], strides = [1, 1]} : (tensor<1x2x3x3xf32>, tensor<2x2x1x1xf32>, tensor<2xf32>) -> tensor<1x2x3x3xf3
2>
  %11 = "onnx.Pad"(%10, %1, %0, %0) {mode = "constant"} : (tensor<1x2x3x3xf32>, tensor<8xi64>, none, none) -> tensor<1x2x3x3xf32>
  %12 = "onnx.Pad"(%11, %1, %0, %0) {mode = "constant"} : (tensor<1x2x3x3xf32>, tensor<8xi64>, none, none) -> tensor<1x2x3x3xf32>
  return %12 : tensor<1x2x3x3xf32>
}

before update

func.func @main_graph(%arg0: tensor<1x2x3x3xf32>) -> tensor<1x2x3x3xf32> attributes {input_names = ["onnx::ConvTranspose_0"], output_names = ["3"]} {
  %0 = onnx.Constant dense<[[[[0.32993716]], [[0.293203473]]], [[[-0.614819228]], [[-0.411144763]]]]> : tensor<2x2x1x1xf32>
  %1 = onnx.Constant dense<[-0.0109079415, -0.527520835]> : tensor<2xf32>
  %2 = "onnx.ConvTranspose"(%arg0, %0, %1) {auto_pad = "NOTSET", dilations = [1, 1], group = 1 : si64, kernel_shape = [1, 1], onnx_node_name = "/op/ConvTranspose", pads = [0, 0, 0, 0], strides = [1, 1]} : (tensor<1x2x3x3xf32>,
tensor<2x2x1x1xf32>, tensor<2xf32>) -> tensor<1x2x3x3xf32>
  return %2 : tensor<1x2x3x3xf32>
}

DecomposeONNXToONNXPass has

#ifdef ONNX_MLIR_ENABLE_STABLEHLO
  // ONNXtoStablehlo pass has own rewriting for ConvTranspose Op using
  // stablehlo ops. To avoid conflict with it, decomposing for ConvTranspose
  // is disabled when the target is stablehlo.
  if (this->target != "stablehlo") {
#endif
    target.addDynamicallyLegalOp<ONNXConvTransposeOp>(
        [](ONNXConvTransposeOp op) {
          return !onnx_mlir::shouldDecomposeConvTransposeOp(op);
        });
#ifdef ONNX_MLIR_ENABLE_STABLEHLO
  }
#endif

• ONNX_MLIR_DECOMP_ONNX_CONVTRANSPOSE is not available

[seanshpark]

Conversion failures

loc("/op/InstanceNormalization"): error: failed to legalize operation 'onnx.LayerNormalization'
// -----// IR Dump After mlir::Circle::ConvertONNXToCirclePass Failed (onnx-to-circle) //----- //
func.func @main_graph(%arg0: tensor<1x2x3x3xf32> {onnx.name = "input"}) -> (tensor<1x2x3x3xf32> {onnx.name = "3"}) {
  %0 = onnx.Constant dense<1.000000e+00> : tensor<2xf32>
  %1 = onnx.Constant dense<0.000000e+00> : tensor<2xf32>
  %2 = onnx.Constant dense<[1, 2]> : tensor<2xi64>
  %3 = "onnx.Unsqueeze"(%0, %2) : (tensor<2xf32>, tensor<2xi64>) -> tensor<2x1x1xf32>
  %4 = "onnx.Unsqueeze"(%1, %2) : (tensor<2xf32>, tensor<2xi64>) -> tensor<2x1x1xf32>
  %Y, %Mean, %InvStdDev = "onnx.LayerNormalization"(%arg0, %3, %4) {axis = 2 : si64, epsilon = 9.99999974E-6 : f32, stash_type = 1 : si64} : (tensor<1x2x3x3xf32>, tensor<2x1x1xf32>, tensor<2x1x1xf32>) -> (tensor<1x2x3x3xf32>, none, none)
  return %Y : tensor<1x2x3x3xf32>
}

gmake[3]: *** [circle-mlir/tools-test/onnx2circle-models/CMakeFiles/onnx2circle_models_target.dir/build.make:699: circle-mlir/tools-test/onnx2circle-models/InstanceNormalization_F32_R4.circle] Error 255

[seanshpark]

DecomposeONNXToONNXPass

this pass converts InstanceNormalization to LayerNormalization which we don't want
this pass converts PadV2 to Pad and other version tagged ONNX*V?? to latest ONNX*
so,

• we should run DecomposeONNXToONNXPass
• but manually disable unnecessary oasses for us

[seanshpark]

apply changes to current head

auto &pad_mutable = resize_op.getPadMutable(); gives

error: cannot bind non-const lvalue reference of type 'mlir::MutableOperandRange&' to an rvalue of type 'mlir::MutableOperandRange'

[seanshpark]

onnx-mlir checkout error

ExternalProject_Add(externals-onnx-mlir
  GIT_REPOSITORY "https://github.com/onnx/onnx-mlir.git"
  GIT_TAG        "660bd8efa664bd2d05801a7a793fe29925313b4b" # v0.5.0.0

gives

fatal: reference is not a tree: 660bd8efa664bd2d05801a7a793fe29925313b4b
CMake Error at /home/.../circle-mlir/build/externals/debug/externals-onnx-mlir-prefix/tmp/externals-onnx-mlir-gitupdate.cmake:175 (execute_process):
execute_process failed command indexes:

1: "Child return code: 128"

command line git checkout to 660bd8efa664bd2d05801a7a793fe29925313b4b is OK.
but fails with ExternalProject_Add.

--> clone and checkout with my personal fork is OK

• maybe onnx/onnx-mlir doens't permit checkout with this commit
• currently this commit is not official tag
• giving origin/0.5.0.0 gives different error, still not working

--> No. retried again two days after adding this comment and failed

[seanshpark]

reduce constant dump

this change in llvm-project is to reduce unnecessary dump to first 16 bytes of constants

diff --git a/mlir/lib/IR/AsmPrinter.cpp b/mlir/lib/IR/AsmPrinter.cpp
index c603db450cbd..2a77cf61ddf7 100644
--- a/mlir/lib/IR/AsmPrinter.cpp
+++ b/mlir/lib/IR/AsmPrinter.cpp
@@ -2831,10 +2831,12 @@ void AsmPrinter::Impl::printEscapedString(StringRef str) {
 }

 void AsmPrinter::Impl::printHexString(StringRef str) {
+  if (str.size() > 16) str = str.slice(0, 16);
   os << "\"0x" << llvm::toHex(str) << "\"";
 }
 void AsmPrinter::Impl::printHexString(ArrayRef<char> data) {
-  printHexString(StringRef(data.data(), data.size()));
+  size_t data_size = data.size() > 16 ? 16 : data.size();
+  printHexString(StringRef(data.data(), data_size));
 }

 LogicalResult AsmPrinter::Impl::pushCyclicPrinting(const void *opaquePointer) {
@@ -3294,10 +3296,11 @@ private:
       printFn(key, [&](raw_ostream &os) {
         // Store the blob in a hex string containing the alignment and the data.
         llvm::support::ulittle32_t dataAlignmentLE(dataAlignment);
+        size_t data_size = data.size() > 16 ? 16 : data.size();
         os << "\"0x"
            << llvm::toHex(StringRef(reinterpret_cast<char *>(&dataAlignmentLE),
                                     sizeof(dataAlignment)))
-           << llvm::toHex(StringRef(data.data(), data.size())) << "\"";
+           << llvm::toHex(StringRef(data.data(), data_size)) << "\"";
       });
     }

[seanshpark]

Local Docker image build for testing

for testing with corporate proxy ... need to give proxy and certificate file

• edit Dockerfile for temporary

ADD corporate.crt /usr/local/share/ca-certificates/corporate.crt
RUN chmod 644 /usr/local/share/ca-certificates/corporate.crt && update-ca-certificates
...
RUN wget -e use_proxy=yes -e https_proxy=http://a.b.c.d:e ....

create Docker image as circle-mlir-build-local-u2204

cd circle-mlir/infra/docker/u2204
docker build -t circle-mlir-build-local-u2204 .

run

docker run \
--rm \
-u $(id -u):$(id -g) \
-it \
-v $HOME:/home/circlemlir \
-w /home/circlemlir \
circle-mlir-build-local-u2204

Docker image doesn't have luci-interpreter

• use local build
• Ubuntu version should be same

export ONE_COMPILER_ROOT=$HOME/../one/build/install
make cfgdi
make debug
make test

[seanshpark]

Upgrade is almost ready, but there is an issue that cannot checkout onnx-mlir to 0.5.0.0 branch with CMake.

• [circle-mlir] Upgrade ONNX-MLIR and other externals #15148 (comment)

Need to check this without corporate proxy.
--> it was OK without proxy (checked this at home)

---

Clone of onnx-mlir this morning('25/04/16) is OK.... weird.