Change image to multiArray in case of converted mlmodel from TFlite

[daisymind]

Hellow Matthijs,

Your article has helped me a lot, thanks!, But now I'm stuck and need some clues.

I am trying to use a converted '.mlmodel' from Google MediaPipe '.tflite'.

• input: change image to multiArray
  • with reference to your machinethink.net: How to convert images to MLMultiArray
• output: change multiArray to image
  • with reference to here CoreMLHelpers: MLMultiArray to image conversion

I'm stuck on changing the input type, I was able to change the type but when I load it into Xcode Project, I get a CoreML compile error.

# convert_inputType.py :convert multiArray to image type
import coremltools as ct
from coremltools.proto import FeatureTypes_pb2 as ft

spec = ct.utils.load_spec('model_coreml_float32.mlmodel') # miltiArray type
builder = ct.models.neural_network.NeuralNetworkBuilder(spec=spec)

# check input/output features
print('--- Before change:')
builder.inspect_input_features()
builder.inspect_output_features()

# change the input so the model can accept 256x256 RGB images
input = spec.description.input[0]
input.type.imageType.colorSpace = ft.ImageFeatureType.RGB
input.type.imageType.width = 256
input.type.imageType.height = 256

# converted input/output features
print('--- After change:')
builder.inspect_input_features()
builder.inspect_output_features()

# save inputType-converted model
ct.utils.save_spec(spec, 'selfie_segmentation.mlmodel') # changed type

--- Before change:

[Id: 0] Name: input_1
          Type: multiArrayType {
  shape: 1
  shape: 256
  shape: 256
  shape: 3
  dataType: FLOAT32
}

[Id: 0] Name: activation_10
          Type: multiArrayType {
  dataType: FLOAT32
}

--- After change:

[Id: 0] Name: input_1
          Type: imageType {
  width: 256
  height: 256
  colorSpace: RGB
}

[Id: 0] Name: activation_10
          Type: multiArrayType {
  dataType: FLOAT32
}

• 'model_coreml_float32.mlmodel' :converted from MediaPipe TFlite at PINTO_model_zoo
• 'selfie_segmentation.mlmodel' :changed-type mlmodel

The following error occurs when loading the changed-type mlmodel into Xcode Peoject.

Espresso exception: "Invalid blob shape": generic_elementwise_kernel: cannot broadcast:
----------------------------------------
SchemeBuildError: Failed to build the scheme "testSelfieSegmentation"

compiler error:  Espresso exception: "Invalid blob shape": generic_elementwise_kernel: cannot broadcast:

Compile CoreML model selfie_segmentation.mlmodel:
coremlc: error: compiler error:  Espresso exception: "Invalid blob shape": generic_elementwise_kernel: cannot broadcast:
 (1, 16, 8, 128)
 (1, 16, 2, 128)

I have checked the model configuration with Newtron, and cannot find any nodes like (1,16,8,128) or (1,16,2,128)

Is there something wrong with the change code? Or does it have something to do with the fact that the original is a TFlite model?

Any hints on how to solve this problem?