MXNet engine Shape inconsistent

[AngshumanChakraborty]

I want to perform transfer learning to classify two objects. For training I am doing the following:

Criteria.Builder<Image, Classifications> builder = Criteria.builder()
.optApplication(Application.CV.IMAGE_CLASSIFICATION)
.setTypes(Image.class, Classifications.class)
.optGroupId("ai.djl.mxnet")
.optArtifactId("resnet")
.optFilter("layers", "50")
.optFilter("flavor", "v1");

Model model = builder.build().loadModel();
SequentialBlock newBlock = new SequentialBlock();
SymbolBlock block = (SymbolBlock) model.getBlock();
block.removeLastBlock();
newBlock.add(block);
newBlock.add(Blocks.batchFlattenBlock());
newBlock.add(Linear.builder().setUnits(2).build());
model.setBlock(newBlock);

ImageFolder dataset = ImageFolder.builder()
.optFlag(Flag.COLOR)
.addTransform(new Resize(32, 32))
.addTransform(array -> array.div(255.0f))
.setSampling(64, true)
.build();

DefaultTrainingConfig config = new DefaultTrainingConfig(Loss.softmaxCrossEntropyLoss())
.addEvaluator(new Accuracy())
.addTrainingListeners(TrainingListener.Defaults.logging());

Trainer trainer = model.newTrainer(config);

Shape inputShape = new Shape(1, 3, 32, 32);
trainer.initialize(inputShape);
EasyTrain.fit(trainer, 100, datasets[0], datasets[1]);

model.save(modelDir, "resnet");

For inference I have done the following:

	    Image img = ImageFactory.getInstance().fromFile(new File("inference.jpg").toPath());

	    try (Model model = Model.newInstance("resnetv1")) {
	        model.load(Paths.get("Model.path"), "resnet");
	        
	        Translator<Image, Classifications> translator =
	                ImageClassificationTranslator.builder()
				    .addTransform(new Resize(width, height))
				    .addTransform(array -> array.div(255.0f))
				    .addTransform(new ToTensor())
                    .optApplySoftmax(true)
                    .build();

	        try (Predictor<Image, Classifications> predictor = model.newPredictor(translator)) {
	            Classifications predictResult = predictor.predict(img);
	            System.out.println(predictResult);
	        }
	    }

But I get the following error in the inference part when loading the model:

java.io.FileNotFoundException: Symbol file not found: resnet-symbol.json, please set block manually for imperative model.

[zachgk]

I think your problem might be with the pre-processing. We definitely know from the error message that when calling the loaded resnet model (SymbolBlock block), we are giving it the wrong shape. Can you try switching the dataset definition to use:

            ImageFolder dataset =
                    ImageFolder.builder()
                            .setRepository(repository)
                            .addTransform(new Resize(100, 100))
                            .addTransform(new ToTensor())
                            .setSampling(1, false)
                            .build();

DJL needs the ToTensor() transform because preprocessing happens with images in the format of nhwc because it uses OpenCV and the model uses images in the format of nchw because that is how MXNet prefer it. So, you have to transpose the image NDArray or else it will be in the wrong shape. I think that is what is happening, but I don't know what datasets[0] and datasets[1] are