Skip to content

Make faster matToNDArray safe with non-image number of channels #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alanocallaghan merged 2 commits into from
Feb 11, 2025
Merged

Make faster matToNDArray safe with non-image number of channels #28

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
aed597b
Revert "Faster Mat to NDArray channels-first conversion"
alanocallaghan Feb 10, 2025
a0829a5
Retain faster code for edge case
alanocallaghan Feb 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 6 additions & 14 deletions src/main/java/qupath/ext/djl/DjlTools.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*-
* Copyright 2022-2024 QuPath developers, University of Edinburgh
* Copyright 2022-2025 QuPath developers, University of Edinburgh
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -29,6 +29,7 @@
import java.util.Set;

import ai.djl.Device;
import org.bytedeco.javacpp.Loader;
import org.bytedeco.javacpp.PointerScope;
import org.bytedeco.javacpp.indexer.BooleanIndexer;
import org.bytedeco.javacpp.indexer.ByteIndexer;
Expand Down Expand Up @@ -369,16 +370,6 @@ public static Device getOverrideDevice(String engineName) {
return defaultDevices.getOrDefault(engineName, null);
}

// static ZooModel<Mat, Mat> loadModelCV(URI uri, String ndLayout) throws ModelNotFoundException, MalformedModelException, IOException {
// var criteria = Criteria.builder()
// .setTypes(Mat.class, Mat.class)
// .optModelUrls(uri.toString())
// .optProgress(new ProgressBar())
// .optTranslator(new MatTranslator(ndLayout, ndLayout))
// .build();
// return ModelZoo.loadModel(criteria);
// }


static Mat predict(Model model, Mat mat) throws TranslateException {
try (var predictor = model.newPredictor(new MatTranslator("CHW", "CHW"))) {
Expand All @@ -389,6 +380,7 @@ static Mat predict(Model model, Mat mat) throws TranslateException {

/**
* Convert an Opencv {@link Mat} to a Deep Java Library {@link NDArray}.
* Note that this ass
* @param manager an {@link NDManager}, required to create the NDArray
* @param mat the mat to convert
* @param ndLayout a layout string for the NDArray, e.g. "CHW"; currently, HW must appear together (in that order)
Expand All @@ -403,15 +395,15 @@ public static NDArray matToNDArray(NDManager manager, Mat mat, String ndLayout)
int indHW = ndLayout.indexOf("HW");
if (indHW < 0)
throw new IllegalArgumentException("Expected layout contains HW, but provided layout is " + ndLayout);
long nChannels = shape.get(indC);
// Copy all at once is using the same storage order as OpenCV
// TODO: Check what this order is!!!
NDArray array = null;
if (indC > indHW || shape.get(indC) == 1) {
// Channels-last, or single-channel
var buffer = mat.createBuffer();
array = manager.create(buffer, shape, dataType);
} else if ("NCHW".equals(ndLayout) || "CHW".equals(ndLayout)) {
// Channels-first - an OpenCV blob is defined to have the order NCHW
} else if (("NCHW".equals(ndLayout) || "CHW".equals(ndLayout)) && (nChannels == 3L || nChannels == 4L)) {
// Channels-first - an OpenCV blob is defined to have the order NCHW, but an Image can only have 1, 3 or 4 channels
array = manager.create(opencv_dnn.blobFromImage(mat).createBuffer(), shape, dataType);
} else {
// Really awkward strategy to handle channels in an uncommon place (shouldn't actually occur?)
Expand Down