What input image format does get_sliced_prediction expect?

[huwl404]

Hi all,

This is a very strange issue. I trained a model using Ultralytics on SAHI sliced dataset and set the gray jpg image channel as 1:
channels: 1 names: 0: DMV path: ... train: train val: val

The model seemed to be good. However, when I predict on gray png images using SAHI, there is a dilemma:

If I directly pass the gray picture, it went error: Given groups=1, weight of size [64, 1, 3, 3], expected input[1, 3, 1280, 1280] to have 1 channels, but got 3 channels instead;
If I read it as H*W*1, it went error: Cannot handle this data type: (1, 1, 1), |u1;
If I read read it as H*W, it went error: too many indices for array: array is 2-dimensional, but 3 were indexed;

I dont know what I should pass...

My code:

class YoLoWrapper:
    """Wraps YOLO model interactions."""
    def __init__(self, cfg: dict):
        self.model = AutoDetectionModel.from_pretrained(model_type="ultralytics", model_path=str(cfg["model_path"]), confidence_threshold=cfg["conf"], device=cfg["device"])

    def predict_image(self, img_path: str, cfg: dict) -> Tuple[Optional[List[Detection]], float, float, float]:
        # directly read picture: Given groups=1, weight of size [64, 1, 3, 3], expected input[1, 3, 1280, 1280] to have 1 channels, but got 3 channels instead
        img_gray = cv2.imread(img_path, cv2.IMREAD_UNCHANGED)  # (4096, 4096, 1) uint8: 'Cannot handle this data type: (1, 1, 1), |u1'
        img_gray = np.squeeze(img_gray)  # (4096, 4096) uint8: too many indices for array: array is 2-dimensional, but 3 were indexed   
        img_gray = cv2.cvtColor(img_gray, cv2.COLOR_GRAY2RGB)  # (4096, 4096, 3) uint8: 'Given groups=1, weight of size [64, 1, 3, 3], expected input[1, 3, 1280, 1280] to have 1 channels, but got 3 channels instead'
        results = get_sliced_prediction(img_gray, self.model, slice_height=cfg["img_size"], slice_width=cfg["img_size"], overlap_height_ratio=cfg["overlap"], overlap_width_ratio=cfg["overlap"],)

[huwl404]

I test Ultralytics prediction, it works well on cpu/gpu for gray image but it can not work for BGR image:

img_gray = cv2.imread(img_path, cv2.IMREAD_GRAYSCALE)  # only this line, ultraltics works well.
# img_gray = cv2.cvtColor(img_gray, cv2.COLOR_GRAY2BGR)  # Given groups=1, weight of size [64, 1, 3, 3], expected input[1, 3, 1280, 1280] to have 1 channels, but got 3 channels instead
results = self.model.predict(source=img_gray, conf=cfg["conf"], iou=cfg["iou"], imgsz=cfg["img_size"], device=device, max_det=cfg["max_detection"], save_conf=True, verbose=False)

When I input a gray image (4096, 4096, 1) to SAHI, the line 3308 of Image.py would cause error: Cannot handle this data type: (1, 1, 1)
mode, rawmode = _fromarray_typemap[typekey]

When I input a (4096, 4096) image to SAHI, the line 85 of ultralytics.py would cause error: too many indices for array: array is 2-dimensional, but 3 were indexed
prediction_result = self.model(image[:, :, ::-1], **kwargs) # YOLO expects numpy arrays to have BGR

When I input a (4096, 4096, 3) image to SAHI, it would cause the same error above: Given groups=1, weight of size [64, 1, 3, 3], expected input[1, 3, 1280, 1280] to have 1 channels, but got 3 channels instead

YOLO does not expect numpy arrays to have BGR, I think this might be a bug?
Any suggestion would be appreciated!

[huwl404]

So I solved this issue by modifying line 85 of ultralytics.py to
prediction_result = self.model(image[:, :], **kwargs) # YOLO does not expect numpy arrays to have BGR