`ImageFeatureTest`s are failing

[GaetanLepage]

Short description
Running the test suite with pillow 12.0.0 gives the following failures:

=========================== short test summary info ============================
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images34 - TypeError: Cannot handle this data type: (1, 1, 3), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images35 - TypeError: Cannot handle this data type: (1, 1, 4), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images4 - TypeError: Cannot handle this data type: (1, 1, 3), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images5 - TypeError: Cannot handle this data type: (1, 1, 4), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images10 - TypeError: Cannot handle this data type: (1, 1, 3), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images11 - TypeError: Cannot handle this data type: (1, 1, 4), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images28 - TypeError: Cannot handle this data type: (1, 1, 3), <u2
FAILED tensorflow_datasets/core/features/image_feature_test.py::ImageFeatureTest::test_images29 - TypeError: Cannot handle this data type: (1, 1, 4), <u2
========== 8 failed, 3332 passed, 588 skipped, 773 warnings in 56.61s ==========

Environment information

• Operating System: NixOS

• Python version: 3.13.9

• tensorflow-datasets/tfds-nightly version: tensorflow-datasets 4.9.9

• tensorflow/tf-nightly version: tensorflow 2.20

• Does the issue still exists with the last tfds-nightly package (pip install --upgrade tfds-nightly) ?
  -> Not tested

Reproduction instructions

$ pytest

Link to logs

________________________ ImageFeatureTest.test_images34 ________________________
[gw32] linux -- Python 3.13.9 /nix/store/7gl4khq26h625mpqzkiiqsify3hclar1-python3-3.13.9/bin/python3.13

obj = array([[[ 27,  68, 156],
        [108,  16,  90],
        [ 76,  43, 200],
        ...,
        [189,  72, 101],
     ...   ...,
        [159,  21,  92],
        [ 48,  35,  23],
        [ 40, 150, 248]]], shape=(128, 100, 3), dtype=uint16)
mode = 'I;16'

    def fromarray(obj: SupportsArrayInterface, mode: str | None = None) -> Image:
        """
        Creates an image memory from an object exporting the array interface
        (using the buffer protocol)::

          from PIL import Image
          import numpy as np
          a = np.zeros((5, 5))
          im = Image.fromarray(a)

        If ``obj`` is not contiguous, then the ``tobytes`` method is called
        and :py:func:`~PIL.Image.frombuffer` is used.

        In the case of NumPy, be aware that Pillow modes do not always correspond
        to NumPy dtypes. Pillow modes only offer 1-bit pixels, 8-bit pixels,
        32-bit signed integer pixels, and 32-bit floating point pixels.

        Pillow images can also be converted to arrays::

          from PIL import Image
          import numpy as np
          im = Image.open("hopper.jpg")
          a = np.asarray(im)

        When converting Pillow images to arrays however, only pixel values are
        transferred. This means that P and PA mode images will lose their palette.

        :param obj: Object with array interface
        :param mode: Optional mode to use when reading ``obj``. Since pixel values do not
          contain information about palettes or color spaces, this can be used to place
          grayscale L mode data within a P mode image, or read RGB data as YCbCr for
          example.

          See: :ref:`concept-modes` for general information about modes.
        :returns: An image object.

        .. versionadded:: 1.1.6
        """
        arr = obj.__array_interface__
        shape = arr["shape"]
        ndim = len(shape)
        strides = arr.get("strides", None)
        try:
            typekey = (1, 1) + shape[2:], arr["typestr"]
        except KeyError as e:
            if mode is not None:
                typekey = None
                color_modes: list[str] = []
            else:
                msg = "Cannot handle this data type"
                raise TypeError(msg) from e
        if typekey is not None:
            try:
>               typemode, rawmode, color_modes = _fromarray_typemap[typekey]
                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
E               KeyError: ((1, 1, 3), '<u2')

/nix/store/96vld9s60486ifj2pq1v3gdbq19gx145-python3.13-pillow-12.0.0/lib/python3.13/site-packages/PIL/Image.py:3285: KeyError

The above exception was the direct cause of the following exception:

self = <tensorflow_datasets.core.features.image_feature_test.ImageFeatureTest testMethod=test_images34>
make_lib_fail = <function make_pil_fail at 0x7ffb8c17c400>
dtypes = (tf.uint16, <class 'numpy.uint16'>), channels = 3

    @parameterized.product(
        make_lib_fail=[make_none_fail, make_cv2_fail, make_pil_fail],
        dtypes=[
            (np.uint8, np.uint8),
            (np.uint16, np.uint16),
            (tf.uint8, np.uint8),
            (tf.uint16, np.uint16),
        ],
        channels=[1, 3, 4],
    )
    def test_images(self, make_lib_fail, dtypes, channels):
      dtype, np_dtype = dtypes
      with make_lib_fail() as failing_lib:
        if _unsupported_images_for_pil(np_dtype, failing_lib):
          return
        img = randint(256, size=(128, 100, channels), dtype=np_dtype)
        img_other_shape = randint(256, size=(64, 200, channels), dtype=np_dtype)

        filename = {
            np.uint8: {
                1: '6pixels_grayscale.png',
                3: '6pixels.png',
                4: '6pixels_4chan.png',
            },
            np.uint16: {
                1: '6pixels_grayscale_16bit.png',
                3: '6pixels_16bit.png',
                4: '6pixels_16bit_4chan.png',
            },
        }[np_dtype][channels]

        img_file_path = os.path.join(
            os.path.dirname(__file__), '../../testing/test_data', filename
        )
        with tf.io.gfile.GFile(img_file_path, 'rb') as f:
          img_byte_content = f.read()
        img_file_expected_content = np.array(
            [
                [[0, 255, 0, 255], [255, 0, 0, 255], [255, 0, 255, 255]],
                [[0, 0, 255, 255], [255, 255, 0, 255], [126, 127, 128, 255]],
            ],
            dtype=np_dtype,
        )[
            :, :, :channels
        ]  # Truncate (h, w, 4) -> (h, w, c)
        if dtype == np.uint16 or dtype == tf.uint16:
          img_file_expected_content *= 257  # Scale int16 images

        numpy_array = testing.FeatureExpectationItem(
            value=img,
            expected=img,
            expected_np=img,
        )
        file_path_as_string = testing.FeatureExpectationItem(
            value=img_file_path,
            expected=img_file_expected_content,
            expected_np=img_file_expected_content,
        )
        file_path_as_path = testing.FeatureExpectationItem(
            value=pathlib.Path(img_file_path),
            expected=img_file_expected_content,
            expected_np=img_file_expected_content,
        )
        images_bytes = testing.FeatureExpectationItem(
            value=img_byte_content,
            expected=img_file_expected_content,
            expected_np=img_file_expected_content,
        )
        img_shape_can_be_dynamic = testing.FeatureExpectationItem(
            value=img_other_shape,
            expected=img_other_shape,
            expected_np=img_other_shape,
        )
        invalid_type = testing.FeatureExpectationItem(
            value=randint(256, size=(128, 128, channels), dtype=np.uint32),
            raise_cls=ValueError,
            raise_cls_np=ValueError,
            raise_msg='dtype should be',
        )
        tests = [
            numpy_array,
            file_path_as_string,
            file_path_as_path,
            images_bytes,
            img_shape_can_be_dynamic,
            invalid_type,
        ]
        # PIL doesn't support 16-bit images.
        if (failing_lib != LibWithImportError.PIL) and np_dtype != np.uint16:
          tests.append(
              testing.FeatureExpectationItem(
                  value=PIL.Image.open(img_file_path),
                  expected=img_file_expected_content,
                  expected_np=img_file_expected_content,
              )
          )
>       self.assertFeature(
            feature=features_lib.Image(shape=(None, None, channels), dtype=dtype),
            shape=(None, None, channels),
            dtype=dtype,
            tests=tests,
            test_attributes=dict(
                _encoding_format=None,
                _use_colormap=False,
            ),
        )

tensorflow_datasets/core/features/image_feature_test.py:177:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tensorflow_datasets/testing/test_utils.py:421: in decorated
    f(self, *args, **kwargs)
tensorflow_datasets/testing/feature_test_case.py:186: in assertFeature
    self.assertFeatureEagerOnly(
tensorflow_datasets/testing/feature_test_case.py:236: in assertFeatureEagerOnly
    run_tests(serialize_fdict=fdict, deserialize_fdict=fdict, feature=feature)
tensorflow_datasets/testing/feature_test_case.py:318: in _run_tests
    self.assertFeatureTest(
tensorflow_datasets/testing/feature_test_case.py:463: in assertFeatureTest
    self._test_repr(feature, out_numpy)
tensorflow_datasets/testing/feature_test_case.py:483: in _test_repr
    text = f.repr_html(ex)
           ^^^^^^^^^^^^^^^
tensorflow_datasets/core/features/image_feature.py:398: in repr_html
    img = utils.create_thumbnail(ex, use_colormap=self._use_colormap)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tensorflow_datasets/core/utils/image_utils.py:190: in create_thumbnail
    img = PIL_Image.fromarray(ex, mode=mode)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

obj = array([[[ 27,  68, 156],
        [108,  16,  90],
        [ 76,  43, 200],
        ...,
        [189,  72, 101],
     ...   ...,
        [159,  21,  92],
        [ 48,  35,  23],
        [ 40, 150, 248]]], shape=(128, 100, 3), dtype=uint16)
mode = 'I;16'

    def fromarray(obj: SupportsArrayInterface, mode: str | None = None) -> Image:
        """
        Creates an image memory from an object exporting the array interface
        (using the buffer protocol)::

          from PIL import Image
          import numpy as np
          a = np.zeros((5, 5))
          im = Image.fromarray(a)

        If ``obj`` is not contiguous, then the ``tobytes`` method is called
        and :py:func:`~PIL.Image.frombuffer` is used.

        In the case of NumPy, be aware that Pillow modes do not always correspond
        to NumPy dtypes. Pillow modes only offer 1-bit pixels, 8-bit pixels,
        32-bit signed integer pixels, and 32-bit floating point pixels.

        Pillow images can also be converted to arrays::

          from PIL import Image
          import numpy as np
          im = Image.open("hopper.jpg")
          a = np.asarray(im)

        When converting Pillow images to arrays however, only pixel values are
        transferred. This means that P and PA mode images will lose their palette.

        :param obj: Object with array interface
        :param mode: Optional mode to use when reading ``obj``. Since pixel values do not
          contain information about palettes or color spaces, this can be used to place
          grayscale L mode data within a P mode image, or read RGB data as YCbCr for
          example.

          See: :ref:`concept-modes` for general information about modes.
        :returns: An image object.

        .. versionadded:: 1.1.6
        """
        arr = obj.__array_interface__
        shape = arr["shape"]
        ndim = len(shape)
        strides = arr.get("strides", None)
        try:
            typekey = (1, 1) + shape[2:], arr["typestr"]
        except KeyError as e:
            if mode is not None:
                typekey = None
                color_modes: list[str] = []
            else:
                msg = "Cannot handle this data type"
                raise TypeError(msg) from e
        if typekey is not None:
            try:
                typemode, rawmode, color_modes = _fromarray_typemap[typekey]
            except KeyError as e:
                typekey_shape, typestr = typekey
                msg = f"Cannot handle this data type: {typekey_shape}, {typestr}"
>               raise TypeError(msg) from e
E               TypeError: Cannot handle this data type: (1, 1, 3), <u2

/nix/store/96vld9s60486ifj2pq1v3gdbq19gx145-python3.13-pillow-12.0.0/lib/python3.13/site-packages/PIL/Image.py:3289: TypeError
------------------------------ Captured log call -------------------------------
WARNING  absl:feature.py:71 `FeatureConnector.dtype` is deprecated. Please change your code to use NumPy with the field `FeatureConnector.np_dtype` or use TensorFlow with the field `FeatureConnector.tf_dtype`.

Expected behavior
All tests pass

Additional context
I am working as a nixpkgs maintainer.
Relevant PR: NixOS/nixpkgs#463025

[0AnshuAditya0]

Hi! I can fix this issue and create a PR shortly.

Root Cause Analysis:
Pillow 12.0.0 removed support for creating multi-channel (RGB/RGBA) images from uint16 numpy arrays via fromarray(). The _fromarray_typemap no longer includes keys like ((1, 1, 3), '<u2') and ((1, 1, 4), '<u2').

Issue Location:
The failure occurs in tensorflow_datasets/core/utils/image_utils.py:190 in the create_thumbnail() function when calling PIL_Image.fromarray(ex, mode=mode) on uint16 arrays with 3 or 4 channels.

Proposed Fix:
Convert uint16 multi-channel images to uint8 before passing to PIL's fromarray() in the thumbnail creation path, or handle them channel-by-channel. This maintains backward compatibility while supporting Pillow 12.0.0.

I have the environment set up and will submit a PR with tests within a few hours.