Skip to content

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 0: invalid start byte #931

@Anas0x45

Description

@Anas0x45

Description
I encountered a UnicodeDecodeError when running the Triton Model Analyzer in a Docker container with GPU support. The issue seems to occur when retrieving the GPU device name. Here’s the traceback:
Traceback (most recent call last):
File "/usr/local/bin/model-analyzer", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/model_analyzer/entrypoint.py", line 263, in main
gpus = GPUDeviceFactory().verify_requested_gpus(config.gpus)
File "/usr/local/lib/python3.10/dist-packages/model_analyzer/device/gpu_device_factory.py", line 39, in init
self.init_all_devices()
File "/usr/local/lib/python3.10/dist-packages/model_analyzer/device/gpu_device_factory.py", line 71, in init_all_devices
device_name = device_atrributes.deviceName
File "/usr/local/lib/python3.10/dist-packages/model_analyzer/monitor/dcgm/dcgm_structs.py", line 464, in getattribute
return value.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf8 in position 0: invalid start byte

Triton Information
Docker image: nvcr.io/nvidia/tritonserver:24.07-py3-sdk

To Reproduce

  • Running Triton Model Analyzer in Docker with GPU support (docker run --gpus all).
  • The error occurs when retrieving the GPU device name using dcgm_structs.py.

Solution that works for me:
Adding errors="ignore" or errors="replace" to the decoding logic temporarily fixes the crash but may cause data loss.

class _DcgmStructure(Structure):

    def __getattribute__(self, key):
        value = super().__getattribute__(key)
        if isinstance(value, bytes):
            try:
                return value.decode("utf-8")
            except UnicodeDecodeError as e:
                print(f"Failed to decode bytes: {e}")
                return value.decode("utf-8", errors="replace")
        if isclass(value):
            return _WrappedStructure(value)
        return value
...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions