Remove `pydantic<2.0` pin in requirements.txt by glenn-jocher · Pull Request #1398 · SonySemiconductorSolutions/mct-model-optimization

glenn-jocher · 2025-04-01T14:23:56Z

Request to unpin this dependency as this is causing installation conflicts in the Ultralytics package when used with other export dependencies that require more recent versions of Pydantic.

I think the main changes may be needed in model_compression_toolkit here as this error shows when running IMX export using model_compression_toolkit with the latest pydantic:

# pip install ultralytics
$ yolo export format=imx model=yolov8n.pt

Downloading https://github.com/ultralytics/assets/releases/download/v8.3.0/yolov8n.pt to 'yolov8n.pt'...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6.25M/6.25M [00:00<00:00, 153MB/s]
Ultralytics 8.3.99 🚀 Python-3.11.10 torch-2.6.0+cpu CPU (AMD EPYC 7502 32-Core Processor)
WARNING ⚠️ IMX only supports int8 export, setting int8=True.
WARNING ⚠️ INT8 export requires a missing 'data' arg for calibration. Using default 'data=coco8.yaml'.
YOLOv8n summary (fused): 72 layers, 3,151,904 parameters, 0 gradients, 8.7 GFLOPs

PyTorch: starting from 'yolov8n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) ((1, 8400, 4), (1, 8400, 80)) (6.2 MB)
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1743514794.166912      60 cuda_dnn.cc:8579] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1743514794.173526      60 cuda_blas.cc:1407] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
W0000 00:00:1743514794.190136      60 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1743514794.190156      60 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1743514794.190160      60 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
W0000 00:00:1743514794.190164      60 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.
IMX: export failure ❌ 8.7s: If you use `@root_validator` with pre=False (the default) you MUST specify `skip_on_failure=True`. Note that `@root_validator` is deprecated and should be replaced with `@model_validator`.

For further information visit https://errors.pydantic.dev/2.11/u/root-validator-pre-skip
Traceback (most recent call last):
  File "/usr/local/bin/yolo", line 10, in <module>
    sys.exit(entrypoint())
             ^^^^^^^^^^^^
  File "/ultralytics/ultralytics/cfg/__init__.py", line 987, in entrypoint
    getattr(model, mode)(**overrides)  # default args from model
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ultralytics/ultralytics/engine/model.py", line 728, in export
    return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ultralytics/ultralytics/engine/exporter.py", line 458, in __call__
    f[13], _ = self.export_imx()
               ^^^^^^^^^^^^^^^^^
  File "/ultralytics/ultralytics/engine/exporter.py", line 182, in outer_func
    raise e
  File "/ultralytics/ultralytics/engine/exporter.py", line 177, in outer_func
    f, model = inner_func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ultralytics/ultralytics/engine/exporter.py", line 1230, in export_imx
    import model_compression_toolkit as mct
  File "/usr/local/lib/python3.11/site-packages/model_compression_toolkit/__init__.py", line 17, in <module>
    from model_compression_toolkit import target_platform_capabilities
  File "/usr/local/lib/python3.11/site-packages/model_compression_toolkit/target_platform_capabilities/__init__.py", line 16, in <module>
    from model_compression_toolkit.target_platform_capabilities.targetplatform2framework.attribute_filter import AttributeFilter
  File "/usr/local/lib/python3.11/site-packages/model_compression_toolkit/target_platform_capabilities/targetplatform2framework/__init__.py", line 17, in <module>
    from model_compression_toolkit.target_platform_capabilities.targetplatform2framework.framework_quantization_capabilities import FrameworkQuantizationCapabilities
  File "/usr/local/lib/python3.11/site-packages/model_compression_toolkit/target_platform_capabilities/targetplatform2framework/framework_quantization_capabilities.py", line 22, in <module>
    from model_compression_toolkit.target_platform_capabilities.schema.schema_functions import \
  File "/usr/local/lib/python3.11/site-packages/model_compression_toolkit/target_platform_capabilities/schema/schema_functions.py", line 18, in <module>
    from model_compression_toolkit.target_platform_capabilities.schema.mct_current_schema import OpQuantizationConfig, \
  File "/usr/local/lib/python3.11/site-packages/model_compression_toolkit/target_platform_capabilities/schema/mct_current_schema.py", line 1, in <module>
    import model_compression_toolkit.target_platform_capabilities.schema.v1 as schema
  File "/usr/local/lib/python3.11/site-packages/model_compression_toolkit/target_platform_capabilities/schema/v1.py", line 515, in <module>
    class Fusing(TargetPlatformModelComponent):
  File "/usr/local/lib/python3.11/site-packages/model_compression_toolkit/target_platform_capabilities/schema/v1.py", line 558, in Fusing
    @root_validator(allow_reuse=True)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic/deprecated/class_validators.py", line 240, in root_validator
    raise PydanticUserError(
pydantic.errors.PydanticUserError: If you use `@root_validator` with pre=False (the default) you MUST specify `skip_on_failure=True`. Note that `@root_validator` is deprecated and should be replaced with `@model_validator`.

Pull Request Description:

Checklist before requesting a review:

I set the appropriate labels on the pull request.
I have added/updated the release note draft (if necessary).
I have updated the documentation to reflect my changes (if necessary).
All function and files are well documented.
All function and classes have type hints.
There is a licenses in all file.
The function and variable names are informative.
I have checked for code duplications.
I have added new unittest (if necessary).

Request to unpin this dependency as this is causing installation conflicts in the Ultralytics package when used with other export dependencies that require more recent versions of Pydantic. I think the main changes may be needed in `model_compression_toolkit` here:

glenn-jocher · 2025-04-01T14:25:12Z

@ServiAmirPM @ambitious-octopus @lakshanthad FYI

ofirgo · 2025-04-02T10:13:08Z

Hi @glenn-jocher ,
Thank you for your contribution!
After looking into it, removing the version constraint from the pydantic package requires additional modification in the MCT, specifically in the TPC schema mechanism.
We want to investigate and modify it appropriately before removing the constraint from the requirement file.

So, I'm putting this PR on hold, and we will continue to update it with all the necessary changes once all verifications are done.
Thanks,
Ofir

lior-dikstein · 2025-04-02T14:05:37Z

Hi,
I investigated the issue and confirmed @ofirgo 's findings. The reason we pinned pydantic < 2 was due to a conflict between pydantic and tensorflow < 2.14, specifically around the typing-extensions dependency. Since we plan to drop support for tensorflow < 2.14 in the upcoming release, I’ll proceed to work on your branch to enable support for pydantic >= 2 in MCT.

glenn-jocher · 2025-04-06T22:14:17Z

@ofirgo @lior-dikstein great guys, understood, thank you for the updates!

ofirgo · 2025-04-09T14:37:49Z

model_compression_toolkit/target_platform_capabilities/schema/v1.py

-    tpc_minor_version: Optional[int]
-    tpc_patch_version: Optional[int]
-    tpc_platform_type: Optional[str]
+    operator_set: Optional[Tuple[OperatorsSet, ...]] = None


Are these default values required?

we prefer no to set default values to TPC attributes

It is required

Ok I think there is no risk in adding this default value in this case

...it_tests/core/quantization/node_quantization_config/test_node_weights_quantization_config.py

ofirgo closed this Apr 2, 2025

ofirgo reopened this Apr 2, 2025

ofirgo requested a review from Idan-BenAmi April 2, 2025 11:31

liord added 3 commits April 8, 2025 14:04

Merge to main

5b05c21

Add support for pydantic v2

612593e

Fix test that fails due to upgrade to pydantic v2

a824569

ofirgo self-requested a review April 9, 2025 11:25

liord added 2 commits April 9, 2025 14:54

Remove tests for keras\tensorflow < 2.14. Update requirements.txt

00bba00

merge main

78145ee

ofirgo reviewed Apr 9, 2025

View reviewed changes

...it_tests/core/quantization/node_quantization_config/test_node_weights_quantization_config.py Show resolved Hide resolved

liord added 4 commits April 10, 2025 14:13

merge main

07c18a3

update schema v2 with pydantic v2

4ef531a

update schema v2 with pydantic v2

1703e3a

Suppress specific warning regarding pydantic v2 in docsrc creation.

e717616

github-actions bot added the auto:docsrc label Apr 14, 2025

ofirgo approved these changes Apr 14, 2025

View reviewed changes

lior-dikstein merged commit 660ba89 into SonySemiconductorSolutions:main Apr 14, 2025
28 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove `pydantic<2.0` pin in requirements.txt#1398

Remove `pydantic<2.0` pin in requirements.txt#1398
lior-dikstein merged 10 commits intoSonySemiconductorSolutions:mainfrom
glenn-jocher:patch-1

glenn-jocher commented Apr 1, 2025 •

edited

Loading

Uh oh!

glenn-jocher commented Apr 1, 2025

Uh oh!

ofirgo commented Apr 2, 2025 •

edited

Loading

Uh oh!

lior-dikstein commented Apr 2, 2025

Uh oh!

glenn-jocher commented Apr 6, 2025

Uh oh!

ofirgo Apr 9, 2025

Uh oh!

lior-dikstein Apr 10, 2025

Uh oh!

ofirgo Apr 11, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

glenn-jocher commented Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Description:

Checklist before requesting a review:

Uh oh!

glenn-jocher commented Apr 1, 2025

Uh oh!

ofirgo commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lior-dikstein commented Apr 2, 2025

Uh oh!

glenn-jocher commented Apr 6, 2025

Uh oh!

ofirgo Apr 9, 2025

Choose a reason for hiding this comment

Uh oh!

lior-dikstein Apr 10, 2025

Choose a reason for hiding this comment

Uh oh!

ofirgo Apr 11, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

glenn-jocher commented Apr 1, 2025 •

edited

Loading

ofirgo commented Apr 2, 2025 •

edited

Loading