ML pipelines: RunInference - OSS Image Object detection, OSS Image Captioning, OSS Image Classification #37186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open

Amar3tto wants to merge 20 commits into master from oss-image-detection

+3,092 −20

GitHub Actions / Python 3.13 Test Results (ubuntu-latest) failed Jan 2, 2026 in 0s

7 errors, 7 skipped, 10 pass in 7m 8s

2 files ±0 2 suites ±0 7m 8s ⏱️ - 1m 33s
24 tests ±0 10 ✅ - 7 7 💤 ±0 0 ❌ ±0 7 🔥 +7
31 runs ±0 10 ✅ - 7 14 💤 ±0 0 ❌ ±0 7 🔥 +7

Results for commit 64187d9. ± Comparison against earlier commit a22d694.

Annotations

Check failure on line 0 in apache_beam.ml.rag.ingestion.milvus_search_it_test.TestMilvusVectorWriterConfig

github-actions / Python 3.13 Test Results (ubuntu-latest)

test_invalid_write_on_missing_primary_key_in_entity (apache_beam.ml.rag.ingestion.milvus_search_it_test.TestMilvusVectorWriterConfig) with error

sdks/python/test-suites/tox/py313/build/srcs/sdks/python/pytest_py313-ml.xml [took 16s]

Raw output


            failed on setup with "testcontainers.core.exceptions.ContainerStartException: Container should be started before getting logs"
self = <docker.api.client.APIClient object at 0x7f59083696a0>
response = <Response [500]>

    def _raise_for_status(self, response):
        """Raises stored :class:`APIError`, if one occurred."""
        try:
>           response.raise_for_status()

target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/docker/api/client.py:275: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [500]>

    def raise_for_status(self):
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )
    
        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.48/containers/0135b0267c935feb707b0c1f44e5dfca91973aeb538c41024f63f24ddde39121/start

target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/requests/models.py:1026: HTTPError

The above exception was the direct cause of the following exception:

image = 'milvusdb/milvus:v2.5.10', max_vec_fields = 5
vector_client_max_retries = 3, tc_max_retries = None

    @staticmethod
    def start_db_container(
        image="milvusdb/milvus:v2.5.10",
        max_vec_fields=5,
        vector_client_max_retries=3,
        tc_max_retries=None) -> Optional[VectorDBContainerInfo]:
      service_container_port = TestHelpers.find_free_port()
      healthcheck_container_port = TestHelpers.find_free_port()
      user_yaml_creator = MilvusTestHelpers.create_user_yaml
      with user_yaml_creator(service_container_port, max_vec_fields) as cfg:
        info = None
        original_tc_max_tries = testcontainers_config.max_tries
        if tc_max_retries is not None:
          testcontainers_config.max_tries = tc_max_retries
        for i in range(vector_client_max_retries):
          try:
            vector_db_container = CustomMilvusContainer(
                image=image,
                service_container_port=service_container_port,
                healthcheck_container_port=healthcheck_container_port)
            vector_db_container = vector_db_container.with_volume_mapping(
                cfg, "/milvus/configs/user.yaml")
>           vector_db_container.start()

apache_beam/ml/rag/test_utils.py:158: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/testcontainers/milvus/__init__.py:82: in start
    super().start()
target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/testcontainers/core/container.py:192: in start
    self._container = docker_client.run(
target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/testcontainers/core/docker_client.py:47: in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/testcontainers/core/docker_client.py:103: in run
    container = self.client.containers.run(
target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/docker/models/containers.py:883: in run
    container.start()
target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/docker/models/containers.py:420: in start
    return self.client.api.start(self.id, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/docker/utils/decorators.py:19: in wrapped
    return f(self, resource_id, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/docker/api/container.py:1136: in start
    self._raise_for_status(res)
target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/docker/api/client.py:277: in _raise_for_status
    raise create_api_error_from_http_exception(e) from e
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

e = HTTPError('500 Server Error: Internal Server Error for url: http+docker://localhost/v1.48/containers/0135b0267c935feb707b0c1f44e5dfca91973aeb538c41024f63f24ddde39121/start')

    def create_api_error_from_http_exception(e):
        """
        Create a suitable APIError from requests.exceptions.HTTPError.
        """
        response = e.response
        try:
            explanation = response.json()['message']
        except ValueError:
            explanation = (response.text or '').strip()
        cls = APIError
        if response.status_code == 404:
            explanation_msg = (explanation or '').lower()
            if any(fragment in explanation_msg
                   for fragment in _image_not_found_explanation_fragments):
                cls = ImageNotFound
            else:
                cls = NotFound
>       raise cls(e, response=response, explanation=explanation) from e
E       docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.48/containers/0135b0267c935feb707b0c1f44e5dfca91973aeb538c41024f63f24ddde39121/start: Internal Server Error ("failed to set up container networking: driver failed programming external connectivity on endpoint mystifying_wiles (b0518e7c1a3e71866977570fcd254ee7a8910f0cbc01c1844b7a36e5c1ba8275): Bind for 0.0.0.0:36331 failed: port is already allocated")

target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/docker/errors.py:39: APIError

During handling of the above exception, another exception occurred:

cls = <class 'apache_beam.ml.rag.ingestion.milvus_search_it_test.TestMilvusVectorWriterConfig'>

    @classmethod
    def setUpClass(cls):
>     cls._db = MilvusTestHelpers.start_db_container()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

apache_beam/ml/rag/ingestion/milvus_search_it_test.py:175: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
apache_beam/ml/rag/test_utils.py:165: in start_db_container
    stdout_logs, stderr_logs = vector_db_container.get_logs()
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <apache_beam.ml.rag.test_utils.CustomMilvusContainer object at 0x7f5908368ec0>

    def get_logs(self) -> tuple[bytes, bytes]:
        if not self._container:
>           raise ContainerStartException("Container should be started before getting logs")
E           testcontainers.core.exceptions.ContainerStartException: Container should be started before getting logs

target/.tox-py313-ml/py313-ml/lib/python3.13/site-packages/testcontainers/core/container.py:283: ContainerStartException

Check failure on line 0 in apache_beam.ml.rag.ingestion.milvus_search_it_test.TestMilvusVectorWriterConfig

github-actions / Python 3.13 Test Results (ubuntu-latest)

test_invalid_write_on_non_existent_collection (apache_beam.ml.rag.ingestion.milvus_search_it_test.TestMilvusVectorWriterConfig) with error