Skip to content

MLFlow ResponseError('too many 500 error responses') if try to log to deleted experiment #20555

@millskyle

Description

@millskyle

Bug description

If one tries to log to a deleted MLFlow experiment, (e.g. you delete the experiment in the MLFlow UI, go on holidays, come back renewed and refreshed and are greeted by your previously working code no longer working), Lightning throws a ResponseError('too many 500 error responses').

I will create a PR with a proposed simple fix.

What version are you seeing the problem on?

v2.5

How to reproduce the bug

Error messages and logs

mlflow.exceptions.MlflowException: API request to http://localhost:5000/api/2.0/mlflow/runs/create failed with exception HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: /api/2.0/mlflow/runs/create (Caused by ResponseError('too many 500 error responses'))

Environment

Current environment
#- PyTorch Lightning Version (e.g., 2.5.0):
#- PyTorch Version (e.g., 2.5):
#- Python version (e.g., 3.12):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):

More info

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingneeds triageWaiting to be triaged by maintainersver: 2.5.x

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions