Skip to content

LightningCLI fails loading jsonnet config files #20898

@muthissar

Description

@muthissar

Bug description

jsonargparse supports jsonnet config files. LightningCli does not: Adding parser_kwargs={"parser_mode": "jsonnet"} and inputting a jsonnet file fails. This is caused by LightningCLI._dump_config not enforcing dumping to yaml.

I do not know if jsonnet is supposed to be supported by lightning, but the bug prevents silently using jsonnet with LightningCli. Adding format="yaml" to dump fixes issue, and I don't believe it's less general (since yaml.safe_load is used after).

What version are you seeing the problem on?

master

Reproduced in studio

No response

How to reproduce the bug

class MainModule(BoringModel):
        def __init__(self, main_param: int = 1):
            super().__init__()
            

    config = """{
        "model":{
            "main_param": 2
        }
    }"""
    config_path = Path("config.jsonnet")
    config_path.write_text(config)

    cli_args = [f"--config={config_path}"]
    with mock.patch("sys.argv", ["any.py"] + cli_args):
        cli = LightningCLI(MainModule, run=False, parser_kwargs={"parser_mode": "jsonnet"})

Error messages and logs

# Error messages and logs here please

Environment

Current environment
  • CUDA:
    - GPU:
    - NVIDIA A40
    - NVIDIA A40
    - NVIDIA A40
    - NVIDIA A40
    - available: True
    - version: 12.6
  • Lightning:
    - lightning-utilities: 0.14.3
    - pytorch-lightning: 2.5.1rc2
    - torch: 2.7.1
    - torchmetrics: 1.7.2
    - torchvision: 0.22.1
  • Packages:
    - absl-py: 2.3.0
    - aiohappyeyeballs: 2.6.1
    - aiohttp: 3.12.12
    - aiosignal: 1.3.2
    - alabaster: 1.0.0
    - antlr4-python3-runtime: 4.9.3
    - anyio: 4.9.0
    - argon2-cffi: 25.1.0
    - argon2-cffi-bindings: 21.2.0
    - arrow: 1.3.0
    - asttokens: 3.0.0
    - async-lru: 2.0.5
    - attrs: 25.3.0
    - autocommand: 2.2.2
    - babel: 2.17.0
    - backcall: 0.2.0
    - backports.tarfile: 1.2.0
    - beautifulsoup4: 4.13.4
    - bitsandbytes: 0.46.0
    - black: 25.1.0
    - bleach: 6.2.0
    - certifi: 2025.4.26
    - cffi: 1.17.1
    - charset-normalizer: 3.4.2
    - click: 8.2.1
    - cloudpickle: 3.1.1
    - coloredlogs: 15.0.1
    - comm: 0.2.2
    - contourpy: 1.3.2
    - coverage: 7.8.2
    - curio: 1.6
    - cycler: 0.12.1
    - debugpy: 1.8.14
    - decorator: 5.2.1
    - deepspeed: 0.9.3
    - defusedxml: 0.7.1
    - docstring-parser: 0.16
    - docutils: 0.21.2
    - executing: 2.2.0
    - fastapi: 0.115.12
    - fastjsonschema: 2.21.1
    - filelock: 3.18.0
    - flatbuffers: 25.2.10
    - fonttools: 4.58.2
    - fqdn: 1.5.1
    - frozenlist: 1.7.0
    - fsspec: 2025.5.1
    - grpcio: 1.73.0
    - h11: 0.16.0
    - hjson: 3.1.0
    - httpcore: 1.0.9
    - httpx: 0.28.1
    - humanfriendly: 10.0
    - hydra-core: 1.3.2
    - idna: 3.10
    - imagesize: 1.4.1
    - importlib-metadata: 8.0.0
    - importlib-resources: 6.5.2
    - inflect: 7.3.1
    - iniconfig: 2.1.0
    - ipykernel: 6.29.5
    - ipyparallel: 9.0.1
    - ipython: 8.1.1
    - ipywidgets: 8.1.7
    - isoduration: 20.11.0
    - jaraco.collections: 5.1.0
    - jaraco.context: 5.3.0
    - jaraco.functools: 4.0.1
    - jaraco.text: 3.12.1
    - jedi: 0.19.2
    - jinja2: 3.1.6
    - joblib: 1.5.1
    - json5: 0.12.0
    - jsonargparse: 4.40.0
    - jsonnet: 0.21.0
    - jsonpointer: 3.0.0
    - jsonschema: 4.24.0
    - jsonschema-specifications: 2025.4.1
    - jupyter-client: 8.6.3
    - jupyter-core: 5.8.1
    - jupyter-events: 0.12.0
    - jupyter-lsp: 2.2.5
    - jupyter-server: 2.16.0
    - jupyter-server-terminals: 0.5.3
    - jupyterlab: 4.4.3
    - jupyterlab-pygments: 0.3.0
    - jupyterlab-server: 2.27.3
    - jupyterlab-widgets: 3.0.15
    - kiwisolver: 1.4.8
    - lightning-utilities: 0.14.3
    - markdown: 3.8
    - markdown-it-py: 3.0.0
    - markupsafe: 3.0.2
    - matplotlib: 3.10.3
    - matplotlib-inline: 0.1.7
    - mdurl: 0.1.2
    - mistune: 3.1.3
    - more-itertools: 10.3.0
    - mpmath: 1.3.0
    - multidict: 6.4.4
    - mypy-extensions: 1.1.0
    - nbclient: 0.10.2
    - nbconvert: 7.16.6
    - nbformat: 5.10.4
    - nest-asyncio: 1.6.0
    - networkx: 3.5
    - ninja: 1.11.1.4
    - notebook: 7.4.3
    - notebook-shim: 0.2.4
    - numpy: 2.3.0
    - nvidia-cublas-cu12: 12.6.4.1
    - nvidia-cuda-cupti-cu12: 12.6.80
    - nvidia-cuda-nvrtc-cu12: 12.6.77
    - nvidia-cuda-runtime-cu12: 12.6.77
    - nvidia-cudnn-cu12: 9.5.1.17
    - nvidia-cufft-cu12: 11.3.0.4
    - nvidia-cufile-cu12: 1.11.1.6
    - nvidia-curand-cu12: 10.3.7.77
    - nvidia-cusolver-cu12: 11.7.1.2
    - nvidia-cusparse-cu12: 12.5.4.2
    - nvidia-cusparselt-cu12: 0.6.3
    - nvidia-nccl-cu12: 2.26.2
    - nvidia-nvjitlink-cu12: 12.6.85
    - nvidia-nvtx-cu12: 12.6.77
    - omegaconf: 2.3.0
    - onnx: 1.18.0
    - onnxruntime: 1.22.0
    - outcome: 1.3.0.post0
    - overrides: 7.7.0
    - packaging: 25.0
    - pandas: 2.3.0
    - pandocfilters: 1.5.1
    - parso: 0.8.4
    - pathspec: 0.12.1
    - pexpect: 4.9.0
    - pickleshare: 0.7.5
    - pillow: 11.2.1
    - pip: 25.1
    - platformdirs: 4.3.8
    - pluggy: 1.6.0
    - prometheus-client: 0.22.1
    - prompt-toolkit: 3.0.51
    - propcache: 0.3.2
    - protobuf: 6.31.1
    - psutil: 7.0.0
    - ptyprocess: 0.7.0
    - pure-eval: 0.2.3
    - py-cpuinfo: 9.0.0
    - pycparser: 2.22
    - pydantic: 1.10.22
    - pygments: 2.19.1
    - pyparsing: 3.2.3
    - pytest: 8.4.0
    - pytest-asyncio: 1.0.0
    - pytest-cov: 6.1.1
    - pytest-random-order: 1.1.1
    - pytest-rerunfailures: 15.1
    - pytest-timeout: 2.4.0
    - python-dateutil: 2.9.0.post0
    - python-json-logger: 3.3.0
    - pytorch-lightning: 2.5.1rc2
    - pytz: 2025.2
    - pyyaml: 6.0.2
    - pyzmq: 26.4.0
    - qtconsole: 5.6.1
    - qtpy: 2.4.3
    - referencing: 0.36.2
    - requests: 2.32.4
    - rfc3339-validator: 0.1.4
    - rfc3986-validator: 0.1.1
    - rich: 14.0.0
    - roman-numerals-py: 3.1.0
    - rpds-py: 0.25.1
    - scikit-learn: 1.7.0
    - scipy: 1.15.3
    - send2trash: 1.8.3
    - setuptools: 78.1.1
    - six: 1.17.0
    - sniffio: 1.3.1
    - snowballstemmer: 3.0.1
    - sortedcontainers: 2.4.0
    - soupsieve: 2.7
    - sphinx: 8.2.3
    - sphinxcontrib-applehelp: 2.0.0
    - sphinxcontrib-devhelp: 2.0.0
    - sphinxcontrib-htmlhelp: 2.1.0
    - sphinxcontrib-jsmath: 1.0.1
    - sphinxcontrib-qthelp: 2.0.0
    - sphinxcontrib-serializinghtml: 2.0.0
    - stack-data: 0.6.3
    - starlette: 0.46.2
    - sympy: 1.14.0
    - tensorboard: 2.19.0
    - tensorboard-data-server: 0.7.2
    - tensorboardx: 2.6.4
    - terminado: 0.18.1
    - testpath: 0.6.0
    - threadpoolctl: 3.6.0
    - tinycss2: 1.4.0
    - tomli: 2.0.1
    - torch: 2.7.1
    - torchmetrics: 1.7.2
    - torchvision: 0.22.1
    - tornado: 6.5.1
    - tqdm: 4.67.1
    - traitlets: 5.14.3
    - trio: 0.30.0
    - triton: 3.3.1
    - typeguard: 4.3.0
    - types-python-dateutil: 2.9.0.20250516
    - typeshed-client: 2.7.0
    - typing-extensions: 4.14.0
    - tzdata: 2025.2
    - uri-template: 1.3.0
    - urllib3: 2.4.0
    - uvicorn: 0.34.3
    - wcwidth: 0.2.13
    - webcolors: 24.11.1
    - webencodings: 0.5.1
    - websocket-client: 1.8.0
    - werkzeug: 3.1.3
    - wheel: 0.45.1
    - widgetsnbextension: 4.0.14
    - yarl: 1.20.1
    - zipp: 3.19.2
  • System:
    - OS: Linux
    - architecture:
    - 64bit
    - ELF
    - processor: x86_64
    - python: 3.13.4
    - release: 6.8.0-60-generic
    - version: Fix typo in README.md #63-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 19:04:15 UTC 2025

More info

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions