Parsing GitHub API OpenAPI spec fails (recursion error)

[alexpovel]

GitHub itself has an API, described using OpenAPI. The source files are in this repository.

However, running

$ datamodel-codegen --url "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json"

fails, with the traceback tail being

  File "/usr/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.9/copy.py", line 137, in deepcopy
    d = id(x)
RecursionError: maximum recursion depth exceeded while calling a Python object

(there's hundreds of these from the recursion, so I'm omitting the full traceback).

Running datamodel-codegen version 0.13.0 in Python 3.9.2 on Debian 11. I also have a poetry.lock file if need be.

Is this an issue with the JSON file itself? Or maybe with pydantic? This is the head of the traceback (omitting the hundreds of recursive calls in the middle again), where pydantic seems to be the origin of the calls:

/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py:260: UserWarning: format of 'uri-template' not understood for 'string' - using default
  warn(
Traceback (most recent call last):
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/__main__.py", line 541, in main
    generate(
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/__init__.py", line 367, in generate
    results = parser.parse()
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/base.py", line 449, in parse
    self.parse_raw()
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/openapi.py", line 393, in parse_raw
    self.parse_raw_obj(
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py", line 1149, in parse_raw_obj
    self.parse_obj(name, JsonSchemaObject.parse_obj(raw), path)
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py", line 1164, in parse_obj
    self.parse_object(name, obj, path)
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py", line 599, in parse_object
    fields=self.parse_object_fields(
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py", line 545, in parse_object_fields
    field_type = self.parse_item(modular_name, field, [*path, field_name])
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py", line 645, in parse_item
    return self.parse_array_fields(
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py", line 753, in parse_array_fields
    data_types=self.parse_list_item(
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py", line 720, in parse_list_item
    return [
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py", line 721, in <listcomp>
    self.parse_item(
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py", line 639, in parse_item
    return self.get_ref_data_type(item.ref)
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/parser/jsonschema.py", line 436, in get_ref_data_type
    return self.data_type(reference=reference)
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/types.py", line 209, in __init__
    super().__init__(**values)
  File "/home/alex/.cache/pypoetry/virtualenvs/avcn-7lhM2XME-py3.9/lib/python3.9/site-packages/datamodel_code_generator/reference.py", line 45, in __init__
    super().__init__(**values)
  File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1038, in pydantic.main.validate_model
  File "pydantic/fields.py", line 857, in pydantic.fields.ModelField.validate
  File "pydantic/fields.py", line 1074, in pydantic.fields.ModelField._validate_singleton
  File "pydantic/fields.py", line 1121, in pydantic.fields.ModelField._apply_validators
  File "pydantic/class_validators.py", line 313, in pydantic.class_validators._generic_validator_basic.lambda12
  File "pydantic/main.py", line 679, in pydantic.main.BaseModel.validate
  File "pydantic/main.py", line 605, in pydantic.main.BaseModel._copy_and_set_values
  File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.9/copy.py", line 205, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/usr/lib/python3.9/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib/python3.9/copy.py", line 270, in _reconstruct
    state = deepcopy(state, memo)

Perhaps I'm doing something wrong as well? Sorry too if this is wrong for 'discussions', I can also post it as an issue if desired. Please also let me know if the issue is not rooted in datamodel-code-generator, I'll pack my bags.

[alexpovel]

Changing uri-template to uri fixes the warning (UserWarning: format of 'uri-template' not understood for 'string' - using default) but doesn't otherwise alleviate the problem. (I have no idea how OpenAPI works!)

[k2bd]

I am not sure if this is helpful to you, but I had the same issue parsing the GitHub Webhook events spec (which is in jsonschema) using pydantic 1.9.1, but downgrading to 1.9.0 works fine.

[alexpovel]

Thanks for the hint. I had converted this discussion into an issue, in which I eventually got it working: #781 .