Airflow:No module named 'metadata.generated'

[stoveZhu]

python version: 3.8.3
OM version: 1.3.1.1
OS: Linux version 4.18.0-348.7.1.el8_5.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-4) (GCC)) #1 SMP Wed Dec 22 13:25:12 UTC 2021 (CentOS Linux release 8.5.2111)

I'm trying to run the airflow service for open metadata localy instead of in a docker container. I have followed the steps in /ingestion/Makefile and /ingestion/dockerfile to create a python environment but error below is showed when trying to run the command airflow standalone. How to fix it?

ERROR [airflow.plugins_manager] Failed to import plugin openmetadata_managed_apis 

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/site-packages/airflow/plugins_manager.py", line 237, in load_entrypoint_plugins

    plugin_class = entry_point.load()

  File "/usr/local/lib/python3.8/site-packages/importlib_metadata/__init__.py", line 184, in load

    module = import_module(match.group('module'))

  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module

    return _bootstrap._gcd_import(name[level:], package, level)

  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import

  File "<frozen importlib._bootstrap>", line 991, in _find_and_load

  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked

  File "<frozen importlib._bootstrap_external>", line 783, in exec_module

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

  File "/usr/local/lib/python3.8/site-packages/openmetadata_managed_apis/plugin.py", line 15, in <module>

    from openmetadata_managed_apis.api.app import get_blueprint

  File "/usr/local/lib/python3.8/site-packages/openmetadata_managed_apis/api/app.py", line 16, in <module>

    from openmetadata_managed_apis.api.utils import import_path

  File "/usr/local/lib/python3.8/site-packages/openmetadata_managed_apis/api/utils.py", line 24, in <module>

    from openmetadata_managed_apis.utils.logger import api_logger

  File "/usr/local/lib/python3.8/site-packages/openmetadata_managed_apis/utils/logger.py", line 8, in <module>

    from metadata.generated.schema.metadataIngestion.application import (

ModuleNotFoundError: No module named 'metadata.generated'

[stoveZhu]

Now I try to copy one generated folder from a existed airflow container(same om version, same airflow version,but python 3.10.13 exactly the same as the dockerfile which is running well) to the site-packages/metadata but have error below:

ERROR - Failed to import plugin openmetadata_managed_apis
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/airflow/plugins_manager.py", line 237, in load_entrypoint_plugins
    plugin_class = entry_point.load()
  File "/usr/local/lib/python3.8/site-packages/importlib_metadata/__init__.py", line 184, in load
    module = import_module(match.group('module'))
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/openmetadata_managed_apis/plugin.py", line 31, in <module>
    api_blueprint = get_blueprint()
  File "/usr/local/lib/python3.8/site-packages/openmetadata_managed_apis/api/app.py", line 36, in get_blueprint
    module = import_path(file)
  File "/usr/local/lib/python3.8/site-packages/openmetadata_managed_apis/api/utils.py", line 41, in import_path
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.8/site-packages/openmetadata_managed_apis/api/routes/run_automation.py", line 22, in <module>
    from metadata.automations.runner import execute
  File "/usr/local/lib/python3.8/site-packages/metadata/automations/runner.py", line 23, in <module>
    from metadata.ingestion.ometa.ometa_api import OpenMetadata
  File "/usr/local/lib/python3.8/site-packages/metadata/ingestion/ometa/ometa_api.py", line 45, in <module>
    from metadata.ingestion.ometa.mixins.mlmodel_mixin import OMetaMlModelMixin
  File "/usr/local/lib/python3.8/site-packages/metadata/ingestion/ometa/mixins/mlmodel_mixin.py", line 42, in <module>
    from metadata.ingestion.ometa.mixins.lineage_mixin import OMetaLineageMixin
  File "/usr/local/lib/python3.8/site-packages/metadata/ingestion/ometa/mixins/lineage_mixin.py", line 25, in <module>
    from metadata.ingestion.lineage.parser import LINEAGE_PARSING_TIMEOUT
  File "/usr/local/lib/python3.8/site-packages/metadata/ingestion/lineage/parser.py", line 25, in <module>
    from collate_sqllineage.runner import LineageRunner
  File "/usr/local/lib/python3.8/site-packages/collate_sqllineage-1.3.2-py3.8.egg/collate_sqllineage/runner.py", line 6, in <module>
    from collate_sqllineage.core.holders import SQLLineageHolder
  File "/usr/local/lib/python3.8/site-packages/collate_sqllineage-1.3.2-py3.8.egg/collate_sqllineage/core/holders.py", line 4, in <module>
    import networkx as nx
  File "/usr/local/lib/python3.8/site-packages/networkx-3.3-py3.8.egg/networkx/__init__.py", line 19, in <module>
    from networkx import utils
  File "/usr/local/lib/python3.8/site-packages/networkx-3.3-py3.8.egg/networkx/utils/__init__.py", line 7, in <module>
    from networkx.utils.backends import *
  File "/usr/local/lib/python3.8/site-packages/networkx-3.3-py3.8.egg/networkx/utils/backends.py", line 258, in <module>
    backends = _get_backends("networkx.backends")
  File "/usr/local/lib/python3.8/site-packages/networkx-3.3-py3.8.egg/networkx/utils/backends.py", line 234, in _get_backends
    items = entry_points(group=group)
TypeError: entry_points() got an unexpected keyword argument 'group'

[Prajwal214]

@stoveZhu Can you confirm the Airflow Version you are using? Also can you try using latest OM version or supported versions.