Seeking Clarification on MySQL Query Log File Support in OpenMetadata 1.8.2????

[ZYH654]

Subject: Seeking Clarification on MySQL Query Log File Support in OpenMetadata 1.8.2

Dear Community,

I’m currently using OpenMetadata version 1.8.2 and noticed in the documentation that for databases not natively supported, we can use query log files (e.g., CSV) to generate lineage.

I placed the MySQL query log file (CSV format) on the OpenMetadata server under /opt/, but when running the lineage task, it doesn’t seem to process the file. Could you kindly clarify:

Is this an issue with my configuration, or does OpenMetadata 1.8.2 not support MySQL log files for lineage extraction yet?

If supported, are there specific formatting requirements for the CSV file?

Any guidance or suggestions would be greatly appreciated. Thank you for your time and support!

Best regards,
Mr zhang

Here is my configuration

import yaml

from metadata.workflow.metadata import MetadataWorkflow

CONFIG = """
source:
  type: query-log-lineage
  serviceName: mysql-demo
  sourceConfig:
    config:
      type: DatabaseLineage
      queryLogDuration: 1
      parsingTimeoutLimit: 300
      resultLimit: 1000
      queryLogFilePath: /opt/
      overrideViewLineage: false
      processViewLineage: true
      processQueryLineage: true
      processStoredProcedureLineage: true
      threads: 1
sink:
  type: metadata-rest
  config: {}
workflowConfig:
  loggerLevel: INFO  # DEBUG, INFO, WARNING or ERROR
  openMetadataServerConfig:
    hostPort: 'http://x.x.x.x:8585/api'
    authProvider: openmetadata
    securityConfig:
      jwtToken: eyJraWQiOiJHYjM4OWEtOWY3Ni1nZGpzLWE5MmotMDI0MmJrOTQzNTYiLCJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJvcGVuLW1ldGFkYXRhLm9yZyIsInN1YiI6ImxpbmVhZ2UtYm90Iiwicm9sZXMiOlsiTGluZWFnZUJvdFJvbGUiXSwiZW1haWwiOiJsaW5lYWdlLWJvdEBvcGVuLW1ldGFkYXRhLm9yZyIsImlzQm90Ijp0cnVlLCJ0b2tlblR5cGUiOiJCT1QiLCJpYXQiOjE3NTI0OTA1NjksImV4cCI6bnVsbH0.vQAEwwP6u-DfibnQ3EjTUB3oYCL8QVtHRduW3fdp-NICWPz2KQBo4mgfh29O8u8jT3QPNjvHJD1qGzFqmTbrqDgASLpIFaoBrACGnblG21dY73K2QiNrli9aWW-XaPFkGYmor5MRXv1gqBL2lVWBgFkLFIPfQonl5YqitMQCwVOvBSqUaKMwwM5OUchbZa6FrTVsfQLHh58kwY7qm9KO9dI_G04K3TuTpjyI8oyNAHFem79XszMoqj83W9JLDniiSqJxyM7qUpSmezA2tzjJmXN_ZUz38eF3fHp1cZZhlRd0v7u0u1w6j8EVrm7_9fs6jh1m76wVq1kDe-MxvJ0hBQ
    storeServiceConnection: true
    
"""


def run():
    workflow_config = yaml.safe_load(CONFIG)
    workflow = MetadataWorkflow.create(workflow_config)
    workflow.execute()
    workflow.raise_from_status()
    workflow.print_status()
    workflow.stop()

if __name__ == "__main__":
  run()