How to use the file sink without openMetadataServerConfig field

[tonxxd]

Hi, I am trying to run the ingestion for testing using the file sink without having a full openmetadata instance running. But the script fails if it cannot reach a valid openmetadata instance

raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: /v1/version (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10e09e860>: Failed to establish a new connection: [Errno 61] Connection refused'))

Any idea if and how this can be achieved? Thanks!

[harshach]

@pmbrull looks like we made serverConfig mandatory?

[tonxxd]

After playing with it a bit it seems the ingestion cli tries to fetch data from the openmetadata apis even before starting the ingestion and connecting to the source. Thus, unless there is a different way I am not aware of it seems it's not possible to use the ingestion framework disjointed from the full platform (e.g. the file sink used by datahub https://datahubproject.io/docs/metadata-ingestion/sink_docs/file/)

[pmbrull]

hi @tonxxd thanks for raising this. This looks like a side effect from when we introduced the version validation. Could you please open an issue? We'll handle it. Thanks