Integrate spark-lineage with data in S3 #16095
Closed
mykola-yesypchuk-inflection
started this conversation in
General
Replies: 2 comments 1 reply
-
|
@ulixius9 can you chime in here |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
hey @mykola-yesypchuk-inflection, just a couple of days back we added the support for s3 container lineage with spark agent as part of this pr: open-metadata/openmetadata-spark-agent#17 we will soon be releasing this fix, once that's out you would be able to see the lineage between tables and containers 🙏 |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Great news. Thanks! |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hello.
I am looking for the data lineage solution and trying to integrate OpenMetadata with my data pipeline.
Currently I checking how spark-lineage and s3 storage are working together, but it seems not working. After my pipeline run I can see that UI get "spark pipeline entry" but empty lineage threads inside it.
Any suggestion?
Beta Was this translation helpful? Give feedback.
All reactions