Skip to content

Commit 2d81412

Browse files
committed
[DOP-22410] Update README
1 parent d7175b3 commit 2d81412

File tree

1 file changed

+5
-4
lines changed

1 file changed

+5
-4
lines changed

README.rst

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -41,16 +41,17 @@ Goals
4141
* Collect lineage events produced by OpenLineage clients & integrations (Spark, Airflow).
4242
* Support consuming large amounts of lineage events, by using Kafka as event buffer and storing data in tables partitioned by event timestamp.
4343
* Store operation-grained events (instead of job grained `Marquez <https://marquezproject.ai/>`_), for better detalization.
44-
* Provide API for building run ↔ dataset lineage, as well as parent run → children run lineage.
45-
* Ability to build lineage graph with specific time boundaries (unlike Marquez there lineage is build only for last job run).
46-
* Ability to build lineage graph with different granularity. e.g. merge all individual Spark operations into Spark applicationId or Spark applicationName.
44+
* Provide API for fetching run ↔ dataset lineage.
45+
* Allow building lineage graph with specific time boundaries (unlike Marquez there lineage is build only for last job run).
46+
* Allow building lineage graph with different granularity. e.g. merge all individual Spark operations into Spark applicationId or Spark applicationName.
47+
* Include column-level lineage into lineage graph.
4748

4849
Non-goals
4950
---------
5051

5152
* This is **not** a Data Catalog. Use `Datahub <https://datahubproject.io/>`_ or `OpenMetadata <https://open-metadata.org/>`_ instead.
5253
* Static Data Lineage like view → table is not supported.
53-
* Currently column-level lineage is collected by OpenLineage, but not yet consumed by Data.Rentgen.
54+
* Job/run/operation are always a part of lineage graph. Hiding them to produce dataset → dataset lineage is not supported for now.
5455

5556
.. documentation
5657

0 commit comments

Comments
 (0)