Lineage notebook 00-UC-lineage updated limitations

[mattb-db]

The limitations listed in the 00-UC-lineage notebook need updating to reflect the current state of the product.

Current limitations listed are:

• Streaming operations are not yet supported
• Lineage will not be captured when data is written directly to files in cloud storage even if a table is defined at that location (eg spark.write.save(“s3:/mybucket/mytable/”) will not produce lineage)
• Lineage is not captured across workspaces (eg if a table A > table B transformation is performed in workspace 1 and table B > table C in workspace 2, each workspace will show a partial view of the lineage for table B)
• Lineage is computed on a 90-day rolling window, meaning that lineage will not be displayed for tables that have not been modified in more than 90 days ago

Lineage is not computed on a one-year rolling window. Also lineage is now aggregated for all workspaces attached to the same UC metastore. Maybe it's better to link to the limitations in the documentation?

[QuentinAmbard]

@mattb-db do you want to fork & send a PR with your fixes ?

[mattb-db]

Yes will do.