Create Reporting layer

[gueniai]

Let's start building our reporting (or platinum) layer. We should start with the main dimensions (job, cluster, notebok, etc), plus a simplified and aggregated view of clsf and jrcp, to make it easy to get daily aggregates.
Something we'll have to take into account will be in identifying changes to rows we've already processed, since there could be slight changes in gold due to late arriving events.

Stories

• closes Create Reporting Layer for Clusters #1276
• closes Reporting cluster 1276 #1289
• closes Create Reporting Layer for Jobs #1277
• closes Migrate legacy canned dashboards to new reporting layer #1294

[neilbest-db]

Is there a branch and/or PR for this? Just curious. Could not find anything.

[neilbest-db]

I guess the folder here(/Workspace/Users/sourav.banerjee@databricks.com/1. OVERWATCH/Platinum Layer) in overwatch-field must be the work in progress.