[FEATURE]: Identify owners for inventory types that we track the history of

[asnare]

Is there an existing issue for this?

• I have searched the existing issues

Problem statement

We will shortly be tracking history for inventory types that are routinely refreshed during migration. The history journal that we maintain will require that for each record an owner is available: the owner is the person (or group) responsible for the underlying resource being migrated. If this is not available, the workspace administrator should be used instead.

Proposed Solution

Each crawler that is responsible for a refreshable class will need to be updated to have code that can identify the owner for its Result type.

Documentation

• [TECH DEBT]: Document the owners for inventory types  #3067

Additional Context

Related issues:

• [FEATURE] Create a ucx.history table #2572
• Add UCX history schema and table for storing UCX's artifact #2744

Blocks:

• [FEATURE] Persist the migration-progress crawled objects in the ucx.history table #2573
• Crawlers: append snapshots to history journal, if available #2743

[asnare]

We need to track owners for the following inventory types:

• ClusterInfo:

  1. Existing creator field, which is optional.
  2. Workspace admin.

• DirectFsAccess:

  1. For:

     • Queries: the user attribute of the query¹, which is optional.
     • Jobs: the owner of the DBFS or Workspace path¹² for the notebook or source, which is optional.

  2. Workspace admin.

• Grant, Table and UDF:

  1. Workspace admin.

  Note: No attempt to determine creator via table/UDF properties.

• JobInfo:

  1. Existing creator field, which is optional.
  2. Workspace admin.

• PipelineInfo:

  1. Existing creator_name field, which is optional.
  2. Workspace admin.

• PolicyInfo:

  1. Existing creator field, which is optional.
  2. Workspace admin.

• TableMigrationStatus:

  1. The owner of the source table, as defined above.

Where the workspace admin is needed (because a more appropriate owner cannot be determined) the algorithm is as follows:

1. Query all active workspace admins, sort alphabetically by user-name, use the first one.
2. If there are no workspace admins, query all active account admins associated with the workspace, sort alphabetically by user-name, use the first one.
3. Raise an error if we still don't have an identity. (Note: this is possible, especially due to accounts being decommissioned and leaving a workspace without an active admin.)

Footnotes

1. This is not yet available on the DirectFsAccess instances; a schema change will probably be required along with code to expose this information. ↩ ↩²

2. The APIs for DBFS and Workspace paths don't expose the owner/creator information, so this information is unavailable. If it were available, this would first be exposed via the .owner attribute of our pathlib emulation. ↩