Introduce the layer table and remove layer information from the partition table

[abey79]

Related

• part of https://linear.app/rerun/project/demo-ready-oss-server-for-data-transformation-pipelines-d0a110693312/
• part of https://linear.app/rerun/issue/RR-2532/pr-tracking-demo-read-oss-server-for-data-transformation-pipelines
• part of https://linear.app/rerun/issue/RR-2476/remove-layer-specific-columns-from-the-partition-table-and-introduce-a
• sibling of TODO

What

Introduces new grpc endpoint for layer table:

rpc GetLayerTableSchema(GetLayerTableSchemaRequest) returns (GetLayerTableSchemaResponse) {}
rpc ScanLayerTable(ScanLayerTableRequest) returns (stream ScanLayerTableResponse) {}

• add new grpc endpoints to proto
• update ext utilities
• fix OSS server
•  add table provider
• rename everything to DatasetManifest (from LayerTable)
• update Python SDK API
• update catalog?

[github-actions]

Web viewer built successfully.

Result	Commit	Link	Manifest
✅	abc37b0	https://rerun.io/viewer/pr/11423	+nightly +main

^{Note: This comment is updated whenever you push a commit.}