feat: Introducing Dataset Permissions

[JHoelli]

Purpose

Intoducing first simple permissions for Dataset.

Assumptions

• A newly created dataset inherits the permissions of the pipeline admin who created it.
• Permissions at the dataset level are user-specific, as in Connect and Pipelines.
• Option to add additional permissions (similar to e.g., Connect)

Concept

Role	Read	Write
Admin	X	X
Asset Admin	-	-
Asset User	-	-
Connect Admin	-	-
Data Explorer Admin	X	-
Data Explorer User	X	-
Dashboard Admin	X	-
Dashboard User	X	-
Pipeline User	X	-
Pipeline Admin	X	X

Example

A dashboard user or admin, the uses a chart which on the other hand is based on a dataset, also needs to be given explicit rights on that dataset to be able to see the chart with values.

Remarks

Backend

• Permission for DataLakeMeasureResources
• Permissions for DataLakeResources

UI

• Section Dataset Permission, so that everyone can see the datasets assigned
• Add Option Share Dataset in the Dataset Section

Migration

• Migrate all existing measures with new permission (--> setting them to public)

Testing

• Write End2End Tests

PR introduces (a) breaking change(s): no

PR introduces (a) deprecation(s): no

[tenthe]

Hi @JHoelli,

thanks a lot for the PR, this is a really cool new functionality.
I tested it locally and everything worked as expected, including the migration.

I only have a few minor code comments.

[tenthe]

Hi @JHoelli,

I just realized we need to change the logic for setting data lake permissions. Currently, this is handled in PipelineManager.addPipeline, but this is not the only place where a measurement can be created. A measurement can also be created via the API, so this logic should be moved to the REST interface, DataLakeMeasureResource.

[JHoelli]

Current Issue

When moving permission management for datasets from the Pipeline Manager to the DataLakeResource, we are facing an issue where the principal ID is lost if a data lake is created from a pipeline. This occurs because the getAuthenticatedUserSid() method only returns the user ID of the service principal (sp-service-client), which results in incorrect ownership assignments.

On the other hand, when a data lake is created via API, the correct ownership is assigned, as the user token is properly used.

Creating a data lake via API results in the correct ownership due to the provided user token.

Possible Solutions

Move the Measurement Creation to the DataSinkInvocation

Idea: Split the measurement creation and update process. Only create the measurement and pipeline permissions within the Pipeline Manager.

Pros:

• Separates concerns, keeping pipeline management cleaner.

• Allows permissions to be set specifically for the pipeline context (using the SchemaManager not the ClientAPI).

Cons:

• more complex flow management.

• Reusability

Add an additional Header

Idea: Introduce a new HTTP header to specify if the authenticated user is an admin. If the user is an admin, allow the system to use the user ID from the token (via TokenAuthenticationFilter) instead of the service principal ID or introduce an 'onbehalfof'.

Reasoning: If a user already has admin rights, setting another user reduces those rights.

Pros:

• Reusable solution for other use cases.

• Ensures the correct user ownership.

Cons:

• may be harder to understand.
• no other usage of such logic.

Create Permissions in the Pipeline Manager and only check if Permissions are available in the DataMeasureResource

Idea: Revert to the old approach: create the permissions in the Pipeline Manager with the authenticated user. In the DataMeasureResource, simply check whether permissions already exist. If not, use the currently authenticated user to set the permission.

Pros:

• Easy to implement.

• Ensures functionality with the StreamPipes client and core systems.

Cons:

• Permissions are managed in two separate places, making the implementation less clean.

• Reduces reusability, as permission management is not centralized.