Add platform-related metadata in Lineage records

[jorgee]

close #5077

This PR retrieves the platform metada during the onWorkflowCreate and stores in the seqeraPlatform property in the WorkflowMetadata class. The lineage module has been extended adding a metadata field in the WorkflowRun record including the platform metadata.

This PR includes the changes of #6069.

To tests the whole integration, it is required to generate a nf-launcher image with the nextflow changes as in e2e tests. Then , use tw launch to run a workflow including the options to use a modified launcher image (--launcher-image) and a nextflow.config file (--config) with the following lineage configuration

lineage.enabled=true
lineage.store.location='s3://<bucket>/lineage'

Use a <bucket> associated to the CE running the pipeline to ensure you have the proper permissions.

Once finished, you can use the nextflow lineage commands to inspect the workflow lineage records.

Example of the generated metadata extracted form a test execution:

"metadata" : {
    ...
    "seqeraPlatform": {
        "workflowId": "4mjnvcrG15bCMp",
        "user": {
          "id": "13898",
          "userName": "jorge-ejarque",
          "email": "jorge.ejarque@seqera.io",
          "firstName": "Jorge",
          "lastName": "Ejarque",
          "organization": null
        },
        "workspace": {
          "id": "20121123931769",
          "name": "jorge-test-workspace",
          "fullName": "Jorge test workspace",
          "organization": "test-org-jorge"
        },
        "computeEnv": {
          "id": "3xjr2I8RofsG5Kx0pjZQoQ",
          "name": "test-ce",
          "platform": "aws-cloud"
        },
        "pipeline": {
          "id": "51175491284294",
          "name": "https://github.com/nextflow-io/hello",
          "revision": "master",
          "commitId": null
        },
        "labels": []
      },
}

TODO:

• Check if version upgrade is required

[netlify]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	0a84715
🔍 Latest deploy log	https://app.netlify.com/projects/nextflow-docs-staging/deploys/69973324337d170008243a73

[jorgee]

Updates:

Regarding increasing the version, I have tested reading WorkflowRun generated with this PR with 25.10.0 and olded versions and viceversa.

• Records generated wih 25.04.x can be read with this PR metadata appear as null as expected.
• Records generated with this PR can be read since 25.07.0-edge, but metadata field is not shown.
• We can't read in 25.04.x because of the spec change. They could be read if we backport commit 6328613 where we added the spec management.

Apart form that, a minor comment is that there are some duplicated data such as run name, session id, repository, script files or revision. They are already Workflow or WorkflowRun properties and they are also inside metadata. Should these properties be removed form metadata ?

[pditommaso]

Would not make sense to pass both TowerCommonApi and HxClient via the constructor?

[jorgee]

fixed in 27e3124

[jorgee]

I have updated the code master and make it also compatible with changes in #6818 (renaming the variable seqeraPlatform to platform and other minor changes).

Example of what is included in WorkflowRun

"platform": {
        "workflowId": "PgoP1b2Di3aBq",
        "user": {
          "id": "13898",
          "userName": "jorge-ejarque",
          "email": "jorge.ejarque@seqera.io",
          "firstName": "Jorge",
          "lastName": "Ejarque",
          "organization": "Seqera"
        },
        "workspace": {
          "id": "20121123931769",
          "name": "jorge-test-workspace",
          "fullName": "Jorge test workspace",
          "organization": "test-org-jorge"
        },
        "computeEnv": {
          "id": "4xKOsGp63s3SHeJ4MTThwT",
          "name": "test-ce-aws",
          "platform": "aws-batch"
        },
        "pipeline": {
          "id": "51175491284294",
          "name": "https://github.com/nextflow-io/hello",
          "revision": "master",
          "commitId": null
        },
        "labels": null
      }