6.0 Back End: PGS Metadata Template Upload and Validation

[sprintell]

• We need to extend the metadata file upload ...endpoint of the deposition service to recognize different data-types
• The data-type can have either value "PGS" or "GWAS"
• Whenever the data-type is "PGS", the deposition service will trigger the PGS Managed Validation Rest API on Google Cloud Engine here - https://deposition-validator-dot-pgs-catalog.appspot.com/validate_metadata
• After validation the response, payload containing validation status and all generated PCST-IDs should be returned to the front end from the deposition service a

[sandsebi]

@sprintell do the PGS metadata files need to go through sumstats and metadata validation? Similar to what happens with GWAS type files? Or should I ignore these steps for PGS type files and just rely on the API response of https://deposition-validator-dot-pgs-catalog.appspot.com/validate_metadata for validation?

[sprintell]

Hi @sandsebi

Do the PGS metadata files need to go through GWAS sumstats and metadata validation? NO

As you mentioned, they are metadata files, hence its only meant to go through metedata validation only, and not GWAS's metadata validation but PGS's

The (https://deposition-validator-dot-pgs-catalog.appspot.com/validate_metadata) Rest API Call and response is the PGS metedata validation hosted by PGS Team, hence the Rest API call and response is all that is required at this step.

[sandsebi]

Thanks for confirming @sprintell
Currently there are no PCST-IDs being returned from the response of the PGS metadata validator:

curl -X POST -F "file=@Test_valid.xlsx" https://deposition-validator-dot-pgs-catalog.appspot.com/validate_metadata
{
  "errorMessages": {},
  "valid": true,
  "warningMessages": {}
}

curl -X POST -F "file=@Test_invalid.xlsx" https://deposition-validator-dot-pgs-catalog.appspot.com/validate_metadata
{
  "errorMessages": {
    "Performance Metrics": [
      "(Lines: 4) Score name 'SCORE_NAME2' from the Performance Metrics spreadsheet can't be found in the Score(s) spreadsheet!"
    ],
    "Sample Descriptions": [
      "(Lines: 2) Can't fetch the GWAS information for the study GCST003219A",
      "No correct Sample Score entries found in this spreadsheet (from GWAS or used in Score Development)"
    ]
  },
  "valid": false,
  "warningMessages": {
    "Sample Descriptions": [
      "(Lines: 4) Can't find a corresponding cohort ID in the Cohort spreadsheet for 'AUGURRR'"
    ]
  }
}

Just wanted to confirm if this is expected from our side or the PGS API?

[sandsebi]

Regarding the PCT-IDs, there was this discussion on Slack: https://gwas-catalog.slack.com/archives/C07NNQ513GV/p1754643022266659
Where Florent mentioned that the validation API has been modified to return:

{
  "errorMessages": {},
  "scoreNames": [
    "SCORE_NAME1"
  ],
  "valid": true,
  "warningMessages": {}
}

But when I tested this, I could not seem to get the scoreNames field in the response.

[sandsebi]

Other valid points to consider for the future from @ala-ebi :

• Should the validation URL have a Dev instance and Prod instance?
• Do we need to have an auth mechanism before calling the API? Currently the API is open
• Should we add new audit events for PGS?

[sandsebi]

To test the API on Local:

curl -v -X POST http://localhost:8082/v1/submissions/689a0759ae4ad1000171cba4/uploads/ -F "file=@Test_valid.xlsx" -F "data-type=PGS"

curl -v -X POST http://localhost:8082/v1/submissions/689a0759ae4ad1000171cba4/uploads/ -F "file=@Test_invalid.xlsx" -F "data-type=PGS"

[sprintell]

@ala-ebi will review this to be sure, the error messages from the template validation are properly preented on the U.I

[ala-ebi]

API looks good to me.

Works in local but not on sandbox, probably caused by a firewall policy.

[sprintell]

We need to raise an ITS ticket to remove firewal policy toacces - https://deposition-validator-dot-pgs-catalog.appspot.com/validate_metadata