metadata-schema API calls don't work

[markwilkinson]

What components are related to the issue?

Metadata Schemas

Which FDP are you using?

My local instance

Version

v1.16.2~51911d6

What happened?

A bunch of problems.

1. the port is stripped from the request (I am mapping 9000:80, and all requests are going to the domain without any port)

2. I changed to run the FDP on port 80. The SWAGGER GUI says that the path is "metadata-schema", but the path seems to be "schema" in the Web UI

3. the following curl fails:

   curl -v -X 'POST' --data-binary @guideline.fdp  -H 'content-type: application/json'  -H 'Authorization: Bearer e***w'  "http://localhost/schemas/"

   and so does

   curl -v -X 'POST' --data-binary @guideline.fdp  -H 'content-type: application/json'  -H 'Authorization: Bearer e***w'  "http://localhost/metadata-schemas/"

   with log saying

   fdp-1              | 2025-04-15 19:12:11,909 1244528 [http-nio-80-exec-10] INFO  nl.dtls.fairdatapoint.api.filter.LoggingFilter - http://localhost/schemas/
   fdp-1              | 2025-04-15 19:12:11,999 1244618 [http-nio-80-exec-10] ERROR nl.dtls.fairdatapoint.api.controller.exception.ExceptionControllerAdvice - Resource with provided uri prefix ('schemas') is not defined
   fdp_client-1       | 172.23.0.1 - - [15/Apr/2025:19:12:12 +0000] "POST /schemas/ HTTP/1.1" 404 71 "-" "curl/7.71.1" "-"
   fdp-1              | 2025-04-15 19:12:30,913 1263532 [http-nio-80-exec-1] INFO  nl.dtls.fairdatapoint.api.filter.LoggingFilter - http://localhost/metadata-schemas/
   fdp-1              | 2025-04-15 19:12:30,995 1263614 [http-nio-80-exec-1] WARN  nl.dtls.fairdatapoint.api.controller.exception.ExceptionControllerAdvice - Metadata schemas not found: []
   fdp_client-1       | 172.23.0.1 - - [15/Apr/2025:19:12:31 +0000] "POST /metadata-schemas/ HTTP/1.1" 400 41 "-" "curl/7.71.1" "-"
   

The Swagger GUI also fails, but it seems to be calling a URL that doesn't exist (metadata-schema, versus schema)

[kburger]

For one of our tools, the path /metadata-schemas seem to work. This runs on 1.16, so far without problems.

For example:
https://github.com/Health-RI/FairDataPointSchemaTool/blob/5a2a7eb8a8151b4cf28ba027b6681a5ce690f1b5/src/main/java/nl/healthri/fdp/uploadschema/FDP.java#L158-L170

[markwilkinson]

Hmmmm.... ok, a few questions.

1. In the Web UI, when I am browsing the registered schemas, the URL prefix is "/schemas". In the SwaggerUI, the prefix is "metadata-schemas"... is this normal/expected?

2. What is the UUID for the parent? Is it the full URL, or only the "code" component?

3. Can a parent be absent? If so, do you pass "", or do you leave the element out completely? (that might be the explanation for my second error message)

4. The closest I have come is to get the FDP to complain about unescaped " characters. I thought it was consuming JSON?

5. Is there an "Accept" type? I have tried a few obvious things and got 406's, so now I just omit that header.

Thanks!

[dennisvang]

5. Is there an "Accept" type? I have tried a few obvious things and got 406's, so now I just omit that header.

For example, I sometimes forget and use Accept: text/ttl, which would result in a 406.

However, afaik, Accept: text/turtle should work.

[dennisvang]

1. In the Web UI, when I am browsing the registered schemas, the URL prefix is "/schemas". In the SwaggerUI, the prefix is "metadata-schemas"... is this normal/expected?

This is normal. The client and API are separate applications. When you browse the client path /schemas, the client makes some requests to the API path /metadata-schemas, in the background.

[dennisvang]

@markwilkinson To see what's going on, we can use the browser network console to inspect what the client application does.

For example, here's a POST from client v1.16.3 /schemas/create to the /metadata-schemas API endpoint, in order to create a new schema (running locally using this compose file):

Request headers (as sent by the browser):

POST /metadata-schemas HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:137.0) Gecko/20100101 Firefox/137.0
Accept: application/json
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br, zstd
Content-Type: application/json
Authorization: Bearer ***
Content-Length: 1098
Origin: http://localhost
DNT: 1
Sec-GPC: 1
Connection: keep-alive
Referer: http://localhost/schemas/create
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
Priority: u=0
Pragma: no-cache
Cache-Control: no-cache

Request body (JSON generated by the client based on content I copied from the default catalog schema):

{"name":"test","description":"","abstractSchema":false,"definition":"@prefix :         <http://fairdatapoint.org/> .\n@prefix dash:     <http://datashapes.org/dash#> .\n@prefix dcat:     <http://www.w3.org/ns/dcat#> .\n@prefix dct:      <http://purl.org/dc/terms/> .\n@prefix foaf:     <http://xmlns.com/foaf/0.1/> .\n@prefix sh:       <http://www.w3.org/ns/shacl#> .\n@prefix xsd:      <http://www.w3.org/2001/XMLSchema#> .\n\n:CatalogShape a sh:NodeShape ;\n  sh:targetClass dcat:Catalog ;\n  sh:property [\n    sh:path dct:issued ;\n    sh:datatype xsd:dateTime ;\n    sh:maxCount 1 ;\n    dash:viewer dash:LiteralViewer ;\n  ], [\n    sh:path dct:modified ;\n    sh:datatype xsd:dateTime ;\n    sh:maxCount 1 ;\n    dash:viewer dash:LiteralViewer ;\n  ], [\n    sh:path foaf:homePage ;\n    sh:nodeKind sh:IRI ;\n    sh:maxCount 1 ;\n    dash:editor dash:URIEditor ;\n    dash:viewer dash:LabelViewer ;\n  ], [\n    sh:path dcat:themeTaxonomy ;\n    sh:nodeKind sh:IRI ;\n    dash:viewer dash:LabelViewer ;\n  ] .","extendsSchemaUuids":[],"suggestedResourceName":null,"suggestedUrlPrefix":null}

[markwilkinson]

Solved, thanks! I've been doing it the hard way for literally years!! My life has just turned towards the sun :-)

[markwilkinson]

(though I notice that you don't return a Location header after the creation of the new resource, which is an unexpected departure from common REST behaviours ;-) )

[dennisvang]

Good point.
I suppose the Location header is only associated with a status 201 (or 3xx), whereas the current implementation returns a 200, for some reason.
Perhaps the original idea was to return status 200 with the location in the message body?
I'll open another issue for this.

[markwilkinson]

Sorry... re-opening just to keep the conversation in one place.

I am able to create a new schema by POST, as you suggest above (with the caveat that there is no Location header in the response, and the HTTP code is 200, not 201 Created, which I would have expected).

The new schema appears in the WebUI (though labelled as "draft"?)

It does NOT appear when I do a GET on /metadata-schemas/

My suspicion is that it needs to get out of draft state, but when I do the usual thing (PUT "Published" to metadata-schemas/{UUID}/meta/state) it also tells me that /metadata-schemas/{UUID} doesn't exist.

So... I'm stuck again :-(

Getting closer, though!

[dennisvang]

It does NOT appear when I do a GET on /metadata-schemas/

According to the API docs (/swagger-ui/index.html#/Metadata%20Model/getSchemas) there's a drafts query parameter that determines whether drafts are included in the GET response.

Something like http://localhost/metadata-schemas?drafts=true.

This does require authentication (admin).

[dennisvang]

My suspicion is that it needs to get out of draft state, but when I do the usual thing (PUT "Published" to metadata-schemas/{UUID}/meta/state) it also tells me that /metadata-schemas/{UUID} doesn't exist.

@markwilkinson Judging from what the fdp-client does, I guess you'd need to POST to

/metadata-schemas/{UUID}/versions

with a request body like this:

{"major":"1","minor":"0","patch":"0","description":"test","published":true,"version":"1.0.0"}

I'm not sure why the fdp-client posts redundant version identifiers, as in both version and separate major/minor/patch values.

Actually, the API docs (/swagger-ui/index.html#/Metadata Model/releaseSchemaVersion) suggest only the version is needed:

{"version": "1.0.0", "description": "string", "published": true}