Too restrictive media-type links (again)

[fmigneault]

As previously raised in #181 and (partially) fixed in #182, the type property of item/collection links are limited to a specific subset, ie:

stac-pydantic/stac_pydantic/shared.py

Lines 46 to 73 in 270a7da

	geotiff = "image/tiff; application=geotiff"
	cog = "image/tiff; application=geotiff; profile=cloud-optimized"
	jp2 = "image/jp2"
	png = "image/png"
	jpeg = "image/jpeg"
	# Vector
	geojson = "application/geo+json"
	geojsonseq = "application/geo+json-seq"
	geopackage = "application/geopackage+sqlite3"
	kml = "application/vnd.google-earth.kml+xml"
	kmz = "application/vnd.google-earth.kmz"
	pbf = "application/x-protobuf"
	mvt = "application/vnd.mapbox-vector-tile"
	# Others
	hdf = "application/x-hdf"
	hdf5 = "application/x-hdf5"
	xml = "application/xml"
	json = "application/json"
	ndjson = "application/ndjson"
	html = "text/html"
	text = "text/plain"
	openapi = "application/vnd.oai.openapi+json;version=3.0"
	openapi_yaml = "application/vnd.oai.openapi;version=3.0"
	jsonschema = "application/schema+json"
	pdf = "application/pdf"
	csv = "text/csv"
	parquet = "application/vnd.apache.parquet"
	octet_stream = "application/octet-stream"

We are trying to integrate link references that contains application/x-netcdf type, and the STAC-FastAPI refuses the pushed STAC Item because of this. I would expect many more types to be causing problem, such as text/markdown, text/x-rst, application/vnd.restful+json, etc.

Therefore, the fix should be revisited, either to allow any types, but preferably with a string pattern check, to avoid revisiting this again when yet another type is needed later on. A potential pattern could be something similar to [\w-]+/[\w\-.]+\s*(;.+)* to allow the type/subtype check, and the optional parameters (eg: profile=cloud-optimized, charset=UTF-8, etc.).

[fmigneault]

@gadomski @vincentsarago FYI
This is currently blocking our integration of climate datasets with proper attribution and documentation referencing of source data, which is required by the terms-of-use of upstream providers.

[gadomski]

Can you provide more information on the error? As of #182 Link should allow arbitrary strings for the type field ... if it's not, then it's a bug.

[vincentsarago]

Happy to review any PR 🙏

[fmigneault]

Our STAC API + PgSTAC is running v6.0.0.

pydantic==2.11.7
pydantic-settings==2.10.1
pydantic_core==2.33.2
pypgstac==0.9.8
stac-fastapi.api==6.0.0
stac-fastapi.extensions==6.0.0
stac-fastapi.pgstac==6.0.0
stac-fastapi.types==6.0.0
stac-pydantic==3.4.0

We POST a STAC Item with a link similar to:

...
"links": [
  {"href": "https://.../some-netcdf.nc", "type": "application/x-netcdf", "rel": "source"}
]

The API responds with this type not being one of the above Enum media-types.

[henriaidasso]

@gadomski here is the JSON error obtained with a status code 400 bad request, indicating that application/x-netcdf is not allowed as the type attribute of object links[0].

{
                "type": "enum",
                "loc": [
                    "body",
                    "Item",
                    "links",
                    0,
                    "type"
                ],
                "msg": "Input should be 'image/tiff; application=geotiff', 'image/tiff; application=geotiff; profile=cloud-optimized', 'image/jp2', 'image/png', 'image/jpeg', 'application/geo+json', 'application/geo+json-seq', 'application/geopackage+sqlite3', 'application/vnd.google-earth.kml+xml', 'application/vnd.google-earth.kmz', 'application/x-protobuf', 'application/vnd.mapbox-vector-tile', 'application/x-hdf', 'application/x-hdf5', 'application/xml', 'application/json', 'application/ndjson', 'text/html', 'text/plain', 'application/vnd.oai.openapi+json;version=3.0', 'application/vnd.oai.openapi;version=3.0', 'application/schema+json', 'application/pdf', 'text/csv', 'application/vnd.apache.parquet' or 'application/octet-stream'",
                "input": "application/x-netcdf",
                "ctx": {
                    "expected": "'image/tiff; application=geotiff', 'image/tiff; application=geotiff; profile=cloud-optimized', 'image/jp2', 'image/png', 'image/jpeg', 'application/geo+json', 'application/geo+json-seq', 'application/geopackage+sqlite3', 'application/vnd.google-earth.kml+xml', 'application/vnd.google-earth.kmz', 'application/x-protobuf', 'application/vnd.mapbox-vector-tile', 'application/x-hdf', 'application/x-hdf5', 'application/xml', 'application/json', 'application/ndjson', 'text/html', 'text/plain', 'application/vnd.oai.openapi+json;version=3.0', 'application/vnd.oai.openapi;version=3.0', 'application/schema+json', 'application/pdf', 'text/csv', 'application/vnd.apache.parquet' or 'application/octet-stream'"
                }
            },

[gadomski]

It would be helpful to provide a minimum reproducible error example, as I cannot reproduce with stac-pydantic==3.4.0:

$ uv pip install stac-pydantic==3.4.0
Audited 1 package in 2ms
$ cat test.py
from stac_pydantic.links import Link

print(
    Link.model_validate(
        {
            "href": "https://.../some-netcdf.nc",
            "type": "application/x-netcdf",
            "rel": "source",
        }
    )
)
$ python test.py
href='https://.../some-netcdf.nc' rel='source' type='application/x-netcdf' title=None

I would recommend double-checking the stac-pydantic version in your server environment. Also, I'd look to ensure the "request model" is correct in your FastAPI implementation, as that error looks like it's getting turned aside by FastAPI's request validation.

There's a chance that

stac-pydantic/stac_pydantic/links.py

Line 18 in 270a7da

type: Optional[Union[MimeTypes, str]] = None

doesn't translate correctly to FastAPI request validation, but that would be surprising to me.

[fmigneault]

@henriaidasso
The problem must be originating from somewhere else.
I have successfully created the STAC Item manually:
https://hirondelle.crim.ca/stac/collections/test/items/CMIP_CCCma_CanESM5_historical_r1i1p1f1_day_snw_gn

@gadomski
Thanks for the cross-check of link definition.