Some primary data is delivered with RPC files. Would it be a good fit to extend this with the metadata? Here is the specification: http://geotiff.maptools.org/rpc_prop.html
existent in rasterio as a model here https://github.com/rasterio/rasterio/blob/95f1f5fb55be6763bf987f42eb54c644604c6d3d/rasterio/rpc.py#L5