Inference API #9
Description
Moving the discussion from #1 (comment) to here
Current challenges blocking this:
- there is no way to know the format of the expected request/response. Especially for custom handlers.
(I prefer not do model_name->type matching manually)
If a request/response schema is added to the returned OpenAPI definitions, I can probably auto generate something like SwaggerUI.