OpenAI compatible minimal API using ORT-GenAI #995
bpietroiu
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I have created a minimal implementation, including a full port of a Jinja parser/interpreter
Docker and docker compose for deployment
OpenTelemetry for logging and monitoring
Dynamic Batching
Chat prompt template processing using Jinja JS to C# ported by me
can be found here https://github.com/bpietroiu/GenAI.Server
Beta Was this translation helpful? Give feedback.
All reactions