Skip to content

feature: enable passing custom default response class #440

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
AleksanderWWW wants to merge 3 commits into from
Closed

feature: enable passing custom default response class #440

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6f0c40d
feature: enable passing custom default response class
AleksanderWWW Feb 27, 2025
aef0017
register /predict with ORJSON response class
AleksanderWWW Feb 28, 2025
2ffb9d3
resolve conflict
aniketmaurya Mar 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions _requirements/perf.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
uvloop
tenacity
jsonargparse
orjson==3.10.15
Copy link
Copy Markdown
Collaborator

@aniketmaurya aniketmaurya Mar 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requirements.txt contains the main dependency. These are optional.

Copy link
Copy Markdown
Author

@AleksanderWWW AleksanderWWW Mar 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You asked to make this dependency optional in one of the comments

1 change: 1 addition & 0 deletions _requirements/test.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ openai>=1.12.0
pillow
numpy <2.0
pytest-retry>=1.6.3
orjson==3.10.15
Copy link
Copy Markdown
Collaborator

@aniketmaurya aniketmaurya Mar 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

2 changes: 2 additions & 0 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
fastapi >=0.100
uvicorn[standard] >=0.29.0
pyzmq >=22.0.0
starlette<0.46.0 # TODO: fix root cause for buffer error
orjson==3.10.15
5 changes: 3 additions & 2 deletions src/litserve/server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@

import uvicorn
from fastapi import Depends, FastAPI, HTTPException, Request, Response
from fastapi.responses import JSONResponse, StreamingResponse
from fastapi.responses import JSONResponse, ORJSONResponse, StreamingResponse
from fastapi.security import APIKeyHeader
from starlette.formparsers import MultiPartParser
from starlette.middleware.gzip import GZipMiddleware
Expand Down Expand Up @@ -221,7 +221,7 @@ def __init__(
lit_api.request_timeout = self.timeout
lit_api.pre_setup(max_batch_size, spec=spec)
self._loop.pre_setup(lit_api, spec=spec)
self.app = FastAPI(lifespan=self.lifespan)
self.app = FastAPI(lifespan=self.lifespan, default_response_class=ORJSONResponse)
self.app.response_queue_id = None
self.response_queue_id = None
self.response_buffer = {}
Expand Down Expand Up @@ -501,6 +501,7 @@ async def stream_predict(request: self.request_type) -> self.response_type:
stream_predict if stream else predict,
methods=methods,
dependencies=[Depends(self.setup_auth())],
response_class=ORJSONResponse,
)

for spec in self._specs:
Expand Down