cannot pickle '_io.BufferedRandom' object

[ywh-my]

litserve 0.2.4

🐛 Bug

To Reproduce

I meet the " cannot pickle '_io.BufferedRandom' object" when I request a simple file

The server code is

import litserve as ls
class SimpleLitAPI(ls.LitAPI):
    # Called once at startup. Setup models, DB connections, etc...
    
    def setup(self, device):
        self.model = lambda x: x**2
        self.deviceid = f"我的设备 { device }"

    # Convert the request payload to model input.
    def decode_request(self, request):
        file_bytes = request['file'].file.read()
        
        print('接收文件的bytes:',len(file_bytes))
        print(self.deviceid)

        return "1"

    # Run inference on the the model, return the output.
    def predict(self, x):
        return ""

    # Convert the model output to a response payload.
    def encode_response(self, output):
        return {"output": output}


if __name__ =="__main__":
    # LitServe scales your API!
    server = ls.LitServer(SimpleLitAPI(),timeout= False , accelerator="gpu" , devices=[0,0,1,1], track_requests=True)
    server.run(port=8999)

The request code is

url  = f"http://127.0.0.1:8999/predict"
# 发送字节流
file_path = "aaaa.pdf"

# 读取文件内容为字节数据
with open(file_path, 'rb') as file:
    file_bytes = file.read()

# 将字节数据发送到 API
response = requests.post(url, files = {'file': ('filename.pdf', file_bytes, 'application/pdf')})
out = response.json()

The bug is

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/litserve/middlewares.py", line 69, in __call__
    await self.app(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/litserve/middlewares.py", line 69, in __call__
    await self.app(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 29, in __call__
    await responder(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 126, in __call__
    await super().__call__(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 46, in __call__
    await self.app(scope, receive, self.send_with_compression)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/routing.py", line 714, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/routing.py", line 734, in app
    await route.handle(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/site-packages/litserve/server.py", line 359, in predict
    self.request_queue.put_nowait((response_queue_id, uid, time.monotonic(), payload))
  File "<string>", line 2, in put_nowait
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/multiprocessing/managers.py", line 817, in _callmethod
    conn.send((self._id, methodname, args, kwds))
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/kemove/miniconda3/envs/minerU120/lib/python3.10/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_io.BufferedRandom' object

[bhimrazy]

Hi @ywh-my, thank you for raising the issue.
It seems that the error occurs when using a PDF file (or any file) greater than 1 MB.
We will get back to you on this issue.

[ywh-my]

maybe! I found that some PDF files can be requested successfully as "Uploadfile" object , but the other can not be . Please solve this .Thank U very much . Now ,I just simply encode the PDF files to base64 , and request them by "requests.post( json = )" . It can solve this problem , but not very good .

[bhimrazy]

Hi @ywh-my,
Could you please try once using the version starlette==0.45.3 ?

The latest version seems to have the issue.

[ywh-my]

It helps. When the issue happend , the version of starlette is 0.46.0

[deependujha]

@ywh-my can you check with this pr: #456