Skip to content

Fix the bge embedding model callback error#119

Open
joy369 wants to merge 1 commit intotriton-inference-server:mainfrom
joy369:fix_bge_emb_error
Open

Fix the bge embedding model callback error#119
joy369 wants to merge 1 commit intotriton-inference-server:mainfrom
joy369:fix_bge_emb_error

Conversation

@joy369
Copy link
Copy Markdown

@joy369 joy369 commented Jan 13, 2026

  • Add missing attr of PoolingParams

Hi all, I try to deploy BAAI/bge-large-en. It establishes server properly but fail to inference.

curl -X POST http://localhost:8000/v2/models/bge_emb/generate \
  -H "Content-Type: application/json" \
  -d '{
    "embedding_request": "{\"input\": \"Test\"}"
  }'

Below is the error log

(EngineCore_DPO pid=4141) Exception in thread Thread-5 (process_input_sockets):
(EngineCore_DPO pid=4141) Traceback (most recent call last):
  File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.12/threading.py", line 1010, in run
    self.target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 1025, in process_input_sockets
    request = add_request_decoder.decode(data_frames)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/serial_utils.py", line 311, in decode
    return self.decoder.decode(bufs[0])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
msgpec.ValidationError: Expected 'bool', got 'None' - at '$[4][10]'

After assign Boolean value the model can inference properly. Thanks

- Add missing attr of PoolingParams
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant