-
Notifications
You must be signed in to change notification settings - Fork 500
Open
Description
🐛 Describe the bug
error message
I1031 06:10:26.097871 1 gateway.go:94] "processing request" requestID="b826c66e-f8c8-48a9-b5e4-39daca741c67"
I1031 06:10:26.103904 1 gateway_req_body.go:75] "request start" requestID="b826c66e-f8c8-48a9-b5e4-39daca741c67" requestPath="/v1/chat/completions" model="deepseek-r1-distill-llama-8b" stream=false
E1031 06:10:26.175809 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:26.175817 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:27.176961 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:27.243634 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:27.243640 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:28.244455 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:28.321206 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:28.321216 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:29.322189 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:29.405118 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:29.405124 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:30.405709 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:30.490204 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:30.490214 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:31.490696 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:31.577800 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:31.577818 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:32.577936 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:32.667135 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:32.667142 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:33.668012 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:33.757004 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:33.757012 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:34.757265 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:34.847027 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:34.847035 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:35.847699 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:35.937384 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:35.937393 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:36.938080 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:37.028746 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:37.028755 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:38.029074 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:38.120026 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:38.120033 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:39.120952 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:39.212065 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:39.212075 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:40.212466 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:40.305438 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:40.305446 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:41.305667 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:41.400401 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:41.400414 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:42.400931 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:42.497127 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:42.497137 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:43.497250 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
E1031 06:10:43.594125 1 zmq_client.go:214] Event consumption error for default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8: failed to process message: failed to decode event batch: failed to unmarshal event batch: msgpack : invalid code 93 decoding map
I1031 06:10:43.594131 1 zmq_client.go:224] Attempting to reconnect to default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 after 1s
I1031 06:10:44.572151 1 gateway_rsp_body.go:189] request end, requestID: b826c66e-f8c8-48a9-b5e4-39daca741c67 -
I1031 06:10:44.594296 1 zmq_client.go:136] Successfully connected to vLLM pod default/deepseek-r1-distill-llama-8b-79ff779c9c-nb6g8 at 172.16.149.117
Steps to Reproduce
- modify gateway-plugins to enable kv_events
# v0.4.1
kubectl set env deployment/aibrix-gateway-plugins -n aibrix-system \
AIBRIX_PREFIX_CACHE_METRICS_ENABLED=true \
AIBRIX_PREFIX_CACHE_TOKENIZER_TYPE=remote \
AIBRIX_USE_REMOTE_TOKENIZER=true \
AIBRIX_REMOTE_TOKENIZER_ENDPOINT="http://deepseek-r1-distill-llama-8b.default.svc.cluster.local:8000"
# v0.4.1
kubectl set env deployment/aibrix-gateway-plugins -n aibrix-system \
AIBRIX_KV_EVENT_SYNC_ENABLED=true- label vllm pod
labels:
model.aibrix.ai/kv-events-enabled: "true"
-
request /v1/chat/completions
-
watch gateway-plugins log error
docs refer: https://aibrix.readthedocs.io/latest/features/kv-event-sync.html
Expected behavior
- kv_events sync success
Environment
- AIBrix 0.4.1
- Kubernetes
- LLM 0.9.1
- Docker Image: vllm-openai-aibrix-kvcache:v0.9.1-20250724
Metadata
Metadata
Assignees
Labels
No labels