Skip to content

Commit 2df74b8

Browse files
authored
Fix OpenAI endpoint on Kubernetes gateways (#2147)
The endpoint didn't work both in the old and the new gateway implementations. The model proxy attempted forwarding requests to `localhost` that resolves to `::1` on Kubernetes gateways, while Nginx is only listening on `127.0.0.1`.
1 parent 9f9aefa commit 2df74b8

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/dstack/_internal/proxy/lib/services/service_connection.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,7 @@ async def get_service_replica_client(service: Service, repo: BaseProxyRepo) -> h
132132
if service.domain is not None:
133133
# Forward to Nginx so that requests are visible to StatsCollector in the access log
134134
return httpx.AsyncClient(
135-
base_url="http://localhost",
135+
base_url="http://127.0.0.1",
136136
headers={"Host": service.domain},
137137
timeout=HTTP_TIMEOUT,
138138
)

0 commit comments

Comments
 (0)