can not configure envoy ai gateway with mcp and connected open-ai compatible provider?

[Dericko681]

I have deployed the envoy stack from https://aigateway.envoyproxy.io/docs/getting-started/basic-usage and deployed an open-ai compatible backend and curl requests return 200 OK

I have tried to deploy context7 mcp server and use it to query some tools and also to inject context into llm requests but the ai gateway doc says llittle about this.
I face difficulties in achieving this. Can someone document this for me pls?

[nacx]

Can you share the configuration you're using to deploy context7 and the AIGW config you're using to configure it?

[Dericko681]

Hi @nacx Here is the deployment manifest
and mcp-route config

apiVersion: apps/v1
kind: Deployment
metadata:
  name: context7-http
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: context7-http
  template:
    metadata:
      labels:
        app: context7-http
    spec:
      imagePullSecrets:
        - name: ghcr-creds
      containers:
        - name: context7-http
          image: ghcr.io/lrstanley/context7-http:latest
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8000
          env:
            - name: MCP_HTTP_ADDR
              value: "0.0.0.0:8000"
          readinessProbe: null
            # httpGet:
            #   path: /
            #   port: 8000
            # initialDelaySeconds: 5
            # periodSeconds: 10
          livenessProbe: null
            # httpGet:
            #   path: /
            #   port: 8000
            # initialDelaySeconds: 10
            # periodSeconds: 20

---
apiVersion: v1
kind: Service
metadata:
  name: context7-http
  namespace: default
spec:
  selector:
    app: context7-http
  ports:
    - name: http
      port: 8000
      targetPort: 8000

sections were commented out for debugging

cat mcp-route.yaml
apiVersion: aigateway.envoyproxy.io/v1alpha1
kind: MCPRoute
metadata:
  name: mcp-route
  namespace: default
spec:
  parentRefs:
    - name: envoy-ai-gateway-basic
      kind: Gateway
      group: gateway.networking.k8s.io
  path: "/mcp" # Clients connect to http://gateway-address/mcp
  backendRefs:
    - name: context7
      kind: Backend
      group: gateway.envoyproxy.io
      path: "/"    # Context7 MCP root
      securityPolicy:
        apiKey:
          secretRef:
            name: context7-token
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: Backend
metadata:
  name: context7
  namespace: default
spec:
  endpoints:
    - fqdn:
        hostname: context7-http.default.svc.cluster.local
        port: 8000
---
apiVersion: v1
kind: Secret
metadata:
  name: context7-token
  namespace: default
type: Opaque
stringData:
  apiKey:

[nacx]

Have you tried using the public context7 MCP server? It is free and already in the AIGW examples: https://github.com/envoyproxy/ai-gateway/blob/main/examples/mcp/mcp_example.yaml#L31-L34
Can you try first with the public one, to discard issues with the context7 deployment?

[Dericko681]

Alright, i'm checking it out, i'l get back to you

[Dericko681]

Hey @nacx from https://github.com/envoyproxy/ai-gateway/blob/main/examples/mcp/mcp_example.yaml#L31-L34 I have this deployment manifest

################################################################################
# 1. MCPRoute: Routes /mcp to remote context7 MCP server
################################################################################
apiVersion: aigateway.envoyproxy.io/v1alpha1
kind: MCPRoute
metadata:
  name: mcp-route
  namespace: default
spec:
  parentRefs:
    - name: aigw-run
      kind: Gateway
      group: gateway.networking.k8s.io
  path: "/mcp"
  backendRefs:
    - name: context7 # our remote Context7 backend
      kind: Backend
      group: gateway.envoyproxy.io
      path: "/mcp"
      # optional: restrict which tools go to context7
      # toolSelector: <if desired, define regex here>
      securityPolicy:
        apiKey:
          secretRef:
            name: context7-secrets

################################################################################
# 2. Backend: remote Context7 MCP server
################################################################################
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: Backend
metadata:
  name: context7
  namespace: default
spec:
  endpoints:
    - fqdn:
        hostname: api.context7.com
        port: 443

################################################################################
# 3. BackendTLS Policy: validate TLS for remote endpoints
################################################################################
apiVersion: gateway.networking.k8s.io/v1alpha3
kind: BackendTLSPolicy
metadata:
  name: context7-tls
  namespace: default
spec:
  targetRefs:
    - group: "gateway.envoyproxy.io"
      kind: Backend
      name: context7
  validation:
    wellKnownCACertificates: "System"
    hostname: api.context7.com

################################################################################
# 4. GatewayClass / Gateway definitions (ensure these exist just once)
################################################################################
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: aigw-run
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller

---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: aigw-run
  namespace: default
spec:
  gatewayClassName: aigw-run
  listeners:
    - name: http
      protocol: HTTP
      port: 1975
  infrastructure:
    parametersRef:
      group: gateway.envoyproxy.io
      kind: EnvoyProxy
      name: envoy-ai-gateway

i get this repsonse

curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc":"2.0",
    "id":1,
    "method":"initialize",
    "params":{
      "protocolVersion":"2024-11-05",
      "capabilities":{},
      "clientInfo":{"name":"curl-test","version":"1.0"}
    }
  }'

upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused% 

and the secret exists

orsys kubectl get secrets
NAME                                        TYPE     DATA   AGE
ai-eg-mcp-brf-mcp-route-github-credential   Opaque   1      46m
context7-secrets                            Opaque   1      38m
envoy-ai-gateway-basic-openai-apikey        Opaque   1      48m
github-access-token                         Opaque   1      46m
github-secrets                              Opaque   1      38m
➜  adorsys 

[nacx]

With that configuration the Gateway is configured to listen on port 1975, not 8080. Can you double-check the ports and make sure you're connecting to the right endpoint?

[Dericko681]

Yea port 1975 is forwarded to localhost:8080?