Skip to content

Conversation

JaredforReal
Copy link
Collaborator

What this PR does / why we need it:
Adds a lightweight local OpenAI-compatible model (llm-katan) to the default Docker Compose stack for a fully self-contained out‑of‑the‑box experience.

Key Changes:

  • New llm-katan service (Qwen/Qwen3-0.6B served as qwen3) with static IP 172.28.0.20:8002.
  • Router config: set qwen3 as the default model.
  • Added pre-download of Qwen3-0.6B to model bootstrap targets.
  • New make target: docker-compose-up-core (stack without local model).
  • Updated README with usage.
  • Lint config: exclude models/ from markdown lint.

Copy link

netlify bot commented Oct 14, 2025

Deploy Preview for vllm-semantic-router ready!

Name Link
🔨 Latest commit 51d09e9
🔍 Latest deploy log https://app.netlify.com/projects/vllm-semantic-router/deploys/68ef12812677ca0008979919
😎 Deploy Preview https://deploy-preview-426--vllm-semantic-router.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copy link

github-actions bot commented Oct 14, 2025

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 Root Directory

Owners: @rootfs, @Xunzhuo
Files changed:

  • Dockerfile.extproc
  • README.md

📁 config

Owners: @rootfs
Files changed:

  • config/config.yaml

📁 deploy

Owners: @rootfs, @Xunzhuo
Files changed:

  • deploy/docker-compose/docker-compose.yml

📁 tools

Owners: @yuluo-yx, @rootfs, @Xunzhuo
Files changed:

  • tools/make/docker.mk
  • tools/make/linter.mk
  • tools/make/models.mk

vLLM

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

@rootfs
Copy link
Collaborator

rootfs commented Oct 14, 2025

lgtm, once the markdown link is fixed, it is good to go

cc @yossiovadia

@Xunzhuo Xunzhuo merged commit ce99067 into vllm-project:main Oct 15, 2025
14 of 15 checks passed
@JaredforReal JaredforReal deleted the katan branch October 17, 2025 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants