Skip to content

Fix/sglang kt detection#1875

Merged
ovowei merged 8 commits intomainfrom
fix/sglang-kt-detection
Mar 4, 2026
Merged

Fix/sglang kt detection#1875
ovowei merged 8 commits intomainfrom
fix/sglang-kt-detection

Conversation

@ovowei
Copy link
Collaborator

@ovowei ovowei commented Mar 4, 2026

What does this PR do?

Fixes # (issue)

Before submitting

ovowei and others added 8 commits March 4, 2026 07:38
…d version alignment

- Add kvcache-ai/sglang as git submodule at third_party/sglang (branch = main)
- Add top-level install.sh for one-click source installation (sglang + kt-kernel)
- Add sglang-kt as hard dependency in kt-kernel/pyproject.toml
- Add CI workflow to auto-sync sglang submodule daily and create PR
- Add CI workflow to build and publish sglang-kt to PyPI
- Integrate sglang-kt build into release-pypi.yml (version.py bump publishes both packages)
- Align sglang-kt version with ktransformers via SGLANG_KT_VERSION env var injection
- Update Dockerfile to use submodule and inject aligned version
- Update all 13 doc files, CLI hints, and i18n strings to reference new install methods

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Users can now `pip install ktransformers` to get everything
(sglang-kt is auto-installed as a dependency).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
`pip install ktransformers` now works as a single install command.
It pulls kt-kernel (which in turn pulls sglang-kt).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Prioritize sglang-kt package version (aligned with ktransformers)
  over sglang internal __version__
- Update display name from "sglang" to "sglang-kt"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Recognize sglang-kt package name as proof of kvcache-ai fork installation.
Previously both commands fell through to "PyPI (not recommended)" for
non-editable local source installs. Now version.py reuses the centralized
check_sglang_installation() logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ovowei ovowei merged commit 15c624d into main Mar 4, 2026
6 checks passed
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the integration and installation process for SGLang within the ktransformers ecosystem. By adopting SGLang as a Git submodule and introducing a unified install.sh script, the changes aim to simplify user setup, ensure version compatibility, and provide clearer guidance on using the kvcache-ai fork of SGLang (now sglang-kt). The updates also improve diagnostic tools to better identify the correct SGLang installation, enhancing overall developer and user experience.

Highlights

  • SGLang Integration: The kvcache-ai fork of SGLang is now integrated as a Git submodule under third_party/sglang, streamlining its management and ensuring version consistency.
  • Simplified Installation: A new install.sh script has been added to provide a one-click installation experience for both SGLang and kt-kernel, significantly simplifying the setup process for users.
  • Documentation Updates: All relevant documentation files have been updated to reflect the new installation methods, emphasizing the use of the sglang-kt package or the install.sh script, and clarifying the distinction from the official SGLang.
  • Docker Build Process Refinement: The Dockerfile has been updated to leverage the SGLang submodule and align the sglang-kt version with the main ktransformers version, ensuring consistent environments.
  • Improved SGLang Detection: The kt-kernel CLI's doctor and version commands have been enhanced to accurately detect and report the kvcache-ai fork of SGLang (now sglang-kt), providing clearer diagnostics and installation hints.
  • KTransformers Meta-Package: The root directory now includes pyproject.toml and setup.py to define ktransformers as a meta-package, which will automatically install kt-kernel and sglang-kt with aligned versions.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • .gitmodules
    • Added third_party/sglang as a new Git submodule, pointing to the kvcache-ai/sglang.git repository on the main branch.
  • doc/en/Kimi-K2-Thinking.md
    • Updated SGLang installation instructions to recommend pip install sglang-kt or running ./install.sh, replacing the direct pip install "sglang[all]" command.
    • Added an important note to use sglang-kt (kvcache-ai fork) and to uninstall the official sglang package if present.
  • doc/en/Kimi-K2.5.md
    • Revised SGLang installation steps to use pip install sglang-kt or ./install.sh, removing previous instructions for cloning and installing the custom SGLang repository directly.
  • doc/en/MiniMax-M2.5.md
    • Modified SGLang installation guidance to suggest pip install sglang-kt or ./install.sh, replacing the manual git clone and pip install -e commands.
  • doc/en/Qwen3.5.md
    • Updated SGLang installation instructions to use pip install sglang-kt or ./install.sh, removing the previous manual cloning and branch checkout steps.
  • doc/en/SFT_Installation_Guide_KimiK2.5.md
    • Changed SGLang installation recommendations to use ./install.sh or pip install sglang-kt, replacing the direct git clone and pip install -e commands.
  • doc/en/kt-kernel/GLM-5-Tutorial.md
    • Updated SGLang installation instructions to use pip install sglang-kt or ./install.sh, removing the manual cloning and editable install steps.
  • doc/en/kt-kernel/Kimi-K2-Thinking-Native.md
    • Revised SGLang installation steps to recommend pip install sglang-kt or ./install.sh, replacing the direct git clone and pip install -e commands.
  • doc/en/kt-kernel/MiniMax-M2.1-Tutorial.md
    • Modified SGLang installation guidance to suggest pip install sglang-kt or ./install.sh, removing the manual cloning and editable install steps.
  • doc/en/kt-kernel/Native-Precision-Tutorial.md
    • Updated SGLang installation instructions to use pip install sglang-kt or ./install.sh, replacing the manual cloning and editable install steps.
  • doc/en/kt-kernel/Qwen3-Coder-Next-Tutorial.md
    • Revised SGLang installation steps to recommend pip install sglang-kt or ./install.sh, removing the manual cloning and editable install steps.
  • doc/en/kt-kernel/deepseek-v3.2-sglang-tutorial.md
    • Updated SGLang installation instructions to a concise pip install sglang-kt or ./install.sh.
  • doc/en/kt-kernel/experts-sched-Tutorial.md
    • Modified SGLang installation guidance to suggest pip install sglang-kt or ./install.sh, removing the manual cloning and editable install steps.
  • docker/Dockerfile
    • Removed the direct git clone of the kvcache-ai/sglang repository and its specific branch checkout.
    • Added a symbolic link from /workspace/ktransformers/third_party/sglang to /workspace/sglang to utilize the newly added submodule.
    • Modified the SGLang installation step to export SGLANG_KT_VERSION from ktransformers/version.py and print it, ensuring version alignment.
    • Removed the SGLang version extraction from /workspace/sglang/python/sglang and instead aligned SGLANG_KT_VERSION with KTRANSFORMERS_VERSION in versions.env.
  • install.sh
    • Added a new shell script to provide a unified, one-click installation process for SGLang and kt-kernel, including submodule initialization and dependency installation.
    • Implemented subcommands (all, sglang, kt-kernel, deps) and options (--skip-sglang, --skip-kt-kernel, --editable, --manual, --no-clean) for flexible installation.
    • Included logic to read the ktransformers version from version.py and export it as SGLANG_KT_VERSION for consistent versioning.
  • kt-kernel/README.md
    • Updated SGLang installation instructions to provide options for one-click install (./install.sh), pip install (pip install sglang-kt), or source install from the ktransformers submodule.
    • Added an important note to use sglang-kt (kvcache-ai fork) and to uninstall the official sglang package if present.
  • kt-kernel/README_zh.md
    • Updated Chinese SGLang installation instructions to provide options for one-click install (./install.sh), pip install (pip install sglang-kt), or source install from the ktransformers submodule.
    • Added an important note in Chinese to use sglang-kt (kvcache-ai fork) and to uninstall the official sglang package if present.
  • kt-kernel/pyproject.toml
    • Added sglang-kt as a dependency under the dependencies section.
  • kt-kernel/python/cli/commands/doctor.py
    • Modified the check_sglang_installation logic to specifically detect if the installed SGLang is the kvcache-ai fork (sglang-kt).
    • Updated the sglang_source_value and sglang_source_hint messages to clearly distinguish between PyPI sglang and sglang-kt and provide correct installation instructions.
    • Revised the hint for unsupported SGLang versions to recommend pip uninstall sglang -y && pip install sglang-kt or running ./install.sh.
  • kt-kernel/python/cli/commands/version.py
    • Refactored _get_sglang_info to use sglang_checker for installation details and prioritize sglang-kt when determining the version.
    • Updated the version reporting format to clearly indicate if sglang-kt is installed from source, as an editable package, or as a standard sglang-kt package.
  • kt-kernel/python/cli/i18n.py
    • Updated the version_sglang string to sglang-kt in both English and Chinese translations.
    • Modified various SGLang-related warning and hint messages to refer to sglang-kt and provide updated installation commands using pip install sglang-kt or ./install.sh.
  • kt-kernel/python/cli/utils/sglang_checker.py
    • Modified check_sglang_installation to first attempt pip show sglang-kt to identify the kvcache-ai fork.
    • Added a new is_kvcache_fork boolean field to the returned dictionary to explicitly indicate if the sglang-kt package is installed.
  • pyproject.toml
    • Added a new pyproject.toml file at the root to define ktransformers as a meta-package, specifying its name, description, authors, license, and Python requirements.
  • setup.py
    • Added a new setup.py file at the root to create a ktransformers meta-package that depends on kt-kernel and sglang-kt, ensuring they are installed with the same version as ktransformers.
  • third_party/sglang
    • Added third_party/sglang as a Git submodule, pointing to a specific commit (6b8b5f4649c18afac6a5491bdfa69cf6746d714a).
  • version.py
    • Updated the __version__ string from 0.5.1 to 0.5.2.post1.
Ignored Files
  • Ignored by pattern: .github/workflows/** (3)
    • .github/workflows/release-pypi.yml
    • .github/workflows/release-sglang-kt.yml
    • .github/workflows/sync-sglang-submodule.yml
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request significantly refactors the installation and dependency management process, introducing sglang as a git submodule, a unified install.sh script, and a ktransformers meta-package to simplify setup and improve maintainability. However, a critical security concern has been identified: a potential code injection vulnerability in the install.sh script where a directory path is unsafely embedded into a Python command string. While the overall code quality is high, this vulnerability requires immediate attention.

read_kt_version() {
local version_file="$REPO_ROOT/version.py"
if [ -f "$version_file" ]; then
KT_VERSION=$(python3 -c "exec(open('$version_file').read()); print(__version__)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-medium medium

The script uses python3 -c to extract the version from version.py by embedding the absolute path directly into the Python script string. This can lead to arbitrary code execution if the repository is cloned into a directory with a name that contains malicious Python code (e.g., ');import os;os.system('id')#).

To remediate this, pass the file path as a command-line argument to the Python script instead of embedding it in the script string.

Suggested change
KT_VERSION=$(python3 -c "exec(open('$version_file').read()); print(__version__)")
KT_VERSION=$(python3 -c "import sys; ns={}; exec(open(sys.argv[1]).read(), ns); print(ns['__version__'])" "$version_file")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant