fix ci and update docs #940
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @shihaobai, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request focuses on two main areas: improving the continuous integration (CI) process by streamlining dependencies and significantly enhancing the project's documentation. The documentation updates provide more comprehensive details on API server arguments and offer a much more detailed guide for deploying DeepSeek models, especially in multi-node and disaggregated setups.
Highlights
- CI Pipeline Improvements: I've updated the
Dockerfile.deepepto remove outdated proxy environment variables and changed the Flash Attention installation method to use a pre-built wheel from a GitHub release, which should improve build reliability and speed. - Comprehensive API Argument Documentation: I've significantly expanded the documentation for API server arguments in
docs/EN/source/tutorial/api_server_args_zh.rst. This includes new sections for scheduling, output constraints, multimodal features, performance optimizations, quantization, sampling, MTP multi-prediction, DeepSeek redundant experts, and monitoring parameters, providing a much more complete reference. - Enhanced DeepSeek Deployment Tutorial: The DeepSeek deployment tutorial (
docs/EN/source/tutorial/deepseek_deployment.rst) has been greatly enhanced. It now includes detailed instructions for launching the decode service in single PD master mode, and entirely new sections for setting up and testing a multi-PD master environment, including config server, multiple PD masters, and corresponding prefill/decode services. - Documentation Consistency and Cleanup: I've performed several minor documentation updates across the board. This includes updating a tutorial link in
README.md, removing an outdated NCCL installation note from both English and Chinese installation guides, translating a comment in the Chinese DeepSeek deployment tutorial, and standardizing terminology from 'separation' to 'disaggregation' and 'Machine' to 'node' in English documentation for better clarity and consistency.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
| RUN pip install -r /lightllm/requirements.txt --no-cache-dir --ignore-installed --extra-index-url https://download.pytorch.org/whl/cu124 | ||
|
|
||
| RUN pip install --no-cache-dir nvidia-nccl-cu12==2.25.1 # for allreduce hang issues in multinode H100 | ||
| RUN pip install --no-cache-dir https://github.com/ModelTC/flash-attn-3-build/releases/download/v2.7.4.post1/flash_attn-3.0.0b1-cp39-cp39-linux_x86_64.whl |
There was a problem hiding this comment.
Consider combining the pip install commands into a single RUN statement to reduce the number of layers in the Docker image.
RUN pip install --no-cache-dir https://github.com/ModelTC/flash-attn-3-build/releases/download/v2.7.4.post1/flash_attn-3.0.0b1-cp39-cp39-linux_x86_64.whl nvidia-nccl-cu12==2.25.1
| Output Constraint Parameters | ||
| --------------------------- | ||
|
|
||
| .. option:: --token_healing_mode |
There was a problem hiding this comment.
The parameter --token_healing_mode is missing a description. Add a description to explain its purpose and usage.
No description provided.