Skip to content

Commit 954eee2

Browse files
ParagEkboteParagEkbote
authored
Enhance Docs (#867)
## Summary Fixes #766 As described in the issue, this PR aims to enhance docs by enabling auto-generate docs. Additionally, I've set the paths to only look for changes in docs/, so the doc workflow is not executed on every commit. Additionally, I've also exposed doc-strings to be visible as part of the docs. The site is here: https://paragekbote.github.io/Liger-Kernel/ Also, what do you think can be done about the low-level api page, can something similar be performed? cc: @shimizust ## Testing Done <!-- Replace BLANK with your device type. For example, A100-80G-PCIe Complete the following tasks before sending your PR, and replace `[ ]` with `[x]` to indicate you have done them. --> - Hardware Type: <BLANK> - [ ] run `make test` to ensure correctness - [ ] run `make checkstyle` to ensure code style - [ ] run `make test-convergence` to ensure convergence
1 parent fa24166 commit 954eee2

File tree

3 files changed

+109
-26
lines changed

3 files changed

+109
-26
lines changed

‎.github/workflows/docs.yml‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,14 @@ on:
33
push:
44
branches:
55
- main
6+
paths:
7+
- 'docs/**'
8+
- 'mkdocs.yml'
69

710
permissions:
811
contents: write
912
jobs:
1013
deploy:
11-
if: False
1214
runs-on: ubuntu-latest
1315
steps:
1416
- uses: actions/checkout@v4
@@ -26,5 +28,5 @@ jobs:
2628
path: .cache
2729
restore-keys: |
2830
mkdocs-material-
29-
- run: pip install mkdocs-material
31+
- run: pip install mkdocs-material mkdocstrings[python]
3032
- run: mkdocs gh-deploy --force

‎docs/High-Level-APIs.md‎

Lines changed: 78 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,93 @@
1+
# High-Level APIs
12

2-
### AutoModel
3+
## AutoModel
34

45
| **AutoModel Variant** | **API** |
5-
|-----------|---------|
6+
|------------------------|---------|
67
| AutoModelForCausalLM | `liger_kernel.transformers.AutoLigerKernelForCausalLM` |
78

89
This API extends the implementation of the `AutoModelForCausalLM` within the `transformers` library from Hugging Face.
910

11+
::: liger_kernel.transformers.AutoLigerKernelForCausalLM
12+
options:
13+
extra:
14+
show_docstring: true
15+
show_signature: true
16+
show_source: true
17+
1018
!!! Example "Try it Out"
1119
You can experiment as shown in this example [here](https://github.com/linkedin/Liger-Kernel?tab=readme-ov-file#1-use-autoligerkernelforcausallm).
1220

13-
### Patching
21+
---
1422

15-
You can also use the Patching APIs to use the kernels for a specific model architecture.
23+
## Patching
1624

17-
!!! Example "Try it Out"
18-
You can experiment as shown in this example [here](https://github.com/linkedin/Liger-Kernel?tab=readme-ov-file#2-apply-model-specific-patching-apis).
25+
You can also use the Patching APIs to use the kernels for a specific model architecture.
1926

2027
| **Model** | **API** | **Supported Operations** |
2128
|-------------|--------------------------------------------------------------|-------------------------------------------------------------------------|
22-
| LLaMA 2 & 3 | `liger_kernel.transformers.apply_liger_kernel_to_llama` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
23-
| LLaMA 3.2-Vision | `liger_kernel.transformers.apply_liger_kernel_to_mllama` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
24-
| Mistral | `liger_kernel.transformers.apply_liger_kernel_to_mistral` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
25-
| Mixtral | `liger_kernel.transformers.apply_liger_kernel_to_mixtral` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
26-
| Gemma1 | `liger_kernel.transformers.apply_liger_kernel_to_gemma` | RoPE, RMSNorm, GeGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
27-
| Gemma2 | `liger_kernel.transformers.apply_liger_kernel_to_gemma2` | RoPE, RMSNorm, GeGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
28-
| Qwen2, Qwen2.5, & QwQ | `liger_kernel.transformers.apply_liger_kernel_to_qwen2` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
29-
| Qwen2-VL | `liger_kernel.transformers.apply_liger_kernel_to_qwen2_vl` | RMSNorm, LayerNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
30-
| Phi3 & Phi3.5 | `liger_kernel.transformers.apply_liger_kernel_to_phi3` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
29+
| LLaMA 2 & 3 | `liger_kernel.transformers.apply_liger_kernel_to_llama` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
30+
| LLaMA 3.2-Vision | `liger_kernel.transformers.apply_liger_kernel_to_mllama` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
31+
| Mistral | `liger_kernel.transformers.apply_liger_kernel_to_mistral` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
32+
| Mixtral | `liger_kernel.transformers.apply_liger_kernel_to_mixtral` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
33+
| Gemma1 | `liger_kernel.transformers.apply_liger_kernel_to_gemma` | RoPE, RMSNorm, GeGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
34+
| Gemma2 | `liger_kernel.transformers.apply_liger_kernel_to_gemma2` | RoPE, RMSNorm, GeGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
35+
| Qwen2, Qwen2.5, & QwQ | `liger_kernel.transformers.apply_liger_kernel_to_qwen2` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
36+
| Qwen2-VL | `liger_kernel.transformers.apply_liger_kernel_to_qwen2_vl` | RMSNorm, LayerNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
37+
| Phi3 & Phi3.5 | `liger_kernel.transformers.apply_liger_kernel_to_phi3` | RoPE, RMSNorm, SwiGLU, CrossEntropyLoss, FusedLinearCrossEntropy |
38+
39+
### Function Signatures
40+
41+
::: liger_kernel.transformers.apply_liger_kernel_to_llama
42+
options:
43+
extra:
44+
show_docstring: true
45+
show_signature: true
46+
47+
::: liger_kernel.transformers.apply_liger_kernel_to_mllama
48+
options:
49+
extra:
50+
show_docstring: true
51+
show_signature: true
52+
53+
::: liger_kernel.transformers.apply_liger_kernel_to_mistral
54+
options:
55+
extra:
56+
show_docstring: true
57+
show_signature: true
58+
59+
::: liger_kernel.transformers.apply_liger_kernel_to_mixtral
60+
options:
61+
extra:
62+
show_docstring: true
63+
show_signature: true
64+
65+
::: liger_kernel.transformers.apply_liger_kernel_to_gemma
66+
options:
67+
extra:
68+
show_docstring: true
69+
show_signature: true
70+
71+
::: liger_kernel.transformers.apply_liger_kernel_to_gemma2
72+
options:
73+
extra:
74+
show_docstring: true
75+
show_signature: true
76+
77+
::: liger_kernel.transformers.apply_liger_kernel_to_qwen2
78+
options:
79+
extra:
80+
show_docstring: true
81+
show_signature: true
82+
83+
::: liger_kernel.transformers.apply_liger_kernel_to_qwen2_vl
84+
options:
85+
extra:
86+
show_docstring: true
87+
show_signature: true
88+
89+
::: liger_kernel.transformers.apply_liger_kernel_to_phi3
90+
options:
91+
extra:
92+
show_docstring: true
93+
show_signature: true

‎mkdocs.yml‎

Lines changed: 27 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ site_name: Liger-Kernel Docs
22
site_url: https://linkedin.github.io/Liger-Kernel/
33
site_author: LinkedIn
44
site_description: Efficient Triton Kernels for LLM Training
5+
56
theme:
67
name: material
78
font:
@@ -12,15 +13,6 @@ theme:
1213
- toc.follow
1314
- navigation.top
1415
- navigation.sections
15-
nav:
16-
- Home: index.md
17-
- Examples: Examples.md
18-
- Getting Started: Getting-Started.md
19-
- High Level APIs: High-Level-APIs.md
20-
- Low Level APIs: Low-Level-APIs.md
21-
- Contributing: contributing.md
22-
- Acknowledgment: acknowledgement.md
23-
- License: license.md
2416
palette:
2517
# Dark Mode
2618
- scheme: slate
@@ -38,6 +30,16 @@ theme:
3830
primary: blue
3931
accent: deep purple
4032

33+
nav:
34+
- Home: index.md
35+
- Examples: Examples.md
36+
- Getting Started: Getting-Started.md
37+
- High Level APIs: High-Level-APIs.md
38+
- Low Level APIs: Low-Level-APIs.md
39+
- Contributing: contributing.md
40+
- Acknowledgment: acknowledgement.md
41+
- License: license.md
42+
4143
markdown_extensions:
4244
- attr_list
4345
- toc:
@@ -58,6 +60,22 @@ markdown_extensions:
5860
- admonition
5961
- pymdownx.details
6062

63+
plugins:
64+
- search
65+
- mkdocstrings:
66+
handlers:
67+
python:
68+
paths: [src]
69+
options:
70+
show_root_heading: true
71+
show_source: true
72+
docstring_style: google
73+
docstring_section_style: table
74+
heading_level: 3
75+
show_signature_annotations: false # Hides type annotations to save space
76+
separate_signature: true # Separates signature from description
77+
78+
6179
# Repository
6280
repo_name: linkedin/Liger-Kernel
6381
repo_url: https://github.com/linkedin/Liger-Kernel

0 commit comments

Comments
 (0)