Skip to content

Commit ee62cff

Browse files
committed
docs: scaffold tensor parallelism optimization blog post
Adds a placeholder page and Tech Blogs navigation section so the TP optimization write-up has a home before the content lands.
1 parent 3911392 commit ee62cff

2 files changed

Lines changed: 33 additions & 0 deletions

File tree

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Optimizing Tensor Parallelism in JAX and XLA
2+
3+
> **Draft** — this page is a placeholder. Content is in progress.
4+
5+
## Background
6+
7+
<!-- Why TP matters: model sizes past single-GPU memory, the compute/communication
8+
tradeoff versus FSDP and pipeline parallelism. -->
9+
10+
## Where tensor parallelism costs performance
11+
12+
<!-- All-gather / reduce-scatter on the critical path, exposed communication,
13+
small GEMM tiles at high TP degree, NVLink versus inter-node bandwidth. -->
14+
15+
## Optimizations
16+
17+
<!-- One subsection per optimization, with the mechanism and how to enable it. -->
18+
19+
## Results
20+
21+
<!-- Model, hardware, TP degrees swept, throughput and step-time deltas. -->
22+
23+
## Reproducing
24+
25+
<!-- Container tag, framework config flags, launch command. -->
26+
27+
## Takeaways

docs/index.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,12 @@ navigation:
5555
- page: nsys-jax
5656
path: nsys-jax.md
5757

58+
# ==================== Tech Blogs ====================
59+
- section: Tech Blogs
60+
contents:
61+
- page: Optimizing Tensor Parallelism in JAX and XLA
62+
path: blog/tensor-parallelism-optimization.md
63+
5864
# ==================== Resiliency ====================
5965
- section: Resiliency
6066
contents:

0 commit comments

Comments
 (0)