Skip to content

Commit 775ef5c

Browse files
Davda-JamesDavda-James
committed
refactored cli commands to directly import in cli.mdx and also refactored cli generation to fix markdownlinter issues
1 parent 6ca74a1 commit 775ef5c

File tree

4 files changed

+41
-137
lines changed

4 files changed

+41
-137
lines changed

‎dev/README.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,13 +18,16 @@ python dev/generate_cli_docs.py
1818

1919
- Extracts help messages from all Click commands in `python/cocoindex/cli.py`
2020
- Generates comprehensive Markdown documentation with properly formatted tables
21-
- Saves the output to `docs/docs/core/cli-reference.md`
21+
- Saves the output to `docs/docs/core/cli-commands.md` for direct import into CLI documentation
2222
- Only updates the file if content has changed (avoids unnecessary git diffs)
23+
- Automatically escapes HTML-like tags to prevent MDX parsing issues
24+
- Wraps URLs with placeholders in code blocks for proper rendering
2325

2426
**Integration:**
2527

2628
- Runs automatically as a pre-commit hook when `python/cocoindex/cli.py` is modified
27-
- The generated documentation is imported into `docs/docs/core/cli.mdx` via MDX import
29+
- The generated documentation is directly imported into `docs/docs/core/cli.mdx` via MDX import
30+
- Provides seamless single-page CLI documentation experience without separate reference pages
2831

2932
**Dependencies:**
3033

‎dev/generate_cli_docs.py‎

Lines changed: 18 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,13 @@
2727

2828

2929
def clean_usage_line(usage: str) -> str:
30-
"""Clean up the usage line to remove 'cli' and make it generic."""
31-
# Replace 'cli' with 'cocoindex' in usage lines
32-
return usage.replace("Usage: cli ", "Usage: cocoindex ")
30+
"""Clean up the usage line to remove 'cli' and make it generic, and remove the 'Usage:' prefix."""
31+
# Replace 'cli' with 'cocoindex' in usage lines and remove 'Usage:' prefix
32+
cleaned = usage.replace("Usage: cli ", "cocoindex ")
33+
# Handle case where it might be "Usage: cocoindex" already
34+
if cleaned.startswith("Usage: cocoindex "):
35+
cleaned = cleaned.replace("Usage: cocoindex ", "cocoindex ")
36+
return cleaned
3337

3438

3539
def escape_html_tags(text: str) -> str:
@@ -40,6 +44,12 @@ def escape_html_tags(text: str) -> str:
4044
text = re.sub(r"http://localhost:<([^>]+)>", r"`http://localhost:<\1>`", text)
4145
text = re.sub(r"https://([^<\s]+)<([^>]+)>", r"`https://\1<\2>`", text)
4246

47+
# Handle comma-separated URL examples specifically (e.g., "https://site1.com,http://localhost:3000")
48+
text = re.sub(r"(?<!`)(\bhttps?://[^\s,`]+,https?://[^\s`]+)(?!`)", r"`\1`", text)
49+
50+
# Handle standalone URLs that aren't already wrapped in backticks
51+
text = re.sub(r"(?<!`)(?<!,)(\bhttps?://[^\s,`]+)(?!`)(?!,)", r"`\1`", text)
52+
4353
# Split text into code blocks and regular text
4454
# Pattern matches: `code content` (inline code blocks)
4555
parts = re.split(r"(`[^`]*`)", text)
@@ -203,60 +213,18 @@ def generate_command_docs(docs: List[Dict[str, Any]]) -> str:
203213

204214
markdown_content = []
205215

206-
if main_cli:
207-
# Generate main CLI documentation
208-
help_text = main_cli["help"]
209-
usage = clean_usage_line(main_cli["usage"])
210-
description = extract_description(help_text)
211-
212-
markdown_content.append("# CLI Commands Reference")
213-
markdown_content.append("")
214-
markdown_content.append(
215-
"This page contains the detailed help information for all CocoIndex CLI commands."
216-
)
217-
markdown_content.append("")
218-
219-
if description:
220-
markdown_content.append(f"## Overview")
221-
markdown_content.append("")
222-
markdown_content.append(description)
223-
markdown_content.append("")
224-
225-
# Add usage
226-
markdown_content.append("## Usage")
227-
markdown_content.append("")
228-
markdown_content.append(f"```sh")
229-
markdown_content.append(usage)
230-
markdown_content.append("```")
231-
markdown_content.append("")
232-
233-
# Add global options
234-
options_section = format_options_section(help_text)
235-
if options_section:
236-
markdown_content.append("## Global Options")
237-
markdown_content.append("")
238-
markdown_content.append(options_section)
239-
markdown_content.append("")
240-
241-
# Add commands overview
242-
commands_section = format_commands_section(help_text)
243-
if commands_section:
244-
markdown_content.append("## Commands")
245-
markdown_content.append("")
246-
markdown_content.append(commands_section)
247-
markdown_content.append("")
248-
249-
# Generate subcommand documentation
250-
markdown_content.append("## Command Details")
216+
# Add top-level heading to satisfy MD041 linting rule
217+
markdown_content.append("# CLI Commands")
251218
markdown_content.append("")
252219

220+
# Generate only the command details section (remove redundant headers)
253221
for doc in sorted(subcommands, key=lambda x: x["command"].name):
254222
command_name = doc["command"].name
255223
help_text = doc["help"]
256224
usage = clean_usage_line(doc["usage"])
257225
description = extract_description(help_text)
258226

259-
markdown_content.append(f"### `{command_name}`")
227+
markdown_content.append(f"## `{command_name}`")
260228
markdown_content.append("")
261229

262230
if description:
@@ -274,11 +242,9 @@ def generate_command_docs(docs: List[Dict[str, Any]]) -> str:
274242
# Add options if any
275243
options_section = format_options_section(help_text)
276244
if options_section:
277-
# Remove the "## Options" header since it's a subsection
278245
markdown_content.append("**Options:**")
279246
markdown_content.append("")
280247
markdown_content.append(options_section)
281-
markdown_content.append("")
282248

283249
markdown_content.append("---")
284250
markdown_content.append("")
@@ -302,7 +268,7 @@ def main():
302268

303269
# Determine output path
304270
docs_dir = project_root / "docs" / "docs" / "core"
305-
output_file = docs_dir / "cli-reference.md"
271+
output_file = docs_dir / "cli-commands.md"
306272

307273
# Ensure directory exists
308274
docs_dir.mkdir(parents=True, exist_ok=True)

‎docs/docs/core/cli-reference.md‎ renamed to ‎docs/docs/core/cli-commands.md‎

Lines changed: 16 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,6 @@
1-
# CLI Commands Reference
1+
# CLI Commands
22

3-
This page contains the detailed help information for all CocoIndex CLI commands.
4-
5-
## Overview
6-
7-
CLI for Cocoindex.
8-
9-
## Usage
10-
11-
```sh
12-
Usage: cocoindex [OPTIONS] COMMAND [ARGS]...
13-
```
14-
15-
## Global Options
16-
17-
| Option | Description |
18-
|--------|-------------|
19-
| `--version` | Show the version and exit. |
20-
| `--env-file FILE` | Path to a .env file to load environment variables from. If not provided, attempts to load '.env' from the current directory. |
21-
| `--app-dir TEXT` | Load apps from the specified directory. Default to the current directory. [default: ""] |
22-
| `--help` | Show this message and exit. |
23-
24-
25-
## Commands
26-
27-
| Command | Description |
28-
|---------|-------------|
29-
| `drop` | Drop the backend setup for flows. |
30-
| `evaluate` | Evaluate the flow and dump flow outputs to files. |
31-
| `ls` | List all flows. |
32-
| `server` | Start a HTTP server providing REST APIs. |
33-
| `setup` | Check and apply backend setup changes for flows, including... |
34-
| `show` | Show the flow spec and schema. |
35-
| `update` | Update the index to reflect the latest data from data sources. |
36-
37-
38-
## Command Details
39-
40-
### `drop`
3+
## `drop`
414

425
Drop the backend setup for flows.
436

@@ -50,7 +13,7 @@ Modes of operation:
5013
**Usage:**
5114

5215
```bash
53-
Usage: cocoindex drop [OPTIONS] [APP_TARGET] [FLOW_NAME]...
16+
cocoindex drop [OPTIONS] [APP_TARGET] [FLOW_NAME]...
5417
```
5518

5619
**Options:**
@@ -60,10 +23,9 @@ Usage: cocoindex drop [OPTIONS] [APP_TARGET] [FLOW_NAME]...
6023
| `-f, --force` | Force drop without confirmation prompts. |
6124
| `--help` | Show this message and exit. |
6225

63-
6426
---
6527

66-
### `evaluate`
28+
## `evaluate`
6729

6830
Evaluate the flow and dump flow outputs to files.
6931

@@ -90,7 +52,7 @@ flow.
9052
**Usage:**
9153

9254
```bash
93-
Usage: cocoindex evaluate [OPTIONS] APP_FLOW_SPECIFIER
55+
cocoindex evaluate [OPTIONS] APP_FLOW_SPECIFIER
9456
```
9557

9658
**Options:**
@@ -101,10 +63,9 @@ Usage: cocoindex evaluate [OPTIONS] APP_FLOW_SPECIFIER
10163
| `--cache / --no-cache` | Use already-cached intermediate data if available. [default: cache] |
10264
| `--help` | Show this message and exit. |
10365

104-
10566
---
10667

107-
### `ls`
68+
## `ls`
10869

10970
List all flows.
11071

@@ -119,7 +80,7 @@ backend.
11980
**Usage:**
12081

12182
```bash
122-
Usage: cocoindex ls [OPTIONS] [APP_TARGET]
83+
cocoindex ls [OPTIONS] [APP_TARGET]
12384
```
12485

12586
**Options:**
@@ -128,10 +89,9 @@ Usage: cocoindex ls [OPTIONS] [APP_TARGET]
12889
|--------|-------------|
12990
| `--help` | Show this message and exit. |
13091

131-
13292
---
13393

134-
### `server`
94+
## `server`
13595

13696
Start a HTTP server providing REST APIs.
13797

@@ -142,7 +102,7 @@ APP_TARGET: path/to/app.py or installed_module.
142102
**Usage:**
143103

144104
```bash
145-
Usage: cocoindex server [OPTIONS] APP_TARGET
105+
cocoindex server [OPTIONS] APP_TARGET
146106
```
147107

148108
**Options:**
@@ -151,7 +111,7 @@ Usage: cocoindex server [OPTIONS] APP_TARGET
151111
|--------|-------------|
152112
| `-a, --address TEXT` | The address to bind the server to, in the format of IP:PORT. If unspecified, the address specified in COCOINDEX_SERVER_ADDRESS will be used. |
153113
| `-c, --cors-origin TEXT` | The origins of the clients (e.g. CocoInsight UI) to allow CORS from. Multiple origins can be specified as a comma-separated list. e.g. `https://cocoindex.io,http://localhost:3000`. Origins specified in COCOINDEX_SERVER_CORS_ORIGINS will also be included. |
154-
| `-ci, --cors-cocoindex` | Allow https://cocoindex.io to access the server. |
114+
| `-ci, --cors-cocoindex` | Allow `https://cocoindex.io` to access the server. |
155115
| `-cl, --cors-local INTEGER` | Allow `http://localhost:<port>` to access the server. |
156116
| `-L, --live-update` | Continuously watch changes from data sources and apply to the target index. |
157117
| `--setup` | Automatically setup backends for the flow if it's not setup yet. |
@@ -161,10 +121,9 @@ Usage: cocoindex server [OPTIONS] APP_TARGET
161121
| `-r, --reload` | Enable auto-reload on code changes. |
162122
| `--help` | Show this message and exit. |
163123

164-
165124
---
166125

167-
### `setup`
126+
## `setup`
168127

169128
Check and apply backend setup changes for flows, including the internal
170129

@@ -175,7 +134,7 @@ APP_TARGET: path/to/app.py or installed_module.
175134
**Usage:**
176135

177136
```bash
178-
Usage: cocoindex setup [OPTIONS] APP_TARGET
137+
cocoindex setup [OPTIONS] APP_TARGET
179138
```
180139

181140
**Options:**
@@ -185,10 +144,9 @@ Usage: cocoindex setup [OPTIONS] APP_TARGET
185144
| `-f, --force` | Force setup without confirmation prompts. |
186145
| `--help` | Show this message and exit. |
187146

188-
189147
---
190148

191-
### `show`
149+
## `show`
192150

193151
Show the flow spec and schema.
194152

@@ -211,7 +169,7 @@ flow.
211169
**Usage:**
212170

213171
```bash
214-
Usage: cocoindex show [OPTIONS] APP_FLOW_SPECIFIER
172+
cocoindex show [OPTIONS] APP_FLOW_SPECIFIER
215173
```
216174

217175
**Options:**
@@ -222,10 +180,9 @@ Usage: cocoindex show [OPTIONS] APP_FLOW_SPECIFIER
222180
| `--verbose` | Show verbose output with full details. |
223181
| `--help` | Show this message and exit. |
224182

225-
226183
---
227184

228-
### `update`
185+
## `update`
229186

230187
Update the index to reflect the latest data from data sources.
231188

@@ -236,7 +193,7 @@ module:FlowName. If :FlowName is omitted, updates all flows.
236193
**Usage:**
237194

238195
```bash
239-
Usage: cocoindex update [OPTIONS] APP_FLOW_SPECIFIER
196+
cocoindex update [OPTIONS] APP_FLOW_SPECIFIER
240197
```
241198

242199
**Options:**
@@ -250,5 +207,4 @@ Usage: cocoindex update [OPTIONS] APP_FLOW_SPECIFIER
250207
| `-q, --quiet` | Avoid printing anything to the standard output, e.g. statistics. |
251208
| `--help` | Show this message and exit. |
252209

253-
254210
---

‎docs/docs/core/cli.mdx‎

Lines changed: 2 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -53,27 +53,6 @@ CocoIndex CLI supports the following global options:
5353
* `--version`: Show the CocoIndex version and exit.
5454
* `--help`: Show the main help message and exit.
5555

56-
## Subcommands
56+
import CliCommands from './cli-commands.md';
5757

58-
The following subcommands are available:
59-
60-
| Subcommand | Description |
61-
| ---------- | ----------- |
62-
| `ls` | List all flows present in the given file/module. Or list all persisted flows under the current app namespace if no file/module specified. |
63-
| `show` | Show the spec and schema for a specific flow. |
64-
| `setup` | Check and apply backend setup changes for flows, including the internal storage and target (to export). |
65-
| `drop` | Drop the backend setup for specified flows. |
66-
| `update` | Update the index defined by the flow. |
67-
| `evaluate` | Evaluate the flow and dump flow outputs to files. Instead of updating the index, it dumps what should be indexed to files. Mainly used for evaluation purpose. |
68-
| `server` | Start a HTTP server providing REST APIs. It will allow tools like CocoInsight to access the server. |
69-
70-
Use `--help` to see the full list of subcommands, and `subcommand --help` to see the usage of a specific one.
71-
72-
```sh
73-
cocoindex --help # Show all subcommands
74-
cocoindex show --help # Show usage of "show" subcommand
75-
```
76-
77-
## Detailed Command Reference
78-
79-
For complete details on all commands and their options, see the auto-generated [CLI Commands Reference](./cli-reference).
58+
<CliCommands />

0 commit comments

Comments
 (0)