Skip to content

Commit 26a0121

Browse files
committed
Update docs
1 parent 4f24e2d commit 26a0121

File tree

16 files changed

+118
-46
lines changed

16 files changed

+118
-46
lines changed

docs/src/explanation/architecture.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ coordination, while clients create workflows and job runners execute tasks on co
77

88
**Key Components:**
99

10-
- **Server**: REST API service that manages workflow state via a SQLite database
10+
- **Server**: HTTP API service that manages workflow state via a SQLite database
1111
- **Client**: CLI tool and library for creating and managing workflows
1212
- **Job Runner**: Worker process that pulls ready jobs, executes them, and reports results
1313
- **Database**: Central SQLite database that stores all workflow state and coordinates distributed
@@ -18,7 +18,7 @@ coordination, while clients create workflows and job runners execute tasks on co
1818
```mermaid
1919
graph TB
2020
subgraph Server["Torc Server"]
21-
API["REST API (Tokio 1-thread)<br/>/workflows /jobs /files<br/>/user_data /results"]
21+
API["HTTP API (Tokio 1-thread)<br/>/workflows /jobs /files<br/>/user_data /results"]
2222
DB["SQLite Database (WAL)<br/>• Workflow state<br/>• Job dependencies<br/>• Resource tracking<br/>• Execution results"]
2323
API --> DB
2424
end
@@ -27,7 +27,7 @@ graph TB
2727
Runner1["Job Runner 1<br/>(compute-01)<br/>• Poll for jobs<br/>• Execute tasks<br/>• Report results"]
2828
RunnerN["Job Runner N<br/>(compute-nn)<br/>• Poll for jobs<br/>• Execute tasks<br/>• Report results"]
2929
30-
Client -.HTTP/REST.-> API
31-
Runner1 -.HTTP/REST.-> API
32-
RunnerN -.HTTP/REST.-> API
30+
Client -.HTTP.-> API
31+
Runner1 -.HTTP.-> API
32+
RunnerN -.HTTP.-> API
3333
```

docs/src/explanation/design/interfaces.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ graph TD
4646
end
4747
4848
subgraph "Server"
49-
HTTP["HTTP REST API<br/>(torc-server)"]
49+
HTTP["HTTP API<br/>(torc-server)"]
5050
DB[(SQLite Database)]
5151
end
5252

docs/src/explanation/job-states.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,5 @@ stateDiagram-v2
3232
- **running** (4) - Currently executing
3333
- **completed** (5) - Finished successfully (exit code 0)
3434
- **failed** (6) - Finished with error (exit code != 0)
35-
- **canceled** (7) - Explicitly canceled by user or system
36-
- **terminated** (8) - Explicitly terminated by system, such as for checkpointing before wall-time
37-
timeout
35+
- **canceled** (7) - Explicitly canceled by user or torc. Never executed.
36+
- **terminated** (8) - Explicitly terminated by system, such as at wall-time timeout

docs/src/getting-started.md

Lines changed: 50 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -10,51 +10,82 @@ coordination, while clients create workflows and job runners execute tasks on co
1010

1111
Choose the interface that fits your workflow:
1212

13-
- **Dashboard** — Web UI ([torc-dash](./how-to/dashboard.md)) for visual monitoring
13+
- **CLI** — Primary interface which provides access to all torc functionality
14+
- **Dashboard** — Web UI ([torc-dash](./how-to/dashboard.md)) for visual configuration and
15+
monitoring
16+
- **TUI** — Terminal User Interface (`torc tui`) for visual monitoring in a terminal
1417
- **AI Assistants** — Use [Claude Code](https://claude.ai/code) or GitHub Copilot to manage
1518
workflows through natural language. _"Create a workflow with 10 parallel jobs"_ or _"Why did job 5
1619
fail?"_
17-
- **TUI** — Terminal User Interface (`torc tui`) for visual monitoring in a terminal
18-
- **CLI** — The `torc` command for scripting and automation
1920
- **Spec Files** — YAML, KDL, or JSON5 files for version-controlled workflow definitions
2021
- **Python/Julia APIs** — Libraries for programmatic workflow generation
2122

22-
All interfaces work with the same servermix and match as needed.
23+
All interfaces work with the same servermix and match as needed.
2324

2425
## Architecture
2526

26-
```
27-
┌─────────────────────────────────────────────────────────────────┐
28-
│ Torc Server │
29-
│ (REST API + SQLite DB) │
30-
└─────────────────────────────────────────────────────────────────┘
31-
▲ ▲ ▲ ▲
32-
│ │ │ │
33-
┌────┴────┐ ┌─────┴─────┐ ┌────┴────┐ ┌─────┴─────┐
34-
│ CLI │ │ Dashboard │ │ AI │ │ Python │
35-
│ torc │ │ torc-dash │ │Assistant│ │ API │
36-
└─────────┘ └───────────┘ └─────────┘ └───────────┘
27+
```mermaid
28+
flowchart TB
29+
subgraph ai["AI Integration"]
30+
AI_CLIENTS["AI Clients<br/>(Claude Code, Copilot)"]
31+
MCP["torc-mcp-server"]
32+
AI_CLIENTS -->|"MCP"| MCP
33+
end
34+
35+
subgraph clients["Client Applications"]
36+
TUI["torc tui"]
37+
DASH["torc-dash<br/>(web UI)"]
38+
PY["Python Client"]
39+
JL["Julia Client"]
40+
end
41+
42+
CLI["torc CLI"]
43+
44+
subgraph server["Server"]
45+
HTTP["HTTP API<br/>(torc-server)"]
46+
DB[(SQLite)]
47+
HTTP --> DB
48+
end
49+
50+
subgraph workers["Workers (local, remote, or HPC)"]
51+
LOCAL["Local Runner"]
52+
SLURM["Slurm Runner"]
53+
end
54+
55+
TUI -->|"executes"| CLI
56+
DASH -->|"executes"| CLI
57+
MCP -->|"executes"| CLI
58+
59+
CLI -->|"HTTP"| HTTP
60+
TUI -->|"HTTP"| HTTP
61+
DASH -->|"HTTP"| HTTP
62+
MCP -->|"HTTP"| HTTP
63+
PY -->|"HTTP"| HTTP
64+
JL -->|"HTTP"| HTTP
65+
66+
LOCAL -->|"HTTP"| HTTP
67+
SLURM -->|"HTTP"| HTTP
3768
```
3869

3970
**Key components:**
4071

41-
- **Server**REST API service managing workflow state via SQLite
72+
- **Server**HTTP API service managing workflow state via SQLite
4273
- **Job Runners** — Worker processes that execute jobs on compute resources
4374
- **Clients** — CLI, dashboard, AI assistants, or API libraries
4475

4576
## Features
4677

47-
- **AI-Assisted Management** — Create, debug, and manage workflows through conversation
4878
- **Declarative Workflow Specifications** — Define workflows in YAML, JSON5, JSON, or KDL
49-
- **Automatic Dependency Resolution** — Dependencies inferred from file and data relationships
5079
- **Job Parameterization** — Create parameter sweeps and grid searches with simple syntax
80+
- **Automatic Dependency Resolution** — Dependencies inferred from file and data relationships
5181
- **Distributed Execution** — Run jobs across multiple compute nodes with resource tracking
5282
- **Slurm Integration** — Native support for HPC cluster job submission
5383
- **Automatic Failure Recovery** — Detect OOM/timeout failures and retry with adjusted resources
5484
- **Workflow Resumption** — Restart workflows after failures without losing progress
5585
- **Change Detection** — Automatically detect input changes and re-run affected jobs
56-
- **Resource Management** — Track CPU, memory, and GPU usage across all jobs
86+
- **Resource Management** — Track CPU and memory across all jobs
5787
- **RESTful API** — Complete OpenAPI-specified REST API for integration
88+
- **AI-Assisted Management** — Create, debug, and manage workflows through conversation
5889

5990
## Next: Quick Start
6091

docs/src/how-to/ai-assistants.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Complete guide for configuring AI assistants (Claude Code, GitHub Copilot) to wo
55
## Overview
66

77
Torc provides an MCP (Model Context Protocol) server that enables AI assistants to interact with
8-
workflows. The `torc-mcp-server` binary acts as a bridge between AI assistants and the Torc REST
8+
workflows. The `torc-mcp-server` binary acts as a bridge between AI assistants and the Torc HTTP
99
API.
1010

1111
## Available Tools
@@ -219,7 +219,7 @@ Torc uses the Model Context Protocol (MCP), an open standard for connecting AI a
219219
external tools. The `torc-mcp-server` binary:
220220

221221
1. **Receives tool calls** from the AI assistant via stdio
222-
2. **Translates them** to Torc REST API calls
222+
2. **Translates them** to Torc HTTP API calls
223223
3. **Returns results** in a format the assistant can understand
224224

225225
The server is stateless—it simply proxies requests to your running Torc server. All workflow state

docs/src/how-to/slurm.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -293,7 +293,7 @@ torc slurm schedule-nodes -n 5 $WORKFLOW_ID
293293
### Check Slurm Job Status
294294

295295
```bash
296-
squeue -u $USER
296+
squeue --me
297297
```
298298

299299
### View Torc Worker Logs

docs/src/installation.md

Lines changed: 44 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,45 @@ xattr -cr /path/to/torc*
1414

1515
Alternatively, you can right-click each binary and select "Open" to add a security exception.
1616

17+
## Site-Specific Installations
18+
19+
Some HPC facilities maintain pre-installed Torc binaries and shared servers. Check if your site is
20+
listed below.
21+
22+
### NREL Kestrel
23+
24+
**Pre-installed binaries** are available at:
25+
26+
```
27+
/scratch/dthom/torc/
28+
├── 0.8.0/
29+
├── ...
30+
└── latest -> 0.8.0 (symlink to current version)
31+
```
32+
33+
> **Recommended**: Use the `latest` directory. Torc maintains backwards compatibility, so you'll
34+
> automatically receive updates and bug fixes without changing your configuration.
35+
36+
Add to your PATH:
37+
38+
```bash
39+
export PATH="/scratch/dthom/torc/latest:$PATH"
40+
```
41+
42+
Or add to your `~/.bashrc` for persistence:
43+
44+
```bash
45+
echo 'export PATH="/scratch/dthom/torc/latest:$PATH"' >> ~/.bashrc
46+
```
47+
48+
**Shared server**: A `torc-server` instance runs on a dedicated VM within the Kestrel environment.
49+
Contact Daniel Thom for access credentials and the server URL. Once you have access:
50+
51+
```bash
52+
export TORC_API_URL="http://<server-address>/torc-service/v1"
53+
export TORC_PASSWORD="<your-password>"
54+
```
55+
1756
## Building from Source
1857

1958
### Prerequisites
@@ -67,8 +106,10 @@ cargo build --release -p torc-dash
67106
cargo build --release -p torc-slurm-job-runner
68107
```
69108

70-
Binaries will be in `target/release/`. We recommend adding this directory to your system path so
71-
that you can run all binaries without specifying the full path.
109+
Binaries will be in `target/release/`.
110+
111+
**Required**: Add this directory to your system path or copy the binaries to a directory already in
112+
your path (e.g., `~/.local/bin/`).
72113

73114
## Python Client
74115

@@ -81,7 +122,7 @@ The Python client provides programmatic workflow management for Python users.
81122
### Installation
82123

83124
```bash
84-
pip install "torc @ git+https://github.com/NREL/torc.git#subdirectory=python_client"
125+
pip install torc-client
85126
```
86127

87128
The `pytorc` command will be available after installation.

docs/src/introduction.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@ dependencies, mixed resource requirements, and multiple stages.
88

99
- **Declarative Workflow Definitions** — Define workflows in YAML, JSON, JSON5, or KDL
1010
- **Automatic Dependency Resolution** — Dependencies inferred from file and data relationships
11-
- **Distributed Execution** — Run jobs across local machines and HPC clusters
11+
- **Distributed Execution** — Run jobs across local machines, HPC clusters, and networked compute
12+
nodes
1213
- **Resource Management** — Track CPU and memory usage across all jobs
1314
- **Automatic Failure Recovery** — Detect OOM/timeout failures and retry with adjusted resources
1415
- **Fault Tolerance** — Resume workflows after failures without losing progress

docs/src/quick-start-hpc.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ torc tui
105105
Check Slurm queue:
106106

107107
```console
108-
squeue -u $USER
108+
squeue --me
109109
```
110110

111111
## View Results

docs/src/quick-start-local.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ jobs:
3838
command: echo "Hello again from torc!"
3939
```
4040
41-
> **Note:** Torc also accepts `.json5` and `.kdl` workflow specifications. See
41+
> **Note:** Torc also accepts `.json`, `.json5` and `.kdl` workflow specifications. See
4242
> [Workflow Specification Formats](./reference/workflow-formats.md) for details.
4343

4444
## Run the Workflow

0 commit comments

Comments
 (0)