Merge pull request #30 from NREL/shared-parameters

Allow shared parameters in spec files

Author: daniel-thom

CLAUDE.md

@@ -41,8 +41,6 @@ torc/
 │   ├── lib.rs           # Library root
 │   └── models.rs        # Shared data models
 ├── torc-server/         # Standalone server binary
-├── torc-tui/            # Standalone TUI binary
-├── torc-plot-resources/ # Standalone plotting binary
 ├── torc-slurm-job-runner/ # Slurm job runner binary
 ├── python_client/       # Python CLI client and library
 │   ├── src/torc/        # Python package

docs/README.md

@@ -76,7 +76,6 @@ src/
 │   ├── job-states.md
 │   ├── reinitialization.md
 │   ├── dependencies.md
-│   └── ready-queue.md
 │
 ├── how-to/                 # Problem-oriented
 │   ├── README.md

docs/src/SUMMARY.md

@@ -24,7 +24,6 @@
   - [Design](./explanation/design/README.md)
     - [Server API Handler](./explanation/design/server.md)
     - [Central Database](./explanation/design/database.md)
-    - [Ready Queue](./explanation/design/ready-queue.md)
 
 # How-To Guides
 

docs/src/explanation/dependencies.md

@@ -13,7 +13,7 @@ jobs:
   - name: analyze
     command: analyze.sh
     blocked_by:
-      - job1
+      - preprocess
 ```
 
 ## 2. Implicit Dependencies

docs/src/explanation/environment-variables.md

@@ -53,10 +53,6 @@ curl -X POST "${TORC_API_URL}/files" \
   }"
 ```
 
-## Implementation Details
-
-These environment variables are set by the job runner when spawning job processes. The implementation can be found in `src/client/async_cli_command.rs` in the `start()` method.
-
 ## Complete Example
 
 Here's a complete example of a job that uses all three environment variables:
@@ -83,16 +79,6 @@ jobs:
       # Do some work
       echo "Processing data..." > "${OUTPUT_DIR}/status.txt"
       date >> "${OUTPUT_DIR}/status.txt"
-
-      # Register the output file with Torc
-      curl -X POST "${TORC_API_URL}/files" \
-        -H "Content-Type: application/json" \
-        -d "{
-          \"workflow_id\": ${TORC_WORKFLOW_ID},
-          \"name\": \"job_${TORC_JOB_ID}_output\",
-          \"path\": \"${OUTPUT_DIR}/status.txt\"
-        }"
-
       echo "Job completed successfully!"
 ```
 

docs/src/explanation/job-runners.md

@@ -16,16 +16,28 @@ The job runner supports two different strategies for retrieving and executing jo
 **Used when**: `--max-parallel-jobs` is NOT specified
 
 **Behavior**:
-- Retrieves jobs from the server via `GET /workflows/{id}/claim_jobs_based_on_resources`
+- Retrieves jobs from the server via the command `claim_jobs_based_on_resources`
 - Server filters jobs based on available compute node resources (CPU, memory, GPU)
 - Only returns jobs that fit within the current resource capacity
 - Prevents resource over-subscription and ensures jobs have required resources
-- Defaults to requiring one CPU for each job.
+- Defaults to requiring one CPU an 1 MB of memory for each job.
 
-**Use case**: When you have heterogeneous jobs with different resource requirements and want
+**Use cases**:
+- When you want parallelization based on one CPU per job.
+- When you have heterogeneous jobs with different resource requirements and want
 intelligent resource management.
 
-**Example**:
+**Example 1: Run jobs at queue depth of num_cpus**:
+```yaml
+parameters:
+  i: "1..100"
+jobs:
+  - name: "work_{i}"
+    command: bash my_script.sh {i}
+    use_parameters: {i}
+```
+
+**Example 2: Resource-based parallelization**:
 ```yaml
 resource_requirements:
   - name: "work_resources"
@@ -34,24 +46,28 @@ resource_requirements:
     runtime: "PT4H"
     num_nodes: 1
 
+parameters:
+  i: "1..100"
 jobs:
-  - name: "work1"
-    command: bash my_script.sh
+  - name: "work_{i}"
+    command: bash my_script.sh {i}
     resource_requirements: work_resources  
+    use_parameters: {i}
 ```
 
 ### Simple Queue-Based Allocation
 
 **Used when**: `--max-parallel-jobs` is specified
 
 **Behavior**:
-- Retrieves jobs from the server via `GET /workflows/{id}/claim_next_jobs`
+- Retrieves jobs from the server via the command `claim_next_jobs`
 - Server returns the next N ready jobs from the queue (up to the specified limit)
 - Ignores job resource requirements completely
 - Simply limits the number of concurrent jobs
 
-**Use case**: When all jobs have similar resource needs or when the resource bottleneck is not
-tracked by Torc, such as network or storage I/O.
+**Use cases**: When all jobs have similar resource needs or when the resource bottleneck is not
+tracked by Torc, such as network or storage I/O. This is the only way to run jobs at a queue
+depth higher than the number of CPUs in the worker.
 
 **Example**:
 ```bash
@@ -66,16 +82,17 @@ The job runner executes a continuous loop with these steps:
 
 1. **Check workflow status** - Poll server to check if workflow is complete or canceled
 2. **Monitor running jobs** - Check status of currently executing jobs
-3. **Execute workflow actions** - Check for and execute any pending workflow actions
+3. **Execute workflow actions** - Check for and execute any pending workflow actions, such as
+   scheduling new Slurm allocations.
 4. **Claim new jobs** - Request ready jobs from server based on allocation strategy:
-   - Resource-based: `GET /workflows/{id}/claim_jobs_based_on_resources`
-   - Queue-based: `GET /workflows/{id}/claim_next_jobs`
+   - Resource-based: `claim_jobs_based_on_resources`
+   - Queue-based: `claim_next_jobs`
 5. **Start jobs** - For each claimed job:
-   - Call `POST /jobs/{id}/start_job` to mark job as started in database
-   - Execute job command using `AsyncCliCommand` (non-blocking subprocess)
-   - Track stdout/stderr output to files
+   - Call `start_job` to mark job as started in database
+   - Execute job command in a non-blocking subprocess
+   - Record stdout/stderr output to files
 6. **Complete jobs** - When running jobs finish:
-   - Call `POST /jobs/{id}/complete_job` with exit code and result
+   - Call `complete_job` with exit code and result
    - Server updates job status and automatically marks dependent jobs as ready
 7. **Sleep and repeat** - Wait for job completion poll interval, then repeat loop
 

docs/src/explanation/job-states.md

@@ -33,13 +33,6 @@ stateDiagram-v2
 - **completed** (5) - Finished successfully (exit code 0)
 - **failed** (6) - Finished with error (exit code != 0)
 - **canceled** (7) - Explicitly canceled by user or system
-- **terminated** (8) - Explicitly terminated by user or system
+- **terminated** (8) - Explicitly terminated by system, such as for checkpointing before
+  wall-time timeout
 - **disabled** (9) - Explicitly disabled by user
-
-## Critical State Transitions
-
-1. **initialize_jobs** - Evaluates all dependencies and sets jobs to `ready` or `blocked`
-2. **manage_status_change** - Updates job status and triggers cascade effects:
-   - When a job completes, checks if blocked jobs become ready
-   - Updates workflow status when all jobs complete
-   - Handles `cancel_on_blocking_job_failure` flag

docs/src/explanation/reinitialization.md

@@ -15,7 +15,8 @@ Reinitialization allows workflows to be rerun when inputs change.
 
 The `process_changed_job_inputs` endpoint implements hash-based change detection:
 
-1. For each job, compute SHA256 hash of all inputs (files + user_data).
+1. For each job, compute SHA256 hash of all input parameters. **Note**: files are tracked by
+   modification times, not hashes. User data records are hashed.
 2. Compare to stored hash in the database.
 3. If hash differs, mark job as `uninitialized`.
 4. All updates happen in a single database transaction (all-or-none).