Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ jobs:
run: |
# Run a basic test to ensure the minimal package works
uv run python -c "import wurzel; print('Minimal installation successful')"
# Run only core tests that don't require optional dependencies
uv run pytest tests/ -k "not (qdrant or milvus or docling or openai or transformers or tlsh or splitter)" -v --ignore=tests/splitter --ignore=tests/splitter_test.py
# Core tests only (no optional deps); skip doc examples (run in full install)
uv run pytest tests/ -k "not (qdrant or milvus or docling or openai or transformers or tlsh or splitter)" -v --ignore=tests/splitter --ignore=tests/splitter_test.py --ignore=tests/test_doc_examples.py


- name: Test full installation
Expand Down
11 changes: 9 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,19 @@ wurzel inspect wurzel.<step_path>
### 2. Programmatic Execution (Python)
Run a step using the snippet below:
```python
from wurzel.steps import step
import os
from pathlib import Path

from wurzel.step_executor import BaseStepExecutor
from wurzel.steps.manual_markdown import ManualMarkdownStep

# Create input dir and set folder (required by ManualMarkdownStep)
input_dir = Path("./input")
input_dir.mkdir(exist_ok=True)
abs_input = input_dir.resolve()
os.environ["MANUALMARKDOWNSTEP__FOLDER_PATH"] = str(abs_input)
with BaseStepExecutor() as ex:
ex(step, set(Path("./input")), Path("./output"))
ex(ManualMarkdownStep, {abs_input}, Path("./output"))
```

### Building your one step
Expand Down
14 changes: 6 additions & 8 deletions docs/backends/argoworkflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -364,7 +364,6 @@ container:
Use the Argo backend directly in Python:

```python
from pathlib import Path
from wurzel.backend.backend_argo import ArgoBackend
from wurzel.steps.embedding import EmbeddingStep
from wurzel.steps.manual_markdown import ManualMarkdownStep
Expand All @@ -375,17 +374,16 @@ from wurzel.utils import WZ
source = WZ(ManualMarkdownStep)
embedding = WZ(EmbeddingStep)
step = WZ(QdrantConnectorStep)

source >> embedding >> step
pipeline = step

# Generate Argo Workflows configuration from values file
backend = ArgoBackend.from_values(
files=[Path("values.yaml")],
workflow_name="pipelinedemo"
)
# Generate Argo Workflows configuration (default config, no values file)
backend = ArgoBackend()
argo_yaml = backend.generate_artifact(pipeline)
print(argo_yaml)

# Or from values file (YAML must contain workflows.pipelinedemo):
# from pathlib import Path
# backend = ArgoBackend.from_values(files=[Path("values.yaml")], workflow_name="pipelinedemo")
```

## Deploying Argo Workflows
Expand Down
19 changes: 7 additions & 12 deletions docs/backends/dvc.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,6 @@ dvc repro
Use the DVC backend directly in Python:

```python
from pathlib import Path
from wurzel.backend.backend_dvc import DvcBackend
from wurzel.steps.embedding import EmbeddingStep
from wurzel.steps.manual_markdown import ManualMarkdownStep
Expand All @@ -109,21 +108,17 @@ from wurzel.utils import WZ
source = WZ(ManualMarkdownStep)
embedding = WZ(EmbeddingStep)
step = WZ(QdrantConnectorStep)

source >> embedding >> step
pipeline = step

# Option 1: Generate DVC configuration from values file
backend = DvcBackend.from_values(
files=[Path("values.yaml")],
workflow_name="pipelinedemo"
)
dvc_yaml = backend.generate_artifact(pipeline)
print(dvc_yaml)

# Option 2: Generate with default settings
# Option 1: Generate with default settings (no values file)
dvc_yaml = DvcBackend().generate_artifact(pipeline)
print(dvc_yaml)

# Option 2: Generate from values file (YAML must contain dvc.pipelinedemo)
# from pathlib import Path
# values_file = Path("values.yaml")
# backend = DvcBackend.from_values(files=[values_file], workflow_name="pipelinedemo")
# dvc_yaml = backend.generate_artifact(pipeline)
```

## Running DVC Pipelines
Expand Down
Loading
Loading