Skip to content

Commit a7697a8

Browse files
authored
Style improvements (#73)
* fix: update dagviz, snakevision call now supports native dark mode * perf: attempt to download only Snakefile containing repos * docs: added pixi and new wf page features * fix: just use latest release as before * fix: review comments
1 parent 711f57c commit a7697a8

File tree

7 files changed

+776
-744
lines changed

7 files changed

+776
-744
lines changed

README.md

Lines changed: 50 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Workflows are automatically added to the Workflow Catalog. This is done by regul
2828
- The repository contains a workflow definition named either `Snakefile` or `workflow/Snakefile`.
2929
- If the repository contains a folder `rules` or `workflow/rules`, that folder must at least contain one file ending on `.smk`.
3030
- The repository is small enough to be cloned into a [Github Actions](https://docs.github.com/en/actions/about-github-actions/understanding-github-actions) job (very large files should be handled via [Git LFS](https://docs.github.com/en/repositories/working-with-files/managing-large-files), so that they can be stripped out during cloning).
31-
- The repository is not blacklisted here.
31+
- The repository is not blacklisted.
3232

3333
### Standardized usage workflows
3434

@@ -54,6 +54,23 @@ usage:
5454
5555
Once included in the standardized usage area you can link directly to the workflow page using the URL `https://snakemake.github.io/snakemake-workflow-catalog/docs/workflows/<owner>/<repo>`. Do not forget to replace the `<owner>` and `<repo>` tags at the end of the URL.
5656

57+
### Workflow pages
58+
59+
Each standardized workflow has its own page, which is linked on the summary tables or the 'top workflows' tiles.
60+
Workflow pages are **enhanced by information automatically parsed** from their Github repositories. Right now this includes:
61+
62+
1. **Tube Maps**: A graphical representation of the workflow rulegraph, build using [snakevision](https://github.com/OpenOmics/snakevision).
63+
Tube maps will automatically show up on your workflow page if the following command can be run for your workflow: `snakemake -s <snakefile> -c 1 -d .test --forceall --rulegraph`.
64+
This means, you need to have a working test case defined in the `.test` sub-dir.
65+
66+
2. **Workflow Configuration**: These are simply the configuration instructions from `config/README.md`.
67+
68+
3. **Workflow Parameters**: If the repo contains a `workflow/schemas/config.schema.yaml` OR `config/schemas/config.schema.yaml` file, the parameters defined in this file will be parsed and displayed as a table on the workflow page.
69+
The default fields that are recognized in the schema are `type`, `description`, `default` and `required`.
70+
Config options can be arbitrarily nested in the schema using the `properties` field of an object.
71+
72+
All these features are implemented in the [snakemake-workflow-template](https://github.com/snakemake-workflows/snakemake-workflow-template). If you want to create a standard-compliant workflow (page), the template is the ideal starting point.
73+
5774
### Release handling
5875

5976
If your workflow provides Github releases, the catalog will always just scrape the latest non-preview release. Hence, in order to update your workflow's records here, you need to release a new version on Github.
@@ -82,54 +99,61 @@ Then, clone the forked repository:
8299

83100
Make your changes to the catalog:
84101

85-
1. Create a conda/mamba environment in order to work with the catalog locally.
102+
1. Download all software dependencies using [pixi](https://pixi.prefix.dev/latest/). Read up here [how to set up pixi](https://pixi.prefix.dev/latest/installation/) on your system.
86103

87104
```bash
88-
cd <path-to>/snakemake-workflow-catalog
89-
conda env create -n snakemake-workflow-catalog -f environment.yml
90-
conda activate snakemake-workflow-catalog
105+
pixi shell
91106
```
92107

93-
2. Set required environment variables.
94-
The variable `N_REPOS` can be used to fetch data from a limited number of repos.
95-
The variable `TEST_REPO` can be used to fetch only data from a single workflow.
96-
**Note:** Building the entire catalog from scratch will take several hours due to searching and testing thousands of Github repos.
108+
2. In order to run predefined tests, run one of:
109+
110+
```bash
111+
pixi run test-generate-catalog # tests 3 latest repos
112+
pixi run test-generate-catalog-single-repo # tests a single predefined repo
113+
pixi run test-cleanup-catalog # tests cleanup of the catalog
114+
```
115+
116+
3. In order to run custom tests, set the required environment variables.
117+
The variable `N_REPOS` can be used to fetch data from a limited number of repos.
118+
The variable `LATEST_COMMIT` can be used to limit the search to repos which were updated within the last N days.
119+
The `OFFSET` variable can be used to skip a number of repos in the search results.
120+
The variable `TEST_REPO` can be used to fetch only data from a single workflow. It overrides the other settings.
121+
**Note:** Building the entire catalog from scratch will take several hours due to searching and testing thousands of Github repos.
97122

98123
```bash
99124
export GITHUB_TOKEN="<your-github-token>"
100-
export OFFSET=0
101-
export LATEST_COMMIT=1000
102125
export N_REPOS=3
103-
export TEST_REPO="snakemake-workflows/rna-seq-star-deseq2"
126+
export LATEST_COMMIT=7
127+
export OFFSET=0
128+
export TEST_REPO="snakemake-workflows/dna-seq-varlociraptor"
104129
```
105130

106-
3. Build the catalog data sources using the test repository.
131+
4. Fetch the data for the catalog from Github:
107132

108133
```bash
109-
python scripts/generate-catalog.py
110-
python scripts/cleanup-catalog.py
134+
pixi run generate-catalog
135+
pixi run cleanup-catalog
111136
```
112137

113-
4. Build the catalog web page using sphinx autobuild (live reload).
138+
5. Build the catalog web page using sphinx.
114139

115140
```bash
116-
sphinx-autobuild source/ build/
141+
pixi run render-catalog
117142
```
118143

119-
... or using the make file (static build).
144+
6. Alternatively, build the catalog web page using sphinx autobuild (live reload).
120145

121146
```bash
122-
make html
147+
sphinx-autobuild source/ build/
123148
```
124149

125-
5. Run `git add .` to stage your changes.
126-
6. Run `git commit -m "fix: your commit message"` to commit your changes.
127-
7. Run `git push` to push your changes to your fork on Github.
128-
129-
Finally, create a pull request:
150+
If you are happy with your changes, you can push them to your fork on Github and [start a Pull Request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request):
130151

131-
1. Go to your fork on Github.
132-
2. Follow the instructions on the [Github documentation: Creating a pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).
152+
```bash
153+
git add .
154+
git commit -m "fix: your commit message"
155+
git push
156+
```
133157

134158
## Using workflows from the catalog
135159

0 commit comments

Comments
 (0)