Skip to content

Commit 6eed27f

Browse files
lucalianaslucalianas
committed
docs(usage/02-worklist_creation): added documentation for worklist creation in the annotation manager
1 parent dc73f44 commit 6eed27f

File tree

2 files changed

+125
-1
lines changed

2 files changed

+125
-1
lines changed

‎docs/installation/04-am_config.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ You will be prompted to enter:
2929
Example output:
3030

3131
```bash
32-
~/cdpp-workflows$ ./compose.sh exec promort-web python manage.py createsuperuser
32+
user@host:~/cdpp-workflows$ ./compose.sh exec promort-web python manage.py createsuperuser
3333
WARN[0000] The "CWLDOCKER_ENV" variable is not set. Defaulting to a blank string.
3434
WARN[0000] The "CWLDOCKER_PID" variable is not set. Defaulting to a blank string.
3535
...

‎docs/usage/02-worklist_creation.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
---
2+
layout: default
3+
title: Review worklist creation
4+
parent: Usage
5+
nav_order: 2
6+
---
7+
8+
## Creating the review worklist
9+
10+
Once one or more reviewers have been created, you can generate review worklists to begin reviewing Whole-Slide Images.
11+
This requires assigning ROI and clinical annotation tasks to existing reviewers using the Annotation Manager's administration tools.
12+
13+
### ROIs annotation
14+
15+
To create the ROI annotation worklist, run the `python manage.py build_rois_reviews_worklist` command. By default, the command assigns all existing cases to all reviewers in the `ROIS_MANAGERS` group using a round-robin strategy.
16+
17+
```bash
18+
user@host:~/cdpp-workflows$ ./compose.sh exec promort-web python manage.py build_rois_reviews_worklist
19+
WARN[0000] The "CWLDOCKER_ENV" variable is not set. Defaulting to a blank string.
20+
...
21+
INFO === Starting ROIs worklist creation ===
22+
INFO Creating RANDOM worklist
23+
INFO Processing case P_003
24+
INFO No ROIs Annotation found, creating a new one
25+
INFO Saved new ROIs Annotation with label 782b19dba74440d797f6bbf069f2d8c4 and assigned to test_reviewer
26+
INFO Processing slide P_003-ABC
27+
INFO Creating steps for ROIs Annotation 782b19dba74440d797f6bbf069f2d8c4
28+
INFO Produced label 782b19dba74440d797f6bbf069f2d8c4-ABC
29+
INFO Saved new ROIs Annotation Step with label 782b19dba74440d797f6bbf069f2d8c4-ABC
30+
INFO Processing slide P_003-DEF
31+
INFO Creating steps for ROIs Annotation 782b19dba74440d797f6bbf069f2d8c4
32+
INFO Produced label 782b19dba74440d797f6bbf069f2d8c4-DEF
33+
INFO Saved new ROIs Annotation Step with label 782b19dba74440d797f6bbf069f2d8c4-DEF
34+
INFO Processing case P_001
35+
INFO No ROIs Annotation found, creating a new one
36+
INFO Saved new ROIs Annotation with label cf0e125804664b839e007386220f34d0 and assigned to test_reviewer
37+
INFO Processing slide P_001-DEF
38+
INFO Creating steps for ROIs Annotation cf0e125804664b839e007386220f34d0
39+
INFO Produced label cf0e125804664b839e007386220f34d0-DEF
40+
INFO Saved new ROIs Annotation Step with label cf0e125804664b839e007386220f34d0-DEF
41+
INFO Processing slide P_001-ABC
42+
INFO Creating steps for ROIs Annotation cf0e125804664b839e007386220f34d0
43+
INFO Produced label cf0e125804664b839e007386220f34d0-ABC
44+
INFO Saved new ROIs Annotation Step with label cf0e125804664b839e007386220f34d0-ABC
45+
INFO === ROIs worklist creation completed ===
46+
```
47+
48+
### Clinical annotation
49+
50+
Once the ROI review worklists have been created, you can generate clinical annotation review worklists by running the `python manage.py build_clinical_reviews_worklist` command. By default, a clinical annotation request is created for each ROI annotation and assigned to the same reviewer. To assign additional reviewers, use the `--reviewers-count` option followed by the desired number. These additional reviewers are selected from the `CLINICAL_MANAGERS` group. All ROIs generated during the linked ROI annotation step will be available for review by any user assigned a clinical annotation review linked to that specific step.
51+
52+
```bash
53+
user@host:~/cdpp-workflows$ ./compose.sh exec promort-web python manage.py build_clinical_reviews_worklist
54+
WARN[0000] The "CWLDOCKER_ENV" variable is not set. Defaulting to a blank string.
55+
...
56+
INFO === Starting clinical annotations worklist creation ===
57+
INFO Processing ROIs Annotation 782b19dba74440d797f6bbf069f2d8c4
58+
INFO Assigning review to user test_reviewer
59+
INFO Saved Clinical Annotation with label 782b19dba74440d797f6bbf069f2d8c4
60+
INFO Saved new Clinical Annotation Step with label 782b19dba74440d797f6bbf069f2d8c4-ABC
61+
INFO Saved new Clinical Annotation Step with label 782b19dba74440d797f6bbf069f2d8c4-DEF
62+
INFO Processing ROIs Annotation cf0e125804664b839e007386220f34d0
63+
INFO Assigning review to user test_reviewer
64+
INFO Saved Clinical Annotation with label cf0e125804664b839e007386220f34d0
65+
INFO Saved new Clinical Annotation Step with label cf0e125804664b839e007386220f34d0-DEF
66+
INFO Saved new Clinical Annotation Step with label cf0e125804664b839e007386220f34d0-ABC
67+
INFO === Clinical annotation worklist creation completed ===
68+
```
69+
70+
### Apply pipeline results
71+
72+
If the automatic annotation pipeline has been run on the slides to be reviewed, the regions detected by AI tools can be used as ROIs and included in the review process. To do this, run the command `python manage.py tissue_to_rois`. This will register the H&E-stained tissue regions detected by the pipeline tools as `Slice` and `Core` objects. To distinguish these ROIs from those manually created by reviewers, we recommend marking them explicitly as created by the default workflow manager user by passing `--username $PROMORT_USER` to the script.
73+
74+
```bash
75+
user@host:~/cdpp-workflows$ ./compose.sh exec promort-web python manage.py tissue_to_rois --username $PROMORT_USER
76+
WARN[0000] The "CWLDOCKER_ENV" variable is not set. Defaulting to a blank string.
77+
...
78+
INFO == Starting import job ==
79+
INFO Loaded 4 ROIs annotation steps
80+
INFO Processing ROIs annotation step 782b19dba74440d797f6bbf069f2d8c4-ABC
81+
INFO Loading bounds for slide P_003-ABC
82+
INFO response.status_code 200
83+
INFO Slice saved with ID 1
84+
INFO Loading core 1 of 5
85+
INFO Core saved with ID 1
86+
INFO Loading core 2 of 5
87+
INFO Core saved with ID 2
88+
...
89+
INFO Loading core 2 of 2
90+
INFO Core saved with ID 20
91+
INFO Processing ROIs annotation step 782b19dba74440d797f6bbf069f2d8c4-DEF
92+
INFO Loading bounds for slide P_003-DEF
93+
INFO response.status_code 200
94+
INFO Slice saved with ID 8
95+
INFO Loading core 1 of 3
96+
INFO Core saved with ID 21
97+
INFO Loading core 2 of 3
98+
INFO Core saved with ID 22
99+
...
100+
INFO Loading core 4 of 4
101+
INFO Core saved with ID 41
102+
INFO Processing ROIs annotation step cf0e125804664b839e007386220f34d0-DEF
103+
INFO Loading bounds for slide P_001-DEF
104+
INFO response.status_code 200
105+
INFO Slice saved with ID 16
106+
INFO Loading core 1 of 2
107+
INFO Core saved with ID 42
108+
INFO Loading core 2 of 2
109+
...
110+
INFO Loading core 2 of 2
111+
INFO Core saved with ID 59
112+
INFO Processing ROIs annotation step cf0e125804664b839e007386220f34d0-ABC
113+
INFO Loading bounds for slide P_001-ABC
114+
INFO response.status_code 200
115+
INFO Slice saved with ID 24
116+
INFO Loading core 1 of 2
117+
INFO Core saved with ID 60
118+
INFO Loading core 2 of 2
119+
INFO Core saved with ID 61
120+
...
121+
INFO Loading core 1 of 1
122+
INFO Core saved with ID 77
123+
INFO == Job completed ==
124+
```

0 commit comments

Comments
 (0)