Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<!-- <img src="/docs/assets/preview.gif" alt="example"> -->
<img src="/docs/assets/preview.gif" alt="example">

# ISS Data Collection Tool

Expand Down Expand Up @@ -30,6 +30,7 @@ See [docs/](/docs) for manuals and walkthroughs:
- [Goals](/docs/goals.md)
- [Statistics](/docs/statistics.md)
- [Downloads](/docs/downloads.md)
- [Annotation](/docs/annotation.md)

## ⚙️ Architecture

Expand Down
63 changes: 63 additions & 0 deletions docs/annotation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# 🧾 Annotation
When exporting your dataset—or just annotations, a structured JSON file is generated.

It contains metadata for all files matching the current filters and their annotations—without media.

## 🧬 Example Structure

```json
[
{
"id": "6848a6709aeaf5eb62cf01a8",
"related_duplicates": 0,
"attributes": [
{
"uid": "f35846ca-0c7e-4120-8225-9204b054bb1a",
"attributes": [
[16, 0, "type_1", ""],
[18, 0, "subtype_2", ""],
[20, 1, "r", ""]
]
}
],
"author_name": "Curator",
"validator_name": "Curator",
"file_name": "6788191feac1f1a81ab2fadd.webp",
"file_type": "image",
"status": "a",
"upload_date": "2025-06-10 21:41",
"update_date": "2025-06-10 21:41",
"rebound_project": null,
"rebound": null
}
]
```

Where:
- **id** - is generated file id by which the file could be found in the system
- **related_duplicates** - number of duplicates associated to this file
- **attributes** - list of attribute (label) groups with annotated data
- **author_name** - the username of user who uploaded this file
- **validator_name** - the username of user who validated this file
- **file_name** - original file name
- **file_type** - media type (image/video)
- **status** - validation status, `a - accepted`, `d - declined`, `v - waits for validation`
- **upload_date** - file upload date
- **update_date** - file update date
- **rebound_project** - project / bucket where the file is located in case of file transfer
- **rebound** - file project / bucket (in terms of storage system)

## Accessing a file

Since the project in **wip** state there are some temprorary solutions.
We have a situation when we want to move some files to other project.
Since proper object movement is not implemented yet we could use a trick to just associate
file project to the new one but since file is stored in original bucket we have to specify it.

Using orm / db model you might encounter **rebound** field as well.
This one is proper and intentional and used for rebounding file lookup in cases when uploaded file marked as duplicated one.
Since we use best quality for all the duplicates this best one has related_duplicates count and all the duplicats have this rebound field set with id of "original" file.

So in order to access a file you must follow this: `project_<rebound_project_id | project_id>/<rebound_id | file_id>` object lookup.

When exporting datasets, rebound is resolved internally. Each exported entry is final, and you can access it directly via its id and extension (from file_name).
Binary file added docs/assets/goal_all.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/label_del_copy.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/label_delete.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/label_filters.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/label_group.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/label_payload.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/label_tree.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/preview.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/project_create.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/project_download.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/project_goals.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/project_roles.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/project_stats.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/project_upload.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/project_validate.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/registration.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/stats.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/stats_diff.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/stats_export.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/upload_label.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/upload_zoom.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/validation_duplicate.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 7 additions & 11 deletions docs/downloads.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,12 @@

This section lets you download collected data for a project.

<!-- <img src="/docs/assets/" alt="downloads"> -->
<img src="/docs/assets/project_download.gif" alt="download">

## 🎛️ Filter First

Use the filter block above to define which files should be included in the export.

<!-- <img src="/docs/assets/" alt="downloads"> -->

Once filters are set, you have two options:

- `get archive`:
Expand All @@ -22,8 +20,6 @@ Since it may include many files, this can take some time.
This gives you only annotation data—no media included.
It includes all annotations matching the current filters.

<!-- <img src="/docs/assets/" alt="downloads_filters"> -->

## 📋 Archive Table

Once requested, all archives for the project are listed in a table below.
Expand All @@ -32,17 +28,17 @@ Column **requested** shows the exact filters used to generate each archive.

Click the download button to get the archive.

<!-- <img src="/docs/assets/" alt="downloads_table"> -->

Once the process completes, the row turns green. This means you can get your archive.

<!-- <img src="/docs/assets/" alt="downloads_table_ok"> -->

The archive process may fail.
In that case, the row turns red and the **result message** column shows what went wrong.

<!-- <img src="/docs/assets/" alt="downloads_table_fail"> -->

## ➕ Only New Files
When re-downloading later (e.g., the next day), you might want to include only new files.
Enable the `not downloaded before` flag to skip any files already included in past archives (based on the filters).

---

### ✅ Next Step

[Annotation](/docs/annotation.md)
8 changes: 3 additions & 5 deletions docs/goals.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This tab lets you define and track collection targets for specific labels.

<!-- <img src="/docs/assets/" alt="goals"> -->
<img src="/docs/assets/project_goals.gif" alt="goals">

## ➕ Create a Goal
To define a goal
Expand All @@ -13,8 +13,6 @@ To define a goal
- 🖼️ Images (e.g., 1)
- 🎞️ Videos (e.g., 2)

<!-- <img src="/docs/assets/" alt="goals_create"> -->

A goal is considered fulfilled when the weighted sum of validated data reaches the target:

Example - Goal = 5, image weight = 1, video weight = 2
Expand All @@ -33,11 +31,11 @@ Active goals are listed with:

The table is sorted by progress (least done on top) for better prioritization.

<!-- <img src="/docs/assets/" alt="goals_table"> -->

Goals are hidden once fulfilled, to keep focus on active targets.
Toggle `show all` to view past goals or completed ones.

<img src="/docs/assets/goal_all.gif" alt="goals">

---

### ✅ Next Step
Expand Down
28 changes: 7 additions & 21 deletions docs/labels.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,44 +10,34 @@ Each project uses a label schema to annotate collected data. Labels help ensure

When creating a project or later in the Project → Edit tab:

<!-- <img src="/docs/assets/" alt="labels"> -->
<img src="/docs/assets/label_tree.gif" alt="labels_flags">

1. Add Tree-like Levels

<!-- <img src="/docs/assets/" alt="labels_level"> -->

2. Add Values per level, with optional nesting. Each value is an actual label used during annotation.

<!-- <img src="/docs/assets/" alt="labels_value"> -->

3. Set flags like:
- `required`
- `multiple choice`

<!-- <img src="/docs/assets/" alt="labels_flags"> -->

4. Each label value could have a payload. This is a meta information in valid `json/string` format.
You can set restricted flag which sets if payload is required
Examples:
- `label payload`
- `{"field1": "value1", "type": 1, "list": ["metalist1", "metalist2"]}`
- `["meta1", "meta2"]`

<!-- <img src="/docs/assets/" alt="labels_payload"> -->
<img src="/docs/assets/label_payload.gif" alt="labels_payload">

5. You can do a quick renaming with special form.

<!-- <img src="/docs/assets/" alt="labels_rename"> -->

6. You can change alignment of values.

<!-- <img src="/docs/assets/" alt="labels_align"> -->

7. Deleting may be performed only when no media is assigned to such label or level of labels.
When you hit remove (`minus`) button - the popup will tell you if this item cannot be removed.
If you really want to remove it even when it's restricted you could change labeling at validation tab to remove association

<!-- <img src="/docs/assets/" alt="labels_delete"> -->
<img src="/docs/assets/label_delete.gif" alt="labels_delete">

8. Grouping
Each block represents a separate attribute tree (e.g., color, shape, type).
Expand All @@ -56,7 +46,7 @@ I.E. each feature has its own tree / block/

To remove a group completely you have to delete all the levels.

<!-- <img src="/docs/assets/" alt="labels_add"> -->
<img src="/docs/assets/label_group.gif" alt="labels_add">

## 📤 Applying Schema

Expand All @@ -66,31 +56,27 @@ When [uploading data](/docs/uploads.md), you assign the schema:

These methods can be used simultaneously

<!-- <img src="/docs/assets/" alt="labels_apply"> -->
<img src="/docs/assets/upload_label.gif" alt="labels_apply">

The label hierarchy defined in the project could be set several times by clicking `add object` button.
This will create a new block with its own hierarchy.
This might be useful if your image for example has several annotated objects like cars or animals.

<!-- <img src="/docs/assets/" alt="labels_apply_new"> -->

Each group can be easily deleted or copied.

<!-- <img src="/docs/assets/" alt="labels_apply_del-cp"> -->
<img src="/docs/assets/label_del_copy.gif" alt="labels_apply_del-cp">

## 📝 Validation Stage

During [validation](/docs/validation.md), labels can be you have same tree you assigned on Upload stage with the same management.

<!-- <img src="/docs/assets/" alt="labels_validate"> -->

## 🔍 Filtering

At some pages there is an option to use Schema labels as filters.
The usage is almost the same as you apply labels to the media.
When you are done with tree click `select` button.

<!-- <img src="/docs/assets/" alt="labels_filter"> -->
<img src="/docs/assets/label_filters.gif" alt="labels_filter">

---

Expand Down
22 changes: 7 additions & 15 deletions docs/projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@
Projects are the core unit of organization in the ISS Data Collection Tool.
Each project defines its own label system, goals, and data scope.

<!-- <img src="/docs/assets/" alt="project_main"> -->

## 🔹 What is a Project?

A project groups:
Expand All @@ -27,13 +25,11 @@ To create a project:
- **Project Description**
- **Label Schema**

<!-- <img src="/docs/assets/" alt="project_create"> -->
<img src="/docs/assets/project_create.gif" alt="project_create">

Next you will be redirected back to list of projects.
Select your newly created one.

<!-- <img src="/docs/assets/" alt="project_detail"> -->

Each project has the following sections:
(admin users will see all sections; regular users will only see a subset)

Expand All @@ -54,7 +50,7 @@ Admins have full access to the whole application and items;
Non admin users by default don't have any access to project or its data
so you need to manually edit the permissions for it if needed.

<!-- <img src="/docs/assets/" alt="project_permissions"> -->
<img src="/docs/assets/project_roles.gif" alt="permissions">

See [Users & Roles](/docs/users.md) for how to manage users and set permissions

Expand All @@ -64,7 +60,7 @@ After creating you can upload images or videos.
After labeling and sending media to the server they will appear in validation section.
Note: media processing on the server side may cause a slight delay before visibility.

<!-- <img src="/docs/assets/" alt="upload"> -->
<img src="/docs/assets/project_upload.gif" alt="upload">

See [Uploads](/docs/uploads.md) for how to upload media

Expand All @@ -73,7 +69,7 @@ See [Uploads](/docs/uploads.md) for how to upload media
When you are done with uploading you can validate this data whether it's applicable to your project or not.
Labels could be corrected there too.

<!-- <img src="/docs/assets/" alt="validate"> -->
<img src="/docs/assets/project_validate.gif" alt="validate">

See [Validation](/docs/validation.md) for how to validate/ see uploaded images

Expand All @@ -86,15 +82,13 @@ Each project might have a label schema, defining what’s being collected. Label
- Required
- Multiple items

<!-- <img src="/docs/assets/" alt="labels"> -->

See [Labels](/docs/labels.md) for how to create and assign them.

## 🎯 Goals

Optionally, you can define target counts for labels (e.g. "Need 100 images of `Dog`").

<!-- <img src="/docs/assets/" alt="goals"> -->
<img src="/docs/assets/project_goals.gif" alt="goals">

See [Goals](/docs/goals.md) for how to create and track them.

Expand All @@ -105,15 +99,15 @@ Each project has a brief dashboard shows the collected count grouped by:
- Media type
- Validation type

<!-- <img src="/docs/assets/" alt="stats"> -->
<img src="/docs/assets/project_stats.gif" alt="stats">

See [Statistics](/docs/statistics.md)

## 🗃 Downloading Results

You can export annotated datasets as `.zip` archives with optional filters

<!-- <img src="/docs/assets/" alt="download"> -->
<img src="/docs/assets/project_download.gif" alt="download">

See [Downloads](/docs/downloads.md) for how to request archives

Expand All @@ -126,8 +120,6 @@ After you are done with editing click `SUBMIT EDIT`

User roles could set with dedicated button.

<!-- <img src="/docs/assets/" alt="project_edit"> -->

---

### ✅ Next Step
Expand Down
10 changes: 3 additions & 7 deletions docs/statistics.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,16 @@

This tab displays how labels are used and how annotations are validated, broken down by media type.

<!-- <img src="/docs/assets/" alt="stats"> -->
<img src="/docs/assets/stats.gif" alt="stats">

## 🔍 Explore by Attribute or User

You can group stats by:

- **Attribute** (label and its nested structure)

<!-- <img src="/docs/assets/" alt="stats_label"> -->

- **User** (who uploaded the data)

<!-- <img src="/docs/assets/" alt="stats_user"> -->

Click on a `parent row` to expand child items.

## 📥 Export Options
Expand All @@ -26,7 +22,7 @@ Export the table in:
- `json`
- `xlsx`

<!-- <img src="/docs/assets/" alt="stats_export"> -->
<img src="/docs/assets/stats_export.gif" alt="stats_export">

Useful for further processing, reporting, or audits.

Expand All @@ -37,7 +33,7 @@ The diff view splits data around a reference upload date.
For example, setting `Diff from`: *2025-01-01* highlights what’s been uploaded since then—helpful for tracking changes over time.
No data is displayed until a reference date is selected.

<!-- <img src="/docs/assets/" alt="stats_diff"> -->
<img src="/docs/assets/stats_diff.gif" alt="stats_diff">

---

Expand Down
Loading