Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
132f9dc
feat: Added pgvector extension
mohamedelabbas1996 Apr 14, 2025
dc48ccc
feat: Added features field to the Classification model
mohamedelabbas1996 Apr 14, 2025
8dc0c00
changed taxon and detection to autocomplete fields in the Classificat…
mohamedelabbas1996 Apr 16, 2025
4bf07b3
feat: added similar action to the ClassificationViewset
mohamedelabbas1996 Apr 16, 2025
b258c9b
chore: changed features vector field name to features_2048
mohamedelabbas1996 Apr 16, 2025
9490045
chore: changed features vector field name to features_2048
mohamedelabbas1996 Apr 16, 2025
0ff569f
feat: read features vector from processing service ClassificationResp…
mohamedelabbas1996 Apr 16, 2025
89a3b6c
test: added tests for PGVector distance metrics
mohamedelabbas1996 Apr 16, 2025
9e13cc4
updated docker-compose.ci.yml to use the same postgres image
mohamedelabbas1996 Apr 17, 2025
5a51593
updated docker-compose.ci.yml to use the same postgres image as docke…
mohamedelabbas1996 Apr 17, 2025
9efff5f
updated docker-compose.ci.yml to use the same postgres image as docke…
mohamedelabbas1996 Apr 17, 2025
1c66f34
feat: Added support for clustering detections for source image collec…
mohamedelabbas1996 Apr 29, 2025
99a7f3f
feat: Allowed triggering collection detections clustering from admin …
mohamedelabbas1996 Apr 29, 2025
83f2c08
fix: show unobserved Taxa in view for now
mihow Apr 29, 2025
5420f85
fix: create & update occurrence determinations after clustering
mihow Apr 29, 2025
6b0020d
feat: add unknown species filter to admin
mihow Apr 29, 2025
856035d
Merge branch 'deployments/ood.antenna.insectai.org' of github.com:Rol…
mihow Apr 30, 2025
036d81d
Merge branch 'deployments/ood.antenna.insectai.org' of github.com:Rol…
mihow Apr 30, 2025
4f8b09b
fix: circular import
mihow Apr 30, 2025
d255085
fix: update migration ordering
mihow Apr 30, 2025
2e12b56
Integrated Agglomerative clustering
mohamedelabbas1996 Apr 30, 2025
a301dc7
Merge branch 'feat/add-clustering' of https://github.com/RolnickLab/a…
mohamedelabbas1996 Apr 30, 2025
10820bb
updated clustering request params
mohamedelabbas1996 Apr 30, 2025
225529e
fixed Agglomerative clustering
mohamedelabbas1996 Apr 30, 2025
0423523
fix: disable missing clustering algorithms
mihow May 1, 2025
cb894f4
fix: syntax when creating algorithm entry
mihow May 1, 2025
39d9b6c
feat: command to create clustering job without starting it
mihow May 1, 2025
abd9cf1
feat: increase default batch size
mihow May 1, 2025
b2a7b3f
fix: better algorithm name
mihow May 1, 2025
bf67d06
feat: allow sorting by OOD score
mihow May 1, 2025
ce08f6a
Merge branch 'deployments/ood.antenna.insectai.org' of github.com:Rol…
mihow May 1, 2025
853b69d
feat: add unknown species and other fields to Taxon serializer
mihow May 1, 2025
6586872
fix: remove missing field
mihow May 1, 2025
b242079
fix: migration conflicts
mihow May 1, 2025
fe744f0
feat: fields for investigating occurrence classifications in admin
mihow May 2, 2025
4ac88da
fix: filter by feature extraction algorithm
mohamedelabbas1996 May 5, 2025
6d44bdb
chore: Used a serializer to handle job params instead of reading them…
mohamedelabbas1996 May 5, 2025
12b4ee4
set default ood threshold to 0.0
mohamedelabbas1996 May 5, 2025
2c73795
test: added tests for clustering
mohamedelabbas1996 May 5, 2025
e5d7ff0
chore: migration for new algorithm type
mihow May 6, 2025
9ad77f7
Merge branch 'deployments/ood.antenna.insectai.org' of github.com:Rol…
mihow May 6, 2025
fdbbf75
fix: remove cluster action in Event admin until its ready
mihow May 6, 2025
0e92904
chore: move algorithm selection to dedicated function
mihow May 7, 2025
b26fbe0
fix: update clustering tests and types
mihow May 7, 2025
4032aff
chore: remove external network config in processing services
mihow May 7, 2025
0f8c544
feat: update GitHub workflows to run tests on other branches
mihow May 7, 2025
5fb5c43
fix: hide unobserved taxa by default
mihow May 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions ami/main/api/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -1550,3 +1550,11 @@ class Meta:
"total_size",
"last_checked",
]


class ClusterDetectionsSerializer(serializers.Serializer):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel much better about using a serializer for the params! thank you 🙏

ood_threshold = serializers.FloatField(required=False, default=0.3)
feature_extraction_algorithm = serializers.CharField(required=False, allow_null=True)
algorithm = serializers.CharField(required=False, default="agglomerative")
algorithm_kwargs = serializers.DictField(required=False, default={"distance_threshold": 0.5})
pca = serializers.DictField(required=False, default={"n_components": 384})
13 changes: 6 additions & 7 deletions ami/main/api/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
from ami.base.serializers import FilterParamsSerializer, SingleParamSerializer
from ami.base.views import ProjectMixin
from ami.jobs.models import DetectionClusteringJob, Job
from ami.main.api.serializers import ClusterDetectionsSerializer
from ami.utils.requests import get_active_classification_threshold, project_id_doc_param
from ami.utils.storages import ConnectionTestResult

Expand Down Expand Up @@ -750,19 +751,17 @@ def cluster_detections(self, request, pk=None):
"""
Trigger a background job to cluster detections from this collection.
"""

collection: SourceImageCollection = self.get_object()
serializer = ClusterDetectionsSerializer(data=request.data)
serializer.is_valid(raise_exception=True)
params = serializer.validated_data
job = Job.objects.create(
name=f"Clustering detections for collection {collection.pk}",
project=collection.project,
source_image_collection=collection,
job_type_key=DetectionClusteringJob.key,
params={
"ood_threshold": request.data.get("ood_threshold", 0.3),
"feature_extraction_algorithm": request.data.get("feature_extraction_algorithm", None),
"algorithm": request.data.get("algorithm", "agglomerative"),
"algorithm_kwargs": request.data.get("algorithm_kwargs", {"distance_threshold": 0.5}),
"pca": request.data.get("pca", {"n_components": 384}),
},
params=params,
)
job.enqueue()
logger.info(f"Triggered clustering job for collection {collection.pk}")
Expand Down