[FEAT]: Benchmarking against other public datasets

[gcielniak]

Description

There are two public datasets for strawberries and tomatoes available so benchmarking against those would be desirable. This can be implemented either through an internship or a student project.

• LaboroTomato
• StrawDI

Additional Information

No response

[tiru-venkatesh]

Hi @gcielniak,

I’m interested in working on benchmarking aoc_fruit_detector
against public datasets (LaboroTomato and StrawDI).

I plan to:

• Adapt datasets to Detectron2 format
• Run inference using existing Mask-RCNN models
• Report metrics (mAP, precision, recall)
• Document results and reproducibility steps

Please let me know if this approach aligns with your expectations.

[gcielniak]

Hi @tiru94397, thankyou for your initiative and willingness to explore this benchmarking step. Your proposed list sounds very reasonable.

Just to be clear, this feature was meant for someone from the Lincoln team, but since it is an open repository and you are keen, then this is not a limiting factor. Go for it!

[tiru-venkatesh]

Hi @gcielniak, Thank you for the clarification and encouragement — much appreciated. I’ll proceed with the benchmarking work as outlined: - Adapting *LaboroTomato* and *StrawDI* to Detectron2 format - Running inference using the existing Mask R-CNN models - Reporting standard metrics (mAP, precision, recall) - Documenting results, assumptions, and reproducibility steps I’ll start by setting up the dataset conversion and will update the issue with progress and any questions that come up. Looking forward to contributing! Best regards, Tiru

…

On Tue, 6 Jan 2026 at 15:55, gcielniak ***@***.***> wrote: *gcielniak* left a comment (LCAS/aoc_fruit_detector#18) <#18 (comment)> Hi @tiru94397 <https://github.com/tiru94397>, thankyou for your initiative and willingness to explore this benchmarking step. Your proposed list sounds very reasonable. Just to be clear, this feature was meant for someone from the Lincoln team, but since it is an open repository and you are keen, then this is not a limiting factor. Go for it! — Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BTTASK5O7HFSDI6GQBFXJH34FOEP7AVCNFSM6AAAAABVJOPDIKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTOMJUGEZDSOBUGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[tiru-venkatesh]

Hi @gcielniak,

Quick update on progress:

Planned execution steps

1. Download and inspect LaboroTomato and StrawDI annotations
2. Write dataset conversion scripts → Detectron2 (COCO-style JSON)
3. Validate class mappings and mask integrity
4. Run inference using existing Mask R-CNN configs from the repo
5. Compute metrics (mAP@[.5:.95], precision, recall)
6. Document results, dataset assumptions, and reproducibility steps

Deliverables

• Dataset conversion scripts
• Benchmark results table (per-dataset)
• Reproducible evaluation instructions (README section)

I’ll start with the dataset conversion and share an initial update once
the Detectron2 loaders are verified.

Please let me know if you’d like any additional metrics or comparisons.

Best regards,
Tiru

[gcielniak]

it all looks good @tiru-venkatesh!

[tiru-venkatesh]

Hi @gcielniak Sir!

Quick check before I go deeper on this —
does this project support any very small bounties or paid tasks?
Even something in a small range (like ~$10) would be totally fine.

If not, no worries at all — I’m happy to contribute just for learning and improving the project.
Just wanted to clarify expectations early.

[tiru-venkatesh]

Hi @gcielniak,

Final update from my side on StrawDI benchmarking:

• Successfully converted StrawDI masks to COCO-format annotations
• Registered the dataset with Detectron2 DatasetCatalog
• Verified data loading and annotation integrity (sample inspection + loader test)
• CPU-based evaluation pipeline is reproducible and ready for benchmarking with Mask R-CNN

All conversion scripts and testing utilities are working locally.
I can proceed with:
– running full quantitative benchmarks (mAP, AP50, AP75), or
– opening a PR with the dataset conversion + evaluation scripts,
whichever you prefer.

Thanks again for the opportunity to contribute.

[gcielniak]

Hi @tiru-venkatesh,

Unfortunately, we have no financial means to support any of the voluntary developments.

When it comes to the next steps, please use your own judgement. Both sound fine!

[tiru-venkatesh]

Hi @gcielniak,

Thanks for the clarification — no problem at all.

I’ll proceed from here based on learning and contribution value.
I’ll start by organizing the StrawDI conversion and evaluation into a clean PR so it’s easy to review and reuse.

Thanks again for the guidance and encouragement.