Skip to content

Low validation accuracy with 69M checkpoint on COCO val2017 (12% vs paper's 48.9%) #76

@Yohann-Ian

Description

@Yohann-Ian

Hello,
I'm trying to reproduce the results from your paper. My accuracy is low and I cannot get good results. Appreciate your help.

Questions:

  1. Is the 69M_best.pt checkpoint the same model used for the paper results?
  2. Are there any special inference parameters or settings required?
  3. Could you share the exact validation command/configuration you used?

These are my results:
Setup:

Checkpoint: 69M_best.pt (T=1, D=4) from Google Drive
Dataset: COCO val2017 (5000 images)
Evaluation: Standard ultralytics validation protocol
Hardware: RTX 3090, CUDA 11.3

Results:

My result: [email protected] = 12.0%, [email protected]:0.95 = 8.81%
Paper reports: [email protected] = 66.2%, [email protected]:0.95 = 48.9%

Observations:

First ~12 classes (person, car, etc.) perform well (50-80% mAP)
Most other classes show near-zero detection
Only 4270/5000 images successfully validated

Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions