Vision Metric Lab

Prompt-driven video analytics for detection, segmentation, tracking, and approximate object measurement.

Vision Metric Lab is a portfolio-grade computer vision project that turns a raw video into structured analytics: annotated video, object tracks, per-frame measurements, and JSON/CSV reports. The current release ships with a lightweight classical backend that runs on a laptop, plus modular interfaces for upgrading the detector, segmenter, tracker, and depth estimator to foundation models such as Grounding DINO, SAM 2, ByteTrack, and Depth Anything V2.

Why This Project Exists

Most computer vision demos stop at drawing boxes. This project treats vision as a deployable system:

• user-selected target prompts
• frame sampling and preprocessing
• region proposal and segmentation
• multi-object tracking
• approximate size estimation
• analytics export
• reproducible configuration
• an interactive Streamlit demo

The architecture is designed for real-world domains such as aquaculture monitoring, drone landing-pad inspection, industrial safety, and traffic analytics.

Demo

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e .
python scripts/make_demo_video.py
vision-metric-lab --video examples/synthetic_fish.mp4 --output-dir outputs/demo --prompt "bright fish" --mode bright
streamlit run app/streamlit_app.py

If your shell cannot find the installed console command, use the module form:

python -m vision_metric_lab.cli --video examples/synthetic_fish.mp4 --output-dir outputs/demo --prompt "bright fish" --mode bright

Then upload a short video and choose a target mode. The default MVP backend supports:

• bright object tracking
• dark object tracking
• motion-based object tracking
• largest object tracking

To run the optional foundation-model backend:

pip install -r requirements-foundation.txt
python -m vision_metric_lab.cli \
  --config configs/foundation.yaml \
  --video examples/synthetic_fish.mp4 \
  --output-dir outputs/foundation-demo \
  --prompt "fish" \
  --detector grounding_dino \
  --segmenter sam2 \
  --tracker bytetrack \
  --device auto

The first run downloads model weights and may be slow. Use short clips while tuning.

--device auto prefers CUDA when available, then Apple Silicon MPS, then CPU. You can force Apple Silicon acceleration with --device mps.

On CPU, SAM2 can take several seconds per frame. For portfolio demos, process short clips first, then run longer videos on a CUDA/MPS-capable machine.

Note: Supervision 0.28.0 marks its built-in ByteTrack wrapper as deprecated and recommends the separate trackers package for future releases. This project keeps the Supervision adapter because it is still functional and easy to integrate, but the tracker module is isolated so it can be swapped cleanly.

Example Resume Bullet

Built a prompt-driven video analytics system with modular detection, segmentation, tracking, and measurement components, exporting annotated videos plus structured object trajectory and size reports for real-world computer vision workflows.

Future upgraded bullet after adding foundation models:

Designed an open-vocabulary video understanding pipeline using Grounding DINO, SAM 2, ByteTrack, and Depth Anything V2 to detect, segment, track, and approximately measure text-specified objects across real-world videos.

Architecture

flowchart LR
    A["Input video"] --> B["Frame sampler"]
    B --> C["Prompt-aware detector"]
    C --> D["Segmenter"]
    D --> E["Tracker"]
    E --> F["Measurement"]
    F --> G["Annotated video"]
    F --> H["CSV/JSON analytics"]

Loading

Repository Layout

vision-metric-lab/
  app/
    streamlit_app.py
  configs/
    default.yaml
  examples/
    README.md
  src/
    vision_metric_lab/
      analytics.py
      config.py
      pipeline.py
      video_io.py
      detectors/
      measurement/
      segmentation/
      tracking/
      visualization/
  tests/
    test_tracking.py
    test_measurement.py
  requirements.txt
  pyproject.toml

Model Roadmap

The project is intentionally modular. Replace one component at a time:

Component	MVP implementation	Upgrade path
Detector	OpenCV threshold/motion proposals	Grounding DINO adapter
Segmenter	Box-to-mask + contour refinement	SAM 2 adapter
Tracker	IoU tracker	ByteTrack adapter
Depth	Pixel-to-unit proxy	Depth Anything-style adapter / calibrated stereo
App	Streamlit	FastAPI + React dashboard

See ROADMAP.md for the staged implementation plan.

Outputs

Each run creates:

• annotated.mp4: video with boxes, IDs, tracks, and measurements
• tracks.csv: frame-level object analytics
• summary.json: aggregate run statistics

Evaluation Ideas

To make the GitHub project stronger, add:

• FPS and latency benchmarks
• ID switch count
• track length distribution
• mAP/IoU evaluation on labeled clips
• robustness tests under blur, turbidity, low light, and occlusion

License

MIT