diff --git a/subjects/ai/vision-track/README.md b/subjects/ai/vision-track/README.md index 0470829e1c..dae49b891b 100644 --- a/subjects/ai/vision-track/README.md +++ b/subjects/ai/vision-track/README.md @@ -84,6 +84,74 @@ The primary goal of **VisionTrack** is to develop practical skills in building a - Evaluate the app's performance with multi-stream support using metrics like **precision**, **recall**, and **F1-score**. - Display performance analysis within the app to inform users of the detection and tracking accuracy. +#### Validation + +To ensure project completeness and audit validation, include the following: + +1. **Model Artifacts**: + - Save all trained and optimized YOLO model weights in: + ``` + models/checkpoints/ + ├── best.pt + ├── best_quantized.onnx + └── config.yaml + ``` + - Include logs or configuration files documenting training and optimization steps. + +2. **Evaluation Metrics**: + - Generate and save a report file: + reports/performance_metrics.json + - Example format: + ```json + { + "detection_precision": 0.92, + "detection_recall": 0.9, + "f1_score": 0.91, + "average_fps_per_stream": 18.5, + "average_latency_ms": 85.0 + } + ``` + - Minimum passing thresholds: + Precision ≥ 0.85 + Recall ≥ 0.80 + F1-score ≥ 0.85 + Average FPS ≥ 15 (for 720p video) + +3. **Real-Time App Test** + - The app must run using: + ``` + streamlit run app.py + ``` + - The app should: + Display real-time detection overlays and FPS/latency counters. + Allow toggling of detection and tracking features per stream. + Handle missing or broken video sources gracefully. + +4. **ROI Counting Validation** + - Demonstrate ROI-based counting of people entering/exiting the region. + - Save examples in: + ``` + reports/demo_results/ + ├── roi_counting_example.png + └── multi_stream_demo.mp4 + ``` + +5. **GPU and Fallback Test** + - Check for CUDA availability in your code: + ``` + import torch + print("Using CUDA:", torch.cuda.is_available()) + ``` + - The app must still run on CPU if CUDA is unavailable (with lower FPS + +6. **Error Handling** + - The app must not crash on missing files or failed streams. + + - Log errors to: + ``` + logs/app_errors.log + ``` + ### Project Repository Structure ``` @@ -97,15 +165,24 @@ vision-track/ ├── models/ │ ├── yolo_person_detection.py │ └── __init__.py +│ └── /checkpoints/ +│ ├── best.pt +│ ├── best_quantized.onnx +│ └── config.yaml │ ├── utils/ │ ├── data_loader.py │ ├── preprocessing.py │ ├── multi_stream_tracking_helpers.py │ ├── counting_logic.py +│ ├── VisionTrack_Analysis.ipynb │ └── __init__.py │ -├── app.py # Streamlit app for running multi-stream detection, tracking, and counting +├── reports/demo_results/ +│ ├── roi_counting_example.png +│ └── multi_stream_demo.mp4 +│ +├── app.py ├── README.md # Project overview and setup instructions └── requirements.txt # List of dependencies ``` diff --git a/subjects/ai/vision-track/audit/README.md b/subjects/ai/vision-track/audit/README.md index 927cc71266..9ce920f425 100644 --- a/subjects/ai/vision-track/audit/README.md +++ b/subjects/ai/vision-track/audit/README.md @@ -8,6 +8,8 @@ ###### Is a `requirements.txt` file included with all dependencies and specific library versions required to run the project? +###### import test `python -c "import torch, supervision, cv2, streamlit"` + ##### Data Processing and Exploratory Data Analysis ###### Does the Jupyter notebook (`VisionTrack_Analysis.ipynb`) include EDA showcasing data distribution, object detection samples, and preprocessing methods? @@ -16,6 +18,9 @@ ###### Does data preprocessing include resizing and normalization, ensuring compatibility with YOLO model input formats? +- Validation of YOLO-compatible annotations (.txt files with class, x, y, w, h). +- Confirm frames are resized and normalized properly before inference. + ##### Model Implementation ###### Is the YOLO model implemented for person detection with configuration options for detection thresholds and class-specific tuning? @@ -32,6 +37,8 @@ ###### Does the project include logic for tracking and counting entries and exits within specified regions of interest (ROIs)? +###### Check that trained weights are saved in: `models/checkpoints/best.pt` + ##### Streamlit App Development ###### Is the **Streamlit** app implemented to display video feeds with overlaid detection, tracking, and counting information? @@ -56,6 +63,38 @@ ###### Are evaluation metrics presented, showcasing precision, recall, and F1-score to assess the effectiveness of detection and tracking? +###### Check: + +- Require metrics file: + + ``` + reports/performance_metrics.json + ``` + +- Validate JSON includes: + + ```json + { + "detection_precision": ..., + "detection_recall": ..., + "f1_score": ..., + "average_fps_per_stream": ..., + "average_latency_ms": ... + } + ``` + +- Add minimum thresholds: + + Precision ≥ 0.85 + + Recall ≥ 0.80 + + F1 ≥ 0.85 + + FPS ≥ 15 (720p) + +- Add check that metrics are visible in Streamlit dashboard (FPS + latency shown live). + ##### Additional Considerations ###### Does the codebase is documented with comments and explanations for readability and maintainability?