Fix/classification streaming support

[dmjtian]

## Summary

Adds classification result processing to `convertResultToStreamData()` method in both Android's `YOLOView.kt` and iOS's `YOLOView.swift`, enabling real-time classification streaming support on both platforms.

## Problem

Currently, classification task works for single image prediction (via `YOLO.predict()`) thanks to PR #418, but **real-time streaming** (via `YOLOView` with `onResult` callback) returns empty results.

### Current Behavior
```dart
YOLOView(
  task: YOLOTask.classify,
  onResult: (results) => print(results.length), // Always prints 0
)

Logs:

D/Classifier: Classification result: top1Label=corn, conf=0.84, index=0
D/Classifier: Prediction completed successfully
D/YOLOView: ✅ Total detections in stream: 0 (boxes: 0, obb: 0)

Classifier succeeds but results array is empty because convertResultToStreamData() doesn't process result.probs.

Root Cause

The convertResultToStreamData() method in both platforms handles:

• ✅ Detection boxes (result.boxes)
• ✅ Pose keypoints (result.keypointsList)
• ✅ OBB (result.obb)
• ❌ Classification results (result.probs) - MISSING

Affected files:

• android/src/main/kotlin/com/ultralytics/yolo/YOLOView.kt
• ios/Classes/YOLOView.swift

Solution

Add classification result processing in convertResultToStreamData() method on both platforms, similar to how other task types are handled.

Code Changes

Android Implementation

File: android/src/main/kotlin/com/ultralytics/yolo/YOLOView.kt

Location: After line ~1668 (after the includeDetections block ends)

Status: ✅ Completed (Commit: bb98008)

Add this code block:

// Add classification results (if available and enabled for CLASSIFY task)
if (config.includeClassifications && result.probs != null) {
    val probs = result.probs!!
    Log.d(TAG, "🎯 Processing CLASSIFY result: top1Label=${probs.top1Label}, conf=${probs.top1Conf}, index=${probs.top1Index}")

    // Add classification result to detections array (for compatibility with YOLOResult.fromMap)
    val detections = map["detections"] as? ArrayList<Map<String, Any>> ?: ArrayList()

    val classificationDetection = HashMap<String, Any>()
    classificationDetection["classIndex"] = probs.top1Index
    classificationDetection["className"] = probs.top1Label
    classificationDetection["confidence"] = probs.top1Conf.toDouble()

    // Full image bounding box for classification
    val boundingBox = HashMap<String, Any>()
    boundingBox["left"] = 0.0
    boundingBox["top"] = 0.0
    boundingBox["right"] = result.origShape.width.toDouble()
    boundingBox["bottom"] = result.origShape.height.toDouble()
    classificationDetection["boundingBox"] = boundingBox

    // Normalized bounding box (full image)
    val normalizedBox = HashMap<String, Any>()
    normalizedBox["left"] = 0.0
    normalizedBox["top"] = 0.0
    normalizedBox["right"] = 1.0
    normalizedBox["bottom"] = 1.0
    classificationDetection["normalizedBox"] = normalizedBox

    detections.add(classificationDetection)
    map["detections"] = detections

    Log.d(TAG, "✅ Added classification result: ${probs.top1Label} (conf=${probs.top1Conf}, index=${probs.top1Index})")
}

iOS Implementation

File: ios/Classes/YOLOView.swift

Location: After line ~1768 (after the includeDetections block ends)

Status: ✅ Completed (Commit: dced377)

Add this code block:

// Add classification results (if available and enabled for CLASSIFY task)
if config.includeClassifications, let probs = result.probs, result.boxes.isEmpty {
  print("🎯 YOLOView: Processing CLASSIFY result - returning top 5 predictions")

  // Get or create detections array (for compatibility with YOLOResult deserialization)
  var detections = map["detections"] as? [[String: Any]] ?? []

  // For classification models, create detections from top5 predictions
  let top5Labels = probs.top5Labels
  let top5Confs = probs.top5Confs

  for i in 0..<min(top5Labels.count, top5Confs.count) {
    var detection: [String: Any] = [:]

    // Find class index from names array
    var classIndex = i
    if let index = result.names.firstIndex(of: top5Labels[i]) {
      classIndex = index
    }

    detection["classIndex"] = classIndex
    detection["className"] = top5Labels[i]
    detection["confidence"] = Double(top5Confs[i])

    // Classification doesn't have bounding boxes, use full image bounds
    let boundingBox: [String: Any] = [
      "left": 0.0,
      "top": 0.0,
      "right": Double(result.orig_shape.width),
      "bottom": Double(result.orig_shape.height)
    ]
    detection["boundingBox"] = boundingBox

    // Normalized bounding box (full image)
    let normalizedBox: [String: Any] = [
      "left": 0.0,
      "top": 0.0,
      "right": 1.0,
      "bottom": 1.0
    ]
    detection["normalizedBox"] = normalizedBox

    detections.append(detection)
  }

  map["detections"] = detections
  print("✅ YOLOView: Added \(detections.count) classification results (top5)")
}

Key Difference: iOS returns top5 results while Android returns top1 only.

Optional but recommended: Update the log message to include probs count:

Before:

Log.d(TAG, "✅ Total detections in stream: ${detections.size} (boxes: ${result.boxes.size}, obb: ${result.obb.size})")

After:

Log.d(TAG, "✅ Total detections in stream: ${detections.size} (boxes: ${result.boxes.size}, obb: ${result.obb.size}, probs: ${if (result.probs != null) 1 else 0})")

Testing

Test Environment

• Model: yolo11n-cls.tflite (Android) / yolo11n-cls.mlpackage (iOS)
• Task: YOLOTask.classify
• Platform: Android & iOS
• Flutter: 3.x

Before Fix

D/YOLOView: ✅ Total detections in stream: 0
I/flutter: Results: 0

After Fix

Android:

D/YOLOView: 🎯 Processing CLASSIFY result: top1Label=corn, conf=0.84, index=0
D/YOLOView: ✅ Added classification result: corn (conf=0.84, index=0)
D/YOLOView: ✅ Total detections in stream: 1 (boxes: 0, obb: 0, probs: 1)
I/flutter: Results: 1
I/flutter:   className: corn
I/flutter:   confidence: 0.84

iOS:

🎯 YOLOView: Processing CLASSIFY result - returning top 5 predictions
✅ YOLOView: Added 5 classification results (top5)
Results: 5
  className: corn
  confidence: 0.84
  (+ 4 more results)

Test Code

YOLOView(
  modelPath: 'assets/yolo11n-cls.tflite',
  task: YOLOTask.classify,
  confidenceThreshold: 0.3,
  onResult: (results) {
    print('Results: ${results.length}');
    if (results.isNotEmpty) {
      print('  className: ${results[0].className}');
      print('  confidence: ${results[0].confidence}');
    }
  },
)

Design Decisions

Why add to detections array?

• Maintains consistency with other task types (detection, pose, OBB)
• Compatible with existing YOLOResult.fromMap() deserialization
• Allows Flutter layer to handle all results uniformly

Why use full image bounding box?

• Classification applies to the entire image, not a specific region
• Provides consistent data structure with other task types
• Simplifies Flutter layer processing

Why check config.includeClassifications?

• Respects YOLOStreamingConfig settings
• Allows users to disable classification results if needed
• Follows the same pattern as other task types

Impact

Fixed

• ✅ Real-time classification streaming now works on both Android and iOS
• ✅ YOLOView with onResult returns classification results
• ✅ Consistent behavior across all task types
• ✅ Android returns top1 result, iOS returns top5 results

Not Affected

• Single image classification (YOLO.predict()) - already works (PR Fix Android CLASSIFY task: ensure FloatArray compatibility (Issue #413 follow-up) #418)
• Detection, segmentation, pose, OBB tasks - unchanged

Platform Differences

• Android: Returns top1 classification result only
• iOS: Returns top5 classification results (more comprehensive)

Related

• Related to: PR Fix Android CLASSIFY task: ensure FloatArray compatibility (Issue #413 follow-up) #418 (fixed single image classification)
• Complements: v0.1.43 release
• Platform: Both Android and iOS
• Commits:
  • Android: bb98008
  • iOS: dced377

Checklist

• Code compiles without errors
• Tested on physical Android device
• Tested on iOS device/simulator
• Follows existing code style and patterns
• Includes appropriate logging
• No breaking changes
• Both platforms implemented

Notes

This PR implements classification streaming support for both Android and iOS platforms.

Implementation Details:

• Android (YOLOView.kt): Returns top1 classification result
• iOS (YOLOView.swift): Returns top5 classification results

Commits:

• bb98008: Android implementation (2026-02-14)
• dced377: iOS implementation (2026-02-25)

Both implementations follow the same design pattern and maintain consistency with other task types.

---

Author: @dmjtian (original contributor of PR #418)

## 🛠️ PR Summary

<sub>Made with ❤️ by [Ultralytics Actions](https://www.ultralytics.com/actions)</sub>

### 🌟 Summary
Enable classification results to flow through streaming outputs on both Android and iOS 📡

### 📊 Key Changes
- Added CLASSIFY-task support in Android streaming by appending a synthetic “detection” built from `result.probs` (top-1) ✅
- Added CLASSIFY-task support in iOS streaming by appending “detections” derived from `result.probs` (top-5) ✅
- Standardized output structure for classification by using full-image bounding boxes + normalized boxes for compatibility with `YOLOResult.fromMap` 🧩

### 🎯 Purpose & Impact
- Fixes missing/empty streaming outputs for classification models when `includeClassifications` is enabled 🎯
- Improves cross-platform behavior by ensuring classification predictions can be consumed through the existing `detections` pipeline 📱
- Reduces app-side parsing changes by keeping the serialized result compatible with current deserialization logic 🔄

[UltralyticsAssistant]

👋 Hello @dmjtian, thank you for submitting a ultralytics/yolo-flutter-app 🚀 PR! This is an automated message—an Ultralytics engineer will assist shortly. Please review the checklist below to help us merge smoothly ✅

-✅ Define a Purpose: Clearly explain the purpose of your fix or feature in your PR description, and link to any relevant issues. Ensure your commit messages are clear, concise, and adhere to the project's conventions.
-✅ Synchronize with Source: Confirm your PR is synchronized with the ultralytics/yolo-flutter-app main branch. If it's behind, update it by clicking the 'Update branch' button or by running git pull and git merge main locally.
-✅ Ensure CI Checks Pass: Verify all Ultralytics Continuous Integration (CI) checks are passing. If any checks fail, please address the issues.
-✅ Update Documentation: Update the relevant documentation for any new or modified features.
-✅ Add Tests: If applicable, include or update tests to cover your changes, and confirm that all tests are passing.
-✅ Sign the CLA: Please ensure you have signed our Contributor License Agreement if this is your first Ultralytics PR by writing "I have read the CLA Document and I sign the CLA" in a new message.
-✅ Minimize Changes: Limit your changes to the minimum necessary for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

For more guidance, please refer to our Contributing Guide. Don't hesitate to leave a comment if you have any questions. Thank you for contributing to Ultralytics! 🚀

[UltralyticsAssistant]

🔍 PR Review

_{Made with ❤️ by Ultralytics Actions}

Overall looks good and aligns with the goal. One issue: the Android classification block can drop existing detections due to a narrow cast; switching to a generic list copy will preserve prior detections.

💬 Posted 1 inline comment

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!