How to query OBB box coordinates?

[cyberN80]

Search before asking

• I checked the docs, issues, and discussions and could not find an answer.

Question

Is there a ready way to retrieve OBB coordinates (or x, y, rotation) in detection results -- or do I need to hack yolo_result.dart or YOLOView.kt, etc?

Additional

No response

[UltralyticsAssistant]

👋 Hello @cyberN80, thank you for submitting a ultralytics/yolo-flutter-app 🚀 Issue. This is an automated response to help us triage quickly — an Ultralytics engineer will also assist soon 😊

To help us address your question efficiently, please ensure you've provided the following information:

1. For bug reports:

• A clear and concise description of the bug
• A minimum reproducible example MRE that demonstrates the issue
• Your environment details (OS, Python version, package versions)
• Expected behavior vs. actual behavior
• Any error messages or logs related to the issue

2. For feature requests:

• A clear and concise description of the proposed feature
• The problem this feature would solve
• Any alternative solutions you've considered

3. For questions:

• Provide as much context as possible about your question
• Include any research you've already done on the topic
• Specify which parts of the documentation, if any, you've already consulted

Since this is about retrieving OBB outputs, it would help a lot if you could also add:
-Which model/task you’re running (OBB vs detect) and what output you currently receive in Flutter
-Where you want the coordinates exposed (Dart API, platform channel, Kotlin/Swift, etc.)
-A small snippet showing how you’re calling inference and reading results

Please make sure you've searched existing issues to avoid duplicates. If you need to add any additional information, please comment on this issue. Thanks! 🙌

[pderrenger]

If you’re running an -obb model (e.g., yolo11n-obb.pt), you shouldn’t need to “hack” Ultralytics—OBB coords are already exposed on the results via result.obb, where result.obb.xywhr gives [cx, cy, w, h, angle] (angle in radians) and result.obb.xyxyxyxy gives the 4 corner points; see the OBB Results API reference and the OBB task predict example. If yolo-flutter-app isn’t surfacing those fields yet, then the app bridge will need to pass through the OBB tensor (typically [cx, cy, w, h, angle, conf, cls]) instead of only axis-aligned xyxy, so please also confirm you’re on the latest app/package version and which runtime you’re using (TFLite/ONNX/CoreML).

[cyberN80]

If you’re running an -obb model (e.g., yolo11n-obb.pt), you shouldn’t need to “hack” Ultralytics—OBB coords are already exposed on the results via result.obb, where result.obb.xywhr gives [cx, cy, w, h, angle] (angle in radians) and result.obb.xyxyxyxy gives the 4 corner points; see the OBB Results API reference and the OBB task predict example. If yolo-flutter-app isn’t surfacing those fields yet, then the app bridge will need to pass through the OBB tensor (typically [cx, cy, w, h, angle, conf, cls]) instead of only axis-aligned xyxy, so please also confirm you’re on the latest app/package version and which runtime you’re using (TFLite/ONNX/CoreML).

Thanks @pderrenger! I'm using TFLite and the latest Flutter plugin version (ultralytics_yolo 0.1.42). These are the attributes and functions available for detection results (no "obb" or "obbPoints"):

Can you advise how I can get "the app bridge ... to pass through the OBB tensor"? Thanks very much for your support!

[pderrenger]

Yep, that’s expected in ultralytics_yolo 0.1.42: the public Dart YOLOResult model only exposes an axis-aligned boundingBox/normalizedBox, so OBB geometry isn’t currently surfaced (even if you run task: YOLOTask.obb). To pass OBB through the bridge you’ll need to extend the plugin end-to-end by adding something like xywhr (and/or obbPoints) to YOLOResult and then emitting that extra field from the Android OBB path over the platform channel (the native OBB result is typically [cx, cy, w, h, angle, conf, cls]; see the OBB predictor output format for the expected ordering), and if you want the 4 corners you can compute them from xywhr using the same math as Ultralytics’ xywhr-to-corners helper.

[cyberN80]

Yep, that’s expected in ultralytics_yolo 0.1.42: the public Dart YOLOResult model only exposes an axis-aligned boundingBox/normalizedBox, so OBB geometry isn’t currently surfaced (even if you run task: YOLOTask.obb). To pass OBB through the bridge you’ll need to extend the plugin end-to-end by adding something like xywhr (and/or obbPoints) to YOLOResult and then emitting that extra field from the Android OBB path over the platform channel (the native OBB result is typically [cx, cy, w, h, angle, conf, cls]; see the OBB predictor output format for the expected ordering), and if you want the 4 corners you can compute them from xywhr using the same math as Ultralytics’ xywhr-to-corners helper.

Thanks again @pderrenger ! I'll give that a try.

[pderrenger]

Yep—once you start wiring it up, note that in ultralytics_yolo 0.1.42 the public YOLOResult model only defines boundingBox/normalizedBox (+ mask/keypoints), so the cleanest path is to add a nullable xywhr (and optionally obbPoints) to YOLOResult + YOLOResult.fromMap, then have the Android TFLite event payload include that extra key (keeping it optional preserves backward compatibility for non-OBB tasks); see the current Dart model here: YOLOResult Dart API. (pub.dev)

[cyberN80]

Yep—once you start wiring it up, note that in ultralytics_yolo 0.1.42 the public YOLOResult model only defines boundingBox/normalizedBox (+ mask/keypoints), so the cleanest path is to add a nullable xywhr (and optionally obbPoints) to YOLOResult + YOLOResult.fromMap, then have the Android TFLite event payload include that extra key (keeping it optional preserves backward compatibility for non-OBB tasks); see the current Dart model here: YOLOResult Dart API. (pub.dev)

I'm stumped. I assume the extra key can be included by adding it to yolo_inference.dart (_processObbResults()) and reflecting the same in yolo_result.dart (YOLOResult(), YOLOResult.fromMap()) , but I've failed to get ahold of 'xywhr', 'xyxyxyxy', and 'obbPoints' this way; in fact, even if I comment out all of the logic of _processObbResults(), nothing breaks when I run the app, so I wonder what it even does. Nor do I understand where yolo_overlay.dart is getting its OBB data to draw oriented boxes on screen properly.

[pderrenger]

You’re not crazy—_processObbResults() is in the YOLOInference.predict* (single image) path, but YOLOView’s real-time camera stream delivers results via the native platform view/EventChannel, so commenting it out won’t affect what you see in the live preview, and the rotated boxes you’re seeing are coming from the native overlay, not from YOLOResult. (pub.dev)

Quick sanity check: add an onStreamingData: (m) => debugPrint(m.keys.toString()) on YOLOView and inspect the raw payload—if you don’t see an angle/corners-like key there, you’ll need to add xywhr (and/or 4 corner points) at the Android serialization point where detections are converted into the map that currently only includes boundingBox/normalizedBox, then read that new key in YOLOResult.fromMap.

[cyberN80]

You're a star, @pderrenger! With your advice, I found the OBB points under the "polygon" key. Here's an example:

{boundingBox: {top: 141.14894104003906, left: 441.8502197265625, bottom: 177.75732421875, right: 484.3997497558594}, classIndex: 1, polygon: [{x: 445.6483459472656, y: 141.14894104003906}, {x: 484.3997497558594, y: 150.88661193847656}, {x: 480.60162353515625, y: 177.75732421875}, {x: 441.8502197265625, y: 168.01963806152344}], confidence: 0.7392578125, className: violin, normalizedBox: {top: 0.2205452173948288, left: 0.9205213189125061, bottom: 0.2777458131313324, right: 1.0091661214828491}}

I can now read OBB points after:

1. Adding this member to YOLOResult() (yolo_result.dart): final List<Object?>? polygon;
2. Adding this to factory YOLOResult.fromMap() (yolo_result.dart): List<Object?> polygon = map['polygon'] as List<Object?>;
3. Getting it in my widget by: $result.polygon.toString()

[pderrenger]

Nice catch—polygon is the OBB’s 4 corner points, i.e. the same 4-point format used by YOLO OBB labels, so you can normalize/store it exactly like the format described in the OBB dataset guide, and if you also need (cx, cy, w, h, angle) you can derive it by fitting a min-area rectangle to those 4 points (otherwise using polygon directly is totally fine); if you’re up for it, a small PR that types polygon as List<Offset> (or List<Map<String, num>>) on YOLOResult would make this much more discoverable for others.

[cyberN80]

Sure thing! Request made: #424

[pderrenger]

Thanks for opening PR #424—I'll review it; please keep polygon nullable for backward compatibility and make sure it’s wired consistently for both YOLOView (streaming/EventChannel) and YOLOInference (single-image) paths (and ideally Android/iOS) with a small example/test so we can merge confidently.