modes/export/

[giscus[bot]]

modes/export/

Step-by-step guide on exporting your YOLOv8 models to various format like ONNX, TensorRT, CoreML and more for deployment. Explore now!.

https://docs.ultralytics.com/modes/export/

[alxAgu]

Where can we find working examples of a tf.js exported model?

[pderrenger]

@alxAgu currently, we don't have specific examples of a tf.js exported model directly within the YOLOv8 documentation. However, exporting a model to TensorFlow.js (tf.js) format typically involves converting the model to a format compatible with tf.js and then loading and running it within a web environment. For the export part you can use i.e.

yolo export model=yolov8n.pt format=tensorflowjs

For general guidance on how to use a tf.js model, you can refer to the TensorFlow.js documentation and tutorials, which provide comprehensive instructions on how to import and run models in the browser or Node.js.

If you're looking for examples specific to YOLO models, you might find community-contributed projects or tutorials by searching online for "YOLO TensorFlow.js example" or similar queries. These resources can often provide practical demonstrations of how to integrate YOLO models with tf.js.

[alxAgu]

@pderrenger, are there any plans to provide a working example of a YOLOv8 model exported to work in the browser with TensorFlow.js? There's a big appetite for this and it's honestly the only thing preventing us from recommending Ultralytics HUB to our clients.

[pderrenger]

Hi @alxAgu, thanks for reaching out! 🚀 We currently don't have a specific example for YOLOv8 models exported to work with TensorFlow.js. However, we appreciate the feedback and understand the demand for such examples. Your suggestion is definitely on our radar, and we'll consider it for future updates to better support web-based applications. Meanwhile, you can export the model to TensorFlow.js format using the command:

yolo export model=yolov8n.pt format=tfjs

For integrating the exported model into a browser environment, you might find the TensorFlow.js documentation helpful. Thanks for recommending Ultralytics, and we hope to meet your needs soon!

[Bharathteja07]

How to use exported engine file for inference of images in a directory?

[pderrenger]

Hey there! It looks like you're encountering an error while trying to deserialize a TensorRT engine. This error often occurs if there's a mismatch between the TensorRT version used to create the .engine file and the version you're using to load it. Here are a few steps you can try to resolve this issue:

1. Version Check: Ensure that the TensorRT version used for creating the .engine file is the same as the one you're using to load it. Version mismatches can lead to serialization errors.

2. Re-export the Model: If possible, re-export your YOLOv8 model to the TensorRT .engine format using the same environment where you're attempting to load it. This ensures compatibility.

3. TensorRT Documentation: Consult the TensorRT documentation for guidance on engine serialization and deserialization. There might be specific flags or steps required for your version.

4. Jetson Nano Deployment: For deploying on the NVIDIA Jetson Nano with JetPack 4.6.4, ensure that all components (CUDA, cuDNN, TensorRT) are correctly installed and match the versions expected by your .engine file. The NVIDIA Jetson Linux Developer Guide can be a helpful resource.

5. Sample Code: When loading and running inference with the .engine file, ensure your code correctly handles input preprocessing and output parsing specific to YOLOv8. The TensorRT Developer Guide provides examples on how to perform these steps.

If you continue facing issues, consider sharing more details about your environment and steps followed on the Ultralytics YOLOv8 GitHub Issues page. The community and developers are quite responsive and may offer more targeted advice. Good luck! 🚀

[Bharathteja07]

How can we integrate .engine model file with SAHI. Does SAHI support .engine files? my images are of size 5328*4206

[pderrenger]

Hey there! 😊 Currently, SAHI doesn't natively support .engine files directly. However, you can still use your TensorRT .engine model with SAHI by creating a custom prediction function. Here's a simplified approach:

1. Load Your Engine: Use TensorRT APIs to load your .engine file as you normally would.
2. Custom Prediction Function: Implement a function that takes an image, performs preprocessing, runs it through the TensorRT engine, and then post-processes the outputs to the format SAHI expects.
3. Integrate with SAHI: Use SAHI's get_prediction method to integrate your custom prediction function.

For large images like yours (5328x4206), consider tiling them into smaller patches that match the input size expected by your model, process each tile, and then aggregate the results.

Here's a very basic snippet to get you started:

import tensorrt as trt
import cv2
import numpy as np

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
runtime = trt.Runtime(TRT_LOGGER)

def load_engine(engine_path):
    with open(engine_path, 'rb') as f:
        engine_data = f.read()
    engine = runtime.deserialize_cuda_engine(engine_data)
    return engine

def custom_predict(image, engine):
    # Preprocess your image to match the model's expected input
    # Run inference with the engine
    # Post-process the engine's output to match SAHI's expected format
    pass

# Example usage
engine = load_engine('path/to/your/yolov8n.engine')
image = cv2.imread('path/to/your/image.jpg')
predictions = custom_predict(image, engine)

Remember, you'll need to adapt the custom_predict function to handle your specific preprocessing, inference, and post-processing steps.

For deploying on NVIDIA Jetson Nano, ensure your JetPack version is compatible with the TensorRT version used to generate the .engine file. If you encounter any issues, consider re-exporting your model with the TensorRT version available on your Jetson Nano.

Hope this helps you get started! If you have more questions or need further assistance, feel free to ask. Happy coding! 🚀

[Bharathteja07]

I am trying to run this on dGPU, I tried using ONNX with SAHI but it is taking a lot of time for processing each image.

1. As it is mentioned in the docs that ONNX (3x CPU) and Tensorrt (5X GPU), does that mean onnx doesnt use GPU? (BTW I have choosen cuda:0".
2. Could you go more in details of custom_predict function mentioned above? This I will be using with tenssort engine files

[pderrenger]

Hey there! 🌟 It looks like you're diving deep into optimizing your model's performance on a dGPU. Let's address your questions:

1. ONNX and GPU Usage: While ONNX itself is a format and doesn't dictate where the inference runs, the framework you use to run ONNX models (like ONNX Runtime) can indeed leverage GPUs. Specifying cuda:0 should direct ONNX Runtime to use your GPU. If you're experiencing slow processing, it might be worth exploring the batch size, model optimization, or the specific ONNX Runtime settings you're using.

2. Custom Predict Function Details: The custom_predict function is essentially where you'd handle the loading of your TensorRT engine, preprocessing of your input images (resizing, normalization), executing the inference, and then post-processing the outputs (decoding the detections). Since you're interested in using TensorRT .engine files, here's a simplified structure of what this function might look like:

import tensorrt as trt
import pycuda.driver as cuda
import pycuda.autoinit  # This is needed for initializing CUDA driver

def custom_predict(image, engine):
    # Assuming 'image' is your preprocessed input image ready for the model
    # And 'engine' is your loaded TensorRT engine

    # Allocate buffers for input and output
    # [Your code to allocate buffers]

    # Create execution context
    context = engine.create_execution_context()

    # Copy input data to the buffer
    # [Your code to copy data to input buffer]

    # Run inference
    context.execute_v2(bindings=[int(d_input), int(d_output)])

    # Copy results from buffer
    # [Your code to copy data from output buffer]

    # Post-process the results to extract bounding boxes, etc.
    # [Your post-processing code]

    return processed_results

This is a very high-level overview. You'll need to fill in the details based on your specific model input/output formats and the preprocessing/postprocessing required.

For deploying on the NVIDIA Jetson Nano with JetPack 4.6.4, ensure you've installed the compatible versions of TensorRT and other dependencies. The Jetson platform is fantastic for edge AI applications, and optimizing your model with TensorRT can significantly boost inference speed.

Regarding the error you encountered, it suggests a version mismatch or an issue with the engine file itself. Double-check the TensorRT versions used for creating and loading the engine, and consider regenerating the engine file if possible.

Hope this helps clear things up! If you have more questions or need further assistance, feel free to reach out. Happy coding! 🚀

[biodatasciencearg]

I trained a custom model taking yolov8n.pt (backbone) and I want to do a model registry in MLFLOW of the model in the .engine format. It's possible directly without the export step? Someone deal with something similar? Tks for your help!

[pderrenger]

Hey there! 👋 It seems like there's a discrepancy in the class detection between your custom YOLOv8 model and the TensorRT .engine file. This usually happens due to a mismatch in the class labels or an issue during the conversion process. Here are a couple of things you might want to check:

1. Class Labels File: Ensure that the class labels file used during training is correctly referenced during the export process. The number of classes should match your custom dataset.

2. Export Command: Your export command looks good, but double-check if the model path and other parameters are correctly specified. Also, ensure that the model argument points to your custom trained .pt file.

3. TensorRT Conversion: After converting to .engine, verify that the conversion process didn't introduce any changes to the class predictions. Sometimes, the conversion process might default to a pre-set number of classes (like 91 for COCO).

Here's a simplified version of your export command for clarity:

from ultralytics import YOLO

# Load your custom trained model
model = YOLO('/content/drive/MyDrive/Colab Notebooks/people_count1/head_model.pt')

# Export to TensorRT .engine format
model.export(format='engine', imgsz=640, half=True)

If you're still facing issues, it might be helpful to explicitly specify the number of classes during the export process if the API allows, or check the documentation for any additional steps required for custom models.

Hope this helps! Let us know if you have any more questions. 😊

[sanvir19]

"I am training my custom model using the pre-trained model YOLOv8-L, which has 91 classes. When I perform inference with my custom model, it works correctly. However, after converting the custom model to TensorRT, the converted model no longer contains my custom classes; instead, it retains the 91 classes from the pre-trained model."

I am using the following code snippet to convert my model to TensorRT:
from ultralytics import YOLO
model = YOLO('custom_model.pt')
model.export(format='engine',
imgsz=640,
optimize=False,
half=True,
int8=False,
dynamic=True,
opset=None)
I have provided the correct path to the custom model.

[pderrenger]

Hey there! 👋 It looks like the issue you're encountering with the TensorRT conversion retaining the original 91 classes from the pre-trained YOLOv8-L model, instead of your custom classes, might be related to how the model's class information is handled during the export process.

When exporting your custom model to TensorRT, it's crucial to ensure that the class information from your custom dataset is correctly preserved. The discrepancy in class numbers suggests that the export process might not be correctly applying your custom model's class structure.

Here's a quick suggestion:

• Double-check that your custom dataset's class labels are correctly defined and accessible during the export process. This might involve specifying the path to your dataset's class labels file if required by the export function.

Unfortunately, the Ultralytics export function doesn't explicitly allow for specifying class labels during the export. However, the model should inherently carry over its learned classes from training. It's possible that during the conversion to TensorRT, the default COCO class labels are being assumed.

If you continue to face issues, consider reaching out on the Ultralytics GitHub issues page or forums with details of your training setup and export process. The community or the developers might offer insights specific to your scenario.

For now, ensure your custom model is correctly recognizing classes post-training and pre-export to isolate the issue to the export process. 🛠️

Hope this helps a bit! Let me know if there's anything else I can do for you.

[sanvir19]

Yes, you are correct. When I am trying to perform inference with my model, it loads its default model trained on the COCO dataset. When I found out about this issue, I provided the path to load my custom model, but then I encountered the error "jetson.inference -- detectNet failed to load network," which indicates that the model may not be loading correctly.

Could you please guide me on how to properly load the model through the entire process?

i use this code for inference:
import jetson.inference
import jetson.utils
import cv2
import time

net = jetson.inference.detectNet(argv=['--model=/home/nano/jetson-inference/data/networks/yolov8/head_mode_newl.engine',
'--labels=/home/nano/jetson-inference/data/networks/yolov8/lable.txt',
'--input-blob=input_0',
'--output-cvg=scores',
'--output-bbox=boxes'], threshold=0.5)

num_classes = net.GetNumClasses()
class_names = [net.GetClassDesc(i) for i in range(num_classes)]

[pderrenger]

@sanvir19 hey there! 👋 It looks like you're encountering an issue where your custom model, when converted to TensorRT and used with jetson.inference, defaults back to detecting the original 91 classes from the pre-trained model rather than your custom classes for head detection.

This issue might stem from how the TensorRT engine is being loaded, especially if the class labels aren't being correctly applied or recognized. Here's a suggestion to ensure your custom classes are correctly utilized:

1. Verify Class Labels: Ensure that your labels.txt file correctly lists your custom classes, and it's in the correct format expected by jetson.inference.

2. Correct Model Path: Double-check the path to your .engine file to ensure it's correctly pointing to your custom model.

3. TensorRT Conversion: When converting your model to TensorRT, ensure that the conversion process is aware of your custom classes. This might involve specifying class labels during the conversion process if supported.

Your code snippet for conversion looks good, but you might want to ensure that the --labels argument in your detectNet initialization points to the correct file and that the file is formatted correctly. Each class should be on a new line, corresponding to the class IDs your model predicts.

If you're still facing issues, consider revisiting the conversion step to ensure it's correctly tailored for your custom classes. Unfortunately, the direct integration with jetson.inference might have limitations or specific requirements for class labels and model formats, so checking the documentation for any additional steps or requirements might also help.

Hope this helps you move forward! If you have more questions or need further assistance, feel free to ask. 😊

[mrawding]

Hi, I appreciate the really awesome work within Ultralytics. I have a simple question.

What is the difference between best_int8.tflite and best_integer_quant.tflite is the quant variation simplifed more than int8 for edge devices? Curious on which one is "better" or at least the differences between the two

[pderrenger]

Hey there! Thanks for the kind words and for reaching out with your question. 😊

The best_int8.tflite and best_integer_quant.tflite files are both TensorFlow Lite models that have been quantized to use 8-bit integer weights and activations, which can reduce the model size and speed up inference, particularly on edge devices.

The main difference typically lies in the quantization technique used. best_int8.tflite usually refers to a model quantized using full integer quantization with int8 precision, while best_integer_quant.tflite might be using a different or more specific form of integer quantization.

As for which one is "better," it really depends on your specific use case and the device you're deploying to. Full integer quantization (best_int8.tflite) is generally more compatible with a wider range of devices, including those with dedicated hardware accelerators for int8 operations. However, if you're targeting a device that supports the specific quantization used in best_integer_quant.tflite, that might be the better choice for you.

In terms of simplicity, both are optimized for edge devices, but you might find one performs better than the other in your particular scenario. It's often a good idea to test both on your target device to see which gives you the best balance of speed and accuracy.

Hope that helps clear things up! If you need more detailed guidance, feel free to check out our export documentation or ask here. Happy coding! 🚀

[himanshu1027]

Hello @pderrenger

Can you plz help me out with how can i use Paddlepaddle Format to extract the text from the images?

Your response is very imp to me i am waiting for your reply.

[pderrenger]

Hello @himanshu1027,

To extract text from images using a model exported in PaddlePaddle format, you'll first need to ensure your model is trained for the task of text detection or OCR (Optical Character Recognition). Once you have a suitable model, you can export it to the PaddlePaddle format using the following code snippet:

from ultralytics import YOLO

# Load your trained model
model = YOLO('path/to/your/trained_model.pt')

# Export the model to PaddlePaddle format
model.export(format='paddle')

After exporting, you can use the PaddlePaddle inference engine to load the model and perform text detection on your images. Here's a basic example of how you might set up the inference:

import paddle
from paddle.inference import Config, create_predictor

# Setup the model
config = Config("path/to/your_model/model.pdmodel", "path/to/your_model/model.pdiparams")
config.enable_use_gpu(100, 0)
predictor = create_predictor(config)

# Prepare your image (assuming 'img' is your preprocessed image tensor)
input_names = predictor.get_input_names()
input_tensor = predictor.get_input_handle(input_names[0])
input_tensor.reshape(img.shape)
input_tensor.copy_from_cpu(img.numpy())

# Run inference
predictor.run()
output_names = predictor.get_output_names()
output_tensor = predictor.get_output_handle(output_names[0])
output_data = output_tensor.copy_to_cpu()

# Process the output_data to extract text

Make sure to adjust the preprocessing and postprocessing steps according to your specific model requirements and the format of your input data.

For more detailed guidance on using PaddlePaddle for inference, you might want to check out the PaddlePaddle documentation.

Hope this helps! Let me know if you have any more questions. 😊

[idaymand]

my code

from ultralytics import YOLO
import tensorflow as tf
import onnx2tf as onnx2tf

model = YOLO('yolov8n_web_model/yolov8n.pt') # load an official model

model = YOLO('/path_to_model/best.pt')
model.export(format="tfjs")

i got an error

ERROR: The trace log is below.
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 310, in print_wrapper_func
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/.local/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 383, in inverted_operation_enable_disable_wrapper_func
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/.local/lib/python3.12/site-packages/onnx2tf/ops/Conv.py", line 246, in make_node
input_tensor = get_padding_as_op(
^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/.local/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 2031, in get_padding_as_op
return tf.pad(x, padding)
^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/.local/lib/python3.12/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/ubuntu/.local/lib/python3.12/site-packages/keras/src/backend/common/keras_tensor.py", line 92, in tf_tensor
raise ValueError(
ValueError: A KerasTensor cannot be used as input to a TensorFlow function. A KerasTensor is a symbolic placeholder for a shape and dtype, used when constructing Keras Functional models or Keras Functions. You can only use it as input to a Keras layer or a Keras operation (from the namespaces keras.layers and keras.operations). You are likely doing something like:

x = Input(...)
...
tf_fn(x)  # Invalid.

What you should do instead is wrap tf_fn in a layer:

class MyLayer(Layer):
    def call(self, x):
        return tf_fn(x)

x = MyLayer()(x)

ERROR: input_onnx_file_path: /home/ubuntu/Python/runs/detect/train155/weights/best.onnx
ERROR: onnx_op_name: Conv_0
ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement
ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again.
ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option.
ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.

TensorFlow SavedModel: export failure ❌ 7.4s: SavedModel file does not exist at: /home/ubuntu/Python/runs/detect/train155/weights/best_saved_model/{saved_model.pbtxt|saved_model.pb}

what is wrong and what i need to do for fix?
my env
python 3.8
yolov8
tensorflow 2.16

thanks a lot

[pderrenger]

@idaymand hey there! 👋 It looks like you're encountering an error while trying to export a YOLOv8 model to TensorFlow.js (tfjs) format. The error message suggests an issue with handling KerasTensors directly in TensorFlow functions, which typically happens when attempting operations not directly supported by Keras layers or operations.

From your code snippet, it seems the error occurs during the conversion process, possibly within the onnx2tf conversion step, which is not directly related to the Ultralytics export functionality. However, to address your main concern about exporting to tfjs, here's a simplified approach using Ultralytics YOLO directly for tfjs export:

from ultralytics import YOLO

# Assuming '/path_to_model/best.pt' is your trained model
model = YOLO('/path_to_model/best.pt')  
model.export(format="tfjs") 

Make sure your environment is set up correctly with the required versions of TensorFlow and Ultralytics YOLO. Since you're using Python 3.8 and TensorFlow 2.16, the setup seems appropriate.

If you're still facing issues, consider the following steps:

1. Ensure that your model is correctly loaded before attempting the export.
2. Verify that all dependencies, especially those related to Ultralytics YOLO and TensorFlow, are correctly installed and compatible.
3. If the problem persists, you might want to explore exporting to ONNX format first and then converting to TensorFlow.js using a more direct approach or tool specifically designed for ONNX to TensorFlow.js conversion.

For more detailed guidance on exporting models and troubleshooting, the Ultralytics documentation on model export is a great resource. It covers a wide range of export formats and provides tips for optimizing your exported models.

Hope this helps! If you have further questions or run into more issues, feel free to reach out. 😊

[idaymand]

Hi @pderrenger! I see that by default ultralytics do export to onnx as step in export process yolov8 model to tfjs. I get an error on the step of export from onnx to tfjs. Log is below

my code

from ultralytics import YOLO

model = YOLO('/my_path/best.pt')
model.export(format="tfjs") # export the model to tfjs format

log

PyTorch: starting from '/home/ubuntu/Python/runs/detect/train155/weights/best.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 103, 8400) (6.7 MB)

TensorFlow SavedModel: starting export with tensorflow 2.12.0...

ONNX: starting export with onnx 1.14.0 opset 12...
ONNX: simplifying with onnxsim 0.4.35...
ONNX: export success ✅ 1.8s, saved as '/home/ubuntu/Python/runs/detect/train155/weights/best.onnx' (13.0 MB)
TensorFlow SavedModel: running 'onnx2tf -i "/home/ubuntu/Python/runs/detect/train155/weights/best.onnx" -o "/home/ubuntu/Python/runs/detect/train155/weights/best_saved_model" -nuo --non_verbose'

ERROR: The trace log is below.
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 310, in print_wrapper_func
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/.local/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 383, in inverted_operation_enable_disable_wrapper_func
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/.local/lib/python3.12/site-packages/onnx2tf/ops/Conv.py", line 246, in make_node
input_tensor = get_padding_as_op(
^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/.local/lib/python3.12/site-packages/onnx2tf/utils/common_functions.py", line 2031, in get_padding_as_op
return tf.pad(x, padding)
^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/.local/lib/python3.12/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/ubuntu/.local/lib/python3.12/site-packages/keras/src/backend/common/keras_tensor.py", line 92, in tf_tensor
raise ValueError(
ValueError: A KerasTensor cannot be used as input to a TensorFlow function. A KerasTensor is a symbolic placeholder for a shape and dtype, used when constructing Keras Functional models or Keras Functions. You can only use it as input to a Keras layer or a Keras operation (from the namespaces keras.layers and keras.operations). You are likely doing something like:

x = Input(...)
...
tf_fn(x)  # Invalid.

What you should do instead is wrap tf_fn in a layer:

class MyLayer(Layer):
    def call(self, x):
        return tf_fn(x)

x = MyLayer()(x)

ERROR: input_onnx_file_path: /home/ubuntu/Python/runs/detect/train155/weights/best.onnx
ERROR: onnx_op_name: Conv_0
ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement
ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again.
ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option.
ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.

TensorFlow SavedModel: export failure ❌ 5.7s: SavedModel file does not exist at: /home/ubuntu/Python/runs/detect/train155/weights/best_saved_model/{saved_model.pbtxt|saved_model.pb}

---

OSError Traceback (most recent call last)
Cell In[29], line 5
1 from ultralytics import YOLO
4 model = YOLO('/home/ubuntu/Python/runs/detect/train155/weights/best.pt')
----> 5 model.export(format="tfjs") # export the model to tfjs format

File ~/Python/python-environments/Torch_1.8/lib/python3.8/site-packages/ultralytics/engine/model.py:313, in Model.export(self, **kwargs)
311 custom = {'imgsz': self.model.args['imgsz'], 'batch': 1, 'data': None, 'verbose': False} # method defaults
312 args = {**self.overrides, **custom, **kwargs, 'mode': 'export'} # highest priority args on the right
--> 313 return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model)

File ~/Python/python-environments/Torch_1.8/lib/python3.8/site-packages/torch/autograd/grad_mode.py:28, in _DecoratorContextManager.call..decorate_context(*args, **kwargs)
25 @functools.wraps(func)
26 def decorate_context(*args, **kwargs):
27 with self.class():
---> 28 return func(*args, **kwargs)

File ~/Python/python-environments/Torch_1.8/lib/python3.8/site-packages/ultralytics/engine/exporter.py:261, in Exporter.call(self, model)
259 if any((saved_model, pb, tflite, edgetpu, tfjs)): # TensorFlow formats
260 self.args.int8 |= edgetpu
--> 261 f[5], keras_model = self.export_saved_model()
262 if pb or tfjs: # pb prerequisite to tfjs
263 f[6], _ = self.export_pb(keras_model=keras_model)

File ~/Python/python-environments/Torch_1.8/lib/python3.8/site-packages/ultralytics/engine/exporter.py:122, in try_export..outer_func(*args, **kwargs)
120 except Exception as e:
121 LOGGER.info(f'{prefix} export failure ❌ {dt.t:.1f}s: {e}')
--> 122 raise e

File ~/Python/python-environments/Torch_1.8/lib/python3.8/site-packages/ultralytics/engine/exporter.py:117, in try_export..outer_func(*args, **kwargs)
115 try:
116 with Profile() as dt:
--> 117 f, model = inner_func(*args, **kwargs)
118 LOGGER.info(f"{prefix} export success ✅ {dt.t:.1f}s, saved as '{f}' ({file_size(f):.1f} MB)")
119 return f, model

File ~/Python/python-environments/Torch_1.8/lib/python3.8/site-packages/ultralytics/engine/exporter.py:712, in Exporter.export_saved_model(self, prefix)
709 for file in f.rglob('*.tflite'):
710 f.unlink() if 'quant_with_int16_act.tflite' in str(f) else self._add_tflite_metadata(file)
--> 712 return str(f), tf.saved_model.load(f, tags=None, options=None)

File ~/Python/python-environments/Torch_1.8/lib/python3.8/site-packages/tensorflow/python/saved_model/load.py:836, in load(export_dir, tags, options)
834 if isinstance(export_dir, os.PathLike):
835 export_dir = os.fspath(export_dir)
--> 836 result = load_partial(export_dir, None, tags, options)["root"]
837 return result

File ~/Python/python-environments/Torch_1.8/lib/python3.8/site-packages/tensorflow/python/saved_model/load.py:941, in load_partial(export_dir, filters, tags, options)
936 if tags is not None and not isinstance(tags, set):
937 # Supports e.g. tags=SERVING and tags=[SERVING]. Sets aren't considered
938 # sequences for nest.flatten, so we put those through as-is.
939 tags = nest.flatten(tags)
940 saved_model_proto, debug_info = (
--> 941 loader_impl.parse_saved_model_with_debug_info(export_dir))
943 if (len(saved_model_proto.meta_graphs) == 1 and
944 saved_model_proto.meta_graphs[0].HasField("object_graph_def")):
945 metrics.IncrementReadApi(_LOAD_V2_LABEL)

File ~/Python/python-environments/Torch_1.8/lib/python3.8/site-packages/tensorflow/python/saved_model/loader_impl.py:58, in parse_saved_model_with_debug_info(export_dir)
45 def parse_saved_model_with_debug_info(export_dir):
46 """Reads the savedmodel as well as the graph debug info.
47
48 Args:
(...)
56 parsed. Missing graph debug info file is fine.
57 """
---> 58 saved_model = parse_saved_model(export_dir)
60 debug_info_path = file_io.join(
61 path_helpers.get_debug_dir(export_dir),
62 constants.DEBUG_INFO_FILENAME_PB)
63 debug_info = graph_debug_info_pb2.GraphDebugInfo()

File ~/Python/python-environments/Torch_1.8/lib/python3.8/site-packages/tensorflow/python/saved_model/loader_impl.py:116, in parse_saved_model(export_dir)
114 raise IOError(f"Cannot parse file {path_to_pbtxt}: {str(e)}.")
115 else:
--> 116 raise IOError(
117 f"SavedModel file does not exist at: {export_dir}{os.path.sep}"
118 f"{{{constants.SAVED_MODEL_FILENAME_PBTXT}|"
119 f"{constants.SAVED_MODEL_FILENAME_PB}}}")

OSError: SavedModel file does not exist at: /home/ubuntu/Python/runs/detect/train155/weights/best_saved_model/{saved_model.pbtxt|saved_model.pb}

Help me please to solve this problem.
Thanks a lot

[AttilaSzucs21]

Hello I had similar problem, I downgraded my tensorflow from 2.16.1 to 2.15.1 and now it works. (In my case I tried tflite conversion so maybe it wont work in your case.)

[pderrenger]

Hey! 👋 It's great to hear that downgrading TensorFlow resolved the issue for you. TensorFlow compatibility can indeed be tricky, especially with newer versions sometimes introducing changes that affect certain functionalities.

Since our export process involves intermediate steps like converting to ONNX before reaching TensorFlow formats, there might still be discrepancies or requirements specific to those versions. If downgrading worked for your situation, it definitely might help others facing similar issues. For those still encountering errors, ensuring all conversion tools and dependencies are compatible with their TensorFlow version is crucial, as outlined in our documentation and error logs. Thanks a lot for sharing your solution! 😊

[pderrenger]

Hey there! 🌟

Thanks for sharing your experience. Downgrading TensorFlow has helped resolve compatibility issues for a lot of users, especially with model conversions. While it worked for your tflite conversion, it might indeed help in resolving OPs compatibility issues that might arise with TensorFlow.js exports as well.

If you face any more challenges, consider checking if your ONNX model uses ops supported by the version of TensorFlow you're using. Sometimes, adjusting the opset version during the ONNX export can help!

Happy coding with YOLOv8! 🚀 If you need more assistance, feel free to ask.

[Ibraheem-Ahmad]

Hello!
Would anyone be able to tell me why this code does not work:
model.export(format=='onnx', int8 = True, dyamic = True, imgsz = (width,height))

the error I get is "TypeError: Model.export() takes 1 positional argument but 2 were given"

[Aisha-Hassan1]

erro ...
wasm-core-impl.ts:219
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'byteLength')
at t.run (wasm-core-impl.ts:219:31)
at t.run (proxy-wrapper.ts:215:17)
at t.OnnxruntimeWebAssemblySessionHandler.run (session-handler.ts:84:15)
at i.run (inference-session-impl.js:94:44)
at runModel (main.html:96:55) ........

when inputTensors => array
const inputTensors = [imageTensor1, imageTensor2];

[Aisha-Hassan1]

[batch_size, channels, height, width]. If your model expects not a single image of shape ???

[glenn-jocher]

Hey there! 👋 When converting a YOLOv8 model for batch processing, you generally don't need to specify the batch size during export, especially with dynamic batching enabled. 🔄 Your export command:

model.export(format='onnx', int8=True, dynamic=True)

is perfectly fine for creating an ONNX model that can handle arrays of images. Just ensure your data is prepped correctly when making actual predictions! 📊

For GPU acceleration with ONNX, leveraging frameworks like ONNX Runtime with CUDA support would be the way to go! 🔥 Make sure your environment is set up for GPU use, and you're specifying the execution provider accordingly.

Feel free to check out more on dynamic batching and ONNX export on the Ultralytics docs page! Hope this helps, and happy coding! 🚀

[Aisha-Hassan1]

code

const inputs = {
images: [inputTensors[0], inputTensors[1]]
};

const { output0, output1 } = await yolov8.run(inputs);

erro

Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'byteLength')
at t.run (wasm-core-impl.ts:219:31)
at t.run (proxy-wrapper.ts:215:17)
at t.OnnxruntimeWebAssemblySessionHandler.run (session-handler.ts:84:15)
at i.run (inference-session-impl.js:94:44)
at runModel (main.html:107:53)

[glenn-jocher]

Hey there! 👋 It looks like there might be a slight mix-up in how you're constructing the input to your model run. When dealing with arrays of tensors in ONNX.js, you should pass a single tensor that concatenates your image tensors along the batch dimension if the model is set up for batch processing.

For your issue with inputTensors causing an error about byteLength, it's likely because the model expects one tensor that combines all input images. Instead of:

const inputs = {
    images: [inputTensors[0], inputTensors[1]]
};

You should concatenate your inputTensors into one tensor if your model expects a batch input. Unfortunately, directly passing an array like this might not work as expected with ONNX.js since it seems to expect the input as a single tensor representing the batch. The error you're encountering (TypeError: Cannot read properties of undefined (reading 'byteLength')) usually occurs when the input format to the session.run method is not as expected.

This could look different based on how your model is trained and how ort.Tensor handles batches. Ensure your model is trained with dynamic batch size if you want to process multiple images in a single forward pass. If you're not sure how to concatenate tensors in JavaScript (as direct tensor concatenation might not be as straightforward as in Python), you might have to loop over your images and process them one at a time, or look into adjusting your model to accept variable batch sizes more gracefully.

Given your current setup, if your model does not explicitly support batched inputs, you might need to process each image individually:

for(const tensor of inputTensors) {
    const inputs = { images: tensor };
    const outputs = await yolov8.run(inputs);
    // Process the outputs for each image here
}

Remember, different models expect the input in specific formats, including batch size. Ensure your preprocessing aligns with these expectations! 🚀

[JamesMcCullochDickens]

Are there any examples of getting the output of a pose estimator model in C++ using a torchscript file. I'm getting an output of shape (1, 56, 8400) for an input of size (1, 3, 640, 640) with two people in the sample picture. How should I interpret/post-process this output?

[pderrenger]

Hey there! 👋 It looks like you're diving into pose estimation with a YOLOv8 model exported as a TorchScript file. The output shape (1, 56, 8400) you're seeing is typical for pose estimation models, where 56 likely represents the number of keypoints (x, y coordinates) multiplied by the number of attributes per keypoint, and 8400 is the flattened spatial grid (for a 640x640 input, this might be 80x80 grid cells, each predicting keypoints for potential objects).

To interpret this output, you'd typically:

1. Reshape or slice the tensor to isolate keypoints for each detected person.
2. Scale the keypoints back to the original image dimensions if they're normalized.
3. Apply any necessary post-processing, like thresholding based on confidence scores if included in your output.

Here's a rough idea of how you might start to interpret the output in C++ (pseudo-code):

// Assuming output is a 1x56x8400 tensor
int num_keypoints = 28; // Example: 28 keypoints (x, y) per person
int grid_cells = 8400; // Spatial grid cells
for (int i = 0; i < grid_cells; i++) {
    // Example: Extracting the first person's keypoints
    // This part depends on how your model structures its output
    float x = output[0][i % num_keypoints * 2][i];
    float y = output[0][(i % num_keypoints * 2) + 1][i];
    // Scale x and y back to image dimensions if necessary
    // Apply any post-processing like thresholding
}

This is quite simplified and will need adjustments based on your specific model's output format. If your model includes confidence scores or other attributes per keypoint, you'll need to account for those in your parsing logic.

For more detailed guidance on exporting models and handling their outputs, our Model Export with Ultralytics YOLO page might offer some insights, even though it's more focused on the export process itself.

Hope this helps! Let us know if you have more questions. 😊

[JustinGeorgi]

I trained a yolov5 detection model a little while ago and have successfully converted that model to tensorflowjs. That tfjs model works as expected in code only slightly modified from the example available at https://github.com/zldrobit/tfjs-yolov5-example. My version of the relevant section:

const [modelWidth, modelHeight] = model.inputs[0].shape.slice(1, 3)

const input = tf.tidy(() => {
  return tf.image.resizeBilinear(tf.browser.fromPixels(imageData), [modelWidth, modelHeight]).div(255.0).expandDims(0)
})

var results = model.executeAsync(input).then(res => {
  const [boxes, scores, classes, valid_detections] = res
  const boxes_data = boxes.dataSync()
  const scores_data = scores.dataSync()
  const classes_data = classes.dataSync()
  const valid_detections_data = valid_detections.dataSync()[0]

I have now trained a yolov8 detection model on very similar data. The comments in exporter.py indicate that in that example script v8 models should be a drop in replacement for v5 models:

https://github.com/ultralytics/ultralytics/blob/main/ultralytics/engine/exporter.py#L45-L49

However, that does not seem to be the case. The v5 model output is the 4 length array of tensors (which is why the destructuring assignment works), but the v8 model output is a single tensor of shape [1, X, 8400] thus the example code results in an error complaining that the model result is non-iterable when attempting to destructure.

From what I understand, the [1, X, 8400] is the expected output shape of the v8 model. Is further processing of the v8 model required, or did I do something wrong during the pt -> tfjs export?

[pderrenger]

Hey there! 😊 It looks like you've encountered a difference in output formats between YOLOv5 and YOLOv8 models when exporting to TensorFlow.js. Indeed, YOLOv8 models output a single tensor of shape [1, X, 8400], which is different from the 4-length array of tensors you'd get from YOLOv5 models.

For YOLOv8, you'll need to process the single output tensor to extract the bounding boxes, scores, and class IDs. This involves reshaping and slicing the tensor based on your model's specific output format. Here's a simplified example of how you might start to approach this:

model.executeAsync(input).then(res => {
  // Assuming res is the single tensor output [1, X, 8400]
  const data = res.dataSync();
  
  // Example processing (you'll need to adjust based on your model specifics)
  const numDetections = data[0];
  for (let i = 0; i < numDetections; i++) {
    const offset = i * 7; // Example offset, adjust based on your output format
    const score = data[offset + 2]; // Example index for score
    // Extract other values (x, y, width, height, class) similarly
  }
  // Further processing...
});

This is a very general example, and you'll need to adjust the indices based on how your YOLOv8 model formats its output tensor. The key takeaway is that the output tensor will contain all the information you need, but it will require some additional processing to extract and use effectively.

For more detailed guidance on handling YOLOv8's output format and integrating it with TensorFlow.js, I recommend checking out our Model Export documentation, which provides insights into exporting models and might offer further clues on handling different output formats.

If you have any more questions or need further assistance, feel free to ask. Happy coding! 🚀

[JustinGeorgi]

Thank you, for the reply. I thought reshaping and slicing should be something like that, but was concerned because that doesn't make any sense with the output that I'm getting.

For example:

const numDetections = data[0];

implies that, after reshaping the results, data[0] should be an integer value. As should any values that refer to class indicies of the detected objects. There are no integer values in the result array.

I know the original pt model works, it can correctly identify objects when I do a quick yolo predict ... command line test. So now, I geuss, that leaves me with two options: the pt -> tfjs has gone wrong or my input value for executeAsync is incorrect. I didn't set any additional preprocessing parameters on the origin model build, so the only things my js code does is reshape the image to the size defined in the model (which is correct) and normalize the data.

Do the yolov8 models have any additional pre-processing that I am unaware of or is it most likely that my exported model is faulty?

[pderrenger]

Hey there! 👋 It sounds like you're diving deep into the intricacies of model outputs and preprocessing, which is fantastic! From what you've described, it seems there might be a bit of confusion regarding the output format of YOLOv8 models and how to handle them in TensorFlow.js.

Indeed, YOLOv8 models output a single tensor with a shape like [1, X, 8400], which is quite different from YOLOv5's multi-tensor output. This single tensor packs all the detection information, including bounding boxes, objectness scores, and class probabilities, in a compact form. The key to using this output effectively is to correctly parse and reshape this tensor to extract the information you need.

Regarding preprocessing, YOLOv8 models generally follow the same preprocessing steps as YOLOv5 models: resizing the input image to the model's input dimensions and normalizing pixel values to the range [0, 1]. If you've got that covered, your preprocessing should be on point! 🎯

If the output tensor doesn't seem to contain integer values where you expect them (like class indices), it might be due to how the data is packed in the tensor. The actual values you're looking for are there but might require a different approach to extract correctly.

Given your situation, it doesn't immediately sound like a fault in the pt -> tfjs conversion process, especially if the model works as expected in a yolo predict test. It's more likely about adapting the post-processing step to match YOLOv8's output format.

For a more tailored solution, I'd recommend revisiting the Model Export documentation and focusing on the specifics of handling YOLOv8's output tensor in TensorFlow.js. Unfortunately, without a direct example of parsing YOLOv8's output in TensorFlow.js, I can't provide a precise code snippet right now. However, understanding the structure of the output tensor and applying the correct tensor operations to unpack it will be key.

Keep experimenting, and don't hesitate to reach out if you hit another snag or have more questions. Happy coding! 😄

[Ibraheem-Ahmad]

I was wondering if anyone could help me with this code: I exported my custom trained yolov8n.pt model to .onnx but now my code is not working(model.export(format='onnx', int8=True, dynamic=True)). I am having trouble using the outputs after running inference.

My Code:
"import cv2
import onnxruntime as ort
import numpy as np

def load_image(image_path):
# Load the image from the specified path
original_image = cv2.imread(image_path)
if original_image is None:
raise ValueError(f"Could not load image at path: {image_path}")

# Resize the image to the required dimensions (640x640)
resized_image = cv2.resize(original_image, (640, 640))

# Rearrange the image dimensions from HWC to CHW format and add a batch dimension
image_data = np.transpose(resized_image, (2, 0, 1))
image_data = np.expand_dims(image_data, axis=0).astype(np.float32)

return original_image, image_data

def draw_bounding_boxes(image, detections, confidence_threshold=0.5):
for detection in detections:
x1, y1, x2, y2, score = detection
if score >= confidence_threshold:
start_point = (int(x1), int(y1))
end_point = (int(x2), int(y2))
color = (255, 0, 0) # Bounding box color
thickness = 2 # Bounding box thickness
image = cv2.rectangle(image, start_point, end_point, color, thickness)
return image

def main(model_path, image_path):
# Load ONNX model
ort_session = ort.InferenceSession(model_path)
input_name = ort_session.get_inputs()[0].name
input_shape = ort_session.get_inputs()[0].shape

# Load and preprocess the image
original_image, image_data = load_image(image_path)

# Run inference
detections = ort_session.run(None, {input_name: image_data})[0]

for output in detections[0]:
    print(f"Output {output}: Shape = {output.shape}, Type = {output.dtype}")
    # Print values for smaller tensors
    if output.size < 500:  # Arbitrary size threshold to avoid flooding the console
        print(f"Values:\n{output}")
# Assume detections are in the format [batch, num_detections, 6] where each detection is [x1, y1, x2, y2, score, class]
detections = detections # Assuming batch size of 1 for simplicity


# Draw bounding boxes on the original image
final_image = draw_bounding_boxes(original_image, detections)

# Save or display the image
#cv2.imwrite('output.jpg', final_image)
cv2.imshow('Detected Objects', final_image)
cv2.waitKey(0)
cv2.destroyAllWindows()

if name == "main":
model_path = 'weights\best.onnx' # Replace with your model path
image_path = "Capture.PNG" # Replace with your image path
main(model_path, image_path)
"

Error:
line 28, in draw_bounding_boxes
if score >= confidence_threshold:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

[glenn-jocher]

Hey there! It looks like you're encountering an issue with handling the output from your ONNX model after running inference. The error you're seeing is likely due to attempting to directly compare a numpy array with a scalar value, which numpy doesn't allow without specifying whether you want to use any() or all() for the comparison.

However, the root cause seems to be how you're processing the detections output from the ONNX model. Typically, the output from an ONNX model will be a numpy array, but you need to ensure you're accessing the detection results correctly. Here's a simplified approach to process the model output and draw bounding boxes:

# Assuming detections are in the format [batch, num_detections, 6] where each detection is [x1, y1, x2, y2, score, class]
# And assuming batch size of 1 for simplicity
detections = detections[0][0]  # Adjust based on your model's output format

# Draw bounding boxes on the original image
final_image = draw_bounding_boxes(original_image, detections, confidence_threshold=0.5)

Make sure the detections variable is correctly formatted before passing it to draw_bounding_boxes. The error you're encountering suggests that detections might not be unpacked correctly, leading to the comparison issue.

If you're still having trouble, could you share more details about the shape and type of the detections output right after inference? That might help pinpoint the issue further. 😊

[amarnathreddy0201]

@Ibraheem-Ahmad you got answer. I am also facing same issue.

[amarnathreddy0201]

import onnxruntime as ort
import cv2
classes = [
"person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck",
"boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench",
"bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra",
"giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee",
"skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove",
"skateboard", "surfboard", "tennis racket", "bottle", "wine glass", "cup", "fork",
"knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", "broccoli",
"carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch", "potted plant",
"bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard",
"cell phone", "microwave", "oven", "toaster", "sink", "refrigerator", "book",
"clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"
] # Truncated for brevity; fill in all COCO labels
import numpy as np

image_path = r"C:\Users\amarn\OneDrive\Pictures\father mother.jpg"

image_path = r"C:\Users\amarn\OneDrive\Pictures\Screenshots\Screenshot 2024-07-09 143306.png"
image = cv2.imread(image_path,1)
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
image = cv2.resize(image, (640, 640))

Normalize image to [0, 1]

image_normalized = image.astype(np.float32) / 255.0

Transpose the image to (channels, height, width) and add batch dimension

input_tensor = np.transpose(image_normalized, (2, 0, 1)) # (C, H, W)
input_tensor = np.expand_dims(input_tensor, axis=0) # (1, C, H, W)

ort_sess = ort.InferenceSession('yolov8n.onnx')
input_name = ort_sess.get_inputs()[0].name
output_names = [output.name for output in ort_sess.get_outputs()]
outputs = ort_sess.run(None, {input_name: input_tensor})

Extract data from output

output = outputs[0] # shape (1, 84, 8400)
output = np.squeeze(output) # shape (84, 8400)

num_classes = len(classes)
num_boxes = output.shape[1]

Assuming the format (x_center, y_center, width, height, scores...)

boxes = output[0:4, :]
scores = output[4:5, :].squeeze()
class_scores = output[5:, :]

Apply a confidence threshold

confidence_threshold = 0.4
mask = scores > confidence_threshold

boxes = boxes[:, mask]
scores = scores[mask]
class_scores = class_scores[:, mask]

Calculate class labels and scores

class_ids = np.argmax(class_scores, axis=0)
class_confidences = np.max(class_scores, axis=0)

Draw bounding boxes

for i in range(boxes.shape[1]):
x_center, y_center, width, height = boxes[:, i]
class_id = class_ids[i]
confidence = class_confidences[i]
print("#################")
# Convert from center coordinates to box coordinates
x1 = int((x_center - width / 2) )
y1 = int((y_center - height / 2) )
x2 = int((x_center + width / 2) )
y2 = int((y_center + height / 2))
print(x1, y1, x2, y2)
# Draw the box and label on the image
label = f"{classes[class_id]}: {confidence:.2f}"
print(label)
cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
cv2.putText(image, label, (x1, y1 - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

Display the image with bounding boxes

image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR) # Convert back to BGR for OpenCV
cv2.imshow("Detections", image)
cv2.waitKey(0)
cv2.destroyAllWindows()

I followed this methodology for detecting the person. but it showing airplane in the bounding boxes.

[glenn-jocher]

@amarnathreddy0201 it looks like the issue might be related to how the outputs from the ONNX model are being processed. The error suggests that the scores array is not being handled correctly. Ensure that the outputs are correctly unpacked and processed before drawing bounding boxes. You might want to check the shape and content of the detections array right after the inference to confirm it matches your expectations. If the problem persists, please verify if the issue is reproducible with the latest version of the package. For more detailed guidance, refer to the Ultralytics export documentation.

[larryKra]

"batch_size" is not in arguments as previous versions?

[glenn-jocher]

Hey there! 👋

In YOLOv8, the batch_size parameter is typically set within your dataset YAML file rather than as a direct argument in training commands. This allows for more flexibility and dataset-specific batch size configurations. Here's a quick example of how it's defined in a dataset YAML:

# Example dataset.yaml
train: /path/to/train/images
val: /path/to/val/images

# Define your batch size here
batch_size: 16
...

Then, when you run your training command, YOLOv8 will automatically pick up the batch_size from your dataset configuration. No need to pass it as a separate argument! 😊

Hope this helps! If you have any more questions, feel free to ask.

[yasirgultak]

I converted the model I trained with costum data to tflite format. Before converting, I set the int8 argument to true. But when I examined the tflite format from the netron website, I saw that the input information is still float32. Is this normal or is there a bug?

Also thank you very much for answering every question without getting bored.

[glenn-jocher]

Hey there! 😊 It's great to hear you're exploring the TFLite format for your custom model. Regarding your observation about the input information still being float32 even after setting int8 to true, this is actually expected behavior. The int8 quantization primarily affects the weights and activations within the model to reduce size and potentially speed up inference on compatible hardware. However, the input and output layers often remain as float32 for compatibility reasons, allowing you to feed in standard floating-point data and receive floating-point outputs without additional conversion steps.

If you're looking to further optimize or have specific requirements for input/output data types, you might consider post-processing steps or exploring additional TensorFlow tools for model optimization.

And no worries at all, we're here to help! If you have any more questions or need further assistance, feel free to ask. Happy coding! 🚀

[usama-sherazi]

!yolo export model=/content/drive/MyDrive/best-1-1.pt format=tflite

export failure ❌ 33.0s: generic_type: cannot initialize type "StatusCode": an object with that name is already defined
Traceback (most recent call last):
File "/usr/local/bin/yolo", line 8, in
can anyone help me to export to tflite....

[usama-sherazi]

Firstly, a big thank you to @glenn-jocher for the support.

During my search for a solution, I discovered that TensorFlow versions greater than 2.13.0 have issues with YOLO.

As Google Colab had version 2.15.0 installed, I resolved the issue by adding the following snippet before the conversion:

%pip install tensorflow==2.13.0
This ensured compatibility and resolved the problem. Thank you again for the assistance!

[glenn-jocher]

Hey there! 🎉 Big thanks for sharing your experience and solution - it's super valuable! 🙌 It's really interesting to hear about the version-specific intricacies with TensorFlow and YOLO. Downgrading TensorFlow to 2.13.0 to ensure compatibility is a clever workaround. Your discovery is a great addition to the community knowledge pool, and it might just be the key for others facing similar issues. 👍

If anyone else encounters similar hiccups, trying out different TensorFlow versions based on your model requirements could indeed be a fruitful debugging step.

Super grateful for your input, and cheers for contributing to smoother YOLO model deployments! 🚀 If there's more to ponder or if new challenges pop up, the community is here to help. Keep up the fantastic work!

[usama-sherazi]

yolo export model=/content/yolov8nsss (3).pt format=tflite

/bin/bash: -c: line 1: syntax error near unexpected token (' /bin/bash: -c: line 1: yolo export model=/content/yolov8nsss (3).pt format=tflite'

Hi i am converting Yolov8n file to tflite ia am facing this issue suggest me solution
thank you

[eladoni1]

Isnt that because your file name? I do not use the CLI, but try to add " (quotes) in the parameters, or, change the file name to something without paranthesys.

So, either :
yolo export model="/content/yolov8nsss (3).pt" format="tflite"

Or, if that doesnt work, rename :
yolov8nsss (3).pt

To :

yolov8nsss.pt

(Just without paranthesys).

[glenn-jocher]

@eladoni1 hey! Great catch on figuring out the solution with the file name! 🌟 Yes, file paths with special characters, including spaces and parentheses, can sometimes throw a wrench into command line processing. Wrapping the path in quotes or simplifying the file name are both solid approaches.

For others who might come across similar issues, here's a quick recap based on the awesome tip shared:

To handle file names with spaces or special characters when using the command line, you can either:

• Wrap the path in quotes, like so:

yolo export model="/content/yolov8nsss (3).pt" format="tflite"

• Or, rename the file to remove spaces and special characters, making it:

yolov8nsss.pt

Both methods help ensure the command line correctly understands the file path, aiding in a smooth export process. 🚀 Thanks again for highlighting this solution, and if there's anything more you're curious about or need help with, feel free to ask!

[rajazain2001]

Hi I havr tried all TFLITE export formats to convert the best.pt to .tflite but non is working. I have also checked my runtime and all the latest imports pip install -U ultralytics, and I have also tried the code you gave to someone in the comments but the issue is not resolvig

Step 1: Export to TensorFlow SavedModel

!yolo export model='/content/drive/MyDrive/best-1-1.pt' format=saved_model

Step 2: Convert the exported SavedModel to TensorFlow Lite

import tensorflow as tf
saved_model_dir = 'path/to/exported/saved_model' # Update this path
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
tflite_model = converter.convert()

Save the TFLite model

with open('/content/drive/MyDrive/yolov8_model.tflite', 'wb') as f:
f.write(tflite_model) .

but the same error comes back.
can you please provide me with some specific solution I wil be very thankful to you.
you can reach me at zainayani2001@gmail.com

[pderrenger]

@rajazain2001 hi there,

It sounds like you're encountering issues while trying to convert your best.pt model to TensorFlow Lite format. Let's try a couple of things to troubleshoot:

1. Ensure Compatibility: Make sure that your model architecture and operations are compatible with TFLite. Some custom layers or operations might not be supported directly.

2. Update TensorFlow: Sometimes, updating TensorFlow to the latest version can resolve conversion issues:

   pip install --upgrade tensorflow

3. Use TensorFlow 2.x: Ensure you are using TensorFlow 2.x, as TensorFlow 1.x might not support some functionalities.

4. Check the Error Message: The specific error message you're receiving can give more clues on what might be going wrong.

5. Simplify the Model: If possible, try simplifying the model or using a standard model to test if the issue is with the model complexity.

If these steps don't resolve the issue, could you please share the specific error message you're encountering? This will help in diagnosing the problem more effectively.

For more detailed guidance on exporting models, you might find our export documentation helpful.

[itachi1232gg]

can we export sam/mobile sam model to tensorRT or onnx?

[pderrenger]

Absolutely! You can export SAM/MobileSAM models to both TensorRT and ONNX formats using the Ultralytics YOLOv8 export functionality. Here's how you can do it:

For ONNX:

yolo export model=path/to/your/sam_model.pt format=onnx

For TensorRT:

yolo export model=path/to/your/sam_model.pt format=engine

Just replace path/to/your/sam_model.pt with the actual path to your trained SAM or MobileSAM model. Let me know if you need further assistance! 😊

[itachi1232gg]

Thanks for answering, I tried yolo export model=path/to/your/sam_model.pt format=engine , it raises error : AttributeError: 'Sam' object has no attribute 'args'. SAM and YOLO extend from different base class.

[pderrenger]

Hey there! 👋 Thanks for reaching out with your issue. It seems like you've encountered an attribute error concerning the SAM model during export to TensorRT. This error suggests the SAM class is missing an expected attribute, likely due to differences in the base class structures between SAM and YOLO models.

A potential workaround could involve ensuring that your SAM model object has all necessary attributes before export. However, without diving deeper into the codebase, it's challenging to provide an exact solution.

Could you try exporting the model to ONNX first and then converting that ONNX file to TensorRT? This two-step process might bypass the immediate issue:

# Export to ONNX
yolo export model=path/to/your/sam_model.pt format=onnx

# Convert ONNX to TensorRT (using an external tool or script)

If this doesn't resolve the issue, could you provide more details about your SAM model and the environment you're working in? That could give us better insight into the problem. 🛠️

Also, we're continuously improving our models and export capabilities. So, your feedback is invaluable for making YOLOv8 even better!

[itachi1232gg]

I think exporter only supports yolo models. In order to convert SAM model to ONNX or TensorRT, I should write anther convert script.

[pderrenger]

Hey! 👋 Thanks for reaching out. Yes, the current exporter is primarily designed for YOLO models. However, for SAM/MobileSAM models, converting to ONNX or TensorRT does indeed require a bit of custom handling.

To get started with ONNX export, you might consider something along these lines after your model is loaded:

import torch.onnx

# Assuming `model` is your SAM/MobileSAM model
torch.onnx.export(model, dummy_input, 'model_name.onnx', verbose=True, input_names=['input'], output_names=['output'])

This code snippet assumes a dummy input that matches the expected dimensions of the model.

For TensorRT, after exporting to ONNX, tools like onnx-tensorrt can help with the conversion:

onnx2trt model_name.onnx -o model_name.trt

Although these steps are a good starting point, depending on the SAM model's specifics, adjustments might be necessary. If you encounter challenges along the way or need further assistance, feel free to ask! We're here to help. 😊

[KPLogan]

Glenn, Thank you for the reply. The process of elimination continues. Kevin. Kevin Logan President/CEO ***@***.***> ***@***.*** 860-861-3172 (cell) 860-535-3885 (office) From: Glenn Jocher ***@***.***> Sent: Wednesday, December 4, 2024 4:14 PM To: ultralytics/ultralytics ***@***.***> Cc: KPLogan ***@***.***>; Mention ***@***.***> Subject: Re: [ultralytics/ultralytics] modes/export/ (Discussion #7933) @KPLogan <https://github.com/KPLogan> 1) YOLO models are automatically set to evaluation mode during the export process, so calling model.eval() before export is unnecessary. 2) A dummy input is internally managed by the library during ONNX export, so you don't need to provide it explicitly. If the inference outputs remain unexpected, ensure you've used the correct preprocessing and postprocessing steps in your C#/.NET environment. If the issue persists, you can revisit the ONNX documentation <https://docs.ultralytics.com/integrations/onnx/> or check your ONNX Runtime pipeline. Let us know if you encounter further issues! — Reply to this email directly, view it on GitHub <#7933 (reply in thread)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACQMYD4RRO7QJ6N2B37H7LT2D5WC7AVCNFSM6AAAAABCTE3SQOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNBWGYYTCOI> . You are receiving this because you were mentioned. <https://github.com/notifications/beacon/ACQMYD6TV6FTQZTJRFILYR32D5WC7A5CNFSM6AAAAABCTE3SQOWGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAV32YO.gif> Message ID: ***@***.***>

[glenn-jocher]

@KPLogan thank you for the update, Kevin. If further assistance is needed with ONNX export or runtime issues, I recommend revisiting the ONNX Export Guide for detailed steps and troubleshooting tips. Let me know if there’s anything specific you’d like clarified!

[SasaStomatopoda]

Hi,
I want to change the model "yolov8m-pose.pt" format and use the format "yolov8m-pose.engine",
but when I run the basic test I get an error that there are no key points data.
I used everything listed in the example

model = YOLO("yolov8m-pose.pt") # Load a model
model.export(format="engine")

and even changed some arguments, but without success.

How should I do that?
Thanks in advance.

[glenn-jocher]

@SasaStomatopoda it seems the issue may be related to compatibility or configuration during the export process. Please ensure you are setting imgsz, batch, or dynamic arguments appropriately for your inference needs when exporting to TensorRT (engine). Additionally, verify that keypoint data is correctly included in your dataset and that the model is configured for pose estimation. Consult the TensorRT Export Documentation for detailed argument usage or try including dynamic=True or batch=1 during export. Let me know if the issue persists!

[isabinimam]

Hey there. Im exporting finished YOLOV11 model on custom dataset to TensorflowLite with this code:
from ultralytics import YOLO

from ultralytics import YOLO

# Load a model
model = YOLO('/kaggle/working/yolov11-bs64-wd0005-e50.pt')  # load a custom trained model

# Export the model
model.export(format='tflite', imgsz=320)  int8= 8-bit
model.export(format='tflite', imgsz=320, int8=True ,data='/kaggle/working/Vehicle-Detection-11/data.yaml')

I Got warning when exporting YOLOv11 on custom dataset model from .pt to .tflite

W0000 00:00:1733799417.574774      23 tf_tfl_flatbuffer_helpers.cc:390] Ignored output_format.
W0000 00:00:1733799417.574812      23 tf_tfl_flatbuffer_helpers.cc:393] Ignored drop_control_dependency.
W0000 00:00:1733799419.546880      23 tf_tfl_flatbuffer_helpers.cc:390] Ignored output_format.
W0000 00:00:1733799419.546917      23 tf_tfl_flatbuffer_helpers.cc:393] Ignored drop_control_dependency.
W0000 00:00:1733799421.716608      23 tf_tfl_flatbuffer_helpers.cc:390] Ignored output_format.
W0000 00:00:1733799421.716642      23 tf_tfl_flatbuffer_helpers.cc:393] Ignored drop_control_dependency.
W0000 00:00:1733799425.629228      23 tf_tfl_flatbuffer_helpers.cc:390] Ignored output_format.
W0000 00:00:1733799425.629258      23 tf_tfl_flatbuffer_helpers.cc:393] Ignored drop_control_dependency.

Is this okay?

[glenn-jocher]

@isabinimam if the issue persists despite using consistent image sizes (640x640), it suggests a potential problem with TFLite conversion or post-processing. To debug, ensure quantization (e.g., int8=True) calibration was accurate and matches your dataset. Additionally, validate the intermediate ONNX model to confirm its correctness before TFLite conversion. If ONNX works fine, consider using it for deployment or revisiting the TFLite export process. You can find detailed export guidance here: Export Documentation.

[isabinimam]

Hey there, ive been try using FP16 or Half. And the .tflite model works just fine although it's using different image size on exporting which is 320. The model run correctly. i think the problem only on exporting to INT8. Why this can happens?

I used the quite same command when exporting both to INT8 and FP16.
FP16/Half:

%cd {HOME}
from ultralytics import YOLO
model = YOLO('/kaggle/working/yolov11-bs64-wd0005-e50.pt')  # load a custom trained model

model.export(format='tflite', half=True ,data='/kaggle/working/datasets/Vehicle-Detection-11/data.yaml')

INT8:

%cd {HOME}
from ultralytics import YOLO
model = YOLO('/kaggle/working/yolov11-bs64-wd0005-e50.pt')  # load a custom trained model

model.export(format='tflite', int8=True ,data='/kaggle/working/datasets/Vehicle-Detection-11/data.yaml')

tflite model validation using FP16 on 320 imagesize:

FP16-half.mp4

Accuracy feel's bad on 320. when using 640 imagesize, the detection runs smother and acccuray improve.

[glenn-jocher]

The issue with INT8 quantization likely stems from calibration inaccuracies during the export process. INT8 requires precise calibration data to maintain detection performance. Ensure the data argument provided during export points to a properly labeled dataset representative of your use case. Refer to the Export Documentation for details on INT8 calibration requirements.

[isabinimam]

I've already check my calibration dataset and seen no error. The bounding box already correct. But still, the problem persist. I've tried other people dataset aswell but the problem persist. What should i do? do yo know a good vehicle calibration dataset containing 4 class which is motorcycle, bus, car, and truck?

[glenn-jocher]

Unfortunately, we don't provide specific dataset recommendations or calibration datasets. However, the issue with INT8 quantization might be due to incorrect calibration during export. Ensure the calibration dataset accurately represents your target data distribution and includes diverse samples for all four classes. For more guidance on export and calibration, refer to the Export Documentation.

[rihablahouar]

Issue with Incorrect Predictions from Quantized YOLOv8m-obb Model (TFLite) in TensorFlow Framework

I have exported my YOLOv8m-obb model to TFLite format with INT8 quantization enabled, using an image size of 640x640 and a data.yaml for my dataset. When I use the quantized model for inference with the Ultralytics framework (Oriented Bounding Boxes), the predictions are correct. However, when I use the same model in the TensorFlow framework, I encounter several issues with the output:

Confidence scores are greater than 1, which is incorrect.
The class_id is either -1 or 0, but it should be 0 or 1 since I have only two classes.
The x and y coordinates have values of -1, which is wrong.

I suspect there might be an issue with how the quantization parameters (scale and zero-point) are being applied in TensorFlow, or possibly with the way I'm handling the model's output or the way I exported the model. I would appreciate guidance on how to correctly handle the quantized model in TensorFlow and resolve the issues with incorrect predictions.

[glenn-jocher]

@rihablahouar it seems the issue might stem from how TensorFlow interprets the quantized model's output. Ensure that the quantization parameters are accurately applied and match those used during export. You may also need to preprocess the inputs and postprocess the outputs in TensorFlow to align with the model's expected format. For detailed guidance on handling TensorFlow exports, refer to the Ultralytics Export Documentation. Let me know if you need further clarification!

[rihablahouar]

TFLite Model Export and Implementation Code yolov8m-obb

I exported my YOLO model to TFLite format using the following CLI command:

yolo export obb model="best_obb.pt" format="tflite" imgsz=640 int8=True data="card_detected/data.yaml"

Below is the Python code I plan to use with the exported model:

1. Model Loading

The exported TFLite model is loaded and initialized as follows:

model_path = "path/to/model.tflite"
interpreter = tf.lite.Interpreter(model_path)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
quantization_params = input_details[0]["quantization"]

2. Image Preprocessing

Steps for preprocessing the input image:

Handle EXIF orientation metadata to ensure the correct alignment.
Resize the image to match the model’s expected input dimensions.
Quantize the image using the model's quantization parameters (scale and zero_point).

def preprocess_image(image_path, input_shape, quant_params):
# Load the image and correct orientation
# Convert the image to BGR format and resize
# Quantize the image using (scale, zero_point)
pass

prepared_image = preprocess_image(image_path, (640, 640), quantization_params)

3. Inference Execution

Steps to run inference on the preprocessed image:

Set the prepared image as input for the model.
Invoke the interpreter to run inference.

input_index = input_details[0]['index']
interpreter.set_tensor(input_index, prepared_image)
interpreter.invoke()

4. Post-Processing Results

Steps for processing the output predictions:

Extract the results and dequantize them to floating-point values.
Filter predictions based on a confidence threshold (e.g., confidence > 0.9).

output_index = output_details[0]['index']
output_data_int8 = interpreter.get_tensor(output_index)
scale_f, zero_point_f = output_details[0]["quantization"]
predictions = ((output_data_int8 - zero_point_f) * scale_f).reshape(-1, 7)

for pred in predictions:
if pred[5] > 0.9: # Filter by confidence
print(f"Class: {int(pred[6])}, Confidence: {pred[5]:.4f}")
print(f"Coordinates: (x: {pred[0]}, y: {pred[1]}, w: {pred[2]}, h: {pred[3]}), Angle: {pred[4]}")
Any help or suggestions on these points would be greatly appreciated. Thank you! as the error persists again

[glenn-jocher]

Thank you for sharing your detailed process. Based on the issues you're encountering, ensure that the quantization parameters (scale and zero-point) are correctly applied during preprocessing and dequantization. Verify the input tensor's format, dimensions, and data type match the model's requirements. For exporting and handling TFLite models, refer to Ultralytics Export Documentation for additional insights. Let us know if further clarification is needed!

[omrir]

I have .pth mode (not sure exactly what is it). How can i convert it to .pt model?
(My end goal is to export as .engine model),
Would appreciate any help on that.
Thanks!

[glenn-jocher]

@omrir to convert your .pth model to a .pt model, you need to load the .pth weights into a YOLO model and save it as a .pt file. Here's an example using PyTorch:

import torch
from ultralytics import YOLO

# Load weights from .pth file
weights = torch.load("path/to/your_model.pth")

# Initialize a YOLO model and load the weights
model = YOLO("yolo11n.pt")  # Load a base YOLO model
model.model.load_state_dict(weights)  # Load your custom weights

# Save as .pt
model.save("path/to/your_model.pt")

Once saved, you can then proceed to export it to .engine using the model.export(format="engine") method. For more details, refer to the Export documentation.

[omrir]

Thank you glenn.
I now understand what i acutally have is a yolo-nas .pth. It has 3 classes (and not 80). Is the method you supllied above would be the same.
Just instead of model= YOLO("yolo11n.pt"), i should use model = NAS("yolo_nas_s.pt").
The rest of conversion would to .enging would be the same?
I tired the above, and it doesn't look trivial. Am i missing something?
I played a bit with the yolo-nas library, and it seems i had to define some kind of pre/post process fucntions. Is this necessary here?

[glenn-jocher]

Thank you for the clarification. YOLO-NAS models have specific preprocessing and postprocessing requirements, which can make the export process less straightforward than YOLO11 models. The NAS("yolo_nas_s.pt") method won't directly work in this case. You will need to adapt your workflow with the YOLO-NAS library, ensuring that all required preprocessing and postprocessing steps are properly defined before exporting to .engine. For detailed guidance, please refer to the YOLO-NAS documentation or support channels. If you'd like to explore YOLO11 for a more streamlined experience, feel free to revisit our Export documentation.

[rajhlinux]

What should the 'opset' value be for Yolov8 to onnx 1.20.1 to be used in TensorRT 10.7 support matrix? Thanks.

[glenn-jocher]

@rajhlinux for TensorRT 10.7 compatibility, the recommended ONNX opset version is 17. Ensure you set opset=17 during the export process. For further details, refer to the Export Documentation.

[rihablahouar]

Exporting YOLOv8 models with Oriented Bounding Boxes (OBB) to TensorFlow Lite (TFLite) is unsupported or supported?

[glenn-jocher]

@rihablahouar exporting YOLO models with Oriented Bounding Boxes (OBB) to TensorFlow Lite (TFLite) is supported. For details on exporting OBB models to TFLite or other formats, please refer to the Export documentation. Let me know if you need further clarification!

[DonatelloDonini]

Hi, I'm trying to use a YOLO model on the web by exporting it to tensorflow.js format.
Documentation on the tensorflow.js website doesn't cover object detection very much, and when it comes to models obtained through YOLO it's completely unexistent.
Is it possible to run YOLO directly on the browser?

[DonatelloDonini]

I mean, I exported the model, loaded it with tfjs, but as a detection it returns a ridicolously long list of floats, here's the output:
https://drive.google.com/file/d/1oTppdtLY66xFCV5Czpgdwd0pa6XhGTsj/view?usp=sharing

[DonatelloDonini]

The problem is that I can't find anything on how to parse/plot the results

[glenn-jocher]

Yes, you'll need to process the raw output tensors into detection results. When exporting with nms=True (recommended), the output format becomes [boxes, scores, classes]. For detailed parsing instructions and web implementation examples, see the TF.js integration guide which explains output handling.

[DonatelloDonini]

Thank you very much for your quick responses.
I'm sorry, but I have one last question for you just to be sure:
I need to use the exported tfjs model in my personal portfolio as a way to demonstrate I can work with this product.
Is this considered commercial use? Because yes, I'm making this portfolio to publicize myself as a developer, but I'm not selling an actual product, it's just just an online demo

[glenn-jocher]

For public demos showcasing YOLO models (even in portfolios), AGPL-3.0 compliance requires open-sourcing your entire project. If unwilling to open-source, an Ultralytics Enterprise License is needed. See License terms for commercial/non-public use definitions.

[Ariya-Kh]

when I use nms=True in exporting yolo11n-seg model to torchscript i get 38 features for each box. The first 36 are the box coordinates and 32 masks. what are the last 2 features?

[glenn-jocher]

@Ariya-Kh the last two features when exporting YOLO11n-seg with nms=True are confidence score and class ID. This consolidated format includes post-processed detections directly in the output. For details on segmentation model outputs with NMS, see the TorchScript export documentation.

[voroshazi]

Dear All,

I would like to export my Yolov11 pt model into TFLite one.
When I use the CLI command here:

yolo export model=yolo11n.pt format=tflite

I got the following error (related to "onnx.serialization"):
Ultralytics 8.3.36 🚀 Python-3.10.16 torch-2.6.0.dev20241112+cu121 CPU (Intel Xeon W-2245 3.90GHz)
YOLO11n summary (fused): 238 layers, 2,616,248 parameters, 0 gradients, 6.5 GFLOPs

PyTorch: starting from 'yolo11n.pt' with input shape (1, 3, 640, 640) BCHW and output shape(s) (1, 84, 8400) (5.4 MB)

TensorFlow SavedModel: starting export with tensorflow 2.15.0...
TensorFlow SavedModel: export failure ❌ 1.7s: No module named 'onnx.serialization'
Traceback (most recent call last):
File "/home/anaconda3/envs/yolov11/bin/yolo", line 8, in
sys.exit(entrypoint())
File "/media/Storage2/Personal/yolov11/ultralytics/cfg/init.py", line 969, in entrypoint
getattr(model, mode)(**overrides) # default args from model
File "/media/Storage2/Personal/yolov11/ultralytics/engine/model.py", line 734, in export
return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model)
File "/media/Storage2/Personal/yolov11/ultralytics/engine/exporter.py", line 365, in call
f[5], keras_model = self.export_saved_model()
File "/media/Storage2/Personal/yolov11/ultralytics/engine/exporter.py", line 146, in outer_func
raise e
File "/media/Storage2/Personal/yolov11/ultralytics/engine/exporter.py", line 141, in outer_func
f, model = inner_func(*args, **kwargs)
File "/media/Storage2/Personal/yolov11/ultralytics/engine/exporter.py", line 940, in export_saved_model
import onnx2tf
File "/home/anaconda3/envs/yolov11/lib/python3.10/site-packages/onnx2tf/init.py", line 1, in
from onnx2tf.onnx2tf import convert, main
File "/home/anaconda3/envs/yolov11/lib/python3.10/site-packages/onnx2tf/onnx2tf.py", line 43, in
from onnx2tf.utils.common_functions import (
File "/home/anaconda3/envs/yolov11/lib/python3.10/site-packages/onnx2tf/utils/common_functions.py", line 23, in
from onnx.serialization import ProtoSerializer
ModuleNotFoundError: No module named 'onnx.serialization'

System:
Ubuntu 22.04.5 LTS
Python 3.10.16
torch 2.6.0
Tensorflow (CPU, GPU) 2.15.0
ultralytics 8.3.36
onnx 1.14.0

(note: export to other formats, e.g. ncnn, openvino, onnx works good)

If you have encountered this problem, what is the solution?
Thanks in advance.

[glenn-jocher]

@voroshazi the error occurs because TFLite export requires ONNX dependencies that aren't properly installed. To resolve this, install the correct ONNX version with:

pip install --upgrade onnx>=1.15.0 onnx2tf

This ensures compatibility between ONNX and the TFLite converter. For more details on TFLite export requirements, see the Ultralytics Export Formats documentation.

[voroshazi]

Dear @glenn-jocher
Thanks for your feedback. The command you wrote automatically installed the onnx 1.17.0, but I got another error related to onnx-tf.
I solved it by downgrading to onnx==1.15.0. Now it works.

[glenn-jocher]

Thanks for sharing your solution! Using ONNX 1.15.0 with TFLite export is indeed the recommended approach, as newer ONNX versions can sometimes introduce compatibility issues with conversion tools like onnx2tf. This version alignment ensures smooth operation of our export pipeline. For future exports, you might want to maintain this dependency setup for consistency.

[StefanNa3Shape]

Hi,
I want to test a model on my iPhone 16 through the hub app.
It works fine with all other model that are standard.

Then I trained a model using the hub, made sure it worked in python and exported it in various formats in the hub including CoreML.

I can also load it in the app, but nothing happens.
What is wrong?

[glenn-jocher]

@StefanNa3Shape it sounds like you might be experiencing compatibility issues with your custom-trained model in the HUB app. Several factors could be causing this:

1. Model Format: Even though you've exported to CoreML, there might be specific optimizations needed for mobile deployment. Ensure your export includes proper metadata.

2. Model Size: Custom models that are too large might encounter loading or performance issues on mobile devices.

3. Input Processing: The app might be preprocessing inputs differently than your Python environment.

To troubleshoot, I recommend:

• Verify your CoreML export settings. Try using model.export(format="coreml", imgsz=640) directly in Python to ensure a clean export.
• Check that class names and model task type are correctly included in the export.
• Consider simplifying your model architecture if it's particularly complex.
• Examine the model in the Netron app (https://netron.app/) to confirm it has the expected inputs, outputs, and metadata.

If problems persist, you may want to try the standard YOLOv8 models as a baseline and then incrementally customize them to better understand compatibility boundaries.

[StefanNa3Shape]

I made the mistake of downloading it before it was ready and that corrupted something on the phone.
When I trained again and waited sufficiently, it worked fine.

A feature to delete models from the phone would be great.

[SiluniKeerthiratne]

Hi,

I'm trying to convert yolo_v12_nano from .pt to .tflite, but I'm running into the error shown below. To check if the issue is specific to YOLOv12 being relatively new, I also tried the same process with yolo_v10, but encountered the same error.

ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.

ERROR: The trace log is below.
Traceback (most recent call last):
...
ValueError: Exception encountered when calling layer "tf.math.add_61" (type TFOpLambda).

Dimensions must be equal, but are 32 and 16 for '{{node tf.math.add_61/Add}} = AddV2[T=DT_FLOAT](Placeholder, Placeholder_1)' with input shapes: [1,116,232,32], [1,116,232,16].

Call arguments received by layer "tf.math.add_61" (type TFOpLambda):
• x=tf.Tensor(shape=(1, 116, 232, 32), dtype=float32)
• y=tf.Tensor(shape=(1, 116, 232, 16), dtype=float32)
• name='wa/model.2/m.0/Add'

ERROR: input_onnx_file_path: [redacted]
ERROR: onnx_op_name: wa/model.2/m.0/Add
ERROR: Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement
ERROR: Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again.
ERROR: If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option.
ERROR: Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.
...

Has anyone run into this before or found a workaround? Thank you!

[glenn-jocher]

Hello @SiluniKeerthiratne,

This error occurs due to dimension mismatches during the ONNX to TensorFlow conversion process. YOLO12 uses Attention layers which can often cause compatibility issues during export to TFLite format - this is one of the reasons we don't typically recommend YOLO12 for deployment scenarios.

The error specifically shows a tensor dimension mismatch (32 vs 16 channels) when attempting to add tensors. To potentially resolve this:

1. Try adding the -kat flag to handle irregular channel arrangements as suggested in the error message. This can be done by modifying the export parameters.

2. Consider using YOLO11 instead, which is more stable and optimized for export to various formats including TFLite.

3. If you must use YOLO12, you can try exporting with static shapes by including dynamic=False in your export command:

   model.export(format="tflite", dynamic=False)

The underlying issue is that YOLO12's attention mechanisms have complex tensor operations that don't always translate cleanly through the ONNX → TensorFlow → TFLite conversion pipeline.

For deployment purposes, YOLO11 generally offers better stability and compatibility across export formats while maintaining excellent performance.

[rememberBr]

Does using dynamic=True on Nvidia's Jetson development board not include batch dimension? It seems that only w and h have become dynamic dimensions, while batch is fixed at 1?

[404-NOTFOUND-Coder]

Title: int8 argument not supported when exporting to ONNX in Ultralytics 8.3.162

Description:

I encountered an error when trying to export a trained YOLOv8 model to ONNX format with the following code:

model = YOLO('/kb210/wrs/tennis-analysis/tennis_analysis/training/runs/detect/train27/weights/best.pt')
model.export(format="onnx", imgsz=(1024,1280), nms=True, int8=True, dynamic=True)
The error traceback is as follows:

Traceback (most recent call last):
File "/kb210/wrs/tennis-analysis/tennis_analysis/yolo_inference.py", line 15, in
model.export(format="onnx", imgsz=(1024,1280), nms=True, int8=True, dynamic=True)
File "/kb210/conda/envs/tennis_analysis/lib/python3.9/site-packages/ultralytics/engine/model.py", line 819, in export
return Exporter(overrides=args, _callbacks=self.callbacks)(model=self.model)
File "/kb210/conda/envs/tennis_analysis/lib/python3.9/site-packages/ultralytics/engine/exporter.py", line 311, in call
validate_args(fmt, self.args, fmt_keys)
File "/kb210/conda/envs/tennis_analysis/lib/python3.9/site-packages/ultralytics/engine/exporter.py", line 171, in validate_args
assert arg in valid_args, f"ERROR ❌️ argument '{arg}' is not supported for format='{format}'"
AssertionError: ERROR ❌️ argument 'int8' is not supported for format='onnx'
From the error message, it seems that the int8=True argument is not supported for ONNX export. However, I believe INT8 quantization could be useful for optimizing models even before deployment to certain runtimes (e.g., TensorRT).

Question:

Is there any plan to support INT8 during ONNX export? If not, what would be the recommended way to apply INT8 quantization after exporting to ONNX?

Environment:

Ultralytics version: 8.3.162
Python version: 3.9.19
PyTorch version: 1.13.1
OS: Linux
Hardware: Intel Xeon E5-2680 v4 2.40GHz
Thanks for your help!

[sidshete]

I have trained model for instance segmentation and I ma trying to export to run on DPU. Here is the code mentioned below

from ultralytics import YOLO

best_weights_path = "../weights/yolo11_s/yolo11_s.pt"
data_yaml = "../yolo_seg/dataset.yaml"

model = YOLO(best_weights_path)

Export for DPU (INT8)

model.export(
format="onnx",
data=data_yaml,
int8=True,
simplify=True,
nms=False
)

I get the following error.

UnpicklingError Traceback (most recent call last)
Cell In[18], line 6
3 best_weights_path = "../weights/yolo11_s/yolo11_s.pt"
4 calib_data_yaml = "../dataset.yaml"
----> 6 model = YOLO(best_weights_path)
8 # Export for DPU (INT8)
9 model.export(
10 format="onnx",
11 data=calib_data_yaml, # calibration dataset required for INT8
(...)
14 nms=False # often applied post-inference
15 )

File ~/testing_env/lib/python3.10/site-packages/ultralytics/models/yolo/model.py:23, in YOLO.init(self, model, task, verbose)
20 self.dict = new_instance.dict
21 else:
22 # Continue with default YOLO initialization
---> 23 super().init(model=model, task=task, verbose=verbose)

File ~/testing_env/lib/python3.10/site-packages/ultralytics/engine/model.py:151, in Model.init(self, model, task, verbose)
149 self._new(model, task=task, verbose=verbose)
150 else:
--> 151 self._load(model, task=task)

File ~/testing_env/lib/python3.10/site-packages/ultralytics/engine/model.py:240, in Model._load(self, weights, task)
237 weights = checks.check_model_file_from_stem(weights) # add suffix, i.e. yolov8n -> yolov8n.pt
239 if Path(weights).suffix == ".pt":
--> 240 self.model, self.ckpt = attempt_load_one_weight(weights)
241 self.task = self.model.args["task"]
242 self.overrides = self.model.args = self._reset_ckpt_args(self.model.args)

File ~/testing_env/lib/python3.10/site-packages/ultralytics/nn/tasks.py:806, in attempt_load_one_weight(weight, device, inplace, fuse)
804 def attempt_load_one_weight(weight, device=None, inplace=True, fuse=False):
805 """Loads a single model weights."""
--> 806 ckpt, weight = torch_safe_load(weight) # load ckpt
807 args = {**DEFAULT_CFG_DICT, **(ckpt.get("train_args", {}))} # combine model and default args, preferring model args
808 model = (ckpt.get("ema") or ckpt["model"]).to(device).float() # FP32 model

File ~/testing_env/lib/python3.10/site-packages/ultralytics/nn/tasks.py:732, in torch_safe_load(weight)
724 try:
725 with temporary_modules(
726 {
727 "ultralytics.yolo.utils": "ultralytics.utils",
(...)
730 }
731 ): # for legacy 8.0 Classify and Pose models
--> 732 ckpt = torch.load(file, map_location="cpu")
734 except ModuleNotFoundError as e: # e.name is missing module name
735 if e.name == "models":

File ~/testing_env/lib/python3.10/site-packages/torch/serialization.py:1524, in load(f, map_location, pickle_module, weights_only, mmap, **pickle_load_args)
1516 return _load(
1517 opened_zipfile,
1518 map_location,
(...)
1521 **pickle_load_args,
1522 )
1523 except pickle.UnpicklingError as e:
-> 1524 raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
1525 return _load(
1526 opened_zipfile,
1527 map_location,
(...)
1530 **pickle_load_args,
1531 )
1532 if mmap:

UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint.
(1) In PyTorch 2.6, we changed the default value of the weights_only argument in torch.load from False to True. Re-running torch.load with weights_only set to False will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
(2) Alternatively, to load with weights_only=True please check the recommended steps in the following error message.
WeightsUnpickler error: Unsupported global: GLOBAL ultralytics.nn.modules.block.C3k2 was not an allowed global by default. Please use torch.serialization.add_safe_globals([ultralytics.nn.modules.block.C3k2]) or the torch.serialization.safe_globals([ultralytics.nn.modules.block.C3k2]) context manager to allowlist this global if you trust this class/function.

[glenn-jocher]

That error is coming from PyTorch 2.6’s new default torch.load(weights_only=True), which blocks custom Ultralytics layers like ultralytics.nn.modules.block.C3k2; if you trust the checkpoint, either pin PyTorch <2.6 or allowlist C3k2 before calling YOLO():

import torch
from ultralytics import YOLO
from ultralytics.nn.modules.block import C3k2

torch.serialization.add_safe_globals([C3k2])  # only if you trust the .pt source
model = YOLO("../weights/yolo11_s/yolo11_s.pt")

Also note that int8=True is not supported for format="onnx" export (ONNX export args don’t include int8), so for a DPU workflow you’ll typically export FP32 ONNX and quantize with your vendor toolchain; see the supported export args in the Export mode docs and ONNX export guide.

If you paste !yolo checks (and restart the notebook kernel if you recently upgraded PyTorch/Ultralytics), I can confirm the exact version combo that’s triggering this on your side.