Ultralytics
tasks/segment/ #8734
giscus[bot]
bot
Replies: 101 comments 233 replies
-
|
I have a question about segmentation dataset and really wish you can answer, my dataset contains lots of information including bbox and segmentations, however it has wrong category_id and all categories have been marked as the same category, does this has bad influence on finetuing? If so, how can I fix that? |
Beta Was this translation helpful? Give feedback.
-
|
@jrryzh hey there! 👋 Absolutely, having incorrect To fix this, you'll need to correct the For instance, if you're working with a JSON format similar to COCO, you could use Python to update the category IDs: import json
# Load your dataset
with open('your_dataset.json') as f:
data = json.load(f)
# Assuming all categories are incorrectly marked as category_id 1 and you want to update them to correct IDs
correct_mapping = {1: 'new_correct_id'} # Update this with your correct category IDs
# Update category IDs
for ann in data['annotations']:
if ann['category_id'] in correct_mapping:
ann['category_id'] = correct_mapping[ann['category_id']]
# Save the corrected dataset
with open('your_corrected_dataset.json', 'w') as f:
json.dump(data, f)Remember to replace If you're working with segmentation, our Instance Segmentation guide might be helpful. It covers how to use our segmentation models, like Hope this helps! Let us know if you have any more questions. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
How can I run my code on GPU ? From the processor the recognition is freezing |
Beta Was this translation helpful? Give feedback.
-
|
@Boaruzhanchik hey there! 😊 To run your code on a GPU, you typically just need to ensure your model is set to use the GPU. With Ultralytics YOLO, your model should automatically utilize a CUDA-compatible GPU if available. Here's a quick check to make sure: from ultralytics import YOLO
# Make sure 'device' is set to 'cuda' if you want to use GPU (default if CUDA is available)
model = YOLO('yolov8n.pt', device='cuda')This small addition directs the model to use the GPU. Make sure you have a CUDA-compatible Nvidia GPU and have installed the necessary CUDA and CuDNN libraries. If recognition is freezing, it's usually due to the heavy computation not being efficiently processed by the CPU. Running it on the GPU should significantly speed things up! 🚀 If CUDA is correctly set up and you're still encountering issues, make sure to check your GPU drivers and CUDA installation. Keep me posted on how it goes or if you run into any further questions! |
Beta Was this translation helpful? Give feedback.
-
|
Hi, i'm install pythorch and have problem when program detect object Traceback (most recent call last): CPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\cpu\nms_kernel.cpp:112 [kernel] |
Beta Was this translation helpful? Give feedback.
-
|
Sorry for the inconvenience, everything was solved, thank you ! |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 Glad to hear everything got sorted out. If you ever run into another hiccup or have any questions, please don't hesitate to reach out. Wishing you continued success with your projects! If instance segmentation or anything else piques your interest, our Ultralytics Docs are there to help guide you along. Cheers! 🎉 |
Beta Was this translation helpful? Give feedback.
-
|
I am building a streamlit app on YOLOv8 models, |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 For displaying the result or the annotated image/video in your Streamlit app using YOLOv8 models, you can follow this approach: After running the prediction with a YOLOv8 model, you get a from ultralytics import YOLO
import streamlit as st
from PIL import Image
import numpy as np
# Load a YOLO model
model = YOLO('yolov8n.pt') # Or your custom model
# Run prediction
results = model('path/to/image.jpg') # Replace with your source
# Convert results image to PIL Image for Streamlit
annotated_img = Image.fromarray(results[0].plot()[..., ::-1]) # Convert BGR to RGB
# Display the image in Streamlit
st.image(annotated_img, caption='Annotated Image')Replace Hope this helps you integrate YOLOv8 with your Streamlit app smoothly! 🚀 Let me know if you have any more questions. |
Beta Was this translation helpful? Give feedback.
-
|
has the ability to use image masks instead of contour points as labels ever been implemented ? |
Beta Was this translation helpful? Give feedback.
-
|
Hello! 👋 Yes, Ultralytics YOLO models have the ability to perform instance segmentation, which can utilize image masks to segment individual objects in an image. This is particularly useful for datasets where contour points might not adequately represent the object due to internal gaps or complex shapes. For instance segmentation, you can use models with the Here's a quick example on how to predict with a pretrained segmentation model: from ultralytics import YOLO
# Load a segmentation model
model = YOLO('yolov8n-seg.pt')
# Predict with the model
results = model('https://ultralytics.com/images/bus.jpg')This will give you both the bounding boxes and segmentation masks for detected objects, handling cases where simple contours are not sufficient. 🎭 Let us know if you need further assistance or examples! |
Beta Was this translation helpful? Give feedback.
-
|
Hi there! I am using the pre-trained segmentation model by Yolov8. I have an image that contains several objects, and I only want to receive the mask of any person present in the frame, while ignoring all the other masks. My requirements:
|
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 For your requirement to only get the person mask from an image using the YOLOv8 pre-trained segmentation model, and ignoring other detections, you can do something like this: from ultralytics import YOLO
# Load the YOLOv8 segmentation model
model = YOLO('yolov8n-seg.pt')
# Perform prediction
results = model('path/to/your/image.jpg')
# Filter for 'person' class masks only (class_id for 'person' is usually 0)
person_masks = results.masks[results.boxes.cls == 0]
# Loop through each person mask to process/visualize as needed
for mask in person_masks:
# Here, 'mask' represents individual person masks
# You can proceed to draw contours or other operations as required
# Save or visualize person_masks as needed, maintaining original image dimensionsThis snippet is a starting point. The By utilizing this approach, you'll be able to:
Hope this helps! Let me know if you need further assistance. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
results.masks gives an attribute error. It was fixed with results[0].masks. I have the tensor containing the mask data of the person through person_masks.data. Just help me save the masked image now, that only contains mask of the detected person. (There is always going to be one person in my usecase) |
Beta Was this translation helpful? Give feedback.
-
|
Hey! 👋 Great to hear you've successfully isolated the person mask using YOLOv8 segmentation! To save the mask image with the same spatial dimensions as the original, here is a quick way: import cv2
import torch
# Assuming `person_masks.data` is your mask tensor
# Convert mask to a binary image (0 or 255), change the shape to match the original image height and width
mask_image = person_masks.data.squeeze().cpu().numpy() * 255 # Adjust the multiplication factor as necessary
mask_image = mask_image.astype(np.uint8)
# Save the mask image
cv2.imwrite('path/to/save/person_mask.jpg', mask_image)This code snippet takes your Let me know if this helps or if you have any further questions! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
I have a question, how do you visualize the segmentation mask with the detected label? it seems it's not discussed in the documentation, it just shows here how to predict using it but not visualizing it with an image. I even checked the list of results but it seems its just returning segmentation masks, not the classes that it predicts, with that said my question is can you visualize the segmentation mask with a label only because of what i see in the tutorial it has a bounding box too which i think that's how you put the label there because i think the segmentation masks are not directly related in the class ID, my goal is to detect object with segmentation and put label to it without bounding box. looking forward to your answer thank you |
Beta Was this translation helpful? Give feedback.
-
|
Hello there! 👋 To visualize the segmentation mask with its corresponding label (without bounding boxes), you can use the from ultralytics import YOLO
import matplotlib.pyplot as plt
# Load a pretrained segmentation model
model = YOLO('yolov8n-seg.pt')
# Predict on an image and return a Results object
results = model('https://ultralytics.com/images/bus.jpg')
# Visualize segmentation with labels (without bounding boxes)
fig = plt.figure(figsize=(10, 10))
plt.imshow(results.render()[0]) # `render()` method overlays masks & class labels onto the image
plt.axis('off')
plt.show()This generates an output where each segmentation mask is color-coded, and the class labels are overlaid directly on the masks. The Unfortunately, our default method integrates both segmentation masks and bounding boxes for comprehensive detection visualization. If you specifically want only the masks with labels and without the bounding boxes in your plots, the code tweak above should help. Hope this answers your question! 😊 If you have any further queries or need more insights, feel free to reach out. |
Beta Was this translation helpful? Give feedback.
-
|
I have a question. Can anyone suggest me how can I save the predicted bounding box and segmented mask in the format of : |
Beta Was this translation helpful? Give feedback.
-
|
@utokyo-sm96 hey there! 😊 To save the predicted bounding box and segmentation mask in your specified format, you can iterate over the detected objects and their corresponding masks to build the desired string for each detection. Below is a Python code snippet using the Ultralytics YOLO model to demonstrate how you could achieve this: from ultralytics import YOLO
# Load your YOLOv8 segment model
model = YOLO('yolov8n-seg.pt')
# Run a prediction
results = model('path/to/your/image.jpg')
# Iterate through results to format and save your detections and masks
for i, det in enumerate(results[0].boxes.xywh):
class_id = int(det[4])
bbox_center_x, bbox_center_y, bbox_width, bbox_height = det[:4]
segment = results[0].masks.xyn[i] # Get mask points for this detection
# Build the desired format string
output_str = f"{class_id} {bbox_center_x} {bbox_center_y} {bbox_width} {bbox_height}"
for x, y in zip(segment[0::2], segment[1::2]): # Iterate over points in mask
output_str += f" {x} {y}"
print(output_str) # Or write to a fileThis code will print out the details for each detection in your specified format. Note that Hope this helps! Let me know if you need further assistance. |
Beta Was this translation helpful? Give feedback.
-
|
When I try to predict some images using the best weights of this trained model (!python3 segment/predict.py --save-crop --save-txt) I expect to get predicted segment coordinates in the saved txt files as: |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 It looks like you're encountering an issue where you're expecting segmentation polygon coordinates in the For including segmentation mask coordinates in your output, you'd need to programmatically access the segmentation masks from the model's output and convert them into your desired format. Here's a quick example on how you might approach it: from ultralytics import YOLO
import numpy as np
model = YOLO('yolov8n-seg.pt') # Load your segment model
results = model('path/to/image.jpg') # Perform inference
for i, (box, mask) in enumerate(zip(results[0].boxes, results[0].masks)):
class_id = box[4].int()
bbox = box[:4] # bbox coordinates (center_x, center_y, width, height)
polygon = mask.to_polygons()[0] # Assuming one polygon per mask for simplification
# Convert polygon to your desired format here
# Example: polygon might need flattening and conversion to (x, y) pairs
flat_polygon = np.array(polygon).flatten()
output_line = f"{class_id} {bbox[0]} {bbox[1]} {bbox[2]} {bbox[3]} " + ' '.join(map(str, flat_polygon))
# Now write output_line to your .txt fileThis is just a starter; you might need to adjust the mask processing part depending on your exact needs. Remember, the actual implementation details may vary as this example assumes direct access to methods for converting masks to polygons which might not exist as is and you may need to implement or adapt them based on the format of the masks. Don't hesitate to reach out again if you need more help! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
` ` if len(det): ` Masks Shape: torch.Size([3, 640, 640]) can you help me out where am i wrong. |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 It looks like your script is running into an issue where it's not properly identifying the segmentation masks, as indicated by "Number of non-zero values in class mask: 0" and "Found 0 polygons." This typically happens if the predicted masks are not properly generated or if there's an issue in the conversion process from the mask tensor to the segmentation polygons. A couple of things you might want to check:
If the masks are indeed being generated but not captured correctly, you might need to adjust the process of converting masks to polygons. Remember, masks are typically provided as binary arrays (or tensors), where you need to find contours and then approximate those contours to polygons. Here's a simplified approach to debugging and trying to fix your issue:
Also, for your specific case of wanting to write the segmentation polygons alongside bounding box details, ensure your Lastly, remember that each segmentation mask corresponds to one detected object, so ensure your processing loop correctly matches each mask with its corresponding detection. If after checking these aspects you still face issues, consider sharing a minimal but complete snippet of how you're processing the image through the model and attempting to save the results. It might offer more clues on where things might be going astray. Hope this helps you to debug the situation! Keep experimenting, and don't hesitate to reach out for more guidance. 😊 |
Beta Was this translation helpful? Give feedback.
-
|
I have an important question, i want going to train yolov8 model for instance segmentation but i need to save all the segmented mask on my disk after training the model. how i can get that |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! Saving segmented masks to disk with Ultralytics YOLO is doable, and arranging them by class labels adds a nice level of organization for further training or analysis. 🧐 Here's a quick guide to help you out:
from ultralytics import YOLO
# Load your trained segmentation model
model = YOLO('path/to/yolov8n-seg.pt')
# Predict on an image, replace with your image path or URL
results = model('path/to/image.jpg')
# Save results, this includes saving masks
results.save()
Here's a very basic starter: import os
import numpy as np
# Assuming 'results' from the prediction step
for det in results.xyxy[0]: # Iterate through detections
class_id = int(det[-2]) # Class ID for detected object
class_name = results.names[class_id] # Class name
mask = results.masks[class_id].numpy() # Convert mask to numpy array
# Create directory for the class if it doesn't exist
os.makedirs(class_name, exist_ok=True)
# Filename for the mask
mask_path = os.path.join(class_name, f"{det[5]}.png") # det[5] is the unique ID, you might adjust this
# Save mask, you may need to adjust the saving depending on mask format
np.save(mask_path, mask)Note: You'll need to adjust the looping and saving part based on your actual needs and mask data structure. Remember, getting into the specifics might require tweaking according to your dataset, and the exact implementation might vary. If you're dealing with a custom dataset or more complex requirements, you might need to dive deeper into handling the output data. Happy coding! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Query: |
Beta Was this translation helpful? Give feedback.
-
|
@utokyo-sm96 hello! For saving instance segmentation predictions, including masks, Here's a quick example on how you might go about this for saving segmentation masks as images, but you can adapt the logic to save in your preferred format: from ultralytics import YOLO
import numpy as np
from PIL import Image
# Load a model
model = YOLO('yolov8n-seg.pt') # Assuming you're using a segmentation model
# Predict with the model
results = model('path/to/image.jpg')
# Save masks
for i, mask in enumerate(results[0].masks):
img = Image.fromarray((mask.numpy() * 255).astype(np.uint8))
img.save(f'mask_{i}.png')In this example, each object's mask is saved as a separate Let us know if you need further assistance! 😊 |
Beta Was this translation helpful? Give feedback.
-
|
hi i am trying to get each mask in a separate photo and i used this code why an i getting this error |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! It looks like you're encountering an error because you're trying to directly operate on the You're on the right track with wanting to convert each mask to an image format! However, you need to access the numpy representation of each mask by iterating over the masks within the from ultralytics import YOLO
from PIL import Image
# Assuming you're using a YOLOv8 segmentation model
model = YOLO('yolov8n-seg.pt')
results = model('path/to/your/image.jpg')
# Iterate over each detected object's mask
for i, mask in enumerate(results[0].masks.numpy()):
# Convert mask to an image
mask_img = Image.fromarray((mask * 255).astype(np.uint8))
# Save each mask as an image file
mask_img.save(f'mask_{i}.png')In this adjusted approach, Hope this helps! Let me know if you have further questions. 😄 |
Beta Was this translation helpful? Give feedback.
-
|
Hello, |
Beta Was this translation helpful? Give feedback.
-
|
Hello 😊! To get the mIoU (mean Intersection over Union) for each epoch end during training/validation with our segmentation models (e.g., from ultralytics import YOLO
# Load your segmentation model
model = YOLO('yolov8n-seg.pt') # using an official segmentation model for example
# Perform validation (after training or directly if the model is pre-trained)
metrics = model.val() # Assumption: model and dataset are correctly configured
# Access mIoU (mean IoU for segmentation)
miou = metrics.seg.map # This is your mIoU value
print(f'mIoU: {miou}')This will give you the mIoU for segmentation masks at the end of each validation phase, which is typically run after each training epoch. Hope this helps! If you have further questions or need more assistance, feel free to ask. 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for sharing, but I don't know how to add MIOU evaluation metrics in v8, if you have time, can you give a detailed guide? Thank you very much! |
Beta Was this translation helpful? Give feedback.
-
|
@handsome2456 hello! 😊 To add mIoU (mean Intersection over Union) evaluation metrics in YOLOv8, you can access these metrics directly after each validation phase. Here's a detailed guide to help you get started: Step-by-Step Guide
Example CodeHere is a complete example that includes training and validation with mIoU metrics: from ultralytics import YOLO
# Load a model
model = YOLO("yolov8n-seg.pt")
# Train the model and validate at the end of each epoch
results = model.train(data="coco8-seg.yaml", epochs=100, imgsz=640)
# Validate the model
metrics = model.val()
miou = metrics.seg.map
print(f'mIoU: {miou}')For more detailed information on instance segmentation and other tasks, you can visit the Ultralytics documentation on segmentation. Feel free to reach out if you have any more questions or need further assistance. Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hi there! I'd like to share with you a project I've recently worked on. Together with a colleague, we've created a repository that serves as a tool with a SAHI-like inference but specifically tailored for instance segmentation tasks. Our repository allows for segmenting small objects in images by combining mask predictions from various overlapping patches. We support both YOLOv8-seg and FastSAM. Additionally, we have a variant for object detection tasks, and the key distinction from SAHI is the support for all the current models from the Ultralytics team: YOLOv9, YOLOv8, RTDTR, and others. I'm a huge fan of Ultralytics, so I'd be thrilled to assist you if you're interested in our project. I'm confident that for many people, the task of finding a large number of segments would be beneficial, especially when using standard Ultralytics models. Here's the link to the project: YOLO-Patch-Based-Inference. Honestly, I wasn't sure where to share information about the project, so I decided to start with the YOLO-seg support section, as it seemed like the most obvious choice. I would greatly appreciate your feedback on the potential usefulness of our project for you. Thank you! Example: |
Beta Was this translation helpful? Give feedback.
-
|
Hello there! Wow, your project sounds absolutely fascinating! 🚀 The idea of tailoring SAHI-like inference for instance segmentation, and supporting an impressive range of Ultralytics models, including YOLOv8-seg and FastSAM, is a commendable effort. It's particularly exciting that you're extending support to the latest models like YOLOv9, RTDTR, and so forth. Your repository is a brilliant testament to the passion and expertise in the community. I'm thrilled to see your innovative use of our models, and I'm confident that your project could significantly benefit those needing refined segmentation, especially for small objects. Your approach to combining mask predictions from overlapping patches is a smart solution to a common challenge in instance segmentation. I've taken a look at your YOLO-Patch-Based-Inference repo, and it’s clearly well-thought-out. The instance segmentation community will surely find tremendous value in the tools you've developed. Sharing it in the YOLO-seg support section was a great choice! If there's anything more we can do to support your project or if you have further questions or insights, feel free to reach out. Feedback like yours is invaluable—it helps us continue to innovate and support projects that enhance the capabilities of Ultralytics models. Once again, thank you for your enthusiasm for Ultralytics and for sharing your project with us. Keep up the fantastic work! 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much for your feedback. |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Thank you so much for reaching out and offering to collaborate on documenting your project for Ultralytics documentation. Your initiative aligns perfectly with our goal of enriching the community with valuable resources and tools. 😊 Given the potential impact and the innovative approach of your SAHI-like tool for instance segmentation, featuring support for Ultralytics models including YOLOv8-seg, FastSAM, and more, it sounds like something that would greatly interest and benefit our users. Documenting your solution in a manner similar to our SAHI guide could indeed foster wider adoption and assist others in their instance segmentation endeavors. Let's connect to discuss how we can work together on creating this documentation. Please feel free to reach out to us via our repository's "Discussions" section or directly via email with a brief outline or proposal for the documentation content. This will help us kickstart the process and determine how best to feature your work within our documentation ecosystem. Again, thank you for your enthusiasm and looking forward to collaborating with you! 🌟 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much. I wrote a proposal in the "Discussions" section - https://github.com/orgs/ultralytics/discussions/9381 |
Beta Was this translation helpful? Give feedback.
-
|
@Koldim2001 that's fantastic! 😊 Your proposal in the "Discussions" section here is a great step forward, and I appreciate your willingness to contribute and help enhance the Ultralytics documentation. Your project's unique approach to instance segmentation, especially for small objects and full support for our model range, sounds like a valuable resource for our users. Collaborating on similar documentation to what we've done with SAHI would not only benefit our community but also raise awareness of your innovative solution. Let's definitely explore this further. Please expect a follow-up in the "Discussions" section as we discuss next steps and how we can integrate your project into our ecosystem effectively. Thank you for reaching out and looking forward to working together! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
YOLO V8 has pre-trained models that are performed on the COCO dataset. However, I am working on a project for image segmentation where the classes are not available in the COCO dataset. In this case, do I need to train a model from scratch? About the dataset: Looking for an answer. Thankyou! |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 😊 It's great that you've already annotated your custom dataset in YOLOv8 format, especially for such an impactful project involving property damage assessment from satellite and aerial imagery. Given your custom classes are outside the COCO dataset, you're on the right track thinking about training a model specifically for your use case. The good news is, you don't need to start completely from scratch! You can leverage one of our pre-trained segmentation models like Here's a quick example of how you might kick off your training process: from ultralytics import YOLO
# Initialize your model for training
model = YOLO('yolov8n-seg.pt') # Load a pre-trained segmentation model
# Fine-tune the model on your custom dataset
results = model.train(data='your_custom_dataset.yaml', epochs=100, imgsz=640)Ensure your dataset YAML file is correctly set up with paths to your training and validation data, classes, etc. This method will help your model learn the specific classes and damage levels you're interested in! Best of luck with your project, and don't hesitate to reach out if you have more questions. Happy detecting! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Thanks a lot again. |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 👋 Since your custom dataset annotations are ready and in the proper YOLOv8 format, you're all set to start fine-tuning one of our pre-trained segmentation models, such as from ultralytics import YOLO
# Load the pre-trained model
model = YOLO('yolov8n-seg.pt')
# Fine-tune the model on your custom dataset
results = model.train(data='your_custom_dataset.yaml', epochs=100, imgsz=640)Make sure your |
Beta Was this translation helpful? Give feedback.
-
|
Is Panoptic Segmentation supported by Yolov8 or in any version of Yolo? |
Beta Was this translation helpful? Give feedback.
-
|
Hello! At the moment, YOLOv8 specifically supports Instance Segmentation with models like For panoptic segmentation, which combines both semantic segmentation (segmenting all pixels in an image) and instance segmentation (distinguishing between individual object instances), we're always exploring new features and capabilities to incorporate into our models. Your interest is definitely noted! 🚀 If you're specifically looking to work with panoptic segmentation, I'd encourage staying tuned to our updates. Also, feel free to contribute or suggest improvements; we welcome collaboration from the community! 🤝 Here's a quick example of how you can work with our segmentation models: from ultralytics import YOLO
# Load a pretrained segment model
model = YOLO('yolov8n-seg.pt')
# Segment objects in an image
results = model('https://ultralytics.com/images/bus.jpg')
# Visualize results
results.show()Let us know if there's anything else we can help you with! |
Beta Was this translation helpful? Give feedback.
-
|
I have trained custom yolov8 model for Instance segmentation on video. Now I have to make video inference for the same please guide me. And I have to deploy it in mobile application which was developed in flutter give me guidance on these for Yolov8 .tflite format model for instance segmentation. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @mayurkatre18, Thank you for reaching out! It's fantastic to hear that you've successfully trained a custom YOLOv8 model for instance segmentation on video. Let's guide you through the process of making video inferences and deploying your model in a Flutter mobile application. Video InferenceTo perform video inference with your trained YOLOv8 model, you can use the following Python code: from ultralytics import YOLO
# Load your trained model
model = YOLO('path/to/your/custom-model.pt')
# Predict on a video
results = model('path/to/your/video/file.mp4')
# Iterate over results if needed
for result in results:
# Process your results here
result.show() # Display the frame with predictionsThis code will load your custom model and run predictions on each frame of the video. If you need to process real-time video streams, you can read frames from the stream and pass them to the Exporting to TensorFlow Lite (.tflite)To deploy your model in a Flutter mobile application, you need to export it to TensorFlow Lite format. Here’s how you can do it: from ultralytics import YOLO
# Load your trained model
model = YOLO('path/to/your/custom-model.pt')
# Export the model to TensorFlow Lite format
model.export(format='tflite')This command will create a Integrating with FlutterOnce you have the
dependencies:
tflite: ^1.0.3
import 'package:flutter/material.dart';
import 'package:tflite/tflite.dart';
void main() => runApp(MyApp());
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
home: Scaffold(
appBar: AppBar(
title: Text('YOLOv8 Instance Segmentation'),
),
body: Center(
child: Text('Running YOLOv8 Model'),
),
),
);
}
}
class MyHomePage extends StatefulWidget {
@override
_MyHomePageState createState() => _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
@override
void initState() {
super.initState();
loadModel();
}
Future<void> loadModel() async {
String res = await Tflite.loadModel(
model: "assets/yolov8n-seg.tflite",
labels: "assets/labels.txt",
);
print(res);
}
Future<void> runInference() async {
var recognitions = await Tflite.detectObjectOnImage(
path: "path/to/your/image.jpg",
model: "YOLO",
threshold: 0.5,
);
print(recognitions);
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text('YOLOv8 Instance Segmentation'),
),
body: Center(
child: Text('Running YOLOv8 Model'),
),
);
}
}This example demonstrates how to load the TensorFlow Lite model and run inference on an image. You can extend this to process video frames as needed. For more detailed instructions on exporting models and integrating them into mobile applications, please refer to the Ultralytics documentation. If you encounter any issues or need further assistance, feel free to ask. Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Hi, |
Beta Was this translation helpful? Give feedback.
-
|
Hi @mayurkatre18, Thank you for reaching out! 😊 To perform video inference using a Step 1: Install TensorFlow LiteFirst, ensure you have TensorFlow Lite installed: pip install tflite-runtimeStep 2: Load and Run Inference on Video FramesHere's a Python script to load your import cv2
import numpy as np
import tflite_runtime.interpreter as tflite
# Load the TFLite model and allocate tensors
interpreter = tflite.Interpreter(model_path="path/to/your/model.tflite")
interpreter.allocate_tensors()
# Get input and output tensors
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Open the video file
cap = cv2.VideoCapture("path/to/your/video.mp4")
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
# Preprocess the frame
input_data = cv2.resize(frame, (input_details[0]['shape'][2], input_details[0]['shape'][1]))
input_data = np.expand_dims(input_data, axis=0)
input_data = input_data.astype(np.float32) / 255.0
# Set the tensor to point to the input data to be inferred
interpreter.set_tensor(input_details[0]['index'], input_data)
# Run the inference
interpreter.invoke()
# Get the results
output_data = interpreter.get_tensor(output_details[0]['index'])
# Process the results (e.g., draw bounding boxes and masks on the frame)
# This part will depend on your specific model's output format
# Display the frame with predictions
cv2.imshow('Frame', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()Step 3: Deploying in FlutterTo deploy your
For more detailed instructions, you can refer to the TensorFlow Lite Flutter Plugin documentation. If you encounter any issues or need further assistance, feel free to ask. Happy coding! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
I have trained YOLOv8 segmentation model in .tflite format. Now I have to integrate it into mobile application with flutter langauge. For this I have to make all configurations in laptop only to make integration of video and providing result and after this we have to make an API to have integration of this video processing in mobile app. Please provide the neccesary steps. |
Beta Was this translation helpful? Give feedback.
-
|
To integrate your YOLOv8 |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I have a question. I have trained a custom model based on a pre trained one to segment the shirts from cyclists. The problem however is that it recognizes 2 slightly different shirt on 1 image. My idea was to only use the one with the highest confidence score, but this object appears to be none (results.probs or results.keypoint are both none objects). Is it possible to get the confidence score or do I have to apply a different methode? |
Beta Was this translation helpful? Give feedback.
-
|
@pietervhij hey there! 👋 With only 100 training images, there's indeed a risk of overfitting when training over a large number of epochs like 100, especially for complex tasks like instance segmentation. Here are a couple of tips to mitigate that:
Training on a smaller Keep innovating! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the response, i have a little followup question. You say Ultralytics supports augmentations directly in the training pipeline. What do you mean by this? Is there a parameter that could implement this? Or is it you should adjust your dataset beforehand? |
Beta Was this translation helpful? Give feedback.
-
|
Hey there! 🤗 Great follow-up question. Ultralytics YOLO integrates data augmentations directly into the training pipeline, meaning you don't have to modify your dataset beforehand. These augmentations are applied on-the-fly during training to enhance the diversity of your dataset and improve model generalization. When you train your model using the No need to set a specific parameter for basic augmentations; they're handled for you by default! However, you can customize augmentations to some extent using the training configuration file ( If you've missed it, don't worry. It's a fairly common question, and the details are indeed more implicit within the documentation. Always feel free to ask for clarifications! 😄 Happy training! |
Beta Was this translation helpful? Give feedback.
-
|
So if I understand correctly, when i train my model with following code, the ultralytics YOLO is already implementing augmentations? from ultralytics import YOLO model = YOLO("yolov8n-seg.pt") # load a pretrained model (recommended for training) freeze_layers = 20 results = model.train(data="shirt.yaml", epochs=50) # train the model The yaml file looks as following: train: "train/images" roboflow: |
Beta Was this translation helpful? Give feedback.
-
|
Yes, you've got it right! 😊 When you train your model with the Ultralytics YOLO framework, data augmentations are indeed automatically applied during training. This helps enhance the diversity of your dataset, which is especially valuable for small datasets or to improve model generalization. Here's a quick look at your training setup: from ultralytics import YOLO
model = YOLO("yolov8n-seg.pt") # Pretrained model loaded
# ... your freezing logic ...
results = model.train(data="shirt.yaml", epochs=50) # Augmentations are applied here!In this setup, augmentations such as random scaling, cropping, and flipping are employed. This is seamlessly handled by the Ultralytics framework, so there's no need for manual augmentation of your dataset before training. Your training configuration through Keep up the great work on your project! If you have more questions or need further assistance, feel free to ask. |
Beta Was this translation helpful? Give feedback.
-
|
Hey, Would like to ask a few questions I encountered in my tinkering with the segmentation task -
d] How should I freeze the pre-trained weights you learned, if I would like to keep the feature extraction (backbone) of your model? Thanks alot! |
Beta Was this translation helpful? Give feedback.
-
|
@eladoni1 here's a concise response to your segmentation questions: 1. Key Parameters: 2a. Data Representation: Correct - masks are stored as normalized XY contours (0.0-1.0 relative to image size). See Dataset Guide. 2b. Overlap Handling: When 2c. Mask Ratio: 2d. Freezing Backbone: Use 2e. Training Process: Our pretrained models use the commands shown in Train YOLO11n-seg on COCO8 with default hyperparameters. Full training configuration details are in Configuration docs. For commercial use, remember all Ultralytics integrations require an Enterprise License unless fully open-sourced under AGPL-3.0. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @glenn-jocher, Let's just recap the things I understood from this -
What do you mean by 'smaller mask overlaid on the top'? you mean that you combine the masks in the training? Really want to understand what 'overlap_mask' does. 2c. After reading YOLACT, I understood that if I trained the model in 640x640, and put a factor of 4 in mask_ratio, then 640 / 4 = 160, and this is the size the mask should be - 160x160. 2d. Yea, but I would like to know the amount of the first layers related to the backbone of the model - is it 10 like the YAML says? or is there more hidden layers? 2e It doesn't give an idea about what were your parameters in your training of the model. ANOTHER questions to top it off with - Thanks |
Beta Was this translation helpful? Give feedback.
-
|
And, would like to ask another question - Why do I get prototype masks that are not between 0.0 to 1.0, and why do I get coefficients that are not either between 0.0 to 1.0? |
Beta Was this translation helpful? Give feedback.
-
|
Here's a concise response to your segmentation questions: 2b: During training with 2d: The backbone typically comprises the first 10 layers in YOLO11 architectures. You can verify this in the model YAML configuration file and freeze using 2e: Our pretrained models use the default training configuration shown in the Train YOLO11n-seg example, with COCO pretraining followed by custom dataset fine-tuning. 3a: The 32 prototype masks represent learned feature basis vectors that combine through the mask coefficients to produce final instance masks, as described in the YOLO11 architecture. 3b: Prototype masks and coefficients use unbounded values since they're intermediate network outputs. Final masks are normalized via sigmoid activation (0-1 range) during post-processing. For commercial implementations, remember all integrations require an Enterprise License unless fully open-sourced under AGPL-3.0. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I have a question. I would like to calculate the Dice score using our predicted masks and ground truth masks while working with the YOLO11n-seg model. Could you help me with this? Thank you very much in advance! |
Beta Was this translation helpful? Give feedback.
-
|
@cbasquitt to calculate the Dice score with YOLO11n-seg masks, you'll need to manually compare your predicted binary masks against ground truth masks. After prediction, access the mask tensor with def dice_score(pred, gt):
intersection = (pred & gt).sum()
return (2.0 * intersection) / (pred.sum() + gt.sum())For full implementation details, see our Segmentation Predict documentation on accessing mask outputs. |
Beta Was this translation helpful? Give feedback.
-
|
Question 1: The resolution of my dataset is 1280x720 and the target is in the middle of the image, if I set imgsz=640 during training, is it taking 640x640 from the middle of the image? Or is there another way to do it? |
Beta Was this translation helpful? Give feedback.
-
|
@LwpZoe for Q1: The model automatically resizes images to |
Beta Was this translation helpful? Give feedback.
-
|
i have a problem with my meshes. i have images which are about 2000*6000 pixels in size. i have trained a mesh with 1000 images with a resolution of 1920p. however, i don't get everything segmented. every now and then it just cuts it off. and as soon as i lower the resolution during inference it recognises more, but not as sharp as before. what could be the reason for this? |
Beta Was this translation helpful? Give feedback.
-
|
@dembski21 thanks for your question! This behavior often relates to input resolution mismatches between training and inference. Since YOLO11 models process fixed-size inputs (like 640x640), high-resolution images are downsampled, which can cause missed detections if objects become too small. Lower resolutions increase receptive field but reduce mask precision. Two solutions to try:
For optimal results, consider training on higher-resolution data with mosaic augmentation enabled. Let us know if you need more specific guidance! 🚀 |
Beta Was this translation helpful? Give feedback.
-
|
how can i perform the tile inference? |
Beta Was this translation helpful? Give feedback.
-
|
For tile inference with your high-resolution microscope images, use the For training welds (single class), we recommend: |
Beta Was this translation helpful? Give feedback.
-
|
thanks! |
Beta Was this translation helpful? Give feedback.
-
|
? |
Beta Was this translation helpful? Give feedback.
-
|
Hi there! |
Beta Was this translation helpful? Give feedback.
-
|
@mw-boop Hello! To ensure your YOLO11 segmentation model predicts only one object per frame, you can use the Here's how you can apply it: CLI: yolo segment predict model=your_model.pt source=your_image.jpg max_det=1Python: from ultralytics import YOLO
# Load your trained segmentation model
model = YOLO('your_model.pt')
# Predict with max_det=1
results = model('your_image.jpg', max_det=1)
# Process results
# ...This approach leverages the model's confidence scores, typically selecting the object with the highest confidence if multiple potential detections exist, but ultimately capping the output at one. You can find more details on prediction arguments in the Predict Mode documentation. |
Beta Was this translation helpful? Give feedback.
-
|
For the Python API, is there a way to disable the segmentation output to save time but just use the bounding boxes? |
Beta Was this translation helpful? Give feedback.
-
|
set |
Beta Was this translation helpful? Give feedback.
-
|
@shaform yes, you can disable the segmentation masks while keeping the bounding box predictions by setting from ultralytics import YOLO
# Load a segmentation model
model = YOLO('yolov8n-seg.pt')
# Run prediction with masks disabled
results = model('path/to/image.jpg', masks=False)
# Access only the bounding boxes
boxes = results[0].boxesThis approach is particularly useful when you only need object locations but not their exact shapes, resulting in faster inference speeds. For more details on available prediction arguments, check out the Predict Mode documentation. |
Beta Was this translation helpful? Give feedback.
-
|
Just to confirm, while auto annotating, instead of using SAM one can also use one's custom build model. right ?? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, you can annotate using custom build model but please check the annotation class id after annotation. |
Beta Was this translation helpful? Give feedback.
-
|
Good day all, I am working on a project with instance segmentation, I want to get the area of the segmented mask in cm2 and the depth. What’s the best approach for this? Accuracy really matters here. Thabk you |
Beta Was this translation helpful? Give feedback.
-
|
Can I use YOLOv9 and YOLOv12 segmentation using: I have tried using this for one of my semantic segmentation task. I am also not sure about the evaluation metrics on training the segmentation model. I have seen mAP for segmentation as well. I want to use Precision, |
Beta Was this translation helpful? Give feedback.
-
|
Hello Ultralytics team, I would appreciate your clarification on the loss functions used in the following segmentation models: YOLOv8-seg, YOLOv9-seg and YOLO11-seg Specifically, could you please specify which types of losses are applied to the bounding box, classification, and segmentation (mask) branches in each version? Thank you very much for your time and support. |
Beta Was this translation helpful? Give feedback.
-
|
@cyberpunk-edgerunner yes, from ultralytics.data.annotator import auto_annotate
auto_annotate(data="path/to/images", det_model="path/to/best.pt", sam_model="sam_b.pt")@esssyjr once you have a binary mask, the area in cm² is @ratom you can load YOLOv9/YOLO12 segmentation models exactly as you wrote, as long as you have the corresponding @mobin3399 across YOLOv8‑seg, YOLOv9‑seg and YOLO11‑seg the structure is similar: the box branch uses an IoU‑style regression loss combined with a distribution‑based loss on box offsets, the classification branch uses binary cross‑entropy with logits, and the mask branch uses a binary cross‑entropy loss between predicted prototype‑based masks and target masks; YOLOv9 mainly adds the PGI training scheme, while YOLO11 adjusts architecture and loss weighting, so for exact formulas in your version the most reliable reference is the |
Beta Was this translation helpful? Give feedback.
-
|
Dear All, I'm using Yolo8n to classify only persons. I'm using an NVIDIA Jetson Orin Nano for inference and a Raspberry Pi for alarm activation. My question about segmentation is: Can I use segmentation to identify a person when they are more than 20 m from the camera where the event happened? If yes, could you guide me on how to do that? At this distance, I can't yet detect the person. The event must be detected at night, not in daylight, using night-vision cameras. |
Beta Was this translation helpful? Give feedback.
-
|
zoom in
…On Thu, 15 Jan 2026 at 21:22, Andrés Rojas ***@***.***> wrote:
Dear All, I'm using Yolo8n to classify only persons. I'm using an NVIDIA
Jetson Orin Nano for inference and a Raspberry Pi for alarm activation. My
question about segmentation is: Can I use segmentation to identify a person
when they are more than 20 m from the camera where the event happened? If
yes, could you guide me on how to do that? At this distance, I can't yet
detect the person. The event must be detected at night, not in daylight,
using night-vision cameras.
—
Reply to this email directly, view it on GitHub
<#8734 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ATLJGEHOEPDYVM6H6WPC2LL4G6ZUTAVCNFSM6AAAAABEKFCKICVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKNJQHA2TKMY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
If the person isn’t detectable at ~20 m, switching to segmentation won’t fix it (seg models still need enough pixels/features to localize the person first), so the real lever is exactly what you noted: zoom/narrower FOV (longer focal length) and/or higher input resolution plus fine-tuning on your night-vision data; once detection is solid you can move to masks via the instance segmentation docs (or add tracking via the instance segmentation + tracking guide). yolo detect predict model=yolo26s.pt source=your_video.mp4 imgsz=1280 classes=0
yolo segment predict model=yolo26n-seg.pt source=your_video.mp4 imgsz=1280 classes=0 |
Beta Was this translation helpful? Give feedback.
-
|
Hello, I am trying to train on a pre-trained yolo segmentation model. I'm using yolo26n-seg.pt and a dataset from roboflox. I keep getting a no labels found warning. I have verified that the folder structure is as it needs to be - images > train, val and labels > train, val. I've verified that file stem names are the same in image and label folders, and I've visually verified that the labels are polygons outlining book spines. The output I'm getting includes:
Could you please help me debug this? |
Beta Was this translation helpful? Give feedback.
-
|
That warning typically means the loader is discarding all your mask labels as invalid/empty (common with Roboflow exports if polygon coords are not normalized to yolo checksAfter you fix the label format, delete any cached label files and rerun: find path/to/dataset/labels -name "*.cache" -deleteFor reference, each segmentation row must be |
Beta Was this translation helpful? Give feedback.
-
|
Hello Glenn, thank you for the response. Here's a full line from one txt file: Here's my data.yaml file: Directoriestrain_label_dir: labels/train/ names: OS macOS-15.7.3-arm64-arm-64bit numpy ✅ 2.2.6>=1.23.0 Thank you |
Beta Was this translation helpful? Give feedback.
-
|
to @ALL
May I ask a general question,
What is the effect of reducing the image size on classification accuracy.
Suppose my input images are 16:9 (or 4:3) aspect ratio images taken from
mobile camera ( shape is either 2000x4000 or 3000x4000).
I want to build a classification model. I want to know the effects of input
images resolution on the model accuracy.
What will happen if I keep the input imgsz to
1. At imgsz = 640
2. at imgsz = 1280
3. At imgsz = 320 (for smaller VRAM GPUs)
Which one should I prefer or if any of you have already tested this please
tell your logic behind resizing images.
I don't just want to know the appropriate value for imgsz, I want to know
the whole rationale behind it with proper logic.
Thank you in advance for reading this.
…On Fri, 30 Jan, 2026, 4:08 pm Glenn Jocher, ***@***.***> wrote:
That warning typically means the loader is discarding all your mask labels
as invalid/empty (common with Roboflow exports if polygon coords are not
normalized to [0,1], the row has an odd token count, or class ids fall
outside 0..nc-1), which then also triggers the zero-size array... error
during label plotting—please paste (as text) one full line from a
labels/train/*.txt file and your data.yaml, and run this and share the
output:
yolo checks
After you fix the label format, delete any cached label files and rerun:
find path/to/dataset/labels -name "*.cache" -delete
For reference, each segmentation row must be <class-index> <x1> <y1> <x2>
<y2> ... with coordinates normalized and at least 3 (x,y) points—see the Segmentation
dataset format <https://docs.ultralytics.com/datasets/segment/> and the
end-to-end training example in the Instance Segmentation docs
<https://docs.ultralytics.com/tasks/segment/>.
—
Reply to this email directly, view it on GitHub
<#8734 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ATLJGEFBDQ3D2POWDNVSXJD4JMYA7AVCNFSM6AAAAABEKFCKICVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKNRUHE3DQOA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
hello if I wanted to use your segmentation model for inference on a live camera preview, to only detect a single item at a time, how would I go about it? thank you. |
Beta Was this translation helpful? Give feedback.
-
|
Yes — use a from ultralytics import YOLO
model = YOLO("yolo26n-seg.pt")
results = model.predict(source=0, stream=True, classes=[0], max_det=1) |
Beta Was this translation helpful? Give feedback.
-
|
Ok thx. I am actually trying to implement edge detection on android devices for rectangular objects. I have tried OpenCV but it's not enough. So I was thinking to use a segmentation model for my rectangular processor (overlay would only show on rectangular objects) but any time I try to implement this on android, I get serious lagging issues on the camera preview. Plus honestly new to this AI stuff. |
Beta Was this translation helpful? Give feedback.
-
|
If you only need a rectangle-style overlay, a segmentation model is usually heavier than necessary on Android; I’d start with a small OBB or detect model instead, export it with Export mode, and keep |
Beta Was this translation helpful? Give feedback.
and others
Uh oh!
There was an error while loading. Please reload this page.
-
tasks/segment/
Learn how to use instance segmentation models with Ultralytics YOLO. Instructions on training, validation, image prediction, and model export.
https://docs.ultralytics.com/tasks/segment/
Beta Was this translation helpful? Give feedback.
All reactions