tasks/detect/

[giscus[bot]]

tasks/detect/

Official documentation for YOLOv8 by Ultralytics. Learn how to train, validate, predict and export models in various formats. Including detailed performance stats.

https://docs.ultralytics.com/tasks/detect/

[Cheterizzin]

Will color differences in similar objects have an effect? For example, I created a model using black horse data and I will use it for white horses detection

[pderrenger]

Hey there! 🌟

Color differences can indeed affect the performance of your model, especially if it was trained on data that significantly differs in color from the data it's being used to detect. In your case, training on black horses and detecting white horses might pose some challenges due to the color contrast.

However, YOLO models are quite robust and can generalize well if trained with a diverse dataset. If you're seeing issues, consider augmenting your training dataset with images of horses in various colors, lighting conditions, and backgrounds. This can help improve the model's ability to detect horses regardless of their color.

Here's a quick example of how you might augment your data using Python:

from torchvision.transforms import ColorJitter
from PIL import Image

# Load your image
image = Image.open('path/to/your/horse/image.jpg')

# Apply color jitter augmentation
augmented_image = ColorJitter(brightness=0.5, contrast=0.5, saturation=0.5, hue=0.5)(image)

augmented_image.save('path/to/save/augmented_image.jpg')

This code snippet randomly changes the brightness, contrast, saturation, and hue of the image, which could help your model generalize better to different horse colors.

Hope this helps! Let us know if you have any more questions. Happy detecting! 🐎

[sh3mmy]

I've finished trained my v8 detection model and i wanted to used a 3d camera (intel Realsense d435) to do inference on live data and then find the XYZ of the the centroid of the object boundary box in reference to the camera origin. How would i do this?

[pderrenger]

Hey, thanks for reaching out and experimenting with the code! 🌟 It looks like there's some confusion regarding accessing the detection confidence. The error you encountered is due to trying to access an index that doesn’t exist in the detection tensor.

In YOLOv8, the confidence score is actually at det[4], not det[3]. The correct line of code to get xmin, ymin, xmax, ymax, and confidence is:

xmin, ymin, xmax, ymax, conf = map(int, det[:4].tolist() + [det[4]])

The structure for each detection det typically is [xmin, ymin, xmax, ymax, confidence, ...]. Make sure that when accessing elements, the correct indices are used according to the structure returned by your model's prediction.

If you're using a custom structure or have further processed the detection output, make sure to adjust the indices accordingly. Feel free to share more details if you keep running into issues, and we'd be happy to dive deeper into this. Keep up the great work! 💪

[shy0625]

but did you ran it by yourself? actually, I just copied and ran it and got that error : )

[shy0625]

@tomershukhman shared the same error ('det[4] is out of bounds') on Mar 14

[tiffanytgr]

if we want to use a live camera, will converting the images to grayscale, then labelling them and training them in grayscale help to reduce inference time?

[pderrenger]

Hi there! 👋

Thank you for your question. Converting images to grayscale and training a YOLO model on grayscale images can indeed reduce the computational load, potentially speeding up inference time. However, this approach comes with trade-offs:

Pros:

1. Reduced Computational Load: Grayscale images have only one channel compared to three in RGB, which can reduce the amount of data the model needs to process.
2. Faster Inference: With less data to process, the model can run faster, which is beneficial for real-time applications.

Cons:

1. Loss of Information: Grayscale images lack color information, which can be crucial for distinguishing between certain objects. This might lead to a decrease in detection accuracy.
2. Model Training: The model might need to be retrained on grayscale images to adapt to the new input format, which can be time-consuming.

Implementation:

If you decide to proceed with grayscale images, you can convert your images to grayscale before feeding them into the model. Here’s a quick example of how you can do this in Python:

import cv2
from ultralytics import YOLO

# Load your trained model
model = YOLO('path/to/your/model.pt')

# Capture video from the camera
cap = cv2.VideoCapture(0)

while True:
    ret, frame = cap.read()
    if not ret:
        break

    # Convert the frame to grayscale
    gray_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Run YOLOv8 inference
    results = model(gray_frame)

    # Display the results
    cv2.imshow('YOLOv8 Inference', results[0].plot())

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Conclusion:

While converting to grayscale can speed up inference, it's essential to weigh the potential loss in detection accuracy. If color information is not critical for your application, this approach might be beneficial. Otherwise, consider optimizing other aspects of your pipeline for better performance.

Feel free to experiment and see what works best for your specific use case. If you encounter any issues or have further questions, don't hesitate to ask! 😊

[tumitoto007]

So is there a way to 'lock on' one particular detected object? Say I have the yolov8 detected 10 objects, my program then go through these objects and determine one particular object I want to do a 'lock' on it. Is there a way I can use yolov8 from that point to ignore the rest of the detected objects and just keep tracking/locking on the particular object I want? Thanks in advance.

[glenn-jocher]

@tumitoto007 hey there! 👋

Absolutely, you can focus on a specific detected object with YOLOv8. After detecting objects, you can filter out the one you're interested in based on its class, confidence score, or any custom criteria you define. Once identified, you can track this object across frames.

Here's a quick example on how you might do this:

from ultralytics import YOLO

# Load your model
model = YOLO('yolov8n.pt')

# Run detection
results = model('path/to/your/image.jpg')

# Assuming you've identified your object of interest as the first detected object
object_of_interest = results.boxes[0]

# Now, you can track this object in subsequent frames
# For tracking, you might need to implement or use a tracking algorithm

For continuous tracking, you'd typically integrate a tracking algorithm that takes the initial detection as input and follows the object in subsequent video frames. YOLOv8 can detect objects in each frame, and your tracker can help maintain the identity of the object of interest.

Hope this helps! Let us know if you have any more questions. 😊

[tumitoto007]

It looks like I can not tell Yolov8 to stop other detections and just focus/locking on a particular detected object in real-time. Yes, it seems that a different tracking/locking algorithm needs to be in place for locking before I am done with it. I sure wish Yolov could have this ability to do so, I guess I can put this in a wish list.... Thanks for your reply.

[pderrenger]

Hey there! 😊

I understand you're looking for a way to focus or lock on a specific detected object in real-time with YOLOv8. Currently, YOLOv8 excels in detecting multiple objects simultaneously but doesn't natively support locking onto a single object through its detection phase.

For your use case, integrating a tracking algorithm post-detection could be a viable solution. You might consider using the tracking capabilities provided by YOLOv8, such as with the track mode, which can help maintain the identity of objects across frames. Here's a quick example on how you might set it up:

yolo track model=yolov8n.pt source='path/to/video.mp4'

This approach allows you to keep track of specific objects once they're detected. Adding a feature to lock onto a single object directly within YOLOv8 is indeed an interesting idea for our wishlist! 🌟

Thanks for your suggestion, and we always appreciate feedback that can help improve YOLOv8. Let us know if you have any more questions or ideas!

[tumitoto007]

Yes, thank you for your suggestion. I realized the tracking option in Yolov8 is there but still, it tracks multiple instead of just one. My program still needs filter logic to catch what I wanted. I am not sure how Yolov internal works but I presume excluding object by tracking ID will be much more faster than my python filtering logic...anyway, there will be a ton of applications can use the locking facility if provided, I sincerely wish you all can put this into you upcoming enhancements. Thanks for the reply.

[pderrenger]

Hello! Thank you for sharing your thoughts and suggestions regarding the tracking feature in YOLOv8. 🌟

Indeed, the current tracking implementation provides multiple object tracking capabilities. However, I understand your requirement for a more focused tracking approach, possibly tracking a single object or filtering based on specific criteria.

While we continuously work on enhancing YOLOv8's features, including tracking, your suggestion for a "locking" mechanism to track specific objects is quite interesting and could indeed be beneficial for various applications. I'll make sure to relay your feedback to our development team for consideration in our roadmap of upcoming enhancements. 🚀

In the meantime, for a more immediate solution, you might consider using the tracking IDs returned by the tracker to filter out the desired object in your Python code. Here's a quick example of how you might implement such filtering:

from ultralytics import YOLO

# Load your model
model = YOLO('path/to/your/model.pt')

# Run tracking on your video/source
results = model.track('path/to/your/video.mp4')

# Assuming you want to track the object with ID = 1
desired_track_id = 1

# Filter results for your desired object
filtered_results = [res for res in results if res.boxes.id == desired_track_id]

# Now, `filtered_results` will only contain the frames and data for your desired object

This approach allows you to leverage the existing tracking IDs to focus on specific objects, albeit through post-processing.

Thank you again for your valuable feedback, and please stay tuned for future updates! If you have any more questions or suggestions, feel free to reach out. Happy coding! 😊

[Abhinav-Singh-Bisht]

Hey I am very much confused with the overall working of the YOLO model . I mean like when we say each cell is responsible for predicting an image what it actually means in terms of workflow and like what is the concept of anchor boxes like what do you mean by predefined anchor boxes.

What i understand is that we have a image like (480,480,3) we finally convert it to like (7,7,30) by passing through convolutional layers / activation layers / pooiling layers . Assuming we divided it into 7 * 7 grid and for each cell we have a structure like lets say one anchor box x,y,w,h,conf,classess) but it is all prediction right ? we didnt defined anything so what does predefined anchor boxes mean?

Also want to know one thing how are we suppossed to do transfer learning in object detection i dont get it .
Transfer learning what i know is lets say for image classification there is a feature extractor + classifier so in transfer learning we use pre -trained feature extractor like vgg16 and then create our own classifier and in case of fine tuning we even train few layers of feature extractor.

But how do we use transfer-learning in object detection i still don get it ? how is it divided into different stages ? Also in this doc for training yolov8 on custom dataset we are training all the layers which isnt transfer learning right and training all layers on a very small dataset wouldnt make sense right ?

Can you please please please clear my confusion and share some updated/latest resources which could help me with this . I really need help in this.

[glenn-jocher]

Hey there! 😊 Let's try to clear up some of the confusion around YOLO and transfer learning in object detection.

1. YOLO Cells & Anchor Boxes:

   • In YOLO, the image is divided into a grid (e.g., 7x7). Each cell in the grid predicts bounding boxes and their corresponding class probabilities. The idea is that each cell is responsible for predicting objects whose center falls within it.
   • Anchor boxes are predefined bounding boxes of certain heights and widths that are designed to capture the scale and aspect ratio of specific object classes. During training, each cell learns to adjust these anchor boxes to fit the actual objects in the image.

2. Transfer Learning in Object Detection:

   • Similar to image classification, transfer learning in object detection involves using a pre-trained model (like YOLO) and fine-tuning it on your custom dataset. The pre-trained model has learned features from a large dataset that can be useful across many tasks.
   • In YOLO, the feature extractor part (backbone) is pre-trained, and you can fine-tune the entire model or parts of it on your dataset. This is beneficial because the model has already learned a good representation of various features, which can be a great starting point for detecting objects in a new dataset.

3. Training on a Small Dataset:

   • When training on a small dataset, it's crucial to leverage transfer learning to avoid overfitting. By starting with a model pre-trained on a large dataset (like COCO), you're not starting from scratch. Instead, you're adapting a model that already understands a lot about images to work on your specific task.
   • Fine-tuning a few layers or even the entire model (depending on your dataset size and similarity to the original dataset) can be very effective. The key is to use a lower learning rate and potentially freeze some of the earlier layers.

For training YOLOv8 on a custom dataset, you're indeed leveraging transfer learning by starting with a model pre-trained on COCO and then training it further on your dataset. This approach is beneficial even for small datasets because the model has already learned a significant amount of information about various objects, which helps in better generalization.

For more detailed guidance, the Ultralytics YOLO documentation is a great resource. It covers everything from training to exporting models, and it's constantly updated with the latest information.

I hope this helps clarify things! If you have more questions, feel free to ask. Happy learning! 🚀

[Abhinav-Singh-Bisht]

Hey, thanks a lot for this explanation but like i still have doubts so

1. What does pre-defined anchor boxes mean is it like we already have ax,ay,aw,ah i.e points and widht , height hard-coded values for each cell ? and then in the network that 7730 contains just the tx,ty,tw,th using which calculate the bx,by,bw,bh by a formula something like bx = ax + etx something likw this and then we use mean squarred error to calculate loss for bounding box regression and also for classes we have another loss function and then we combine these loss function and using a optimizer we then optimize the weights of our model ?

2. Transfer learning what i assume is difference from fine-tuning as we just change the last layers in classification it is last connected layers after feature extraction and maybe for object detection it would be connected layers for bbox + classification in yolo while in faster-rcnn there are 2 different netowrks .

While fine-tuning would to even train some of the feature -extraction layers too ?

3. Also like how do i improve model for my custom dataset ? and like how am i doing transfer learning here in this case ? because what as per the doc we are doing is training the whole pre-trained model with a low -learning rate i suppose but what if i have very very small data wouldnt it be better just to train the last layers just like classification i.e just training the bbox+classification layers with a low learning rate ? and like how do i do that can i check the model.summary() and where can i see these pre-defined anchor boxes and i suppose these are even created using k-means clustering ig based on the ground-truth boxes i suppose ?

How can i freeze the feature -extraction layers lets say Darknet-53 and then train rest of the layers and like how are anchor boxes being pre-defined how do i get that and also how do i know the functioning of these bbox + classification layers are these fully -connected nn or convolutional layers or something else ?

[glenn-jocher]

Hey there! 🌟 Let's dive into your questions:

1. Pre-defined Anchor Boxes:

   • Yes, pre-defined anchor boxes are essentially preset width and height values that help the model predict bounding boxes more accurately. During training, the model learns to adjust these pre-defined boxes to match the actual objects. The formula you mentioned is on the right track, where bx = ax + etx adjusts the anchor box based on the model's predictions.

2. Transfer Learning vs. Fine-Tuning:

   • You've got it! Transfer learning often involves using the pre-trained weights of a model and modifying the last few layers for your specific task. Fine-tuning goes a step further by also adjusting weights in earlier layers. In object detection, this could mean adjusting both the feature extraction layers and the layers responsible for predicting bounding boxes and classes.

3. Improving Model Performance on Custom Dataset:

   • For a very small dataset, focusing on training the last layers (bounding box regression and classification layers) can indeed be beneficial. You can achieve this by freezing the earlier layers (feature extraction) and only training the later ones. This approach helps prevent overfitting.
   • To see the model architecture, including pre-defined anchor boxes, you might use tools like Netron for visualizing ONNX models or TensorFlow's model.summary() for TF models. Pre-defined anchor boxes are usually determined based on common object sizes in the training dataset and can be adjusted for custom datasets.

For freezing layers and focusing on the last layers, you can modify the training script to set the requires_grad attribute of the parameters in the feature extraction layers to False. This prevents them from being updated during training. Here's a quick example for a PyTorch model:

for param in model.backbone.parameters():
    param.requires_grad = False

This code snippet freezes the backbone (feature extractor) of the model, allowing you to focus on training the rest.

Anchor boxes are indeed often determined using clustering techniques like k-means on the ground truth bounding boxes to capture the most common shapes and sizes.

For more detailed guidance and the latest resources, the Ultralytics YOLO documentation is a fantastic place to start. It's packed with tutorials, examples, and explanations that can help clarify these concepts further. Happy modeling! 🚀

[Abhinav-Singh-Bisht]

hey @glenn-jocher thanks you very much for spending so much time on clearing my doubts it really means a lot. And yes most of my confusion is clear thanks to you. Have also joined the discord community so will try to ask doubts there to .
Just one last question sorry.
Like if i want to do this transfer learning for YOLOv8 than can it be done rather than fine-tuning ? If yes can you share resources for the same ?

[glenn-jocher]

Hey! 🤗 I'm super glad to hear that your doubts are clearing up and it's awesome you've joined our Discord community. Always feel free to drop your questions there too!

Regarding your question on transfer learning for YOLOv8 as opposed to fine-tuning: Yes, it's entirely possible and often a great approach, especially when working with a small dataset or aiming to adapt the model to a new domain. Transfer learning in this context means taking a pre-trained YOLOv8 model (like yolov8n.pt) and further training it on your specific dataset, which can involve adjusting the final layers and possibly some mid-level features depending on your requirements and dataset similarity.

For resources, our docs include training YOLOv8 on a custom dataset, which inherently leverages the concept of transfer learning by utilizing pre-trained weights. You might particularly focus on the sections about adjusting the learning rate and fine-tuning the model to fit your data better.

For a more targeted approach on just the last layers (like you might see in classification tasks), the workflow slightly differs in object detection due to the complex output structure (bounding box + classification). But, starting with yolo detect train using a pre-trained model and a custom dataset is straight on the path of transfer learning. To focus more on the concept of fine-tuning only specific parts, you might consider manually adjusting the model architecture in code to freeze early layers or look into advanced tutorials within the community, like those found in our Discord or GitHub discussions.

Hope this helps! And remember, there's no such thing as too many questions; we're all learning together! 😊

[tomershukhman]

Hi,
I would like to know what is the best way to access a box coordinates,
I saw this snippent in the comments here but I don't see how that works (and it failes why i tested it)

for det in results[0].boxes.xyxy:
    xmin, ymin, xmax, ymax, conf = map(int, det[:4].tolist() + [det[4]])

xyxy in 1,4 dimensional and therefor det[4] is out of bounds.

what i did manage to get working but it feels convoluted since I am using [0] so many times for such a short code is

from ultralytics import YOLO
import cv2

model = YOLO('./runs/detect/train/weights/best.pt')
result = model('tennis.jpg')
for result in result[0].boxes.numpy():
    xyxy = result.xyxy[0]
    x1, y1, x2, y2 = map(float, xyxy)
    cls = int(result.cls[0])
    conf = float(result.conf[0])
    print("x1: ", x1, "y1: ", y1, "x2: ", x2, "y2: ", y2, "cls: ", cls, "conf: ", conf)

[glenn-jocher]

Hey there! 👋

It looks like there's a bit of confusion regarding accessing box coordinates from the results. Your approach is almost there, but let me clarify a bit to help you out.

When you're working with the results from a YOLO model prediction, you can access the bounding box coordinates directly from the results object. Here's a simplified way to do it:

from ultralytics import YOLO

# Load your model
model = YOLO('path/to/your/model.pt')  # For example, 'yolov8n.pt'
# Run prediction
results = model('path/to/your/image.jpg')

# Now, iterate over detected objects
for det in results[0].boxes:
    # det is now a single detection with attributes you can directly access
    xmin, ymin, xmax, ymax = det.xyxy[0]  # Coordinates
    conf = det.conf  # Confidence
    cls = det.cls  # Class ID
    print(f"Box coordinates: {xmin}, {ymin}, {xmax}, {ymax}, Confidence: {conf}, Class ID: {cls}")

This way, you're directly accessing the attributes of each detection without needing to index [0] multiple times or manually map the values. The results[0].boxes gives you a structured way to iterate over each detection and access its properties like xyxy (for coordinates), conf (for confidence), and cls (for class ID).

Hope this helps clear things up! If you have any more questions, feel free to ask. Happy coding! 😊

[tomershukhman]

Alrighty, we are almost there.
The output for the code you provided (modifying the relavent vars) is
Box coordinates: 317.3462829589844, 242.7781982421875, 356.83758544921875, 282.7698669433594, Confidence: tensor([0.9921]), Class ID: tensor([13.])
Notice that the cls and id are tensor objects
so I modified the code to be as below and now it prints as I wanted.

`from ultralytics import YOLO
import cv2

model = YOLO('./runs/detect/train/weights/best.pt')
results = model('tennis.jpg')
for det in results[0].boxes:
# det is now a single detection with attributes you can directly access
xmin, ymin, xmax, ymax = det.xyxy[0] # Coordinates
conf = det.conf[0] # Confidence
cls = det.cls[0] # Class ID
print(f"Box coordinates: {xmin}, {ymin}, {xmax}, {ymax}, Confidence: {conf}, Class ID: {cls}")
`

I do still have a question which bother me (sorry if it's a complete nobby question)
If det represent a single box detaction and a box geometrically has 4 uniqe coordinates then why are we accessing those coordinates with [0], that would indicat that det.xyxy has a list of coordinates sets.
same goes for conf, we should have a single confidence score for a box

[glenn-jocher]

@tomershukhman hey there! 😊 It looks like you've made great progress and have a keen eye for detail. Regarding your question about why det.xyxy and other attributes like det.conf seem to be accessed with [0], it's because these attributes are returned as tensors, often with an extra dimension to support batch processing. Even for a single detection, the output maintains this structure for consistency.

In your case, when you access det.xyxy[0], you're effectively unpacking the tensor to get the coordinates of the bounding box. Similarly, for det.conf[0] and det.cls[0], you're accessing the first (and in this context, only) element of the tensor, which holds the confidence score and class ID, respectively.

Your modified code snippet correctly accesses these values for a single detection. Here's a slightly refined version for clarity:

from ultralytics import YOLO

model = YOLO('./runs/detect/train/weights/best.pt')
results = model('tennis.jpg')
for det in results[0].boxes:
    xmin, ymin, xmax, ymax = det.xyxy[0].tolist()  # Convert tensor to list for easier handling
    conf = det.conf.item()  # Get Python number from tensor
    cls = det.cls.item()  # Get Python number from tensor
    print(f"Box coordinates: {xmin}, {ymin}, {xmax}, {ymax}, Confidence: {conf}, Class ID: {cls}")

This version uses .tolist() for the coordinates and .item() for confidence and class ID, making it clear that you're working with individual values, not tensors. This should make the code a bit more intuitive and maintain the efficiency of your approach.

And no worries about asking questions, no matter how basic they might seem! We're all here to learn and help each other out. Keep up the great work! 🚀

[tomershukhman]

Great tnx.
Then let me ask an additional more intreseting question.

I want to identify complex processes, like tracking the entire events leading to the crash. It's not just about recognizing a crashed the car. I know the basic approach involves gathering crash images, labeling them, and training the model. However, I'm looking for insights on how to capture the dynamics of a crash effectively.

[glenn-jocher]

Hey there! 🌟

For capturing the dynamics of a crash effectively, beyond just recognizing a crashed car, you're venturing into the realm of action recognition and event prediction. This is indeed a more complex task than static object detection. Here are a couple of insights:

1. Temporal Aspect: Consider incorporating video data that captures the sequence leading up to a crash. This allows your model to learn not just from static frames but from the progression of events.

2. Model Choice: Look into models that are designed for action recognition or temporal sequence learning. While YOLO is excellent for object detection, models like RNNs, LSTMs, or 3D CNNs might be more suited for understanding sequences and predicting events over time.

3. Data Annotation: Annotate your video data to include not just the crash itself but also the events leading up to it. This might involve labeling frames or sequences that precede the crash, indicating risky behaviors or conditions that typically lead to a crash.

4. Feature Engineering: Consider extracting features that are indicative of a crash happening, such as sudden changes in speed, erratic movements, or proximity to other objects. These features can be crucial for your model to predict a crash before it happens.

5. Simulation and Synthetic Data: Generating synthetic data through simulations can help model various crash scenarios and conditions that might be hard to capture through real-world data alone.

Here's a simple example of how you might start to approach this with YOLO for detecting potential precursors to a crash:

from ultralytics import YOLO

# Load your model
model = YOLO('yolov8n.pt')

# Predict on a video
results = model.track('path/to/crash_video.mp4')

# Analyze the results for potential crash precursors
# This is a simplified example. In practice, you'd need a more complex analysis.
for frame_results in results:
    for det in frame_results.boxes:
        if det.cls == 'car' and det.conf > 0.5:
            # Check for conditions indicative of a crash risk
            print(f"Potential risk detected at frame {frame_results.frame_id}")

Remember, this is a complex task that might require combining multiple models and approaches. Good luck, and feel free to reach out if you have more questions! 🚀

[tryprncp]

hi, im implementing yolov8 in my android gallery's search algorithm. the yolov8's purpose is to scan media files and return images that has a box label that matches the search query. i can make it work using yolov5s.torchscript.ptl with org.pytorch:pytorch_android_lite:1.10.0 and org.pytorch:pytorch_android_torchvision_lite:1.10.0 but i cant make it work with yolov8s.torchscript

[glenn-jocher]

@tryprncp hey there! 😊 It sounds like you're making great strides with implementing YOLOv5 on Android and looking to upgrade to YOLOv8. Awesome choice!

If you're experiencing troubles moving from YOLOv5s to YOLOv8s TorchScript, here are a couple of things you might want to check:

1. Ensure you're using the latest PyTorch Android dependencies, as updates might have been released that better support the newer YOLOv8 models.
2. Try exporting the YOLOv8 model again with the optimize flag set to True, as this might help with compatibility:

   from ultralytics import YOLO
   model = YOLO('yolov8s.pt')
   model.export(format='torchscript', optimize=True)

3. Check for any TorchScript export or loading error messages. They can often point towards what might be going wrong.

If these steps don't solve the issue, could you provide more details on the error message or behavior you're encountering when trying to use the YOLOv8s TorchScript model?

Keep on exploring and building, and don't hesitate to reach out if you have more questions or need further assistance!

[MeqdadDev]

Can you provide me with the best configurations for human-detection example using YOLOv8?

[pderrenger]

@MeqdadDev absolutely, happy to help with setting up your human detection configuration using YOLOv8! 🚀

For a human detection task, I'd suggest starting with one of the pretrained YOLOv8 Detect models, like yolov8n.pt for efficiency or yolov8l.pt and yolov8x.pt for higher accuracy. These models are pretrained on the COCO dataset, which includes human class among others.

Here's a concise example of how to load a pretrained model and train it on your specific dataset tailored for human detection:

from ultralytics import YOLO

# Assuming you have a custom dataset in YOLO format with human annotations
# Load a pretrained model
model = YOLO('yolov8n.pt')  # or 'yolov8l.pt'/'yolov8x.pt' for better accuracy

# Train the model on your dataset for human detection
results = model.train(data='your_dataset.yaml', epochs=100, imgsz=640)

Make sure your dataset is annotated properly and the your_dataset.yaml is configured correctly to point to your training and validation data paths. For fine-tuning parameters like epochs or imgsz, adjust them according to your computational resources and desired accuracy-speed trade-off.

Feel free to reach out if you need further assistance. Best of luck with your human detection project! 🙌

[mudassir659]

how to reduce predicted bounding box line width in yolov8

[glenn-jocher]

Hello there! 😊 To reduce the line width of the predicted bounding boxes in YOLOv8, you can use the line_thickness argument when you plot or show your results. Here's how you can adjust it in your Python code:

from ultralytics import YOLO

# Load the model
model = YOLO('yolov8n.pt')

# Run predictions
results = model('https://ultralytics.com/images/bus.jpg')

# Plot results with a custom bounding box line thickness
results[0].plot(line_thickness=1)  # Adjust line_thickness as needed
results[0].show()

For CLI, the line thickness is automatically adjusted for optimal viewing experience and currently, there isn't a direct CLI argument to alter it. However, modifying the source code or using the Python API as illustrated above are good alternatives. 🛠️

Hope this helps! Let us know if you have any other questions.

[arifan-instrumentasi-UB]

Hi, can yolov8 detect color sequence?. i create color sequence of rainbow (red, yellow, green) as a real rainbow. if i change the yellow to orange, can yolov8 detect it as a wrong rainbow?

[glenn-jocher]

@arifan-instrumentasi-UB hi there! 🌈

YOLOv8 is primarily designed for detecting objects and their categories, rather than the sequence or color composition of objects. It can recognize objects (like a rainbow) based on the training data it has been exposed to, but it doesn't inherently understand the "correct" color sequence of a natural rainbow.

However, you could approach the problem creatively by training your model on labeled data specifying "correct" and "incorrect" rainbows. Here’s a simplified example of how your training annotations might look:

• Image 1 (correct rainbow): Class label “rainbow_correct”
• Image 2 (rainbow with yellow replaced by orange): Class label “rainbow_incorrect”

This way, YOLOv8 could potentially learn to distinguish between the different types of rainbows based on your specific definitions of "correct" and "incorrect."

Remember, accurate detection relies heavily on the diversity and quality of the training dataset. 📊

Hope that helps!

[Vinrub]

i need a object detection code using yolov8 for detect the two images

[glenn-jocher]

@Vinrub hey there! For object detection with YOLOv8, here's a simple script to help you detect objects in two images. Assuming you have YOLOv8 installed and set up:

from ultralytics import YOLO

# Load YOLOv8 model
model = YOLO('yolov8n.pt')  # Here we're using the YOLOv8n model

# Run detection on images
results1 = model('path/to/your/first/image.jpg')
results2 = model('path/to/your/second/image.jpg')

# You can view or save the results
results1.show()  # View the first image with detections
results2.show()  # View the second image with detections

# Or save them to files
results1.save('path/to/save/first/image')  
results2.save('path/to/save/second/image')  

This script loads the YOLOv8 model, runs it on two images, and then displays and saves the results. Don't forget to replace 'path/to/your/first/image.jpg' and 'path/to/your/second/image.jpg' with your actual image paths. Hope this helps! 🚀

[Vinrub]

here is my code import cv2
from ultralytics import YOLO

Load the YOLO model

model = YOLO('yolov8n.pt') # Assuming 'yolov8n.pt' is the YOLO model you want to use

Specify the path to your image

image_path = '/media/XavierSSD250/22_08_23/ui.png'

Perform object detection

results = model(image_path)

Convert image to OpenCV format

image = cv2.imread(image_path)

Extract bounding boxes, labels, and confidence scores from results

bboxes = results.pred[0][:, :4] # Extract bounding boxes
labels = results.names # Extract labels
confidences = results.pred[0][:, 4] # Extract confidence scores

Draw bounding boxes and labels on the image

for bbox, label, confidence in zip(bboxes, labels, confidences):
bbox = bbox.astype(int)
cv2.rectangle(image, (bbox[0], bbox[1]), (bbox[2], bbox[3]), (0, 255, 0), 2)
cv2.putText(image, f'{label}: {confidence:.2f}', (bbox[0], bbox[1] - 10),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

Show the image with bounding boxes

cv2.imshow('Object Detection', image)
cv2.waitKey(0)
cv2.destroyAllWindows()
the error is Traceback (most recent call last):
File "frames5.py", line 17, in
bboxes = results.pred[0][:, :4] # Extract bounding boxes
AttributeError: 'list' object has no attribute 'pred'

[glenn-jocher]

Hi there! It looks like you're encountering an issue while trying to extract bounding boxes, labels, and confidence scores from your results. The results object has been updated, so you'll want to access these attributes a bit differently.

To extract bounding boxes, labels, and confidences from your object detection results, you can use the .xyxy[0] for bounding boxes, .names for label names, and index into .pred[0] for confidence scores, like so:

# Corrected way to extract information
bboxes = results.xyxy[0]  # xyxy format bounding boxes
labels = [results.names[int(cls)] for cls in results.pred[0][:, 5]]
confidences = results.pred[0][:, 4]  # Extract confidence scores

# Now, you iterate like this to draw your bounding boxes and put text
for *xyxy, conf, cls in results.xyxy[0]:
    label = results.names[int(cls)]
    xyxy = [int(x) for x in xyxy]  # Convert tensor to int
    cv2.rectangle(image, (xyxy[0], xyxy[1]), (xyxy[2], xyxy[3]), (0, 255, 0), 2)
    cv2.putText(image, f'{label}: {conf:.2f}', (xyxy[0], xyxy[1] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

cv2.imshow('Object Detection', image)
cv2.waitKey(0)
cv2.destroyAllWindows()

This should resolve the AttributeError you're experiencing. 😊 Let us know if you need more help!

[Vinrub]

i got the same kind of error only

[glenn-jocher]

@Vinrub hey there! 🙌 If you're seeing an AttributeError similar to this when trying to extract bounding boxes or other details from your detections, it seems there might be a slight mix-up in the way you're accessing the results object.

Make sure you're using the updated way to access the bounding boxes and other properties. Here's a quick snippet to guide you:

# Assuming 'results' is your detection result
bboxes = results.xyxy[0]  # Get bounding boxes in xyxy format
# Loop through detections and print out details
for det in bboxes:
    bbox = det[:4].numpy().astype(int)  # Convert bbox to numpy array and then to integer
    conf = det[4].item()  # Get the confidence
    label = results.names[int(det[5])]  # Get the label

    print(f"BBox: {bbox}, Confidence: {conf}, Label: {label}")

If you need more detailed assistance or are looking for something specific, don't hesitate to check out the Object Detection section in the Ultralytics docs. It's packed with useful tips! 😊

[dolfMo]

I am a beginner and have trained an object detection model using my own dataset. If I want to display the results of detection inference in real-time on the web, which method would be better if you recommend?

[pderrenger]

Hey there! 😃 Thanks for reaching out with your questions. I'm here to help clarify things for you!

1. Model choice with modified YAML: The choice between n, s, l, etc., versions of pretraining weights largely depends on the balance you seek between speed and accuracy for your project. If your modified YAML is similar in complexity to the original yolov8n.yaml, starting with yolov8n.pt might make sense. For more complex modifications, l or x versions could provide a better starting point. It's often a good idea to experiment with a couple of versions to see which performs best for your specific dataset and task.

2. Getting detected category names: After running a prediction, you can access the names of detected categories using the names attribute of the Results object in conjunction with the detected class IDs. For example:

   results = model('your_image.jpg')
   for det in results.boxes:  # assuming detection result is stored in results
       print(results.names[det.cls])  # Print detected class name

3. Loading pretrained weights for a custom model: You're on the right track with specifying the pretrained model file along with your custom YAML. This approach leverages the pretrained weights effectively while customizing other aspects of the model. Your code snippet for training is spot-on for this:

   model = YOLO('./models/yolov8l.pt')
   model.train(data='./custom.yaml', epochs=300)

   This code loads the yolov8l.pt pretrained weights and applies them for training with settings and dataset specified in custom.yaml.

And no need to apologize for asking questions—that's what we're here for! If anything remains unclear or you have further inquiries, feel free to reach out. Wishing you success on your project! 🌟

[dolfMo]

Hello😊! I would like to know the information on data processing for the input training image data in YOLOv8's detection task, including preprocessing, feature extraction, etc., or where can I obtain it? And what are the corresponding codes?

[pderrenger]

Hello! 😊 It's great to hear you're diving into YOLOv8's detection tasks! For the preprocessing and feature extraction used in YOLOv8's detection, these processes are inherently part of the model's design, optimizing input images for the best detection performance.

When you load a YOLOv8 model like yolov8n.pt, yolov8s.pt, yolov8l.pt, etc., preprocessing steps such as resizing images to the model size, normalization, and potentially augmentations are executed. Feature extraction is handled by the model's architecture, leveraging layers optimized for detecting objects within various scenes.

To use pretrained weights with your modified YAML, select a version (n, s, l, etc.) that aligns with your performance and complexity requirements. Typically, n (nano) is fastest but least accurate, while x (extra large) is most accurate but slowest. If your modifications are minor, starting with a version closest in size to your alterations might work best.

To get the currently detected categories from prediction results, you can access the names attribute alongside detected class IDs. Here's a quick example:

results = model('your_image.jpg')
detected_names = [model.names[i] for i in results.boxes.cls]
print(detected_names)

To load pretrained weights for training on your custom dataset with YOLOv8-custom.yaml, you've got the right approach. You specify your dataset in the YAML and load the pretrained weights as the starting point:

from ultralytics import YOLO

# Assuming you are using YOLOv8 large model as pretrained weights
model = YOLO('yolov8l.pt')
model.train(data='YOLOv8-custom.yaml', epochs=300)

Ensuring you're correctly specifying your custom dataset and chosen pretrained model will streamline the training process, leveraging the power of YOLOv8 effectively. Thanks for your interest, and don't hesitate to reach out with more questions!

[dolfMo]

apologize for my poor English🥹,I am using translation software to talk you,what I want to express is what data preprocessing process will be carried out after training the image data input network to enhance the model's ability. and then where can I find the corresponding function for the operation?

[pderrenger]

Hello! 😊 No worries at all about your English; you're doing just fine. And yes, YOLOv8 does incorporate several preprocessing steps to enhance model performance during training. These typically include:

• Resizing: Images are resized to the model's input dimensions (e.g., 640x640 for YOLOv8n).
• Normalization: Pixel values might be normalized to aid in the training process.
• Data Augmentation: Techniques like flipping, scaling, or cropping are applied to increase dataset diversity.

To locate the specific functions for these operations in the YOLOv8 GitHub repository, you might want to look into the datasets module, where data loading and preprocessing are handled.

For obtaining the name of the detected category, after you've run your prediction like so:

results = model('your_image.jpg')

You can access the names using:

for det in results.boxes:
    print(results.names[det.cls])  # This prints the detected class names

When looking to use pretrained weights (yolov8n.pt, yolov8s.pt, yolov8l.pt, etc.) for training a model defined in a custom YAML, you do this right at the model creation step. The code snippet you've mentioned already does that effectively:

from ultralytics import YOLO

model = YOLO('yolov8l.pt')  # Using YOLOv8l pretrained weights
model.train(data='./custom.yaml', epochs=300)  # Assuming custom.yaml is your dataset and model configuration

Here, custom.yaml should contain the model configuration tailored for your specific needs along with the dataset information.

Hope these details help streamline your YOLOv8 journey! Feel free to explore further and reach out if you've got more questions. Happy coding! 🚀

[LizBeansprout]

Hi, at the moment I'm trying to used .onnx exported from yolov8n.pt in Unity, using C#, to demonstrate real-time object detection in 3D scene, I got output in the shape of (1,84,8400). I would like to ask how to post process this output to draw bounding box, extracting the information of the output or the output format. Sorry for my english, and if there are any better method please suggest right away. thank you in advance.

[pderrenger]

Hello! Great to hear you're integrating the YOLOv8n model into Unity for 3D object detection. 🚀 The output shape (1, 84, 8400) you mentioned typically contains the detected bounding boxes, object class probabilities, and confidence scores.

To draw bounding boxes, you need to process this output. Generally, the steps involve:

1. Reshaping: The output tensor usually needs reshaping to separate out each attribute (x, y coordinates, width, height, confidence, class scores).

2. Applying a threshold: Filter out detections based on a confidence threshold.

3. Non-maximum suppression (NMS): This removes overlapping bounding boxes, keeping the one with the highest confidence.

Here's a basic guideline in C# (assuming you can translate the logic):

float confThreshold = 0.5f; // Set your confidence threshold
foreach (var detection in detections) {
    float score = detection[4]; // Assuming the 5th element is confidence
    if (score > confThreshold) {
        float x = detection[0];
        float y = detection[1];
        float width = detection[2];
        float height = detection[3];
        // Draw bounding box: Use Unity's drawing utilities or custom rendering
    }
}

Note: Implementation specifics might vary based on how you're processing the output in Unity and the exact output format of your ONNX model.

Also, consider diving into the ONNX Runtime C# documentation for more insights on handling ONNX models in C# environments.

Keep experimenting, and don't hesitate to ask if you have more questions. Your English is just fine, and your pioneering effort in Unity is impressive! 🌟

[kh-diab]

hello all

I have around original images around 74 images dataset with resolution 8192 * 5460
I have tiled the images to 640 and run the model model = YOLO('yolov8x.pt')
so after that I had like 3500 sub images tiled for training and validation.
the point is that my dataset has a similar background similar to the object I want to detect which is weed with many grass

Train the model

model.train(
data=data_path,
epochs=100,
imgsz=640,
batch=-1,
project=project_path, # Specify the directory where results will be saved
name=experiment_name, # Specify the subdirectory name for this run
keras= True,
optimize=True,
cache= True,
amp= True,
workers= 15
)

below is my results

but the model is not performing well, in terms of precision.

please give me hints how to imporve the model and how to make use of all the resources to reduce the training model

also I have access to powerful machine with 5 GPU but I am not able to utilize all the resources
when I set device parameter to 0,1,2,3,4 I got error cuda out of memory

the machine spec:
500 GB RAM
and GPU as followed by the command
nvidia-smi

NVIDIA-SMI 525.147.05 Driver Version: 525.147.05 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA A100-SXM... On | 00000000:01:00.0 Off | Off |
| N/A 56C P0 91W / 275W | 12432MiB / 40960MiB | 100% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA A100-SXM... On | 00000000:47:00.0 Off | Off |
| N/A 57C P0 91W / 275W | 4224MiB / 40960MiB | 100% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 2 NVIDIA A100-SXM... On | 00000000:81:00.0 Off | Off |
| N/A 55C P0 87W / 275W | 4230MiB / 40960MiB | 100% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 3 NVIDIA DGX Display On | 00000000:C1:00.0 Off | N/A |
| 68% 83C P0 N/A / 50W | 3599MiB / 4096MiB | 98% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 4 NVIDIA A100-SXM... On | 00000000:C2:00.0 Off | Off |
| N/A 56C P0 92W / 275W | 4218MiB / 40960MiB | 100% Default |
| | | Disabled |
+-------------------------------+---------------------

[pderrenger]

@kh-diab hello,

For improving your model's performance, consider the following steps:

1. Data Augmentation: Enhance your dataset with techniques like flipping, rotation, and color adjustments to make the model more robust.
2. Hyperparameter Tuning: Experiment with different learning rates, batch sizes, and epochs to find the optimal settings.
3. Class Imbalance: Ensure your dataset is balanced. If not, use techniques like oversampling or class weighting.
4. Multi-GPU Training: To utilize all GPUs, ensure your CUDA setup is correct and try reducing the batch size to fit within GPU memory.

For the CUDA out-of-memory error, check if each GPU has enough memory available and consider using mixed precision training (amp=True) to reduce memory usage.

For more detailed guidance, refer to the Ultralytics documentation.

[kh-diab]

hello @pderrenger
thanks for your reply
the point is the image as shown in the attached has similar background
how to deal with it, do you think the bounding box is ok here or I should go for object segmentation ?
for class imbalance actually I have around 8
5% of the annotation is for one class and the other 15% is for 3 other classes.
can you guide me how can i find the best batch size of epoch and learning rate

thanks

[pderrenger]

@kh-diab hello,

Given the similar background and object in your images, object segmentation might indeed offer better performance than bounding boxes alone. Segmentation can help the model differentiate between the weed and the grass more effectively.

For class imbalance, consider techniques like oversampling the minority classes or using class weights during training. This can help the model learn to recognize less frequent classes better.

To find the best batch size, epoch, and learning rate, you might need to experiment with different values. Start with a smaller batch size to avoid memory issues and gradually increase it. Use a learning rate scheduler to adjust the learning rate dynamically during training.

For utilizing all GPUs, ensure each GPU has enough memory. You might need to reduce the batch size or use mixed precision training (amp=True). If you continue to face issues, try training on fewer GPUs or using gradient accumulation to simulate larger batch sizes.

[nounotabe]

How to disable print outputs like this: “0: 384x640 3 bags, 124.8ms
Speed: 11.2ms preprocess, 124.8ms inference, 25.0ms postprocess per image at shape (1, 3, 384, 640)”?

[pderrenger]

@nounotabe to disable the print outputs, you can set the verbose parameter to False when running your model. This will suppress the detailed output logs during inference. If you are using the CLI, you can add verbose=False to your command. If you are using the Python API, you can set verbose=False in the predict method. Please ensure you are using the latest version of the Ultralytics package to access this feature. For more details, you can refer to the Ultralytics documentation.

[zhendelan-666]

Hello, I want to improve the detection accuracy of specific classes in the dataset without using retraining. My current idea is to increase the weight of specific classes in the loss function. Is this practice effective? Are there other effective ways to do this?

[glenn-jocher]

@zhendelan-666 increasing the weight of specific classes in the loss function can indeed help improve detection accuracy for those classes. This approach adjusts the model's focus during training, making it more sensitive to the specified classes. However, since you mentioned not wanting to retrain, this method won't be applicable directly.

An alternative approach without retraining is to use post-processing techniques, such as adjusting the confidence thresholds for specific classes. This can help prioritize detections of those classes by filtering out lower-confidence detections more aggressively for other classes. Additionally, you can explore techniques like ensemble methods, where multiple models are combined to improve overall detection performance.

For more detailed guidance, you can refer to the Ultralytics documentation on object detection.

[tiffanytgr]

Hi I have a project where we intend to place cameras in warehouses. take periodic snapshots and find differences in images to see what has entered the warehouse. can yolo do this?

[glenn-jocher]

@tiffanytgr yes, YOLO can be used to detect changes in warehouse images by identifying and classifying objects in each snapshot. You can then compare the detected objects across different images to determine what has entered or exited the warehouse. For more details, visit the Object Detection documentation.

[tiffanytgr]

do you have sample codes where it is comparing 1) images with objects that have moved, 2) images that have been added?
Thanks a bunch

[glenn-jocher]

@tiffanytgr we don't have specific sample codes for comparing images with moved or added objects, but you can achieve this by running YOLOv8 on each image to detect objects and then comparing the results. For more details on using YOLOv8 for object detection, please refer to our Object Detection documentation.

[Git-2001]

How can I use yolo V8 with amd Radeon r5 m335

[glenn-jocher]

@Git-2001 yOLOv8 currently supports NVIDIA GPUs for acceleration. For AMD Radeon GPUs, you might consider using CPU inference or exploring compatibility with ROCm, AMD's open software platform. For more details, please refer to the official documentation or community forums.

[sainisatish]

I have trained the YOLO11l model for 1000 epochs, and now I want to continue training for 1500 epochs. I passed resume=True, but it is not working. I used a checkpoint with the saved optimizer state. However, with the YOLOv8 model, I am able to continue the training for additional epochs without any issues.
Please guide me on what silly mistake I might be making.

-> model = YOLO('checkpoint/epoch998.pt')
-> model.train( epochs=1500, resume =True)

[glenn-jocher]

@sainisatish it seems like there might be an issue with the checkpoint path or the resume functionality. Ensure the checkpoint file is correctly specified and contains the optimizer state. If the problem persists, consider re-checking the model's compatibility with the resume feature.

[glenn-jocher]

@swarnalathayv to continue training from a checkpoint, ensure the path is correct and the checkpoint includes the optimizer state. For YOLO11, use:

model = YOLO('checkpoint/epoch998.pt')
model.train(epochs=1500, resume=True)

Regarding YOLO11 vs. YOLOv8, YOLO11 offers state-of-the-art performance improvements. For more details, visit the Ultralytics documentation.

[ajay78123]

I have trained YOLOv8 on my custom dataset, and it’s working well. However, I’m facing a challenge: I’ve captured a video where the objects are in fixed positions, but the camera is moving. My goal is to get the total count of all detected objects. The problem is that I’m getting the object count for every frame, but I need the actual overall count of the objects, not repeated detections from each frame. How can I resolve this?

[glenn-jocher]

@ajay78123 to achieve an overall count of unique objects in your video, consider using object tracking to maintain consistent IDs across frames. This will help differentiate between repeated detections and actual unique objects. You can explore trackers like ByteTrack or BoT-SORT, which are compatible with YOLO models. For more details, please refer to the tracking section in the Ultralytics documentation.

[tylerfrisinger01]

If i use the scale parameter while training my model, will this scale every image in the dataset? or will it train itself on the regular images and then go back through and train itself again with all images scaled? Also is the only difference between YOLO11 and YOLO8 how accurate they are or fast they are? we are currently using YOLOv8 for object detection but if YOLO11 is faster and more accurate we will switch before we train. Thank you!

[Brenda-Caroline]

Hello, good morning.

After doing my detection training in yolov8, I ran my training set images and saved the predicted images with the bounding boxes, saved the txt file with the labels returned from the bounding boxes (returns the ) and saved the file with the boxes that returns all the data for xyxy, xyxyn, xywh, xywhn in a more complete way.

After doing this, I want to calculate if there is overlap of the bounding boxes, I thought about doing it by analyzing if the point of one box is inside another. Do you have any other suggestions on how to obtain this result of the overlap of the boxes? And how can I read the file and obtain the necessary data?

[Brenda-Caroline]

Sorry, I put the question in a new tab

[glenn-jocher]

@Brenda-Caroline to calculate the overlap of bounding boxes, consider using the Intersection over Union (IoU) metric, which measures the overlap between two bounding boxes. You can read the bounding box coordinates from your saved files and compute IoU to determine overlap. For reading and processing the data, you can use Python libraries like NumPy or Pandas to handle the text files and perform the necessary calculations.

[tylerfrisinger01]

@glenn-jocher What is the process that YOLO uses when i use the scale augmentation? Will it scale every image? or will it get trained with all the original images and then retrain itself again with all the scaled images?

[glenn-jocher]

The scale parameter in YOLO applies random scaling to each image during training, enhancing model robustness by simulating different object sizes. YOLO11 offers improved speed and accuracy over YOLOv8, making it a better choice for your object detection tasks.

[Brenda-Caroline]

Hello, good morning.

After doing my detection training in yolov8, I ran my training set images and saved the predicted images with the bounding boxes, saved the txt file with the labels returned from the bounding boxes (returns the ) and saved the file with the boxes that returns all the data for xyxy, xyxyn, xywh, xywhn in a more complete way.

After doing this, I want to calculate if there is overlap of the bounding boxes, I thought about doing it by analyzing if the point of one box is inside another. Do you have any other suggestions on how to obtain this result of the overlap of the boxes? And how can I read the file and obtain the necessary data?

[Brenda-Caroline]

And how is it possible to obtain the result xyxy through the normalized values ​​?

[glenn-jocher]

@Brenda-Caroline to calculate the overlap of bounding boxes, you can use the Intersection over Union (IoU) metric, which measures the overlap between two bounding boxes. You can read the bounding box data from your saved files and compute IoU to determine overlap. For more details on object detection tasks, visit https://docs.ultralytics.com/tasks/detect/.

[201power]

hello, I trained yolo11 with 27 class, and raw output[0] for detection is 51x31x8400, how I do get final bbox, cls and conf from this? I assume there are 51 detection heads, 4 bbox coordinates + 27 class scores, and 8400 anchor points.

[glenn-jocher]

You can find the code for post-processing in the ultralytics/engine/predictor.py file, specifically within functions that handle detection outputs like non-max suppression. This part of the code will help you extract predicted labels and confidence scores for visualization tasks like Grad-CAM. For detailed code reviews, consider exploring the relevant sections in the repository.

[201power]

Hi Glenn,

the post processing function is empty in here: https://github.com/ultralytics/ultralytics/blob/main/ultralytics/engine/predictor.py#L163, please advise on where to find it.

Also, I am interested in commercial license, who should I contact?

Thanks,

[glenn-jocher]

@201power the post-processing logic is embedded in helper functions like non-max suppression (NMS) within the Ultralytics library. For detailed implementation, review the ultralytics/yolo/utils/utils.py file, specifically functions like non_max_suppression. You can adapt them for raw output processing for GradCAM visualization.

For a commercial license, please visit https://ultralytics.com/license to apply or contact the licensing team directly. Thank you!

[201power]

thanks Glenn, now my output result is correct.

I wrapper yolo model.model using YOLOWrapper() below. do you know why x become [51x3x640x640] in forward() function, even the input tensor is only [1x3x640x640]?

class YOLOWrapper(torch.nn.Module):
    def __init__(self, model):
       super(YOLOWrapper, self).__init__()
       self.model = model
    def forward(self, x):
        # Run the forward pass and get the output from YOLO
        outputs = self.model(x)

[glenn-jocher]

The behavior you're observing with the tensor shape ([51x3x640x640]) could be due to YOLO11's internal model structure, especially if it uses multi-scale feature maps generated during the forward pass. To diagnose and verify, inspect the model object passed to your YOLOWrapper. Specifically, check if it's the detection head or backbone that's called, as this may influence input processing.

For information on customizing the forward pass or debugging further, explore the relevant parts of the repository or dive into the model's architecture outlined in the YOLO11 documentation. If you're interested in GradCAM, ensure your wrapper integrates seamlessly with intermediate layers for gradient extraction.

For commercial licensing queries, please visit https://ultralytics.com/license. Let me know if you need anything else!

[Zelyanne]

Hello
i have a dataset on disease for detection it contains different class of diseases, i want the model to detect even unknow disease, like i want to detect if a plant have a disease or not and if it have a disease, assign the proper class to it or if it doesnt know assign labels "unknow disease" to it, do i need too train two differents yolo models ? or is there a simpler way, also i wants to do class weighting can you help me please

Thanks you

[glenn-jocher]

@Zelyanne for detecting unknown diseases and assigning labels accordingly, you can implement an "unknown" class during training or consider using anomaly detection techniques alongside your YOLO model. Class weighting can be applied by adjusting the loss function to emphasize certain classes. For more specific guidance, you might want to explore YOLO11's documentation on training and model configuration.

[towhidmds]

[glenn-jocher]

@towhidmds if you have any questions or specific issues related to YOLO11 or its usage, feel free to elaborate, and I’ll be happy to assist you directly here. For general guidance, I recommend visiting the comprehensive Ultralytics YOLO11 Documentation for detailed resources on training, prediction, and export modes.

[towhidmds]

Saved onnx format model is not working as expected. Getting output shape (1,10,8400). How to fix it to get boxs, and classes.

import onnxruntime as ort
import numpy as np
from PIL import Image
import cv2
import time

# Load the ONNX model
onnx_model_path = "best.onnx"
session = ort.InferenceSession(onnx_model_path)

# Function to preprocess the image
def preprocess_image(image_path, input_size):
    # Load image
    image = Image.open(image_path)
    # Resize image
    image = image.resize((input_size, input_size))
    # Convert to numpy array
    image_np = np.array(image)
    # Normalize the image
    # image_np = image_np / 255.0
    # Transpose the image to match the model's input shape (N, C, H, W)
    image_np = np.transpose(image_np, (2, 0, 1))
    # Add batch dimension
    image_np = np.expand_dims(image_np, axis=0).astype(np.float32)
    return image_np

# Prepare input data
input_image_path = "test.jpg"  # Replace with your actual image path


input_size = 640  # This should match the input size used during training
input_data = preprocess_image(input_image_path, input_size)

# Get model input name
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].shape

print("input_name: ", input_name)
print("output_name: ", output_name)
# Add timing code
start_time = time.time()

# Run inference
outputs = session.run(None, {input_name: input_data})

# Calculate inference time in milliseconds
inference_time = (time.time() - start_time) * 1000
print(f"Inference time: {inference_time:.2f} ms")

# Process the outputs as needed
print("outputs shape: ", len(outputs), outputs[0].shape)
print("outputs: ", outputs[0][0][0])


def process_output(image, outputs, conf_threshold=0.1):
    # Get image dimensions
    image_height, image_width = image.shape[:2]
    
    # YOLOv11 output format: [batch, num_classes+5, num_predictions]
    predictions = outputs[0]  # Shape: [1, 10, 8400]
    
    # Transpose predictions to [num_predictions, num_classes+5]
    predictions = predictions[0].T  # Shape: [8400, 10]
    
    boxes = []
    scores = []
    class_ids = []
    
    # Get confidence scores for all classes
    class_scores = predictions[:, 5:]  # Shape: [8400, num_classes]
    # Get maximum confidence score and class id for each prediction
    max_class_scores = np.max(class_scores, axis=1)
    class_ids_all = np.argmax(class_scores, axis=1)
    # Get objectness scores
    objectness = predictions[:, 4]
    print("objectness: ", objectness)
    
    # Filter based on confidence threshold
    mask = (objectness > conf_threshold) & (max_class_scores > conf_threshold)
    filtered_boxes = predictions[mask]
    filtered_scores = max_class_scores[mask]
    filtered_class_ids = class_ids_all[mask]
    print("filtered_boxes: ", filtered_boxes.shape)
    print("filtered_scores: ", filtered_scores.shape)
    print("filtered_class_ids: ", filtered_class_ids.shape)
    
    for box, score, class_id in zip(filtered_boxes, filtered_scores, filtered_class_ids):
        # Get normalized coordinates
        cx, cy, w, h = box[:4]
        
        # Convert to corner coordinates
        x1 = int((cx - w/2) * image_width)
        y1 = int((cy - h/2) * image_height)
        x2 = int((cx + w/2) * image_width)
        y2 = int((cy + h/2) * image_height)
        
        boxes.append([x1, y1, x2, y2])
        scores.append(float(score))
        class_ids.append(int(class_id))
    
    return boxes, scores, class_ids

# After running inference, process and draw the results
image = cv2.imread(input_image_path)
boxes, scores, class_ids = process_output(image, outputs)

# Draw boxes on the image
for box, score, class_id in zip(boxes, scores, class_ids):
    x1, y1, x2, y2 = box
    
    # Draw rectangle
    cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
    
    # Add label
    label = f'Class {class_id}: {score:.2f}'
    cv2.putText(image, label, (x1, y1-10), 
                cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

# Resize image for display (keeping aspect ratio)
max_display_size = 1024  # Maximum width or height for display
display_height, display_width = image.shape[:2]
scale = min(max_display_size / display_width, max_display_size / display_height)


if scale < 1:
    display_width = int(display_width * scale)
    display_height = int(display_height * scale)
    display_image = cv2.resize(image, (display_width, display_height))
else:
    display_image = image

# Display or save the result
cv2.imshow('Result', display_image)
cv2.waitKey(0)
cv2.destroyAllWindows()

# Optionally save the image
cv2.imwrite('result.jpg', image)

[glenn-jocher]

@towhidmds the output shape (1, 10, 8400) indicates raw YOLO11 predictions. You need to decode these into bounding boxes, class IDs, and confidence scores. Ensure your post-processing aligns with the specific YOLO11 format, where predictions are typically arranged as [batch, channels, grid_cells]. Your decoding function process_output() seems appropriate but cross-verify against the documentation for model-specific adjustments. For detailed instructions, refer to the Ultralytics export documentation. Let me know if further specifics are needed!

[tylerfrisinger01]

Within my tracking file I have the following code,

results = detection.predict(video_path, stream=True, save=False, max_det=2, verbose=False, save_crop=True, project=output, agnostic_nms=True, iou=0.5)
results2 = detection.predict(video_path, stream=True, save=True, max_det=2, verbose=False, save_crop=True, project=output, agnostic_nms=True, iou=0.5)
distance_displayed = 0
for result in results:
annotated_frame = result.plot()
if len(result.boxes) > 1:
print(f'box1 = {(result.boxes[0].xyxy).tolist()[0]}, box2 = {(result.boxes[1].xyxy).tolist()[0]}')
distance_displayed = distance((result.boxes[0].xyxy).tolist()[0], (result.boxes[1].xyxy).tolist()[0])
cv2.putText(annotated_frame, f'distance: {distance_displayed:.2f} cm', (1100, 200), cv2.FONT_HERSHEY_COMPLEX, 4, (255, 255, 255), 3)
video_writer.write(annotated_frame)

This snippet of code is used to calculate the distances of the the two objects in each frame of the output. Our model however is having some trouble distinguishing the two objects (they are very very similar and have the same background), for example it will correctly detect that there are two objects within that frame but will give them the same object label. I was thinking that within my
"if len(result.boxes) > 1" statement I could also say that if the labels of the two detected objects equal each other then change one of their labels to the one that is not being used. I have tried doing a couple different things but i keep getting stuck. if you guys have any suggestions that would be appreciated.

[glenn-jocher]

@tylerfrisinger01 to make prev_coords store both coordinates and labels, you can modify it to hold tuples (coordinates, label). Then, you can check and assign labels based on previous labels. Here’s an example adjustment:

# Initialize prev_coords to hold (coordinates, label)
prev_coords = {1: (None, None), 2: (None, None)}

for result in results:
    current_coords = [(box.xyxy.numpy()[0], box.cls.cpu().item()) for box in result.boxes]

    for coord, label in current_coords:
        if prev_coords[1][0] is None:
            prev_coords[1] = (coord, label)
            continue

        if prev_coords[2][0] is None:
            prev_coords[2] = (coord, label)
            continue

        # Calculate distances to previous coordinates
        distance_to_1 = np.linalg.norm(np.array(coord) - np.array(prev_coords[1][0]))
        distance_to_2 = np.linalg.norm(np.array(coord) - np.array(prev_coords[2][0]))

        # Assign based on proximity and previous labels
        if distance_to_1 < distance_to_2:
            prev_coords[1] = (coord, label)  # Keep label consistent
        else:
            prev_coords[2] = (coord, label)

        if prev_coords[1][1] == prev_coords[2][1]:  # If labels are the same
            prev_coords[2] = (prev_coords[2][0], label + 1)  # Adjust label (example logic)

        # Update the label dynamically
        label = 1 if np.array_equal(coord, prev_coords[1][0]) else 2

This ensures that bounding boxes are consistently labeled based on proximity and adjusts labels if duplicates occur. Test and refine this logic based on your specific needs, and let me know if you need further assistance!

[tylerfrisinger01]

@glenn-jocher thank you, this is pretty close to what i implemented myself. But this does not change the label within the output video, do i need to reference the model directly or something in order to change the labels within the output video itself? either within the last last line where you set "label" equal to something or within the last if statement?

[glenn-jocher]

To dynamically update and display labels in the output video, you need to modify the result.boxes attribute directly before plotting. For example, you can update the cls (class) property of each box:

for box in result.boxes:
    # Update box label (example: assign label 1 or 2 based on your logic)
    box.cls = 1 if np.array_equal(box.xyxy.numpy()[0], prev_coords[1][0]) else 2

# Plot the frame after updating labels
annotated_frame = result.plot()

This ensures the new labels are reflected during visualization. Let me know if you need further clarification!

[tylerfrisinger01]

@glenn-jocher thank you for getting back to me I know im being a pain, however when i try to implement this for loop i get the following error.
"AttributeError: property 'cls' of 'boxes' object has no setter

the entire code section looks like the following:

results = detection.predict(video_path, stream=True, save=False, max_det=2, verbose=False, save_crop=True, project=output, agnostic_nms=True, iou=0.5)

    results2 = detection.predict(video_path, stream=True, save=True, max_det=2, verbose=False, save_crop=True, project=output, agnostic_nms=True, iou=0.5)

    distance_displayed = 0

    print("Creating Distance Video...")


    prev_coords = {1: (None, None), 2:(None, None)}

    for result in results:


        current_coords = [(box.xyxy.numpy()[0], box.cls.cpu().item()) for box in result.boxes]


        for coord, label in current_coords:

            if prev_coords[1][0] is None:

                prev_coords[1] = (coord, label)

                continue


            if prev_coords[2][0] is None:

                prev_coords[2] = (coord, label)

                continue


            distance_to_1 = np.linalg.norm(np.array(coord) - np.array(prev_coords[1][0]))

            distance_to_2 = np.linalg.norm(np.array(coord) - np.array(prev_coords[2][0]))


            if distance_to_1 < distance_to_2:

                prev_coords[1] = (coord, label)

            else:

                prev_coords[2] = (coord, label)


            if prev_coords[1][1] == prev_coords[2][1]:

                prev_coords[2] = (prev_coords[2][0], label + 1)


            for box in result.boxes:

                box.cls = 1 if np.array_equal(box.xyxy.numpy()[0], prev_coords[1][0]) else 2


        annotated_frame = result.plot()

        if len(result.boxes) > 1:

            #print(f'box1 = {(result.boxes[0].xyxy).tolist()[0]}, box2 = {(result.boxes[1].xyxy).tolist()[0]}')

            distance_displayed = distance((result.boxes[0].xyxy).tolist()[0], (result.boxes[1].xyxy).tolist()[0])

        cv2.putText(annotated_frame, f'distance: {distance_displayed:.2f} cm', (1100, 200), cv2.FONT_HERSHEY_COMPLEX, 4, (255, 255, 255), 3)

        video_writer.write(annotated_frame)

[glenn-jocher]

The error occurs because the box.cls attribute in result.boxes is read-only and doesn't have a setter. To modify the labels for visualization, you can update them dynamically during plotting using the annotator.box_label() method instead of attempting to change box.cls directly. For example:

annotator = Annotator(annotated_frame, line_width=2)
for box in result.boxes:
    label = 1 if np.array_equal(box.xyxy.numpy()[0], prev_coords[1][0]) else 2
    annotator.box_label(box.xyxy, label=str(label))  # Dynamically update label here

This ensures the updated labels are reflected in the output video without modifying the cls attribute. Let me know if you need further clarification!

[ds-dinis007]

Can we use tracker mode other than bot sort and bytetrack? for example with deepsort tracker.
how to configure it?

[glenn-jocher]

@ds-dinis007 yes, you can use trackers like DeepSORT with YOLO11 for object tracking. To configure it, specify the tracker argument with the path to the DeepSORT YAML configuration file when using the track mode. For example:

results = model.track(source="path/to/video.mp4", tracker="deepsort.yaml")

Ensure that the DeepSORT tracker is properly set up, and the YAML file matches your requirements. For more details, consult the tracker configuration section in the documentation.