Need to know about the feature extraction procedure from YOLO_v8 backbone

[97mallikan]

I am using YOLO_v8 to detect objects from video files. Now I want to perform feature extraction on frames of video files using the backbone section of that YOLO_v8 model.
Can anyone guide me on how can I do that? If there is any documentation or example that exists please share it with me, it would be very helpful to me.

Thanks

[pderrenger]

@97mallikan certainly! To perform feature extraction using the backbone of a YOLOv8 model, you can utilize the embed functionality provided by the YOLOv8 Python interface. This allows you to extract feature vectors from specified layers of the model.

Here's a concise example of how you can achieve this:

from ultralytics import YOLO

# Load your YOLOv8 model
model = YOLO('yolov8n.pt')  # Replace with your model path if necessary

# Perform feature extraction on a video frame
# Specify the layers from which you want to extract features
layer_indices = [10, 14, 17]  # Example layer indices, adjust as needed
results = model.predict(source='path/to/your/video.mp4', embed=layer_indices)

# Iterate over results to access the embeddings
for result in results:
    embeddings = result.embeddings  # List of feature tensors from the specified layers
    # Now you can use these embeddings for your application

The embed parameter takes a list of layer indices from which you want to extract features. The predict method will return results that include the embeddings from these layers.

For more detailed information on using the embed parameter and other functionalities of YOLOv8, please refer to the Predict section in the Ultralytics Docs. This will provide you with additional insights and examples on how to work with the model for various tasks, including feature extraction.

[97mallikan]

Thank you for your response.

[pderrenger]

@97mallikan absolutely, happy to help! 🚀 To extract features using the YOLOv8 model's backbone, you can use the .predict() method along with the embed argument. This allows you to specify which layers' outputs you'd like to grab. Here's a quick example:

from ultralytics import YOLO

# Load model
model = YOLO('yolov8n.pt')  # Make sure to use your model path

# Perform prediction and extract features
results = model.predict('path/to/image.jpg', embed=[10, 14, 17])  # Adjust layers as needed

# Access the embeddings
features = results.embeddings

This snippet assumes you're interested in features from layers 10, 14, and 17. Just change those layer indices based on your requirements. Hope this clears things up! Let me know if there's anything else you're curious about. 😊

[AI-Elmanawy]

@pderrenger Thanks for your response but I get this error "AttributeError: 'Tensor' object has no attribute 'embeddings'" any suggestions please

[daisy9542]

@pderrenger Thanks for your response but I get this error "AttributeError: 'Tensor' object has no attribute 'embeddings'" any suggestions please

You can check out this issue: #15678. Using the method of registering forward hooks is useful for me.