tasks/pose/

[giscus[bot]]

tasks/pose/

Learn how to use Ultralytics YOLOv8 for pose estimation tasks. Find pretrained models, learn how to train, validate, predict, and export your own.

https://docs.ultralytics.com/tasks/pose/

[KingAsperin]

Do you already have a documentation on how to implement yolov8 landmark detection on android development? if none is it posible for you to make a thread to teach us how to implement yolov8 model into android development please

[pderrenger]

@KingAsperin hey there! 👋

We don't currently have a dedicated guide for implementing YOLOv8 landmark detection on Android, but it's a great suggestion! 🚀

In the meantime, you can export a YOLOv8 pose model to ONNX or TensorFlow formats, which are more compatible with Android development. For example, you can use the following command to export a model:

yolo export model=yolov8n-pose.pt format=onnx

Once you have the ONNX model, you can use tools like ONNX Runtime Mobile to run it on Android devices. For TensorFlow, you can convert the model to TensorFlow Lite and use the TensorFlow Lite Android Support Library.

I hope this helps! If you have more questions or need further assistance, feel free to ask. Happy coding! 😊

[leonidasavila]

How can i calculate angle position between three points? Thanks

[Kindstep]

Whoops, was not meant to be edited like that :D

[glenn-jocher]

To access specific keypoints in YOLO11 pose estimation, you can index the keypoints array directly using the indices corresponding to each body part. For example, to calculate the angle at the right elbow, you can use keypoints[6], keypoints[8], and keypoints[10] for the right shoulder, right elbow, and right wrist, respectively. Let me know if you need further assistance!

[Kindstep]

Okay, i Already figured it out now that I had time to try it. Seems I was on the right track. :)

Have a great weekend everybody!

[glenn-jocher]

Great to hear you figured it out! If you have more questions or need further assistance, feel free to reach out. Have a wonderful weekend!

[leonidasavila]

@Kindstep thanks for sharing. Good luck with your thesis, it's will be fine!
Currently, we are using YOLO8 and we are studying to migrate YOLO11... I hope the deploy of hand pose very soon!

[ozankavcu38]

Sir i just wanna ask you something about live training . I was making some calculations and getting some results that is okay by the way yolov8 is very good about pose estimation issues. I just wanna ask when i am training mp4 files ( my main is live training however i am just using for the testing this ) . The model is drawing everything like boxes poses conf values etc. I dont want to see nothing on the video however i just want to keep operating the results by the way . In addition to this i just want to write something on video like i have counted 3 people on the top of right corner of that video how could i do this i did not see any answer . when i turn off results = model(source="C://Users//Hp//Desktop//videolar//operation_room_1.mp4", show=True, conf=0.3, save=False,
verbose=False, stream=True,show_conf=True). like show false i did not see anything about video it is completely dissapeared or when i turn off stream yeah my calculations is continue however the video is dissapeared . I want to show the video appeared however there must be nothing as image , poses , boxes .

[glenn-jocher]

Hello! It sounds like you're doing some interesting work with live training and pose estimation using YOLOv8. To address your concerns about visualizing the video without drawing boxes, poses, or confidence values, while still being able to operate on the results and add custom text annotations, here's a simplified approach:

1. To process the video without visualizing detections: You can set show=False to prevent displaying the video with detections. This will still allow you to access the detection results programmatically.

2. To add custom text annotations: You can use OpenCV to write text on the video frames. Here's a small code snippet to help you get started:

import cv2
from ultralytics import YOLO

# Load your model
model = YOLO('path/to/your/model.pt')

# Open the video
cap = cv2.VideoCapture("C://Users//Hp//Desktop//videolar//operation_room_1.mp4")

# Process each frame
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Run model prediction
    results = model(frame, show=False)

    # Example: Count people and add text annotation
    people_count = sum(1 for *_, cls in results.xyxy[0] if cls == 0)  # Assuming class 0 is 'person'
    cv2.putText(frame, f"People Count: {people_count}", (50, 50), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2)

    # Show the frame
    cv2.imshow('Frame', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

This script processes each frame to count the number of people (assuming class 0 corresponds to 'person') and displays the count on the top right corner of the video. Adjust the class index as per your model's class labels.

Remember, the show=False parameter in the model() call ensures that the original video is processed without overlaying the detections, while cv2.putText() is used to add custom text annotations.

I hope this helps! Let me know if you have any more questions. 😊

[samverh]

Hi, is it possible to run this model on the OAK-D camera with VPU?

[pderrenger]

Hey there! 😊

After converting your YOLOv8 model for use with the OAK camera, the output format indeed changes, and parsing the results might require some adjustments. For handling the output layer and parsing keypoints correctly, I recommend checking out the Pose Estimation documentation on the Ultralytics Docs page. It provides insights into how keypoints are represented and might give you a good starting point.

For a quick example, if you're working with an ONNX model, you might use something like this to parse the output:

import numpy as np

# Assuming 'output' is the model's output tensor
keypoints = np.where(output > confidence_threshold)  # Apply confidence threshold
# Process 'keypoints' as needed

This is a simplified example, and the exact implementation will depend on your model's output structure. If you need more specific guidance, feel free to ask! 🚀

[samverh]

Hi, your help is really appreciated.

I am still struggling with parsing the output in the proper way. I inspected the model using Netron and the output layer should have a shape of (1, 56, 8400). The actual shape is (470400, ) so I perform the following conversion:

outputs = data.reshape(1, 56, 8400)
outputs = outputs.transpose(0, 2, 1)

Now I assume the bounding boxes can be extracted by iterating over the data as outputs[0][i][:4] where i is the row index. Then, I convert the boxes with the following function and filter the most relevant boxes using NMS.

def xywh2box(bbox_vals):

    center_x = bbox_vals[0]
    center_y = bbox_vals[1]
    w = bbox_vals[2] 
    h = bbox_vals[3]
    x = center_x - 0.5 * w # center_x to top_left_x
    y = center_y - 0.5 * h # center_y to top_left_y
    box = [x, y, x + w, y + h]

    return box

When plotting the results including keypoint coordinates I am getting very small boxes. It seems that my data conversion is incorrect or that I should apply scaling, but the image sizes and input shapes are both (640, 640).

What am I missing here?

[pderrenger]

@samverh hey there! 👋 It looks like you're on the right track with reshaping and transposing the output. However, if the boxes appear too small, it might indeed be a scaling issue. When working with models converted to OpenVINO and running on devices like the OAK-D, remember that the output coordinates are normalized. You might need to scale them back to the original image dimensions.

Here's a quick tip: Ensure you're scaling the bounding box coordinates by the original image width and height before converting them with your xywh2box function. For example:

def scale_boxes(outputs, img_width, img_height):
    scaled_outputs = []
    for i in range(outputs.shape[1]):  # Iterate over each detection
        bbox = outputs[0][i][:4]
        bbox[0] *= img_width  # Scale x
        bbox[1] *= img_height # Scale y
        bbox[2] *= img_width  # Scale width
        bbox[3] *= img_height # Scale height
        scaled_outputs.append(xywh2box(bbox))
    return scaled_outputs

Make sure to replace img_width and img_height with the actual dimensions of your input images. This should help correct the size of the bounding boxes. If you're still facing issues, double-check the model's input and output layer specifications and ensure they align with your preprocessing steps. Keep up the great work! 🚀

[samverh]

The problem was the input from the OAK-D camera. Frames from ColorCamera/MonoCamera are in U8 data type [0, 255]. To obtain [0, 1] values that can be passed to the model, rescaling is needed by setting mean=0 and scale=255. This can be done when converting the model to blob format:

import blobconverter

# convert onnx to blob
blobconverter.from_onnx(
    model=f"{yolo_modelname}.onnx", 
    output_dir=f"{folder}/",
    data_type="FP16",
    shaves=6,
    use_cache=False,

    optimizer_params=[
        "--mean_values=[0,0,0]",
        "--scale_values=[255,255,255]",
    ]
)

Thanks again for your help! 🙌

[pderrenger]

Hey there! 🙌 It looks like you've done a great job converting the model and reshaping the output for use with your OAK-D camera. The issue with small bounding boxes might indeed stem from not scaling the output coordinates back to the original image dimensions.

Given the output shape (1, 56, 8400) and after your reshaping to (1, 8400, 56), it's crucial to remember that the model outputs are normalized to [0, 1]. To map these back to the original image size, you'll need to scale the bounding box coordinates (x, y, width, height) by the width and height of the input image.

Here's a quick adjustment to your xywh2box function to include scaling:

def xywh2box(bbox_vals, img_width, img_height):
    center_x = bbox_vals[0] * img_width
    center_y = bbox_vals[1] * img_height
    w = bbox_vals[2] * img_width
    h = bbox_vals[3] * img_height
    x = center_x - 0.5 * w  # Convert center_x to top_left_x
    y = center_y - 0.5 * h  # Convert center_y to top_left_y
    box = [x, y, x + w, y + h]
    return box

Make sure to pass the img_width and img_height of your original image to this function. This should correct the size of your bounding boxes. If you're still encountering issues, double-check the output layer's interpretation to ensure you're extracting the bounding box and keypoint information correctly.

Keep up the great work, and don't hesitate to reach out if you have more questions! 😊

[shekharsingh8811]

Hi,
Could you please let me know which approach does the yolov8-pose model take? And, how does it output the bounding boxes?
Is it,

1. Top-down approach — which involves a person detector that outputs the bounding boxes and then a pose is calculated for each of these bounding boxes.
   OR
2. Bottom-up approach — which involves detecting all parts of each person within an image and then associating or grouping the parts that belong to each individual. Finally, the estimated pose is then used to calculate the bounding box thus encapsulating all the detected keypoints within it.

My motive here is to figure out, if there would be any difference between the bounding box output that I would get from yolov8 (detection) model and the yolov8-pose model. Would there be any?

Thank You!!

[glenn-jocher]

Hi @shekharsingh8811,

Thank you for your detailed question! 😊

The behavior you're observing, where the YOLOv8-pose model outputs 17 keypoints even for partially visible persons and occasionally produces keypoint coordinates at the edge (640.0), is expected. This is because the model is designed to predict all keypoints regardless of visibility, and it attempts to place non-visible keypoints as close to the edge of the frame as possible.

Regarding the keypoint coordinates reaching 640.0, this can happen due to the model's bounding box normalization and rounding during inference. While the coordinates should ideally be within the image dimensions (0.0 to 639.0), slight deviations can occur.

To address this, you can post-process the keypoints to ensure they fall within the expected range. Here's a quick example in Python:

import numpy as np

# Assuming 'results' is the output from the model
for result in results:
    keypoints = result.keypoints.xy
    keypoints = np.clip(keypoints, 0, 639)  # Ensure keypoints are within the image bounds
    # Further processing or visualization

If you haven't already, please ensure you're using the latest versions of torch and ultralytics. You can update them using:

pip install --upgrade torch ultralytics

If the issue persists, could you provide a minimum reproducible example? This will help us investigate further. You can find guidelines for creating one here: Minimum Reproducible Example.

Feel free to reach out if you have more questions or need further assistance!

[shekharsingh8811]

Hi @glenn-jocher,

Thank you for the response!
I'm already clipping the keypoints for my use case, so this resolves the issue for now.

[glenn-jocher]

Hi @shekharsingh8811,

Thank you for your response! 😊

I'm glad to hear that clipping the keypoints resolves the issue for your use case. The behavior you described, where the model outputs keypoints even for partially visible persons and occasionally produces coordinates at the edge (640.0), is indeed expected due to the model's bounding box normalization and rounding during inference.

If you have any further questions or need additional assistance, feel free to ask. For more detailed information on pose estimation, you can refer to our Pose Estimation Documentation.

Happy coding! 🚀

[aleshem]

Hi Glenn,
Can you please explain why YOLO-v8_pose uses a top down approach and not a bottom up approach? Would it not make more sense to calculate the keypoints for each point in the grid at one instead of performing a two stage solution? this would also make sure that all the post processing and normalization would be with the same scales which may improve problems in quantization. If the calculation in YOLO-v11_pose similar to that of v8?
Thanks!

[glenn-jocher]

@aleshem yOLOv8-pose uses a top-down approach for efficiency and simplicity, focusing on detecting persons first and then estimating keypoints within those bounding boxes. This method aligns well with the architecture's strengths in object detection. While a bottom-up approach could offer different benefits, the top-down method ensures consistent scaling and processing. YOLOv11-pose follows a similar approach. If you have further questions, feel free to ask!

[StavrosAnaniadis]

Hello! I am trying to output all the points detected by yolov8 into a excel or csv file in a video, and am having trouble with it. I am using the code:
from ultralytics import YOLO

model = YOLO('yolov8m-pose.pt')
video_path = #video path here

results = model(source=video_path, show=True, conf = 0.3, save=True)

Is there any way I can save all of the coordinates for the 2 people in the video to an excel file?

Thank you,
Stavros Ananiadis

[pderrenger]

@StavrosAnaniadis hi there! 🙌

For saving the keypoints to an Excel or CSV file, including their labels like "left arm", "right wrist" etc., you'll need to modify the approach slightly. Based on your requirement and the given structure of the Keypoints object, here's a concise example using pandas to save the data in a more structured format, including the labels:

import pandas as pd
from ultralytics import YOLO

# Define keypoint labels for your pose model (order should match the model's output)
labels = ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder",
          "right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist",
          "left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"]

model = YOLO('yolov8m-pose.pt')
results = model(source=video_path, show=True, conf=0.3, save=True)

# Initialize an empty list to store data
data = []

# Loop through results and store keypoints data
for frame_results in results:
    for person_id, person in enumerate(frame_results.keypoints):
        for label, (x, y, conf) in zip(labels, person.numpy()):
            data.append({
                "person_id": person_id,
                "label": label,
                "x": x,
                "y": y,
                "confidence": conf
            })

# Create a DataFrame and save it to an Excel file
df = pd.DataFrame(data)
df.to_excel("keypoints.xlsx", index=False)

This script runs through the keypoints detected on each person in every frame, collecting their labels, coordinates, and confidence levels into a structured format that's then written to an "keypoints.xlsx" Excel file.

Hope this serves your project well, especially with the fascinating application of machine learning in analyzing sword fights! If you need more info on the keypoint indexes and labels, referring to this Ultralytics docs page might be helpful. Good luck with your analysis! 🤺👀

[StavrosAnaniadis]

Thank you so much for your help.

When I run this code, I get the error:
for label, (x, y, conf) in zip(labels, person.numpy()):

ValueError: not enough values to unpack (expected 3, got 1)
Referring to line 26: for label, (x, y, conf) in zip(labels, person.numpy()):

Do you know why this would not be working?

Thank you,
Stavros Ananiadis

[pderrenger]

Hi Stavros,

It looks like the error stems from unpacking the keypoints incorrectly. The keypoints are structured as xy, xyn, and conf for the coordinates, normalized coordinates, and confidences respectively, according to the Ultralytics documentation. Here's how you can modify your loop for correct unpacking:

import pandas as pd

data = []
# Assuming 'results' is from your prediction
for frame_id, result in enumerate(results):
    for person_id, person in enumerate(result.keypoints):
        keypoints = person.xy  # Get keypoints
        confidences = person.conf  # Get confidences
        for kp_id, (xy, conf) in enumerate(zip(keypoints, confidences)):
            label = labels[kp_id]  # Get the label from predefined labels list
            data.append([frame_id, person_id, label, *xy, conf])

# Convert the data into a pandas DataFrame
df = pd.DataFrame(data, columns=['frame_id', 'person_id', 'label', 'x', 'y', 'confidence'])

# Save to an Excel file
df.to_excel('keypoints.xlsx', index=False)

Remember, for this to work, ensure you have labels list correctly ordered as per your model's keypoints output.

This adaptation should resolve the unpacking issue and store coordinates with their corresponding labels like "left arm", "right wrist", etc. 🚀

Best of luck with your project on providing insights to sword fighters. It sounds like a cutting-edge (pun intended) application of AI! If you have more questions or hit another snag, feel free to ask.

[StavrosAnaniadis]

Hello! Thank you for your continued assistance. On running the code, a new error appeared:
ValueError: 6 columns passed, passed data had 21 columns
My labels look like:

Define keypoint labels for your pose model (order should match the model's output)

labels = ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder",
"right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist",
"left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"]
Would you happen to know whats causing that?

Thank you,
Stavros Ananiadis

[pderrenger]

Hello Stavros,

It seems the issue arises because there’s a mismatch in the structure of the data being written to the CSV and the expected format. The error indicates that the DataFrame construction is attempting to handle more columns than specified.

Given the ValueError related to column mismatch and considering the structure of the keypoints in the YOLOv8 pose estimation model, it appears we need to adjust how the keypoints are unpacked before saving them. Specifically, the xy, conf values should be handled correctly.

The correct usage for processing keypoints and confidences involves accessing these attributes directly from the person object as .xy or .conf and ensuring each keypoint and its confidence are correctly represented:

import pandas as pd
from ultralytics import YOLO

model = YOLO('yolov8m-pose.pt')
results = model(source=video_path, conf=0.3)

data = []

for frame_id, (frame_results) in enumerate(results):
    for person_id, person in enumerate(frame_results.keypoints):
        xy = person.xy.numpy()  # keypoints coordinates
        conf = person.conf.numpy()  # confidence scores of keypoints
        for kp_id, (xy, kp_conf) in enumerate(zip(xy, conf)):
            label = labels[kp_id]  # Using the predefined labels list
            x, y = xy
            data.append({"frame_id": frame_id, "person_id": person_id, "label": label, "x": x, "y": y, "confidence": kp_conf})

df = pd.DataFrame(data)
df.to_csv('keypoints.csv', index=False)

Make sure the labels list matches your model's output order. This script aims to correctly handle the extraction of keypoint coordinates and confidence scores, complementing your innovative approach to analyzing sword fighting techniques with AI. Keep pushing the boundaries! 🤺🚀

If you have more questions or need further assistance, feel free to reach out.

[arav1ndb]

Where can I find the mapping for the xy coordinates obtained by the algorithm, as in which point pertains to which body part?

[arav1ndb]

I got it here, for anyone else searching for it

# Define keypoint labels for your pose model (order should match the model's output)
labels = ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder",
          "right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist",
          "left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"]

[glenn-jocher]

@arav1ndb great find! 🌟 For anyone else curious about the keypoint labels for pose estimation models, here they are:

labels = [
    "nose", "left_eye", "right_eye", "left_ear", "right_ear",
    "left_shoulder", "right_shoulder", "left_elbow", "right_elbow",
    "left_wrist", "right_wrist", "left_hip", "right_hip",
    "left_knee", "right_knee", "left_ankle", "right_ankle"
]

These labels correspond to the output keypoints of your pose model. When interpreting [x, y] coordinate predictions, you can match each point to the respective body part listed above. Happy coding! 😊

[dolfMo]

Hello! I would like to ask if it is possible to output the coordinate information of the specified key point, which is the coordinate point of the wrist

[glenn-jocher]

@dolfMo hello! 😊 Yes, it's definitely possible to get the coordinate information of specific keypoints, such as the wrist, using YOLOv8's pose estimation models. If you're working with a model like yolov8n-pose.pt, the keypoint coordinates, including those for the wrist, are outputted as part of the model's predictions.

Here's a quick Python example on how to access these keypoints:

from ultralytics import YOLO

# Load the trained pose estimation model
model = YOLO('yolov8n-pose.pt')

# Run predictions
results = model('path/to/your/image.jpg')

# Access keypoints for each detected person
for person in results.keypoints:
    wrist_coords = person.xy[person.names.index('wrist')]  # Assuming 'wrist' is part of your model's keypoints
    print(f'Wrist coordinates: {wrist_coords}')

Make sure to replace 'path/to/your/image.jpg' with the path to your image. The keypoint names (like 'wrist') and their exact order depend on your model and dataset, so you might need to adjust the person.names.index('wrist') part to fit your specifics.

Let me know if there's anything else I can help with! 🌟

[dolfMo]

Hello, I encountered the following error message after executing your code. I would like to know why?
for person in results.keypoints: AttributeError: 'list' object has no attribute 'keypoints'

[jirivchi1]

result = results[0]
Extract keypoints
keypoints = result.keypoints

It works for me in the previous code but I am got a new error

[glenn-jocher]

Hey there! 👋 It looks like you're encountering an attribute error because results is a list, not directly accessing attributes. For pose estimation with YOLOv8, you'll need to interact with the list of results to get the keypoints. Here's how you can correctly access keypoints:

from ultralytics import YOLO

# Load your model
model = YOLO('yolov8n-pose.pt') 

# Run your prediction
results = model('path/to/image.jpg')

# Correct way to access keypoints 
for result in results:
    if hasattr(result, 'keypoints'):
        for person in result.keypoints:
            wrist_coords = person.xy[person.names.index('wrist')]
            print(f'Wrist coordinates: {wrist_coords}')

This snippet checks if the keypoints attribute exists in each result and then iterates through each person detected to find the wrist coordinates. Let me know if that helps or if you have any more questions!

Keep on exploring! 🚀

[pyrat]

Hi,

I have trained a model based on YOLOv8l-pose, when I infer the model I get None for confidence and the keypoints just have x and y. Is there a reason why there is no confidence? here is contents of the keypoints object:

proto-proto-1 | Initializing LandmarkDetector
proto-proto-1 | YOLOv8l-pose summary (fused): 287 layers, 44464206 parameters, 0 gradients, 168.6 GFLOPs
proto-proto-1 |
proto-proto-1 | 0: 192x640 1 fish, 56.9ms
proto-proto-1 | Speed: 1.3ms preprocess, 56.9ms inference, 164.3ms postprocess per image at shape (1, 3, 192, 640)
proto-proto-1 | ultralytics.engine.results.Keypoints object with attributes:
proto-proto-1 |
proto-proto-1 | conf: None
proto-proto-1 | data: tensor([[[ 7.0377, 94.4488],
proto-proto-1 | [ 35.9414, 107.0425],
proto-proto-1 | [ 1.5750, 111.7595],
proto-proto-1 | [ 55.6107, 92.0907],
proto-proto-1 | [101.0829, 49.3980],
proto-proto-1 | [147.8248, 93.4525],
proto-proto-1 | [112.3623, 138.2682],
proto-proto-1 | [153.1546, 141.8253],
proto-proto-1 | [277.1523, 16.1814],
proto-proto-1 | [343.0081, 22.3252],
proto-proto-1 | [324.8737, 135.0928],
proto-proto-1 | [429.2652, 111.5511],
proto-proto-1 | [470.5245, 98.0572],
proto-proto-1 | [457.7650, 36.1689],
proto-proto-1 | [479.3454, 46.8262],
proto-proto-1 | [507.4113, 54.2151],
proto-proto-1 | [532.2663, 45.4843],
proto-proto-1 | [542.9807, 83.6471],
proto-proto-1 | [521.8376, 102.8704],
proto-proto-1 | [506.2361, 90.8436]]], device='cuda:0')
proto-proto-1 | has_visible: False
proto-proto-1 | orig_shape: (154, 576)
proto-proto-1 | shape: torch.Size([1, 20, 2])
proto-proto-1 | xy: tensor([[[ 7.0377, 94.4488],
proto-proto-1 | [ 35.9414, 107.0425],
proto-proto-1 | [ 1.5750, 111.7595],
proto-proto-1 | [ 55.6107, 92.0907],
proto-proto-1 | [101.0829, 49.3980],
proto-proto-1 | [147.8248, 93.4525],
proto-proto-1 | [112.3623, 138.2682],
proto-proto-1 | [153.1546, 141.8253],
proto-proto-1 | [277.1523, 16.1814],
proto-proto-1 | [343.0081, 22.3252],
proto-proto-1 | [324.8737, 135.0928],
proto-proto-1 | [429.2652, 111.5511],
proto-proto-1 | [470.5245, 98.0572],
proto-proto-1 | [457.7650, 36.1689],
proto-proto-1 | [479.3454, 46.8262],
proto-proto-1 | [507.4113, 54.2151],
proto-proto-1 | [532.2663, 45.4843],
proto-proto-1 | [542.9807, 83.6471],
proto-proto-1 | [521.8376, 102.8704],
proto-proto-1 | [506.2361, 90.8436]]], device='cuda:0')
proto-proto-1 | xyn: tensor([[[0.0122, 0.6133],
proto-proto-1 | [0.0624, 0.6951],
proto-proto-1 | [0.0027, 0.7257],
proto-proto-1 | [0.0965, 0.5980],
proto-proto-1 | [0.1755, 0.3208],
proto-proto-1 | [0.2566, 0.6068],
proto-proto-1 | [0.1951, 0.8978],
proto-proto-1 | [0.2659, 0.9209],
proto-proto-1 | [0.4812, 0.1051],
proto-proto-1 | [0.5955, 0.1450],
proto-proto-1 | [0.5640, 0.8772],
proto-proto-1 | [0.7453, 0.7244],
proto-proto-1 | [0.8169, 0.6367],
proto-proto-1 | [0.7947, 0.2349],
proto-proto-1 | [0.8322, 0.3041],
proto-proto-1 | [0.8809, 0.3520],
proto-proto-1 | [0.9241, 0.2954],
proto-proto-1 | [0.9427, 0.5432],
proto-proto-1 | [0.9060, 0.6680],
proto-proto-1 | [0.8789, 0.5899]]], device='cuda:0')

[glenn-jocher]

Hey there! 🌟 It looks like the keypoints are indeed present but the confidence associated with each keypoint isn't being returned directly in your output. This behavior depends on how the model was trained and how the inference script processes the outputs.

For YOLOv8l-pose, confidence scores are typically associated with the detection rather than individual keypoints. If you need per-keypoint confidence, consider modifying the model's architecture or post-processing script to include this information.

As for the keypoints, x and y in your output represent the normalized positions of each keypoint detected by your model. This is standard for pose estimation tasks, where the main goal is to locate key positional landmarks.

If your application requires confidence scores for each keypoint, you may explore customizing the post-processing step to calculate these based on the model's predictions or the detection's overall confidence score.

For now, you can work with the provided keypoints as they give you the essential information about the position of each landmark detected in the image. Happy coding! 😊

[PolinKun]

can we model pose estimation for excavator work?

[glenn-jocher]

@PolinKun absolutely! Pose estimation can be utilized for modeling excavator work by identifying and tracking keypoints on the excavator. For instance, keypoints could represent the excavator's arm joints, bucket, and other significant parts to analyze its movements and work dynamics.

Here's a quick start on doing so with a YOLOv8n-pose model:

from ultralytics import YOLO

# Load a pretrained pose estimation model
model = YOLO('yolov8n-pose.pt')

# Predict on an image
results = model('path/to/excavator/image.jpg')

# Display the keypoints
results.show()

This example loads a pretrained YOLOv8n-pose model and runs it on an excavator image to detect keypoints. You might want to train the model on a dataset specific to excavators for optimal results. Training on the excavator-specific dataset would involve collecting images of excavators, annotating them with keypoints, and following the training process as detailed in our docs.

Feel free to explore the Pose Estimation section for more insights and guidance on custom training for your specific needs. Happy modeling! 😊

[shukurullo2004]

hello, i hope you can help i am struggled to create logic human falling based on shoulder and knee distance if shoulder is below than knee that means human is fallen, but i can not get keypoints 😭of them and calculate distance, becouse it is first time to using yolov pose estimation. thank you

[shukurullo2004]

another question: can i find velocity of shoulder and knees using keypoints ?
if yes, how can i ?

[pderrenger]

Hello! 😊 Yes, you can calculate the velocity of shoulders and knees using keypoints from pose estimation. Velocity is essentially the change in position over time. If you have keypoints for shoulders and knees detected over sequential frames, you can calculate the distance they move between frames and divide by the time elapsed to get their velocity.

Here's a simple example on how you might approach this:

from ultralytics import YOLO

# Load your pose model
model = YOLO('yolov8n-pose.pt')

# Assume frame1 and frame2 are consecutive frames from your video
results_frame1 = model(frame1)
results_frame2 = model(frame2)

# Extract keypoint locations for shoulders and knees for both frames
# For demonstration, let's say the IDs for shoulders and knees are 2 and 3 respectively
shoulder_frame1 = results_frame1.keypoints[2]
knee_frame1 = results_frame1.keypoints[3]
shoulder_frame2 = results_frame2.keypoints[2]
knee_frame2 = results_frame2.keypoints[3]

# Calculate the distance moved by each keypoint (simple Euclidean distance for demonstration)
distance_shoulder = ((shoulder_frame2[0] - shoulder_frame1[0])**2 + (shoulder_frame2[1] - shoulder_frame1[1])**2)**0.5
distance_knee = ((knee_frame2[0] - knee_frame1[0])**2 + (knee_frame2[1] - knee_frame1[1])**2)**0.5

# Assuming you know the time elapsed between the two frames (time_elapsed), calculate velocity
velocity_shoulder = distance_shoulder / time_elapsed
velocity_knee = distance_knee / time_elapsed

Please adjust the keypoint IDs according to the specific model outputs and ensure the correct handling of the coordinate system used in your data. The velocity calculation here is a basic illustration. Depending on your exact needs, especially if you're dealing with video data, you might need a more robust approach to handle varying frame rates and more complex movement patterns. Good luck with your project!

[shukurullo2004]

thank you helping me, with your help i can detect delta distance but how can i measure time ?

i am doing project human fall detection with using yolov8-pose.pt model, so i can find keypoints but logical part is being difficult, my logic is that comparing velocity of shoulder if shoulder velocity > knee velocity AND if position of shoulder below than knee points that means human is fallen.

so can you help me write this logic

thank you a lot

[pderrenger]

@shukurullo2004 hey there! 😃 I'm glad to hear you've made progress with keypoint detection using the YOLOv8-pose model. For measuring the time, especially in videos, you can use the frame rate (FPS) of your video. If you're processing frame by frame, the time between frames would be 1/FPS. Here's a simplified way to include it in your logic:

fps = 30  # Example FPS of your video
time_elapsed = 1 / fps  # Time between each frame

# Your logic for comparing shoulder and knee velocities
# Assuming you already calculated velocities velocity_shoulder and velocity_knee
if (velocity_shoulder > velocity_knee) and (shoulder_position < knee_position):
    print("Human has fallen")

Remember to adjust fps according to your video's frame rate. This way, you can incorporate time measurement into your logic seamlessly. Best of luck with your project on human fall detection! Feel free to share any further questions. 🚀

[mihkriss]

Hello, I want to ask you for help
My task is to determine in the video whether there is a fight or not, using a model based on key points,
Could you tell me how to implement this using yolov8-pose ^_^

[pderrenger]

Hello there! 😄 It's awesome that you're looking to use YOLOv8-pose for detecting fights in videos based on key points. Here's a simple way to get started:

First, make sure you've installed the Ultralytics YOLOv8 package. If not, you can do so using pip:

pip install ultralytics

Next, you’ll want to use a YOLOv8 pose estimation model. Since your task is quite specific, I’d recommend starting with our pretrained yolov8n-pose.pt model to detect human poses and then apply your logic to determine if a fight is happening based on the poses/keypoints patterns observed.

Here’s how you can run predictions on a video file:

from ultralytics import YOLO

# Load a pretrained pose model
model = YOLO('yolov8n-pose.pt')

# Run predictions on a video file
results = model.predict('path/to/your/video.mp4', stream=True)

# Process your results here to analyze key points and infer fighting scenarios

You might need to develop a custom algorithm that analyzes the keypoints for each person detected in the video frames, looking for patterns that might indicate a fight (like proximity of people, rapid changes in pose, etc.).

Please note, identifying fights based on motion and pose in videos can be complex and may require further fine-tuning or training the model on a dataset specific to fighting scenarios if the results aren’t satisfactory.

Good luck with your project, and feel free to reach out if you have more questions! 👍

[mihkriss]

Do I understand correctly that I need to first determine the key points (their coordinates)? How to do this with video? and how will they be identified if there are several people there?

[pderrenger]

@mihkriss hey there! Yes, you've got the basic idea right. 😊 To work with videos and detect key points for multiple people, here's a streamlined approach using YOLOv8-pose models, like yolov8n-pose.pt:

1. Model Prediction on Video Frames: Pass each frame of your video to the model. It will automatically detect people and their keypoints.

   from ultralytics import YOLO
   
   # Load a pretrained model
   model = YOLO('yolov8n-pose.pt')
   
   # Predict on a video
   results = model.predict('path/to/your/video.mp4', stream=True)

2. Handling Multiple People: The model outputs keypoints for all detected people in a frame. Each person will have their own set of keypoints.

3. Iterating Through Results: Use a loop to iterate through each result's keypoints. You can differentiate between people by their unique set of keypoints.

Here's an example of accessing keypoints for further processing:

for result in results:
    for person in result.keypoints:
        keypoints = person.xy  # Access X, Y coordinates
        # Your logic to analyze keypoints here

YOLOv8's pose estimation models are pretrained on the COCO keypoints dataset and are optimized to handle multiple people in a scene efficiently. They provide the keypoints' locations along with confidence scores, making it easier to analyze complex scenes. 🚀

Hope this helps! Feel free to reach out if you have more questions.

[Alexis191019]

And to determine the coordinates of a key point? For example, the coordinates of the right ankle and then print it on the screen?

[glenn-jocher]

To determine and print the coordinates of a specific keypoint, such as the right ankle, you can access the keypoints from the model's prediction results. Here's a minimal example:

from ultralytics import YOLO

# Load the model
model = YOLO('yolo11n-pose.pt')

# Run prediction
results = model('path/to/your/video.mp4', stream=True)

# Iterate through results
for result in results:
    for person in result.keypoints:
        right_ankle = person.xy[16]  # Index 16 for the right ankle
        print(f"Right Ankle Coordinates: {right_ankle}")

This will print the coordinates of the right ankle for each detected person in the video frames.

[mihkriss]

Hello! I have a task to detect fights by key points, I extracted points from the dataset to train the model to detect a fight. I have several questions:

1. In what format should points be stored so that they can be trained later?
2. And once I save the points in the desired format, how can I train the model to determine the fighting pose?
3. And in the final output I need to display only a frame with a label on the video that there is a fight going on here
   Please help me deal with these questions :(

[glenn-jocher]

Hello! It sounds like an interesting project you're working on. Let's address your questions to help get you started with detecting fights using key points:

1. Point Format: The keypoints should be stored in a format that represents the x, y coordinates (and possibly the visibility) of each keypoint. For YOLOv8 pose estimation, keypoints are typically encoded in a list or array per object instance, each element holding the [x, y, visible] information.

2. Training for Fighting Pose: With your points appropriately formatted and saved, you can train a model like yolov8n-pose to recognize fighting poses. Start by organizing your data according to the YOLO pose dataset format, then train the model using command:

   yolo pose train data=your_dataset.yaml model=yolov8n-pose.pt epochs=100 imgsz=640

   Make sure your your_dataset.yaml points to your training and validation data, configured similarly to the coco-pose.yaml.

3. Displaying Frames with a Label: For the output, after prediction, you can process the results to check if a fighting pose is detected and then annotate the frame with a label indicating the fight. Here's a snippet:

   from ultralytics import YOLO
   model = YOLO('path/to/your_trained_model.pt')
   results = model('path/to/video_or_image')
   if 'fight' in results.pandas().xyxy[0]['name'].values:  # assuming 'fight' is a class name
       # This code assumes you're working with frames;
       # you can annotate the frame here to indicate a fight is detected.

I hope this helps clarify your workflow! If you have any more questions or need further assistance, don't hesitate to ask. Happy coding! 😊

[mihkriss]

Why when withdrawing
result.boxes.xyxy
displays
tensor([[ 85., 335., 755., 1521.]])
which does not correspond to the format from 0 to 1

[glenn-jocher]

Hey there! When you get result.boxes.xyxy, it actually returns the bounding boxes in pixel coordinates, not normalized ones. 😊 If you're looking to work with coordinates normalized between 0 and 1, you can do so by dividing by the image dimensions. Here's a quick example:

# Assuming 'result' is your prediction output and 'img' is your input image.
normalized_boxes = result.boxes.xyxy / torch.tensor([img.shape[1], img.shape[0], img.shape[1], img.shape[0]]).to(result.boxes.device)

This will give you the coordinates in the format you're looking for!

For pose estimation using YOLOv8, keypoints are usually the focus, but knowing how to handle bounding boxes is definitely useful too. If there's anything else you'd like to dive into, feel free to ask! 🌟

[gustavoortiz7]

hola buenas
estoy haciendo un proyecto de correción deportiva , remates de voley , y bueno para poder lograr esto intento usar yolov8-pose para , poder obtener los puntos clases de la persona y luego convertirlos en secuencias para una lstm , quería saber cómo puedo obtener los puntos claves pero solo de la persona que hace el remate , por que yolo de detecta a todas las personas que ve en el frame y los se quiere hacer es que en ves de mandarle un vídeo , mandarle una dirección para así poder leer todos los fotogramas de distintas acciones , bueno en este caso solo son dos (rematebueno y remate malo) no trabajo con video por que , saca muchos fotogramas repetidas aunque al final mi proyecto debe detectar en tiempo real el error que cometió , agradecería mucho si puedes proporcionarme como convertir los puntos clases en secuencias y si tiene algún concejos de cómo abordar el problema de mejor manera gracias

[pderrenger]

¡Hola! Para focalizar la detección de puntos clave únicamente en la persona que realiza el remate en vóley, una estrategia sería primero identificar a esa persona específica usando algún criterio (como la posición en el marco) y luego ejecutar la estimación de pose solamente en esa persona detectada.

Aquí un breve ejemplo de cómo podrías hacerlo:

from ultralytics import YOLO

# Carga tu modelo de estimación de pose
model = YOLO('yolov8n-pose.pt')

# Para procesar cada imagen de tu directorio
for imagen in tus_imagenes:
    # Realiza detecciones
    results = model(imagen)
    
    # De alguna manera determina cuál de los detectados es el jugador haciendo el remate
    # Esto podría implicar filtrar por ubicación si siempre rematan desde una posición similar
    jugador_remate = filtrar_jugador(results)
    
    if jugador_remate:
        # Convierte los puntos clave (keypoints) del jugador a secuencias para tu LSTM
        keypoints = jugador_remate.keypoints
        secuencia = convertir_a_secuencia(keypoints)
        # Usa secuencia para tu LSTM...

Este código es solo un esqueleto para darte una idea. La función filtrar_jugador sería personalizada para identificar al jugador correcto, y convertir_a_secuencia dependerá de cómo requieras estructurar los datos para tu LSTM.

Recuerda, el rendimiento efectivo dependerá mucho de la precisión de tu modelo YOLOv8-pose y cómo de bien puedes filtrar al jugador que realiza el remate. Podría serte útil recortar previamente la región de interés (en donde el jugador lleva a cabo el remate) antes de pasarle la imagen al modelo de pose.

Para leer los fotogramas desde una dirección específica, puedes usar algún método para iterar sobre los archivos en ese directorio. Python y las bibliotecas de manejo de archivos te permitirán hacer esto de manera bastante sencilla.

Finalmente, para el análisis en tiempo real, después de haber entrenado tu modelo LSTM con suficientes datos, podrás clasificar los remates en tiempo real procesando individualmente los frames de vídeo capturado.

Espero que esto te ayude en tu proyecto de corrección deportiva. ¡La combinación de YOLOv8-pose y una LSTM suena prometedora!🚀

[Bekbo01]

Hi there! Is it possible to train pose model for 2 or more classes? If it possible can you share with examples?

[pderrenger]

@Bekbo01 hey there! 🌟 Yes, you can have a .yaml configuration for training models with two or more classes. Here's a simplified example for a pose estimation task with two classes:

# custom-pose-dataset.yaml
train: /datasets/coco128/images/train2017/  # path to training images
val: /datasets/coco128/images/val2017/      # path to validation images

nc: 2  # number of classes
names: ['Class1', 'Class2']  # class names

# Define keypoints for each class
keypoints:
  - [nose, left_eye, right_eye, left_ear, right_ear, left_shoulder, right_shoulder, left_elbow, right_elbow, left_wrist, right_wrist, left_hip, right_hip, left_knee, right_knee, left_ankle, right_ankle]  # Class1 keypoints
  - [point1, point2, point3, point4]  # Class2 keypoints
  
# Example for specific keypoints count per class if different
num_keypoints:
  - 17  # Number of keypoints for Class1
  - 4   # Number of keypoints for Class2

Make sure the keypoints and num_keypoints sections reflect the actual structure of your dataset for each class. You'll need to customize paths and names accordingly. This configuration allows you to define different sets of keypoints for each class, which is especially useful in diverse datasets where different objects require different keypoints for accurate pose estimation. 🚀

[Bekbo01]

hey there! In wich version of yolo it works? I got error KeyError: 'kpt_shape' (if add just kpt_shape it will work inly for one class)

[pderrenger]

Hey there! 👋 It seems you encountered a KeyError: 'kpt_shape' because the kpt_shape parameter might be missing or improperly set for multiple classes in your dataset configuration. For pose estimation with multiple classes in YOLOv8, you should define num_keypoints for each class within your .yaml dataset configuration file, not just kpt_shape.

Here's a quick fix using an example .yaml config tailored for two classes:

# Your custom-pose-dataset.yaml
train: /path/to/train/images
val: /path/to/val/images

nc: 2  # number of classes
names: ['Class1', 'Class2']  # Class names

# Define keypoints for each class
num_keypoints:
  - 17  # Number of keypoints for Class1
  - 5   # Number of keypoints for Class2

Make sure to adjust the train, val paths, names, and num_keypoints according to your specific dataset and classes. This configuration will help you avoid the KeyError and work seamlessly with multiple classes. Hope this helps! Let me know if you have more questions! 🚀

[lilin19890401]

When we training multiple classes and the .yaml as you mentioned above，two classes with different number of keypoints. How do we modify nc and kpt_shape in yolov8n-pose.yaml ?

[pderrenger]

@lilin19890401 hello! For training a YOLOv8 pose model with multiple classes where each class has a different number of keypoints, you don't need to modify nc or kpt_shape directly in the yolov8n-pose.yaml. Instead, you should specify num_keypoints for each class in your dataset configuration YAML file. Here’s how you can structure it:

nc: 2  # Number of classes
names: ['Class1', 'Class2']  # Class names

# Specify keypoints for each class
num_keypoints:
  - 17  # Number of keypoints for Class1
  - 5   # Number of keypoints for Class2

This setup tells the model how many keypoints to expect for each class, allowing for different configurations per class. Make sure your dataset labels match this configuration! Let me know if you need further assistance. 😊

[davidetedd9203]

Hi!
i'm working on a personal project about volleyball. i should use pose estimation to track players movement but also player field, ball and net. how can i do this? Do I have to use two different models? one for pose and another for object detection?

[glenn-jocher]

@davidetedd9203 to track player movements and detect objects like the ball and net, you'll need two models: a pose estimation model (e.g., YOLOv5) for tracking movements, and an object detection model (e.g., YOLOv5) for identifying the ball and net. You can use Ultralytics YOLO for both tasks, as it supports exporting models to various formats. For detailed guidance, check the Ultralytics YOLO documentation. If you have more questions, feel free to ask!

[sensuman7]

how to detect human pose like t-pose, Crouching/Squatting, Lunging, Twisting, Bending Over, Planking ext. I have already trained the bounding box dataset but did not get good results.

[glenn-jocher]

@sensuman7 to improve pose detection for specific actions like T-pose or crouching, consider using YOLO11-pose models, which are optimized for keypoint detection. Ensure your dataset is well-labeled with these poses and retrain using a pose-specific model. For more guidance, visit the Pose Estimation page.

[MarserHK]

Hello, I would like to ask, when using yolov8s-pose.pt for pose estimation, how can I directly exclude people with low confidence scores from being detected?

[glenn-jocher]

@MarserHK to exclude detections with low confidence scores, you can set a confidence threshold during inference. Use the conf parameter in the predict method to specify the minimum confidence level required for detections. For example: results = model.predict(source='image.jpg', conf=0.5). This will filter out detections below the specified confidence level.

[Mrgorlay]

Hello,@glenn-jocher
I'm interested in using YOLOv11-Pose to implement personnel action monitoring, such as detecting when someone is bending over. Do you have any good suggestions and methods? I would appreciate your guidance. Thank you.

[glenn-jocher]

@Mrgorlay for personnel action monitoring using YOLOv11-Pose, you can leverage the model's keypoint detection capabilities to track specific body movements like bending. Consider training the model on a dataset that includes these actions to improve accuracy. For more detailed guidance, please refer to the Pose Estimation documentation.

[Mrgorlay]

Thank you. My current idea is to calculate the angle between each joint. Is this approach feasible?

[glenn-jocher]

Calculating joint angles is a feasible approach for detecting actions like bending over. By analyzing the angles between keypoints, you can identify specific postures. Ensure your dataset includes diverse examples of the actions you want to monitor for better accuracy.

[adripriadana]

Dear the authors,

Hello, I am Adri. I would like to ask related the YOLO11 Pose result. Which train setting, such as number of batch size, learning rate, number of GPU, etc, did you use to train the model to get the result? I tried to train it with the default setting but I cannot achieve the similar result. Could you please provide me the setting?

Thank you so much and I look forward to hearing from you.

Best regards,
Adri

[glenn-jocher]

@adripriadana hi Adri,

For training settings like batch size, learning rate, and GPU usage, we recommend experimenting with values based on your hardware and dataset. The default settings are a good starting point, but adjustments may be necessary for optimal results. For more detailed guidance, please refer to our documentation on pose estimation.

[pranav6670]

Is it possible to leverage YOLOv11 for 6D pose estimation?

[glenn-jocher]

@pranav6670 yOLOv11 currently supports 2D pose estimation. For 6D pose estimation, additional development would be required. You can explore the existing capabilities at Ultralytics Pose Estimation.

[pranav6670]

@glenn-jocher Thank you so much for replying.

[glenn-jocher]

Thank you for your interest! For 6D pose estimation, YOLOv11 currently supports only 2D pose estimation. You might need to explore other specialized solutions for 6D pose tasks.

[liangsheng-xiao]

Why does the predictive model I trained not have skeleton connections, I have already set the skeleton parameter in the plotting.py

[glenn-jocher]

@liangsheng-xiao it sounds like the skeleton connections might not be rendering due to a configuration issue. Please ensure that the model outputs keypoints correctly and that the plotting function is set to visualize these connections. For further guidance, you can refer to the Pose Estimation documentation.

[MarserHK]

Hello, I am using YOLOPose for video inference, and in outdoor scenes, it also recognizes people's shadows as people. How can I resolve this issue?

[glenn-jocher]

@MarserHK to address the issue of shadows being recognized as people, consider adjusting the confidence threshold or using additional preprocessing techniques to filter out shadows. You might also explore fine-tuning the model on a dataset that includes shadow variations. For more details, visit the pose estimation documentation.

[legend-zhen]

Hi,how to use yolo-pose to realise behavior recognition task? Based on pose estimation i mean.Use posec3D?

[glenn-jocher]

@legend-zhen for behavior recognition using YOLO pose models, you can extract keypoints and apply a model like PoseC3D for action recognition. For more details, refer to the Pose Estimation page in the Ultralytics documentation.

[Regitam]

Hi, iam trying to train with yolov8m-pose with custom dataset:
from ultralytics import YOLO
model = YOLO('yolo8m-pose.pt')

Train the model

results = model.train(data="/kaggle/working/thermal_pose_updated.yaml",epochs=50)

but some errors occurs like :
1332 if not hasattr(f, 'readinto') and (3, 8, 0) <= sys.version_info < (3, 8, 2):
1333 raise RuntimeError(
1334 "torch.load does not work with file-like objects that do not implement readinto on Python 3.8.0 and 3.8.1. "
1335 f'Received object of type "{type(f)}". Please update to Python 3.8.2 or newer to restore this '
1336 "functionality.")
-> 1338 magic_number = pickle_module.load(f, **pickle_load_args)
1339 if magic_number != MAGIC_NUMBER:
1340 raise RuntimeError("Invalid magic number; corrupt file?")

UnpicklingError: could not find MARK

is it because of the ultralytics version i installed?
Thanks to anyone who replies

[glenn-jocher]

@Regitam it seems like the error might be related to the Python version you're using. Please ensure you're using Python 3.8.2 or newer. For more details on pose estimation, visit https://docs.ultralytics.com/tasks/pose/.

[Regitam]

HI, thanks for replies. I checked my python version and it shows 8.10.14 ver. Is there anything related to this error beside the python version??

[glenn-jocher]

It seems like the issue might not be related to the Python version since you're using 3.8.10. Please ensure your model file is not corrupted and try re-downloading it. If the issue persists, consider checking for compatibility with the latest Ultralytics version.

[rudro12356]

Hi Ultralytics, I was hoping to get some help with Pose estimation. In one of my research, we are trying to figure out who is showing the actions first and then who is trying to copy the actions. One of my thoughts was to use the time stamps and then do comparison but this doesn't seem to be right exactly. Is there a feature already to see pose estimation object wise and do comparison much easily? Thanks!

[glenn-jocher]

@rudro12356 for comparing pose estimation results object-wise, you can utilize the YOLO11-pose models to track keypoints over time and analyze their movements. This approach can help identify action sequences and their initiators. For more details on pose estimation, please visit our documentation at https://docs.ultralytics.com/tasks/pose/.

[AdadAlShabab]

Hi. What are the keypoints for neck to head?

[glenn-jocher]

@AdadAlShabab the keypoints for the neck to head in the YOLO11 pose model are represented by the nose, left eye, right eye, left ear, and right ear.

[Aianv0]

Hi, I have a question concerning the yolo11-pose model. I am detecting very well the person with that model. My issue is that sometimes some joints aren't seen, therefore the confidence score drop. The result of this (apparently a threshold of confidence 0.5) is that the joint is saved as position (0.0, 0.0). In my case, I need to have a continuous location of the joints.
My question is: Is there a way to lower the threshold of the joint detection and not of the person detection? I tried to change the 'conf' parameter but is seems to only impact the box detection, not the joints. If not, is there a way to obtain these estimation from the model in any way?

Thank you a lotin advance.

[glenn-jocher]

@Aianv0 to adjust the confidence threshold for joint detection specifically, you might need to modify the model's configuration or post-processing steps. Unfortunately, the 'conf' parameter typically affects the entire detection pipeline, not just joints. Consider exploring the model's architecture or custom post-processing to achieve your desired outcome.

[YutoYamazaki1997]

Hello!!!
I am a programmer in Japan.
I want to train a first-person view pose estimation model.
Is it possible to train it based on Yolo's Pose? Does Yolo's Pose require human detection to perform accurate pose estimation?

For object detection tasks, should I label and train right and left hands separately?
Sorry for my cheep English.

[glenn-jocher]

@YutoYamazaki1997 yes, you can train a YOLO11-pose model for first-person view pose estimation. Human detection is typically required for accurate pose estimation. For labeling, it's advisable to label right and left hands separately to improve model accuracy.

[YutoYamazaki1997]

Thank you for replay!!!
In your opinion, if the video is taken from behind a person, then skeletal estimation is possible.

Is it difficult to create a skeletal estimation model of a hand or elbow with a camera hanging from the neck?
I really need to extrapolate the hand or elbow from that footage.

[glenn-jocher]

Training a skeletal estimation model from a first-person view is possible with YOLO11-pose, but it may be challenging due to occlusions and unusual angles. Accurate pose estimation typically benefits from detecting the whole body, but you can focus on specific keypoints like hands or elbows.

[YutoYamazaki1997]

Ok! I will TRY IT!

[glenn-jocher]

Thank you for your interest! You can indeed train a YOLO11-pose model for first-person view pose estimation. While human detection aids in accuracy, focusing on specific keypoints like hands or elbows is feasible. For detailed guidance, please refer to the Pose Estimation documentation.

[hekkwar]

Hi there, I'm trying to export the keypoint coordinates into a csv file but I get the following error:

line 23, in
x, y = xy
ValueError: too many values to unpack (expected 2)

Any help is appreciated. Here's the code I'm running:

import pandas as pd
from ultralytics import YOLO

labels = ["nose", "left_eye", "right_eye", "left_ear", "right_ear", "left_shoulder",
"right_shoulder", "left_elbow", "right_elbow", "left_wrist", "right_wrist",
"left_hip", "right_hip", "left_knee", "right_knee", "left_ankle", "right_ankle"]

model = YOLO('yolov8x-pose-p6.pt')
results = model(source="test2.png", show=True, conf=0.001, save=True)

data = []

for frame_id, (frame_results) in enumerate(results):
for person_id, person in enumerate(frame_results.keypoints):
xy = person.xy.numpy() # keypoints coordinates
conf = person.conf.numpy() # confidence scores of keypoints
for kp_id, (xy, kp_conf) in enumerate(zip(xy, conf)):
label = labels[kp_id] # Using the predefined labels list
x, y = xy
data.append({"frame_id": frame_id, "person_id": person_id, "label": label, "x": x, "y": y, "confidence": kp_conf})

df = pd.DataFrame(data)
df.to_csv('keypoints.csv', index=False)

[glenn-jocher]

The error occurs because enumerate and zip need to be combined correctly and large constructs like for ja, kp_id, (xy, kp_conf) are being improperly unpacked. Instead, iterate over ja_list and handle enumerations separately. Replace your loop with:

for ja in ja_list:
    angle = calculate_angle(xy[ja[0]], xy[ja[1]], xy[ja[2]])
    str_angle = str(angle)
    data.append({"frame_id": frame_id, "person_id": person_id, "xy": xy_str, "angle": str_angle})

This simplifies the logic and should resolve the issue. Let me know of any further concerns!

[MarcKr3]

Thanks!
Your suggested code yields:

frame_id,person_id,xy,angle
0,0,[        0.27054          0.32959] ,  [        0.27079          0.32787] ,  [        0.26698          0.32339] ,  [                    0                      0] ,  [        0.25605          0.33095] ,  [        0.27429          0.39124] ,  [        0.24764          0.34531] ,  [        0.29295          0.40439] ,  [        0.25213            0.2805] ,  [        0.31294          0.41499] ,  [        0.26839          0.21751] ,  [        0.25684          0.49569] ,  [        0.23976          0.47824] ,  [        0.26361          0.59153] ,  [        0.25314          0.57706] ,  [        0.26301          0.68071] ,  [        0.25716          0.68032],172.77576
0,0,[        0.27054          0.32959] ,  [        0.27079          0.32787] ,  [        0.26698          0.32339] ,  [                    0                      0] ,  [        0.25605          0.33095] ,  [        0.27429          0.39124] ,  [        0.24764          0.34531] ,  [        0.29295          0.40439] ,  [        0.25213            0.2805] ,  [        0.31294          0.41499] ,  [        0.26839          0.21751] ,  [        0.25684          0.49569] ,  [        0.23976          0.47824] ,  [        0.26361          0.59153] ,  [        0.25314          0.57706] ,  [        0.26301          0.68071] ,  [        0.25716          0.68032],169.49625
0,0,[        0.27054          0.32959] ,  [        0.27079          0.32787] ,  [        0.26698          0.32339] ,  [                    0                      0] ,  [        0.25605          0.33095] ,  [        0.27429          0.39124] ,  [        0.24764          0.34531] ,  [        0.29295          0.40439] ,  [        0.25213            0.2805] ,  [        0.31294          0.41499] ,  [        0.26839          0.21751] ,  [        0.25684          0.49569] ,  [        0.23976          0.47824] ,  [        0.26361          0.59153] ,  [        0.25314          0.57706] ,  [        0.26301          0.68071] ,  [        0.25716          0.68032],64.3181
0,0,[        0.27054          0.32959] ,  [        0.27079          0.32787] ,  [        0.26698          0.32339] ,  [                    0                      0] ,  [        0.25605          0.33095] ,  [        0.27429          0.39124] ,  [        0.24764          0.34531] ,  [        0.29295          0.40439] ,  [        0.25213            0.2805] ,  [        0.31294          0.41499] ,  [        0.26839          0.21751] ,  [        0.25684          0.49569] ,  [        0.23976          0.47824] ,  [        0.26361          0.59153] ,  [        0.25314          0.57706] ,  [        0.26301          0.68071] ,  [        0.25716          0.68032],179.42664
0,0,[        0.27054          0.32959] ,  [        0.27079          0.32787] ,  [        0.26698          0.32339] ,  [                    0                      0] ,  [        0.25605          0.33095] ,  [        0.27429          0.39124] ,  [        0.24764          0.34531] ,  [        0.29295          0.40439] ,  [        0.25213            0.2805] ,  [        0.31294          0.41499] ,  [        0.26839          0.21751] ,  [        0.25684          0.49569] ,  [        0.23976          0.47824] ,  [        0.26361          0.59153] ,  [        0.25314          0.57706] ,  [        0.26301          0.68071] ,  [        0.25716          0.68032],166.47421
0,0,[        0.27054          0.32959] ,  [        0.27079          0.32787] ,  [        0.26698          0.32339] ,  [                    0                      0] ,  [        0.25605          0.33095] ,  [        0.27429          0.39124] ,  [        0.24764          0.34531] ,  [        0.29295          0.40439] ,  [        0.25213            0.2805] ,  [        0.31294          0.41499] ,  [        0.26839          0.21751] ,  [        0.25684          0.49569] ,  [        0.23976          0.47824] ,  [        0.26361          0.59153] ,  [        0.25314          0.57706] ,  [        0.26301          0.68071] ,  [        0.25716          0.68032],168.8918
0,0,[        0.27054          0.32959] ,  [        0.27079          0.32787] ,  [        0.26698          0.32339] ,  [                    0                      0] ,  [        0.25605          0.33095] ,  [        0.27429          0.39124] ,  [        0.24764          0.34531] ,  [        0.29295          0.40439] ,  [        0.25213            0.2805] ,  [        0.31294          0.41499] ,  [        0.26839          0.21751] ,  [        0.25684          0.49569] ,  [        0.23976          0.47824] ,  [        0.26361          0.59153] ,  [        0.25314          0.57706] ,  [        0.26301          0.68071] ,  [        0.25716          0.68032],175.57079
0,0,[        0.27054          0.32959] ,  [        0.27079          0.32787] ,  [        0.26698          0.32339] ,  [                    0                      0] ,  [        0.25605          0.33095] ,  [        0.27429          0.39124] ,  [        0.24764          0.34531] ,  [        0.29295          0.40439] ,  [        0.25213            0.2805] ,  [        0.31294          0.41499] ,  [        0.26839          0.21751] ,  [        0.25684          0.49569] ,  [        0.23976          0.47824] ,  [        0.26361          0.59153] ,  [        0.25314          0.57706] ,  [        0.26301          0.68071] ,  [        0.25716          0.68032],174.5127
0,1,[        0.68909          0.39644] ,  [          0.6917          0.39235] ,  [        0.68635          0.39157] ,  [        0.69507          0.40165] ,  [                    0                      0] ,  [        0.70123          0.43593] ,  [        0.66924          0.43106] ,  [        0.72618            0.4416] ,  [        0.65064          0.43664] ,  [        0.73248          0.40108] ,  [        0.65353          0.41024] ,  [          0.6872          0.54132] ,  [        0.66779          0.53827] ,  [        0.68265          0.63291] ,  [        0.66896          0.62668] ,  [        0.68056          0.70402] ,  [        0.66196          0.69712],86.04971
0,1,[        0.68909          0.39644] ,  [          0.6917          0.39235] ,  [        0.68635          0.39157] ,  [        0.69507          0.40165] ,  [                    0                      0] ,  [        0.70123          0.43593] ,  [        0.66924          0.43106] ,  [        0.72618            0.4416] ,  [        0.65064          0.43664] ,  [        0.73248          0.40108] ,  [        0.65353          0.41024] ,  [          0.6872          0.54132] ,  [        0.66779          0.53827] ,  [        0.68265          0.63291] ,  [        0.66896          0.62668] ,  [        0.68056          0.70402] ,  [        0.66196          0.69712],67.032646
0,1,[        0.68909          0.39644] ,  [          0.6917          0.39235] ,  [        0.68635          0.39157] ,  [        0.69507          0.40165] ,  [                    0                      0] ,  [        0.70123          0.43593] ,  [        0.66924          0.43106] ,  [        0.72618            0.4416] ,  [        0.65064          0.43664] ,  [        0.73248          0.40108] ,  [        0.65353          0.41024] ,  [          0.6872          0.54132] ,  [        0.66779          0.53827] ,  [        0.68265          0.63291] ,  [        0.66896          0.62668] ,  [        0.68056          0.70402] ,  [        0.66196          0.69712],84.800735
0,1,[        0.68909          0.39644] ,  [          0.6917          0.39235] ,  [        0.68635          0.39157] ,  [        0.69507          0.40165] ,  [                    0                      0] ,  [        0.70123          0.43593] ,  [        0.66924          0.43106] ,  [        0.72618            0.4416] ,  [        0.65064          0.43664] ,  [        0.73248          0.40108] ,  [        0.65353          0.41024] ,  [          0.6872          0.54132] ,  [        0.66779          0.53827] ,  [        0.68265          0.63291] ,  [        0.66896          0.62668] ,  [        0.68056          0.70402] ,  [        0.66196          0.69712],72.502205
0,1,[        0.68909          0.39644] ,  [          0.6917          0.39235] ,  [        0.68635          0.39157] ,  [        0.69507          0.40165] ,  [                    0                      0] ,  [        0.70123          0.43593] ,  [        0.66924          0.43106] ,  [        0.72618            0.4416] ,  [        0.65064          0.43664] ,  [        0.73248          0.40108] ,  [        0.65353          0.41024] ,  [          0.6872          0.54132] ,  [        0.66779          0.53827] ,  [        0.68265          0.63291] ,  [        0.66896          0.62668] ,  [        0.68056          0.70402] ,  [        0.66196          0.69712],175.25697
0,1,[        0.68909          0.39644] ,  [          0.6917          0.39235] ,  [        0.68635          0.39157] ,  [        0.69507          0.40165] ,  [                    0                      0] ,  [        0.70123          0.43593] ,  [        0.66924          0.43106] ,  [        0.72618            0.4416] ,  [        0.65064          0.43664] ,  [        0.73248          0.40108] ,  [        0.65353          0.41024] ,  [          0.6872          0.54132] ,  [        0.66779          0.53827] ,  [        0.68265          0.63291] ,  [        0.66896          0.62668] ,  [        0.68056          0.70402] ,  [        0.66196          0.69712],178.47017
0,1,[        0.68909          0.39644] ,  [          0.6917          0.39235] ,  [        0.68635          0.39157] ,  [        0.69507          0.40165] ,  [                    0                      0] ,  [        0.70123          0.43593] ,  [        0.66924          0.43106] ,  [        0.72618            0.4416] ,  [        0.65064          0.43664] ,  [        0.73248          0.40108] ,  [        0.65353          0.41024] ,  [          0.6872          0.54132] ,  [        0.66779          0.53827] ,  [        0.68265          0.63291] ,  [        0.66896          0.62668] ,  [        0.68056          0.70402] ,  [        0.66196          0.69712],178.84528
0,1,[        0.68909          0.39644] ,  [          0.6917          0.39235] ,  [        0.68635          0.39157] ,  [        0.69507          0.40165] ,  [                    0                      0] ,  [        0.70123          0.43593] ,  [        0.66924          0.43106] ,  [        0.72618            0.4416] ,  [        0.65064          0.43664] ,  [        0.73248          0.40108] ,  [        0.65353          0.41024] ,  [          0.6872          0.54132] ,  [        0.66779          0.53827] ,  [        0.68265          0.63291] ,  [        0.66896          0.62668] ,  [        0.68056          0.70402] ,  [        0.66196          0.69712],173.56549

While I'm getting all data, it's a lot of duplicates obviously...

---

In the mean time I found the following approach:

for kp_id, (ja, xy, kp_conf) in enumerate(zip(ja_list, xy, conf)):
            xy_str = ', '.join(map(str, xy))  # Convert the array to a comma-separated string
            
            angle = calculate_angle(xy[ja[0]],xy[ja[1]],xy[ja[2]])
            str_angle = str(angle)
            
            data.append({"frame_id": frame_id, "person_id": person_id, "xy": xy_str, "angle":str_angle})

Yielding:

frame_id,person_id,nose ,  left_eye ,  right_eye ,  left_ear ,  right_ear ,  left_shoulder ,  right_shoulder ,  left_elbow ,  right_elbow ,  left_wrist ,  right_wrist ,  left_hip ,  right_hip ,  left_knee ,  right_knee ,  left_ankle ,  right_ankle,angle
0,0,[        0.27054          0.32959] ,  [        0.27079          0.32787] ,  [        0.26698          0.32339] ,  [                    0                      0] ,  [        0.25605          0.33095] ,  [        0.27429          0.39124] ,  [        0.24764          0.34531] ,  [        0.29295          0.40439] ,  [        0.25213            0.2805] ,  [        0.31294          0.41499] ,  [        0.26839          0.21751] ,  [        0.25684          0.49569] ,  [        0.23976          0.47824] ,  [        0.26361          0.59153] ,  [        0.25314          0.57706] ,  [        0.26301          0.68071] ,  [        0.25716          0.68032],172.77576
0,1,[        0.68909          0.39644] ,  [          0.6917          0.39235] ,  [        0.68635          0.39157] ,  [        0.69507          0.40165] ,  [                    0                      0] ,  [        0.70123          0.43593] ,  [        0.66924          0.43106] ,  [        0.72618            0.4416] ,  [        0.65064          0.43664] ,  [        0.73248          0.40108] ,  [        0.65353          0.41024] ,  [          0.6872          0.54132] ,  [        0.66779          0.53827] ,  [        0.68265          0.63291] ,  [        0.66896          0.62668] ,  [        0.68056          0.70402] ,  [        0.66196          0.69712],86.04971

Feels like a step in the right direction... Is stringing of all 8 angle values possible?

[glenn-jocher]

You're on the right track with your approach! To string all 8 angles into a single comma-separated string, you can do the following within the loop:

angles = [calculate_angle(xy[ja[0]], xy[ja[1]], xy[ja[2]]) for ja in ja_list]
str_angles = ', '.join(map(str, angles))  # Combine all angles into a single string
data.append({"frame_id": frame_id, "person_id": person_id, "xy": xy_str, "angles": str_angles})

This will provide all joint angles in one column of the CSV. Let me know if further tweaking is necessary!

[MarcKr3]

Glenn I can't thank You enough!!!

You're single handedly enabling me to do my Bachelor's Thesis.

I'm adding the final code here in case anybody needs the same application.

---

Export Keypoint Coordinates & Joint Angles for every frame in a video into .csv format

Additionally, this code can be run on a directory full of images/videos and output unique .csv for each file.

Code:

import csv
import pandas as pd
from ultralytics import YOLO
from pathlib import Path
import numpy as np

input_dir = Path.cwd() / "source material"
files = list(input_dir.glob("*.mp4*"))

model = YOLO("yolo11n-pose.pt")

def calculate_angle(p1, p2, p3):
    # p1, p2, p3 are the points in format [x, y]
    # Calculate the vectors
    v1 = np.array(p1) - np.array(p2)
    v2 = np.array(p3) - np.array(p2)
    
    # Calculate the angle in radians
    angle_rad = np.arccos(np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2)))
    
    # Convert to degrees
    angle_deg = np.degrees(angle_rad)
    
    return angle_deg

ja_list = [[9,7,5], [10,8,6], [7,5,11], [8,6,12], [5,11,13], [6,12,14], [11,13,15], [12,14,16]] #L/R Elbow, L/R Shoulder, L/R Hip, L/R Knee Angles

for file in files:
    results = model.track(source=file, save=False) # save=True to save videos with Skeletons on top

    # Initialize an empty list to store data
    data = []

    for frame_id, (frame_results) in enumerate(results):
        for person_id, person in enumerate(frame_results.keypoints):
            xy = person.xyn.numpy()  # keypoints coordinates (xyn=normalized, xy=raw)
            conf = person.conf.numpy()  # confidence scores of keypoints
            for kp_id, (ja, xy, kp_conf) in enumerate(zip(ja_list, xy, conf)):
                xy_str = ', '.join(map(str, xy))  # Convert the array to a comma-separated string

                angles = [str(calculate_angle(xy[ja[0]], xy[ja[1]], xy[ja[2]])) for ja in ja_list]
                str_angles = ', '.join(map(str, angles)).replace("'", "")
                
                data.append({"frame_id": frame_id, "person_id": person_id, "nose, left_eye, right_eye, left_ear, right_ear, left_shoulder, right_shoulder, left_elbow, right_elbow, left_wrist, right_wrist, left_hip, right_hip, left_knee, right_knee, left_ankle, right_ankle": xy_str, "left_elbow_angle, right_elbow_angle, left_shoulder_angle, right_shoulder_angle, left_hip_angle, right_hip_angle, left_knee_angle, right_knee_angle": str_angles})

        df = pd.DataFrame(data)
        df.to_csv(str(file) + '.csv', escapechar=' ', quoting=csv.QUOTE_NONE, index=False)

.csv Output:

frame_id,person_id,nose ,  left_eye ,  right_eye ,  left_ear ,  right_ear ,  left_shoulder ,  right_shoulder ,  left_elbow ,  right_elbow ,  left_wrist ,  right_wrist ,  left_hip ,  right_hip ,  left_knee ,  right_knee ,  left_ankle ,  right_ankle,left_elbow_angle ,  right_elbow_angle ,  left_shoulder_angle ,  right_shoulder_angle ,  left_hip_angle ,  right_hip_angle ,  left_knee_angle ,  right_knee_angle
0,0,[                    0                      0] ,  [                    0                      0] ,  [                    0                      0] ,  [        0.37043          0.49321] ,  [                    0                      0] ,  [        0.36608          0.53391] ,  [        0.39164          0.52975] ,  [        0.33835          0.50062] ,  [        0.38325          0.49401] ,  [        0.33667          0.44496] ,  [        0.36536          0.42566] ,  [        0.36204          0.66559] ,  [        0.37784          0.66164] ,  [        0.33408          0.76912] ,  [        0.34369          0.77239] ,  [        0.32319          0.87872] ,  [        0.32732          0.88325],141.93419 ,  178.54129 ,  138.44943 ,  160.82266 ,  166.64551 ,  168.83043 ,  170.56398 ,  171.2587
0,1,[                    0                      0] ,  [                    0                      0] ,  [                    0                      0] ,  [        0.83518          0.48545] ,  [                    0                      0] ,  [        0.83313          0.52866] ,  [        0.86121          0.52642] ,  [        0.82727          0.53884] ,  [        0.87441          0.52998] ,  [        0.83882          0.50397] ,  [        0.86689          0.47486] ,  [        0.84591          0.66505] ,  [        0.86339          0.66328] ,  [        0.83149          0.77674] ,  [          0.8419          0.77625] ,  [        0.83638            0.8804] ,  [        0.84344          0.87612],11.577432 ,  67.1431 ,  35.246887 ,  74.00404 ,  167.28818 ,  168.31642 ,  169.94182 ,  168.34938

THANKS GLENN

[glenn-jocher]

Thank you for your kind words and for sharing your work! It's inspiring to see how the YOLO community applies Ultralytics tools in innovative ways like this. Wishing you all the best with your Bachelor's Thesis and future endeavors! If you have further questions, feel free to reach out here.