Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow arbitrary camera frame dimensions #459

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

alexander-soare
Copy link
Contributor

What this does

Allows arbitrary camera dimensions for OpenCVCamera. Previously, if the camera dimensions were not supported by the camera, an exception would be raised. With this PR, the frame is resized as a postprocessing step.

How it was tested

Added test which has full coverage if you have a camera connected, but only partial coverage otherwise (see the test docstring).

I also tested it locally with a camera.

How to checkout & try? (for the reviewer)

Run this script.

import cv2

from lerobot.common.robot_devices.cameras.opencv import OpenCVCamera, OpenCVCameraConfig
from lerobot.common.utils.utils import init_logging

init_logging()

height = 40
width = 40
camera = OpenCVCamera(0, OpenCVCameraConfig(height=height, width=width))
camera.connect()
assert camera.height == height
assert camera.width == width
img = camera.read()
cv2.imshow("window", img)
cv2.waitKey(0)

You should probably get logs like:

INFO 2024-10-03 14:48:44 s/opencv.py:286 The provided width (40) could not be set natively on your camera. The settings are now (width=160, height=90. This class will resize the frames to your desired resolution before returning them.
INFO 2024-10-03 14:48:44 s/opencv.py:288 The provided height (40) could not be set natively on your camera. The settings are now (width=160, height=90. This class will resize the frames to your desired resolution before returning them.

You should see a window with 40x40 image appear.

@alexander-soare alexander-soare self-assigned this Oct 3, 2024
@alexander-soare alexander-soare added ✨ Enhancement New feature or request 🌍 Real world Real-world robotics & controls labels Oct 3, 2024
Copy link
Collaborator

@Cadene Cadene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Could we do the same for intealrealsense.py after merging/rebasing master?
Then I am happy to quickly review again and approve

Comment on lines +148 to +150
def test_camera_resize(
request_resolution: tuple[int, int], read_resolution: tuple[int, int], is_camera_available: bool
):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After rebasing on top of master, could we add the mock version to it?

@michel-aractingi michel-aractingi mentioned this pull request Nov 8, 2024
17 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨ Enhancement New feature or request 🌍 Real world Real-world robotics & controls
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants