v2.4.0

Release Artifacts

• 🐋 Docker container: tag v2.4.0-dgpu and v2.4.0-igpu
• 🐍 Python wheel: pip install holoscan==2.4.0
• 📦️ Debian packages: 2.4.0.1-1
• 📕 Documentation

See supported platforms for compatibility.

Release Notes

New Features and Improvements

• The Holoscan CLI packager has been updated to create application containers that are up to 78% smaller than their previous standard container based size. The new --includes option allows the packager to include only runtime dependencies relevant to the application. Refer to the documentation for more information.

• The Holoscan pipeline metadata feature introduced for the C++ API in release v2.3 is now also available from the Python API. Interaction with metadata can be done with an API very similar to Python's built-in dictionaries. Please see the dynamic application metadata section of the user guide for details.

• The V4L example now supports a YUV configuration to display YUV input directly without RGB conversion.

• Added support to build Holoscan SDK without docker cache in run script e.g., ./run build --no-cache

Core

• It is now possible to set a HOLOSCAN_QUEUE_POLICY environment variable to override the default queue policy that is used by the input and output ports of the SDK. Valid options (case insensitive) are:

  • "pop": a new item that arrives when the queue is full replaces the oldest item
  • "reject": a new item that arrives when the queue is discarded
  • "fail": terminate the application if a new item arrives when the queue is full

  The default behavior remains "fail" if the environment variable is not specified. If an operator's setup method explicitly sets a receiver or transmitter via the IOSpec::connector method, this default value does not override the policy of that connector.

• Improve the handling of extra arguments passed to the application via CLI.

  • Now calls the allow_extras() method to permit extra arguments instead of ignoring them on an ExtrasError in the CLI11 parser.

• The SDK no longer spawns a new process to check for unused network ports for UCX communication when running a distributed application. Previously, this process caused issues such as redundant system resource consumption when the Holoscan distributed application was run as part of a larger application (e.g., importing Holoscan as a Python module after importing other modules) because a new process was created solely for checking unused network ports. This issue has been resolved by performing the network port check in in-process mode.

Operators/Resources

• Two new operators useful for examples and testing were added. PingTensorTxOp will emit a TensorMap containing a single tensor with user-specified name, shape, data type and storage type (e.g. host vs. device). PingTensorRxOp will receive a message containing a TensorMap and print some attributes of any tensors contained within it. Versions of these previously existed in examples and test code, but have now been moved to a common public location (holoscan::ops namespace for C++ and under holoscan.operators for Python).
• The V4L2VideoCaptureOp now supports passing the input buffer unmodified to the output. This can be enabled by using the parameter pass_through, by default this is disabled.
• Handle/enhance various cases of multi-receiver input ports (holoscan::InputContext::receive<std::vector<T>>())
  • Support receiving an array of TensorMap items from the input port.
  • Improve handling of cases where no data or null pointers are received from the input port.
  • Throw an invalid argument exception if the operator attempts to receive non-vector data (op_input.receive<T>()) from an input port with a queue size of IOSpec::kAnySize.
  • Avoid using nvidia::TypenameAsString for the type name in error messages, as it may include characters that are not permitted in the message (e.g., {anonymous}), which could be interpreted as a format specifier. This can result in an exception being thrown during message formatting.
• The HolovizOp now supports YUV (aka YCbCr) images as input. Various 420 and 422 formats are supported.
  • New image formats:
    • y8u8y8v8_422_unorm
    • u8y8v8y8_422_unorm
    • y8_u8v8_2plane_420_unorm
    • y8_u8v8_2plane_422_unorm
    • y8_u8_v8_3plane_420_unorm
    • y8_u8_v8_3plane_422_unorm
    • y16_u16v16_2plane_420_unorm
    • y16_u16v16_2plane_422_unorm
    • y16_u16_v16_3plane_420_unorm
    • y16_u16_v16_3plane_422_unorm
  • YUV color model conversions:
    • yuv_601
    • yuv_709
    • yuv_2020
  • YUV ranges:
    • itu_full
    • itu_narrow
  • Chroma locations in x and y:
    • cosited_even
    • midpoint
• The AJASourceOp now supports the following video formats:
  • 720p @ 50, 59.94, 60Hz
  • 1080i @ 50, 59.94, 60Hz
  • 1080p @ 23.98, 24, 25, 29.97, 30, 50, 59.94, 60Hz
  • 3840x2160 (UHD) @ 23.98, 24, 25, 29.97, 30, 50, 59.94, 60Hz
  • 4096x2160 (4K) @ 23.98, 24, 25, 29.97, 30, 50, 59.94, 60Hz

Holoviz module

• Now supports YUV (aka YCbCr) images and YUV conversion parameters. The functions to specify image layer data have been extended to support planar formats.
  • New entry point ImageYuvModelConversion() to specify the YUV model conversion (BT.601, BT.709, BT.2020)
  • New entry point ImageYuvRange() to specify the YUV range (ITU full and ITU narrow)
  • New entry point ImageChromaLocation() to specify the chroma location (cosited even and midpoint)

Utils

• An aja_build.sh script was added to automate the download, build, and loading of the AJA NTV2 drivers and SDK.

HoloHub

Documentation

Breaking Changes

Bug fixes

Issue	Description
-	Holoviz operator fails with Surface format '29, 0' not supported when enabling sRGB framebuffer in headless mode.
-	Fixed a bug where the run vscode --parallel <num_workers> command was not working as expected, displaying the message arg: unbound variable.
4791938	v4l_camera doesn't work with the USB camera when 800x600 is set, and there are multiple sizes available for width 800.
4792457	Heap memory error was found in GXFParameterAdaptor with AddressSanitizer (ASAN) during dynamic analysis.
4752615	In Python, the operator's parameter values are not available in the initialize() method. This bug was introduced in version 2.1.0.
4510522	V4L2VideoCaptureOp does not work with RGB.

Known Issues

This section supplies details about issues discovered during development and QA but not resolved in this release.

Issue	Description
4062979	When Operators connected in a Directed Acyclic Graph (DAG) are executed in a multithreaded scheduler, it is not ensured that their execution order in the graph is adhered.
4267272	AJA drivers cannot be built with RDMA on IGX SW 1.0 DP iGPU due to missing nv-p2p.h. Expected to be addressed in IGX SW 1.0 GA.
4384768	No RDMA support on JetPack 6.0 DP and IGX SW 1.0 DP iGPU due to missing nv-p2p kernel module. Expected to be addressed in JP 6.0 GA and IGX SW 1.0 GA respectively.
4190019	Holoviz segfaults on multi-gpu setup when specifying device using the --gpus flag with docker run. Current workaround is to use CUDA_VISIBLE_DEVICES in the container instead.
4210082	v4l_camera example seg faults at exit.
4339399	High CPU usage observed with video_replayer_distributed application. While the high CPU usage associated with the GXF UCX extension has been fixed since v1.0, distributed applications using the MultiThreadScheduler (with the check_recession_period_ms parameter set to 0 by default) may still experience high CPU usage. Setting the HOLOSCAN_CHECK_RECESSION_PERIOD_MS environment variable to a value greater than 0 (e.g. 1.5) can help reduce CPU usage. However, this may result in increased latency for the application until the MultiThreadScheduler switches to an event-based multithreaded scheduler.
4318442	UCX cuda_ipc protocol doesn't work in Docker containers on x86_64. As a workaround, we are currently disabling the UCX cuda_ipc protocol on all platforms via the UCX_TLS environment variable.
4325468	The V4L2VideoCapture operator only supports YUYV and AB24 source pixel formats, and only outputs the RGBA GXF video format. Other source pixel formats compatible with V4L2 can be manually defined by the user, but they're assumed to be equivalent to RGBA8888.
4325585	Applications using MultiThreadScheduler may exit early due to timeouts. This occurs when the stop_on_deadlock_timeout parameter is improperly set to a value equal to or less than check_recession_period_ms, particularly if check_recession_period_ms is greater than zero.
4301203	HDMI IN fails in v4l2_camera on IGX Orin Devkit for some resolution or formats. Try the latest firmware as a partial fix. Driver-level fixes expected in IGX SW 1.0 GA.
4384348	UCX termination (either ctrl+c , press 'Esc' or clicking close button) is not smooth and can show multiple error messages.
4481171	Running the driver for a distributed applications on IGX Orin devkits fails when connected to other systems through eth1. A workaround is to use eth0 port to connect to other systems for distributed workloads.
4458192	In scenarios where distributed applications have both the driver and workers running on the same host, either within a Docker container or directly on the host, there's a possibility of encountering "Address already in use" errors. A potential solution is to assign a different port number to the HOLOSCAN_HEALTH_CHECK_PORT environment variable (default: 8777), for example, by using export HOLOSCAN_HEALTH_CHECK_PORT=8780.
4782662	Installing Holoscan wheel 2.0.0 or later as root causes error.
4768945	Distributed applications crash when the engine file is unavailable/generating engine file.
4753994	Debugging Python application may lead to segfault when expanding an operator variable.
	Wayland: holoscan::viz::Init() with existing GLFW window fails.
4394306	When Python bindings are created for a C++ Operator, it is not always guaranteed that the destructor will be called prior to termination of the Python application. As a workaround to this issue, it is recommended that any resource cleanup should happen in an operator's stop() method rather than in the destructor.
4824619	iGPU: Rendering YUV images with HolovizOp fails on first run