Skip to content

Commit

Permalink
Holoscan OpenEmbedded Layer v0.4.0 Release
Browse files Browse the repository at this point in the history
  • Loading branch information
ibstewart committed Dec 20, 2022
1 parent b8872fe commit 64621bf
Show file tree
Hide file tree
Showing 65 changed files with 2,461 additions and 518 deletions.
56 changes: 52 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,49 @@
## [0.4.0] - 2022-12-20
### Added
- ONNX Runtime 1.12.1 recipe
- Missing mlnx-ofed components (libibmad5, libibumad3, libibnetdisc5, etc.)
- mstflint 4.22.0-1 recipe

### Changed
- Renamed layer to meta-tegra-holoscan
- Updated Holsocan SDK to 0.4.0 using new GitHub repository
(https://github.com/nvidia-holoscan/holoscan-sdk)
- Updated GXF to 2.5.0
- glad is now built as a shared library.
- Updated mlnx-ofed to 5.8-1.0.1.1
- Updated Rivermax to 1.20.10

### Fixed
- Added host network option to docker run commands to work with VPN.
- Added missing Vulkan ICD to GPU drivers.
- The AJA kernel module will now load automatically if an AJA device is
found, eliminating the need to add it to `KERNEL_MODULE_AUTOLOAD`.

### Known Issues
- A build-time race condition may lead to Rivermax packaging conflicts.
See the "Building and Flashing" section of README.md for a solution.
- The display may not be detected during first boot.
See the "Building and Flashing" section of README.md for a solution.

### Component Versions
| | iGPU and dGPU |
| --------------- | ------------- |
| L4T | 35.1.0 |
| Holoscan SDK | 0.4.0 |
| GXF | 2.5.0 |
| AJA NTV2 | 16.2.0 |
| MLNX OFED | 5.8-1.0.1.1 |
| Rivermax | 1.20.10 |

| | iGPU | dGPU |
| -------------- | -------- | --------- |
| GPU Drivers | 35.1.0 | 515.65.01 |
| CUDA | 11.4.239 | 11.7.1 |
| cuDNN | 8.4.1.50 | 8.5.0.96 |
| TensorRT | 8.4.1 | 8.4.3.1 |
| nvidia-peermem | N/A | 1.3.0 |


## [0.3.0] - 2022-10-19
### Added
- Support for the NVIDIA IGX Orin Developer Kit (`holoscan-devkit`).
Expand All @@ -12,8 +58,9 @@ Module Source](https://github.com/NVIDIA/open-gpu-kernel-modules).
Repo](https://github.com/NVIDIA/cuda-samples).

### Known Issues
- Holoscan SDK Ultrasound segmentation application is not functional on NVIDIA IGX Orin Developer Kit (holoscan-devkit) with iGPU configuration.
- Holoscan SDK High-Speed Endoscopy application is not supported in deployment stack.
- Holoscan SDK Ultrasound segmentation application is not functional on NVIDIA
IGX Orin Developer Kit (holoscan-devkit) with iGPU configuration.
- Holoscan SDK High-Speed Endoscopy application is not yet supported.

### Component Versions
| | iGPU and dGPU |
Expand Down Expand Up @@ -52,5 +99,6 @@ Repo](https://github.com/NVIDIA/cuda-samples).
| AJA NTV2 | 16.2.0 |
| Holoscan SDK | 0.2.0 |

[0.3.0]: https://github.com/NVIDIA/meta-tegra-clara-holoscan-mgx/compare/v0.2.0...v0.3.0
[0.2.0]: https://github.com/NVIDIA/meta-tegra-clara-holoscan-mgx/releases/tag/v0.2.0
[0.4.0]: https://github.com/nvidia-holoscan/meta-tegra-holoscan/compare/v0.3.0...v0.4.0
[0.3.0]: https://github.com/nvidia-holoscan/meta-tegra-holoscan/compare/v0.2.0...v0.3.0
[0.2.0]: https://github.com/nvidia-holoscan/meta-tegra-holoscan/releases/tag/v0.2.0
167 changes: 90 additions & 77 deletions README.md

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions conf/holoscan-dgpu.conf
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,8 @@ PREFERRED_PROVIDER_tensorrt-plugins = "tensorrt"
PREFERRED_PROVIDER_tensorrt-trtexec = "tensorrt"

# Override the CUDA version (11.7) and GPU arch to:
# 7.5 (Turing) for the RTX 6000 in the Clara AGX Devkit.
# 8.6 (Ampere) for the RTX A6000 in the Holoscan Devkit.
# 8.6 (Ampere) for the RTX A6000 (default).
# 7.5 (Turing) for the RTX 6000 (Clara AGX Devkit).
CUDA_VERSION = "11.7"
TEGRA_CUDA_ARCHITECTURE:clara-agx-xavier-devkit ?= "75"
TEGRA_CUDA_ARCHITECTURE:holoscan-devkit ?= "86"
TEGRA_CUDA_ARCHITECTURE = "86"
TEGRA_CUDA_ARCHITECTURE:clara-agx-xavier-devkit = "75"
12 changes: 6 additions & 6 deletions conf/layer.conf
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@ BBFILES += " \
${LAYERDIR}/recipes-*/*/*.bb \
${LAYERDIR}/recipes-*/*/*.bbappend"

BBFILE_COLLECTIONS += "tegra-clara-holoscan-mgx"
BBFILE_PATTERN_tegra-clara-holoscan-mgx = "^${LAYERDIR}/"
BBFILE_PRIORITY_tegra-clara-holoscan-mgx = "6"
BBFILE_COLLECTIONS += "tegra-holoscan"
BBFILE_PATTERN_tegra-holoscan = "^${LAYERDIR}/"
BBFILE_PRIORITY_tegra-holoscan = "6"

LAYERVERSION_tegra-clara-holoscan-mgx = "1"
LAYERVERSION_tegra-holoscan = "1"

LAYERDEPENDS_tegra-clara-holoscan-mgx = "core tegra openembedded-layer"
LAYERDEPENDS_tegra-holoscan = "core tegra openembedded-layer"

LAYERSERIES_COMPAT_tegra-clara-holoscan-mgx = "kirkstone"
LAYERSERIES_COMPAT_tegra-holoscan = "kirkstone"
2 changes: 1 addition & 1 deletion conf/rivermax.conf
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
# DEALINGS IN THE SOFTWARE.

CORE_IMAGE_EXTRA_INSTALL:append = " \
mlnx-ofed-kernel-dkms \
mlnx-ofed \
rivermax \
rivermax-apps \
rivermax-license \
Expand Down
4 changes: 2 additions & 2 deletions env/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,10 @@ RUN apt-get update -y \

RUN locale-gen en_US.UTF-8

WORKDIR /opt/nvidia/meta-tegra-clara-holoscan-mgx
WORKDIR /opt/nvidia/meta-tegra-holoscan
COPY . .
RUN env/docker_check_packages.sh
RUN ln -s container_setup.sh env/setup.sh
ENV PATH="/opt/nvidia/meta-tegra-clara-holoscan-mgx/env:${PATH}"
ENV PATH="/opt/nvidia/meta-tegra-holoscan/env:${PATH}"

WORKDIR /workspace
30 changes: 15 additions & 15 deletions env/README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# OpenEmbedded/Yocto Build Container for NVIDIA Clara Holoscan
# OpenEmbedded/Yocto Build Container for NVIDIA Holoscan

This container image contains runtime dependencies, scripts, and the
NVIDIA-proprietary binary packages that are required to build an OpenEmbedded
BSP image for NVIDIA Clara Developer Kits with dGPU support.
BSP image for NVIDIA Holoscan Developer Kits with dGPU support.

The following documentation provides information specific to the usage of the
Holoscan Build Container, and may be missing information from the main
documentation that may be useful to know when configuring or using the BSP.
Please see the main [README](../README.md) file for additional documentation.

> Note: the main `README` file can be found at
> `meta-tegra-clara-holoscan-mgx/README.md` after following the `1. Setting up
> `meta-tegra-holoscan/README.md` after following the `1. Setting up
> the Local Development Environment` section, below.
Also note that building a BSP for Clara Holoscan requires a significant
Also note that building a BSP for NVIDIA Holoscan requires a significant
amount of resources, and at least **200GB of free disk space is required to
build**. See the `System Requirements` section in the main
[README](../README.md) for more details.
Expand All @@ -35,8 +35,8 @@ would like to initialize the development environment and run the following
(making sure `IMAGE` matches the name and tag of this container image):

```sh
$ export IMAGE=nvcr.io/nvidia/clara-holoscan/holoscan-mgx-oe-builder:v0.3.0
$ docker run --rm -v $(pwd):/workspace ${IMAGE} setup.sh ${IMAGE} $(id -u) $(id -g)
$ export IMAGE=nvcr.io/nvidia/clara-holoscan/holoscan-oe-builder:v0.4.0
$ docker run --rm -v $(pwd):/workspace --network host ${IMAGE} setup.sh ${IMAGE} $(id -u) $(id -g)
```

This setup processes initializes the following:
Expand All @@ -48,7 +48,7 @@ This setup processes initializes the following:
openembedded-core
meta-openembedded
meta-tegra
meta-tegra-clara-holoscan-mgx
meta-tegra-holoscan
```

2. A sample build configuration in the `build` folder.
Expand Down Expand Up @@ -79,8 +79,8 @@ details.

To see the additional configuration that is added to this file relative to the
standard OpenEmbedded `local.conf`, as well as some documentation as to what
additional components offered by this meta-tegra-clara-holoscan-mgx layer may be
enabled, scroll down to the "BEGIN NVIDIA CONFIGURATION" section in this file.
additional components offered by this meta-tegra-holoscan layer may be enabled,
scroll down to the "BEGIN NVIDIA CONFIGURATION" section in this file.

## 3. Build the Image

Expand Down Expand Up @@ -114,7 +114,7 @@ build/tmp-glibc/deploy/images/holoscan-devkit/core-image-x11-holoscan-devkit.teg
## 4. Flash the Image

The `flash.sh` script can be used to flash the BSP image that is output by the
previous step onto the Clara Developer Kit hardware. For example, to flash the
previous step onto the Holoscan Developer Kit hardware. For example, to flash the
`core-image-x11` image that was produced by the previous step, connect the
developer kit to the host via the USB-C debug port, put it into recovery
mode, then run:
Expand All @@ -123,10 +123,10 @@ mode, then run:
$ ./flash.sh core-image-x11
```

> To put a Clara AGX Developer Kit into recovery mode, first remove the
> left-hand side cover to expose the recover and reset buttons; then while the
> unit is powered on, press the recovery and reset buttons, then release both
> buttons. For more information see the [Clara Developer Kit User Guide](https://developer.nvidia.com/clara-agx-developer-kit-user-guide).
> For instructions on how to put the developer kit into recovery mode, see the
> developer kit user guide:
> - [Clara AGX Developer Kit User Guide](https://developer.nvidia.com/clara-agx-developer-kit-user-guide).
> - [IGX Orin Developer Kit User Guide](https://developer.nvidia.com/igx-orin-developer-kit-user-guide).
Note that flashing the device will require root privileges and so you may be
asked for a sudo password by this script.
Expand All @@ -141,7 +141,7 @@ the endoscopy instrument tracking application using sample recorded video data:

```sh
$ cd /workspace
$ ./apps/endoscopy_tool_tracking_gxf/tracking_replayer
$ ./apps/endoscopy_tool_tracking/cpp/tracking_replayer
```

Note that the first execution of the samples will rebuild the model engine files
Expand Down
22 changes: 11 additions & 11 deletions env/container_setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -62,31 +62,31 @@ META_OPENEMBEDDED_REV="a755af4fb5ca2e158b00214bb18e27ba69c200fd"
git_clone "https://github.com/openembedded/meta-openembedded.git" ${META_OPENEMBEDDED_REV}

# meta-tegra layer
META_TEGRA_REV="b4fa3bafc4efa48248b94d8c69366eb42ac45aef"
git_clone "https://github.com/ibstewart/meta-tegra.git" ${META_TEGRA_REV}
META_TEGRA_REV="81e851ffb66917bdc7e5e9135661f6bfcce47fc3"
git_clone "https://github.com/nvidia-holoscan/meta-tegra.git" ${META_TEGRA_REV}

# meta-tegra-clara-holoscan-mgx layer (copied from build container)
if [ ! -d "meta-tegra-clara-holoscan-mgx" ]; then
cp -r /opt/nvidia/meta-tegra-clara-holoscan-mgx .
chown -R ${user}:${group} meta-tegra-clara-holoscan-mgx
echo "Initialized meta-tegra-clara-holoscan-mgx"
# meta-tegra-holoscan layer (copied from build container)
if [ ! -d "meta-tegra-holoscan" ]; then
cp -r /opt/nvidia/meta-tegra-holoscan .
chown -R ${user}:${group} meta-tegra-holoscan
echo "Initialized meta-tegra-holoscan"
else
echo "meta-tegra-clara-holoscan-mgx already exists; skipping"
echo "meta-tegra-holoscan already exists; skipping"
fi

# Default configuration (copied from build container)
if [ ! -d "build/conf" ]; then
mkdir -p build/conf
cp /opt/nvidia/meta-tegra-clara-holoscan-mgx/env/templates/conf/* build/conf
cp /opt/nvidia/meta-tegra-holoscan/env/templates/conf/* build/conf
chown -R ${user}:${group} build
echo "Initialized configuration defaults in build/conf"
else
echo "build/conf already exists; skipping"
fi

# Host-side scripts (copied from build container)
cp /opt/nvidia/meta-tegra-clara-holoscan-mgx/env/host_bitbake.sh bitbake.sh
cp /opt/nvidia/meta-tegra-clara-holoscan-mgx/env/host_flash.sh flash.sh
cp /opt/nvidia/meta-tegra-holoscan/env/host_bitbake.sh bitbake.sh
cp /opt/nvidia/meta-tegra-holoscan/env/host_flash.sh flash.sh
chown ${user}:${group} bitbake.sh
chown ${user}:${group} flash.sh

Expand Down
4 changes: 2 additions & 2 deletions env/docker_build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
#
# This script is run on the host, and builds the docker image.
#
# Usage: docker_build.sh [image name, default = holoscan-mgx-oe-builder]
# Usage: docker_build.sh [image name, default = holoscan-oe-builder]
#

script_path=$(readlink -f ${0})
Expand All @@ -31,7 +31,7 @@ root_path=$(dirname ${env_path})

image_name="${1}"
if [ -z "${image_name}" ]; then
image_name="holoscan-mgx-oe-builder"
image_name="holoscan-oe-builder"
fi

docker build -f ${env_path}/Dockerfile -t ${image_name} ${root_path}
14 changes: 8 additions & 6 deletions env/docker_check_packages.sh
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,17 @@

#
# This script is run when the docker image is being built, and checks to make
# sure that the binary packages required by the meta-tegra-clara-holoscan-mgx
# layer exist in the current build context.
# sure that the binary packages required by the meta-tegra-holoscan layer
# exist in the current build context.
#

CUDNN_PACKAGE="recipes-devtools/cudnn/files/cudnn-linux-sbsa-8.5.0.96_cuda11-archive.tar.xz"
TENSORRT_PACKAGE="recipes-devtools/tensorrt/files/TensorRT-8.4.3.1.Ubuntu-20.04.aarch64-gnu.cuda-11.6.cudnn8.4.tar.gz"
GXF_PACKAGE="recipes-devtools/gxf/files/gxf_2.4.3_20220811_6ff6ffd4_holoscan-sdk_arm64.tar.gz"
HOLOSCAN_ENDOSCOPY_DATA="recipes-devtools/holoscan/files/holoscan_endoscopy_data.zip"
HOLOSCAN_ULTRASOUND_DATA="recipes-devtools/holoscan/files/holoscan_ultrasound_data.zip"
RIVERMAX_PACKAGE="recipes-connectivity/rivermax/files/rivermax_ubuntu2004_1.11.11.tar.gz"
GXF_PACKAGE="recipes-devtools/gxf/files/gxf_22.11_20221116_8fbe43cb_holoscan-sdk_arm64.tar.gz"
HOLOSCAN_ENDOSCOPY_DATA="recipes-devtools/holoscan/files/holoscan_endoscopy_data_20221121.zip"
HOLOSCAN_ULTRASOUND_DATA="recipes-devtools/holoscan/files/holoscan_ultrasound_data_20220608.zip"
HOLOSCAN_MULTI_AI_DATA="recipes-devtools/holoscan/files/holoscan_multi_ai_ultrasound_data_20221201.zip"
RIVERMAX_PACKAGE="recipes-connectivity/rivermax/files/rivermax_ubuntu2004_1.20.10.tar.gz"

script_path=$(readlink -f ${0})
env_path=$(dirname ${script_path})
Expand All @@ -48,6 +49,7 @@ check_package ${TENSORRT_PACKAGE}
check_package ${GXF_PACKAGE}
check_package ${HOLOSCAN_ENDOSCOPY_DATA}
check_package ${HOLOSCAN_ULTRASOUND_DATA}
check_package ${HOLOSCAN_MULTI_AI_DATA}
check_package ${RIVERMAX_PACKAGE}

exit 0
2 changes: 1 addition & 1 deletion env/host_bitbake.sh
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,4 @@ if [ -z "${1}" ]; then
exit 1
fi

docker run --rm -it -v ${workspace_path}:/workspace --user $(id -u):$(id -g) ${build_image} container_bitbake.sh ${@}
docker run --rm -it -v ${workspace_path}:/workspace --network host --user $(id -u):$(id -g) ${build_image} container_bitbake.sh ${@}
2 changes: 1 addition & 1 deletion env/templates/conf/bblayers.conf
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ BBFILES ?= ""
BBLAYERS ?= " \
/workspace/openembedded-core/meta \
/workspace/meta-openembedded/meta-oe \
/workspace/meta-tegra-clara-holoscan-mgx \
/workspace/meta-tegra-holoscan \
/workspace/meta-tegra \
"
13 changes: 4 additions & 9 deletions env/templates/conf/local.conf
Original file line number Diff line number Diff line change
Expand Up @@ -245,8 +245,8 @@ CONF_VERSION = "2"
# Other than the MACHINE configuration (i.e. "clara-agx-xavier-devkit"),
# everything above mirrors the default configuration provided by OpenEmbedded
# Core. The following adds additional features that are provided or used by the
# meta-tegra-clara-holoscan-mgx layer to support a Clara Holoscan MGX platform
# with a discrete GPU.
# meta-tegra-holoscan layer to support an NVIDIA Holoscan platform with a
# discrete GPU.
#
################################################################################

Expand All @@ -264,17 +264,12 @@ CORE_IMAGE_EXTRA_INSTALL:append = " \
# Add an SSH server for debugging purposes
EXTRA_IMAGE_FEATURES:append = " ssh-server-dropbear"

# The following components are provided by the meta-tegra-clara-holoscan-mgx
# The following components are provided by the meta-tegra-holoscan
# layer and can be built into the image by adding them to
# CORE_IMAGE_EXTRA_INSTALL:append, above:
#
# cuda-samples - CUDA Sample Applications (/usr/bin/cuda-samples)
# tensorrt-samples - TensorRT Sample Applications (/usr/bin/tensorrt-samples)
# kernel-module-ajantv2 - AJA NTV2 Drivers
# ajantv2-sdk - AJA NTV2 SDK and Sample Apps
# holoscan-sdk - Clara Holoscan Embedded SDK

# When the AJA NTV2 drivers are included above (kernel-module-ajantv2), they can
# be loaded automatically when the system boots by uncommenting the following.
#
#KERNEL_MODULE_AUTOLOAD:append = " ajantv2"
# holoscan-sdk - Holoscan Embedded SDK
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,6 @@ LICENSE = "CLOSED"

require mlnx-ofed-package.inc

do_install() {
install -d ${D}${libdir}
install -m 0644 ${S}/usr/lib/*.so.* ${D}${libdir}
}

RDEPENDS:${PN} += " \
ibverbs-providers \
libibverbs1 \
Expand Down
Loading

0 comments on commit 64621bf

Please sign in to comment.