From abe881a6550bcdffb7805d86fc4fcd9d38cd0dad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnar=C3=B6k?= Date: Wed, 21 Sep 2022 13:47:44 -0700 Subject: [PATCH 1/8] Create REP-2013: ROS 2 Client Library Integration --- rep-2013.rst | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 rep-2013.rst diff --git a/rep-2013.rst b/rep-2013.rst new file mode 100644 index 00000000..3c9acd7b --- /dev/null +++ b/rep-2013.rst @@ -0,0 +1,58 @@ +REP: 2013 +Title: ROS 2 Client Library Integration +Author: Steven! Ragnarok +Status: Draft +Type: Informational +Content-Type: text/x-rst +Created: 21-Sep-2022 +Post-History: 21-Sep-2022 + + +Abstract +======== + +This REP will document, retroactively, the aspects and expectations of language-specific client libraries in ROS 2 in order to serve as a template for new language communities and client libraries which can be integrated into ROS. +In particular this REP is an attempt to elaborate on the questions raised on ROS Discourse [1]_ regarding incorporating third-party client libraries into the ROS 2 core. + + +Integration and Infrastructure +============================== + + * rosidl pipeline + * colcon + * rosdep + * bloom + +Languages and ecosystems +======================== + + * C, C++, and CMake + * python and PyPI + * Rust and Crates + +Alternative approaches +====================== + + * document the existing gradle, node, and rust approaches + +References +========== + +.. [1] Process for third-party client libraries to be incorporated into the core ROS 2 distribution + (https://discourse.ros.org/t/process-for-third-party-client-libraries-to-be-incorporated-into-the-core-ros-2-distribution/27325/) + + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: From fc9cef8584742e3004558c22a4c739a451afcb5e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnar=C3=B6k?= Date: Wed, 21 Sep 2022 14:17:06 -0700 Subject: [PATCH 2/8] Start filling out the requirements by different tools. --- rep-2013.rst | 57 ++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 51 insertions(+), 6 deletions(-) diff --git a/rep-2013.rst b/rep-2013.rst index 3c9acd7b..3fae6c3f 100644 --- a/rep-2013.rst +++ b/rep-2013.rst @@ -18,16 +18,56 @@ In particular this REP is an attempt to elaborate on the questions raised on ROS Integration and Infrastructure ============================== - * rosidl pipeline - * colcon - * rosdep - * bloom +rosidl pipeline +^^^^^^^^^^^^^^^ + +TODO introduce the idl pipeline in a way that is accurate and succinct. +Message packages in ROS contain generated code for supported languages to directly interact with message fields. +The rosidl pipeline handles the process of generating language specific code for working with each supported runtime and typesupport. +Generating and distributing code for a rosidl runtime on the ROS build farm requires the rosidl runtime and typesupport packages to be available when message packages are built. + +package build types +^^^^^^^^^^^^^^^^^^^ + +TODO hoist in some of the stuff common to colcon and bloom + +colcon +^^^^^^ + +colcon is the recommended build tool for ROS 2. +It is highly extensible and support for nearly all package build types is provided by extensions rather than built into colcon core. +Support for most build types is also separated into base support and ros-specific support so that colcon can be used with that build type with or without ROS. +The prime examples here are colcon-cmake [2]_ and colcon-ros-cmake [3]_ which provide general support for CMake packages and some ROS specific behavior which builds on the existing CMake support. + +rosdep +^^^^^^ + +If your client library is associated with a language-specific package manager, it can be useful to support that language specific package manager in rosdep. +In order for a package to be released on the ROS build farm, all of its dependencies must be satisifed either in the ROS distribution directly (sometimes referred to as vendoring) or available from the platform package manager. +However, it is still useful to have rosdep support while packages are being developed and to support community members who may be working on packages that do not need to be released on the ROS build farm. + + +bloom +^^^^^ + +When building binary packages we convert ROS package metadata into metadata for each target platform's specific packaging infrastructure. +As a result, a pre-requisite to supporting client library packages in ROS is supporting the target language and library ecosystem in ROS 2's supported platforms. +Bloom templates supporting the new build type associated with the client library language must be added to generate platform packaging infrastructure for use on the ROS build farm. + +At the moment, bloom is also responsible for injecting certain dependencies (such as typesupport and idl pipeline dependencies) into packages containing idl. +Changes to the injected packages may be required in order to support additional client libraries or typesupports. + +ros_buildfarm +^^^^^^^^^^^^^ + +If everything else has been set up successfully, no additional changes should be required in the ros_buildfarm project since it uses the information generated by bloom for packaging jobs, and runs development, PR, and CI jobs with colcon. Languages and ecosystems ======================== - * C, C++, and CMake + * C and C++ with CMake * python and PyPI + * C and C++ with Meson * Rust and Crates Alternative approaches @@ -38,9 +78,14 @@ Alternative approaches References ========== -.. [1] Process for third-party client libraries to be incorporated into the core ROS 2 distribution +.. [1] Process for third-party client libraries to be incorporated into the core ROS 2 distribution (https://discourse.ros.org/t/process-for-third-party-client-libraries-to-be-incorporated-into-the-core-ros-2-distribution/27325/) +.. [2] colcon-cmake project on GitHub + (https://github.com/colcon/colcon-cmake) + +.. [3] colcon-ros-cmake project on GitHub + (https://github.com/colcon/colcon-ros-cmake) Copyright ========= From b049904ca22e8e192cbffb8bff612fc2a00a6076 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnar=C3=B6k?= Date: Wed, 16 Nov 2022 17:09:39 -0800 Subject: [PATCH 3/8] Rewrite abstract to focus on Rust specifically. Hopefully this will focus the rep and make it easier to review and at the same time provide a template for projects like rclada. --- rep-2013.rst | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/rep-2013.rst b/rep-2013.rst index 3fae6c3f..3ce7568e 100644 --- a/rep-2013.rst +++ b/rep-2013.rst @@ -11,13 +11,20 @@ Post-History: 21-Sep-2022 Abstract ======== -This REP will document, retroactively, the aspects and expectations of language-specific client libraries in ROS 2 in order to serve as a template for new language communities and client libraries which can be integrated into ROS. -In particular this REP is an attempt to elaborate on the questions raised on ROS Discourse [1]_ regarding incorporating third-party client libraries into the ROS 2 core. +This REP documents the plan and implementation for incorporating support for the Rust language in ROS 2. +This support requires changes in build tools such as colcon, ros development tools like rosdep, and release tools like bloom, as well as additions to the packaged configurations of ROS 2 interface packages and the standard set of rosidl generators. +This REP was prompted by questions raised on ROS Discourse [1]_ regarding incorporating third-party client libraries into the ROS 2 core. +Rust[2]_ is a programming language which has a strong interest in the ROS community. +Rust projects have been active in the ROS community since 2018. +TODO ros2-rust, r2r, etc + +This REP focuses on ros2_rust[3]_, specifically the rosidl_generator_rs, rosidl_runtime_rs, and rclrs packages. Integration and Infrastructure ============================== + rosidl pipeline ^^^^^^^^^^^^^^^ @@ -37,7 +44,7 @@ colcon colcon is the recommended build tool for ROS 2. It is highly extensible and support for nearly all package build types is provided by extensions rather than built into colcon core. Support for most build types is also separated into base support and ros-specific support so that colcon can be used with that build type with or without ROS. -The prime examples here are colcon-cmake [2]_ and colcon-ros-cmake [3]_ which provide general support for CMake packages and some ROS specific behavior which builds on the existing CMake support. +The prime examples here are colcon-cmake [4]_ and colcon-ros-cmake [5]_ which provide general support for CMake packages and some ROS specific behavior which builds on the existing CMake support. rosdep ^^^^^^ @@ -81,10 +88,16 @@ References .. [1] Process for third-party client libraries to be incorporated into the core ROS 2 distribution (https://discourse.ros.org/t/process-for-third-party-client-libraries-to-be-incorporated-into-the-core-ros-2-distribution/27325/) -.. [2] colcon-cmake project on GitHub +.. [2] Rust programming language web site + (https://www.rust-lang.org) + +.. [3] ros2_rust project on GitHub + (https://github.com/ros2-rust/ros2_rust) + +.. [4] colcon-cmake project on GitHub (https://github.com/colcon/colcon-cmake) -.. [3] colcon-ros-cmake project on GitHub +.. [5] colcon-ros-cmake project on GitHub (https://github.com/colcon/colcon-ros-cmake) Copyright From b6d6390fb42bcfa66d00fefe5046aa8cbcd6fb49 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnaro=CC=88k?= Date: Thu, 17 Nov 2022 08:10:50 -0800 Subject: [PATCH 4/8] Add a section on workstation usage. This is a mostly a description of the status quo but I am pushing to get most direct system build dependencies described via rosdep. --- rep-2013.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/rep-2013.rst b/rep-2013.rst index 3ce7568e..0712b40e 100644 --- a/rep-2013.rst +++ b/rep-2013.rst @@ -21,6 +21,15 @@ TODO ros2-rust, r2r, etc This REP focuses on ros2_rust[3]_, specifically the rosidl_generator_rs, rosidl_runtime_rs, and rclrs packages. + +Developer workstation usage +=========================== + +ros2_rust includes setup instructions for building ros2-rust. +These instructions require the use of two colcon extensions not currently distributed with ros-dev-tools by default: ``colcon-cargo`` and ``colcon-ros-cargo`` as well as the ``cargo-ament-build`` extension for Rust's cargo build system. +Additional system dependencies are also required as installed by ``rosdep`` (TODO: nuclearsandwich) these are not currently installed via rosdep and we should correct that prior to publication) + + Integration and Infrastructure ============================== From ddbb5fa74aa6f06238eddd573e09fd194394e1e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnaro=CC=88k?= Date: Thu, 17 Nov 2022 08:11:43 -0800 Subject: [PATCH 5/8] Introduce the infrastructure integration section. --- rep-2013.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rep-2013.rst b/rep-2013.rst index 0712b40e..5e9bf7a7 100644 --- a/rep-2013.rst +++ b/rep-2013.rst @@ -33,6 +33,8 @@ Additional system dependencies are also required as installed by ``rosdep`` (TOD Integration and Infrastructure ============================== +The ROS build infrastructure, including the ROS 2 build farm, does not currently support building Rust packages. +There are several aspects to that support which we will try to summarize here. rosidl pipeline ^^^^^^^^^^^^^^^ From 30086149f3999f5a540bf2ab28493bf5ea812a55 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnaro=CC=88k?= Date: Thu, 17 Nov 2022 08:11:55 -0800 Subject: [PATCH 6/8] Link to documentation instead of incorporating it. This REP should focus on Rust integration. Documentation providing explanatory material for infrastructure should not be inlined but referenced here. --- rep-2013.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rep-2013.rst b/rep-2013.rst index 5e9bf7a7..accc9e9c 100644 --- a/rep-2013.rst +++ b/rep-2013.rst @@ -39,7 +39,7 @@ There are several aspects to that support which we will try to summarize here. rosidl pipeline ^^^^^^^^^^^^^^^ -TODO introduce the idl pipeline in a way that is accurate and succinct. +TODO link out to documentation describing the idl pipeline in a way that is accurate and succinct. Message packages in ROS contain generated code for supported languages to directly interact with message fields. The rosidl pipeline handles the process of generating language specific code for working with each supported runtime and typesupport. Generating and distributing code for a rosidl runtime on the ROS build farm requires the rosidl runtime and typesupport packages to be available when message packages are built. From 9ff1a3f1d3239fa1b744dbecce0efb31142936a0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnaro=CC=88k?= Date: Mon, 28 Nov 2022 10:11:57 -0800 Subject: [PATCH 7/8] Fix spelling of ros2_rust project. --- rep-2013.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rep-2013.rst b/rep-2013.rst index accc9e9c..2736d730 100644 --- a/rep-2013.rst +++ b/rep-2013.rst @@ -25,7 +25,7 @@ This REP focuses on ros2_rust[3]_, specifically the rosidl_generator_rs, rosidl_ Developer workstation usage =========================== -ros2_rust includes setup instructions for building ros2-rust. +ros2_rust includes setup instructions for building ros2_rust. These instructions require the use of two colcon extensions not currently distributed with ros-dev-tools by default: ``colcon-cargo`` and ``colcon-ros-cargo`` as well as the ``cargo-ament-build`` extension for Rust's cargo build system. Additional system dependencies are also required as installed by ``rosdep`` (TODO: nuclearsandwich) these are not currently installed via rosdep and we should correct that prior to publication) From e453053b543f1cf656a1a81924c0bdb2ef324825 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Steven!=20Ragnaro=CC=88k?= Date: Mon, 28 Nov 2022 10:12:36 -0800 Subject: [PATCH 8/8] Update infrastructure sections. --- rep-2013.rst | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/rep-2013.rst b/rep-2013.rst index 2736d730..60d98ad6 100644 --- a/rep-2013.rst +++ b/rep-2013.rst @@ -40,14 +40,18 @@ rosidl pipeline ^^^^^^^^^^^^^^^ TODO link out to documentation describing the idl pipeline in a way that is accurate and succinct. -Message packages in ROS contain generated code for supported languages to directly interact with message fields. +Message packages in ROS contain generated code (and potentially libraries compiled from that generated code) for supported languages to directly interact with message fields. The rosidl pipeline handles the process of generating language specific code for working with each supported runtime and typesupport. -Generating and distributing code for a rosidl runtime on the ROS build farm requires the rosidl runtime and typesupport packages to be available when message packages are built. +Generating and distributing code for a rosidl runtime on the ROS build farm requires the rosidl generator and typesupport packages to be available when message packages are built. +Interface packages must declare build dependencies on rosidl_default_generators for expected functionality to work. +Bloom currently inserts build dependencies on rosidl static typesupport packages appropriate to each platform and rosdistro. package build types ^^^^^^^^^^^^^^^^^^^ -TODO hoist in some of the stuff common to colcon and bloom +The colcon build tool has an extension system for discovering and analyzing packages which includes detecting how the package must be built. +For ROS packages, this discovery is augmented by reading the ``package.xml`` contents including the ```` field. +These build types are also used by other tools in the ROS ecosystem, most notably bloom, to generate appropriate packaging metadata for various platforms. colcon ^^^^^^ @@ -57,6 +61,13 @@ It is highly extensible and support for nearly all package build types is provid Support for most build types is also separated into base support and ros-specific support so that colcon can be used with that build type with or without ROS. The prime examples here are colcon-cmake [4]_ and colcon-ros-cmake [5]_ which provide general support for CMake packages and some ROS specific behavior which builds on the existing CMake support. +The ros2-rust team has already created colcon extensions supporting the use of cargo-based Rust packages in a colcon workspace. + +colcon-cargo and crates.io +-------------------------- + +The current versions of these tools allow the mix of package dependencies from crates.io, the Rust package registry, and the local workspace through the use of underlying cargo features. + rosdep ^^^^^^