Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Viability of using RCLCPP as the client library for low resource arm based linux systems #398

Open
coalman321 opened this issue Oct 13, 2021 · 9 comments
Labels
question Further information is requested

Comments

@coalman321
Copy link

  • Hardware description: NI Roborio (armv7 softfp, with 256mb ram)
  • RTOS: linux kernel with real-time scheduler patch
  • Installation type: micro_ros_setup to build static library for platform
  • Version or commit hash: galactic

Hi, I have a question about the viability of using RCLCPP in place of RCLC for low power / resource constrained arm systems. So far I have a working version of micro_ros for the platform, as well as a working colcon.meta and toolchain, but was curious if it would be possible to use RCLCPP as the client library for the micro_ros client. Given our use case is already C++ on the device, using RCLCPP seems to be a better fit. Especially rather than than having to write a significant wrapper around RCLC. Would this switch even be possible, or are the two client libraries functionally different enough to a point where it would not work? Thank you for your help! (Sorry if this is the wrong place to pose this question)

@JanStaschulat
Copy link

JanStaschulat commented Oct 14, 2021

Hi @coalman321. Give it a try. The interface for rclcpp is rcl, so there should be no other dependencies (potentially, the type conversion on xrce-dds-client support might be an issue). Mainly you have to watch out for total memory consumption as well as memory fragmentation with RCLCPP. You could make some measurements on a normal linux desktop with ROS 2 Galactic and rclcpp - with your C++ application and check with tools like valgrind, how much (static/dynamic) memory is actually used. This might give you an indication, if 256MB RAM is sufficient.

My question would be - with such a powerfull machine with large memory, why are you not using ROS 2 directly, as you have Linux with Preempt-RT patch already running?

Functionally RCLCPP and RCLC are compliant (RCLC is feature complete: creation of nodes/publisher/subscription/service/client/parameters/...)The RCLC has a (I would argue) a more advanced executor, it offers more deterministic features for real-time behavior. But if you don't need that in your application, start with default RCLCPP Executor. This is the only major difference between RCLCPP and RCLC. Just noting, the rclc-executor pre-allocates memory in the configuration-phase. At runtime there is no dynamic memory allocation. This is not the case for the RCLCPP-Executor (which is using STL-containers).

@coalman321
Copy link
Author

Our biggest issue with RCLC right now is that it doesnt seem to have any real way to deal with C++ and its classing system, something our software is based around. We attempted to build minimal versions of ros2 for the platform, but were unable to get any real traction. Many system dependencies that ROS expects to have access to would not cooperate nor build properly, largely due to the older compiler and kernel being used by the system (something we do not have control over).

That aside, I did make an attempt, by adding RCLCPP into the mcu_ws as well as any other dependencies it has listed and the build does start as expected, but seems to be failing due to missing C++ type support. have not yet had a chance to work out out why, other than the build fails on libstatistics_collector due to missing some of the C++ builtin_interfaces.
This is the current error:

fatal error: builtin_interfaces/msg/time.hpp: No such file or directory
 #include "builtin_interfaces/msg/time.hpp"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This seems strange as C++ typesupport is present, and seems to build okay, but isnt producing results. These are the current build args for the generate_lib build.sh script:

colcon build \
		--merge-install \
		--metas $COLCON_META \
		--cmake-args \
		"--no-warn-unused-cli" \
		-DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=OFF \
		-DTHIRDPARTY=ON \
		-DBUILD_TESTING=OFF \
		-DCMAKE_BUILD_TYPE=Release \
		-DCMAKE_TOOLCHAIN_FILE=$TOOLCHAIN \
		-DCMAKE_VERBOSE_MAKEFILE=ON; \

And this is the list of the current packages in mcu_ws:

colcon list
action_msgs     ros2/rcl_interfaces/action_msgs (ros.ament_cmake)
actionlib_msgs  ros2/common_interfaces/actionlib_msgs   (ros.ament_cmake)
builtin_interfaces      ros2/rcl_interfaces/builtin_interfaces  (ros.ament_cmake)
common_interfaces       ros2/common_interfaces/common_interfaces        (ros.ament_cmake)
composition_interfaces  ros2/rcl_interfaces/composition_interfaces      (ros.ament_cmake)
diagnostic_msgs ros2/common_interfaces/diagnostic_msgs  (ros.ament_cmake)
example_interfaces      ros2/example_interfaces (ros.ament_cmake)
geometry_msgs   ros2/common_interfaces/geometry_msgs    (ros.ament_cmake)
libstatistics_collector libstatistics_collector (ros.ament_cmake)
libyaml_vendor  ros2/libyaml_vendor     (ros.ament_cmake)
lifecycle_msgs  ros2/rcl_interfaces/lifecycle_msgs      (ros.ament_cmake)
micro_ros_msgs  uros/micro_ros_msgs     (ros.ament_cmake)
micro_ros_utilities     uros/micro_ros_utilities        (ros.ament_cmake)
microcdr        eProsima/Micro-CDR      (cmake)
microxrcedds_client     eProsima/Micro-XRCE-DDS-Client  (cmake)
nav_msgs        ros2/common_interfaces/nav_msgs (ros.ament_cmake)
rcl     uros/rcl/rcl    (ros.ament_cmake)
rcl_action      uros/rcl/rcl_action     (ros.ament_cmake)
rcl_interfaces  ros2/rcl_interfaces/rcl_interfaces      (ros.ament_cmake)
rcl_lifecycle   uros/rcl/rcl_lifecycle  (ros.ament_cmake)
rcl_logging_interface   ros2/rcl_logging/rcl_logging_interface  (ros.ament_cmake)
rcl_logging_noop        ros2/rcl_logging/rcl_logging_noop       (ros.ament_cmake)
rcl_yaml_param_parser   ros2/rcl/rcl_yaml_param_parser  (ros.ament_cmake)
rclc    uros/rclc/rclc  (ros.ament_cmake)
rclc_lifecycle  uros/rclc/rclc_lifecycle        (ros.ament_cmake)
rclc_parameter  uros/rclc/rclc_parameter        (ros.ament_cmake)
rclcpp  ros2/rclcpp/rclcpp      (ros.ament_cmake)
rclcpp_components       ros2/rclcpp/rclcpp_components   (ros.ament_cmake)
rclcpp_lifecycle        ros2/rclcpp/rclcpp_lifecycle    (ros.ament_cmake)
rcpputils       ros2/rcpputils  (ros.ament_cmake)
rcutils uros/rcutils    (ros.ament_cmake)
rmw     ros2/rmw/rmw    (ros.ament_cmake)
rmw_implementation      ros2/rmw_implementation/rmw_implementation      (ros.ament_cmake)
rmw_implementation_cmake        ros2/rmw/rmw_implementation_cmake       (ros.ament_cmake)
rmw_microxrcedds        uros/rmw_microxrcedds/rmw_microxrcedds_c        (ros.ament_cmake)
ros2trace       uros/tracetools/ros2trace       (ros.ament_python)
rosgraph_msgs   ros2/rcl_interfaces/rosgraph_msgs       (ros.ament_cmake)
rosidl_adapter  ros2/rosidl/rosidl_adapter      (ros.ament_cmake)
rosidl_cli      ros2/rosidl/rosidl_cli  (ros.ament_python)
rosidl_cmake    ros2/rosidl/rosidl_cmake        (ros.ament_cmake)
rosidl_default_generators       ros2/rosidl_defaults/rosidl_default_generators  (ros.ament_cmake)
rosidl_default_runtime  ros2/rosidl_defaults/rosidl_default_runtime     (ros.ament_cmake)
rosidl_generator_c      ros2/rosidl/rosidl_generator_c  (ros.ament_cmake)
rosidl_generator_cpp    ros2/rosidl/rosidl_generator_cpp        (ros.ament_cmake)
rosidl_generator_dds_idl        ros2/rosidl_dds/rosidl_generator_dds_idl        (ros.ament_cmake)
rosidl_parser   ros2/rosidl/rosidl_parser       (ros.ament_cmake)
rosidl_runtime_c        ros2/rosidl/rosidl_runtime_c    (ros.ament_cmake)
rosidl_runtime_cpp      ros2/rosidl/rosidl_runtime_cpp  (ros.ament_cmake)
rosidl_typesupport_c    uros/rosidl_typesupport/rosidl_typesupport_c    (ros.ament_cmake)
rosidl_typesupport_cpp  uros/rosidl_typesupport/rosidl_typesupport_cpp  (ros.ament_cmake)
rosidl_typesupport_interface    ros2/rosidl/rosidl_typesupport_interface        (ros.ament_cmake)
rosidl_typesupport_introspection_c      ros2/rosidl/rosidl_typesupport_introspection_c  (ros.ament_cmake)
rosidl_typesupport_introspection_cpp    ros2/rosidl/rosidl_typesupport_introspection_cpp        (ros.ament_cmake)
rosidl_typesupport_microxrcedds_c       uros/rosidl_typesupport_microxrcedds/rosidl_typesupport_microxrcedds_c(ros.ament_cmake)
rosidl_typesupport_microxrcedds_c_tests uros/rosidl_typesupport_microxrcedds/test/c     (ros.ament_cmake)
rosidl_typesupport_microxrcedds_cpp     uros/rosidl_typesupport_microxrcedds/rosidl_typesupport_microxrcedds_cpp      (ros.ament_cmake)
rosidl_typesupport_microxrcedds_cpp_tests       uros/rosidl_typesupport_microxrcedds/test/cpp   (ros.ament_cmake)
rosidl_typesupport_microxrcedds_test_msg        uros/rosidl_typesupport_microxrcedds/test/msg   (ros.ament_cmake)
sensor_msgs     ros2/common_interfaces/sensor_msgs      (ros.ament_cmake)
sensor_msgs_py  ros2/common_interfaces/sensor_msgs_py   (ros.ament_python)
shape_msgs      ros2/common_interfaces/shape_msgs       (ros.ament_cmake)
statistics_msgs ros2/rcl_interfaces/statistics_msgs     (ros.ament_cmake)
std_msgs        ros2/common_interfaces/std_msgs (ros.ament_cmake)
std_srvs        ros2/common_interfaces/std_srvs (ros.ament_cmake)
stereo_msgs     ros2/common_interfaces/stereo_msgs      (ros.ament_cmake)
test_interface_files    ros2/test_interface_files       (ros.ament_cmake)
test_msgs       ros2/rcl_interfaces/test_msgs   (ros.ament_cmake)
test_rmw_implementation ros2/rmw_implementation/test_rmw_implementation (ros.ament_cmake)
tracetools      uros/tracetools/tracetools      (ros.ament_cmake)
tracetools_launch       uros/tracetools/tracetools_launch       (ros.ament_python)
tracetools_read uros/tracetools/tracetools_read (ros.ament_python)
tracetools_test uros/tracetools/tracetools_test (ros.ament_cmake)
tracetools_trace        uros/tracetools/tracetools_trace        (ros.ament_python)
trajectory_msgs ros2/common_interfaces/trajectory_msgs  (ros.ament_cmake)
unique_identifier_msgs  ros2/unique_identifier_msgs     (ros.ament_cmake)
visualization_msgs      ros2/common_interfaces/visualization_msgs       (ros.ament_cmake)

@coalman321
Copy link
Author

coalman321 commented Oct 14, 2021

Okay, so that was a failure on my part. I had forgotten Colcon copies all executables into the install directory the uses the copy when the executable is run. I updated my micro_ros_setup build and it no longer fails at libstatistics_collector. Now its failing at RCLCPP. the current error is as follows:

/home/coalman321/code/micro_ros_ws/firmware/mcu_ws/ros2/rclcpp/rclcpp/src/rclcpp/context.cpp: In member function 'virtual void rclcpp::Context::init(int, const char* const*, const rclcpp::InitOptions&)':
/home/coalman321/code/micro_ros_ws/firmware/mcu_ws/ros2/rclcpp/rclcpp/src/rclcpp/context.cpp:220:24: error: 'using element_type = struct rcl_context_t {aka struct rcl_context_t}' has no member named 'global_arguments'
         &rcl_context_->global_arguments,
                        ^~~~~~~~~~~~~~~~

It looks like the context manager for micro_ros's RCL may not be strictly compatible with the context manager RCLCPP is expecting. The direct cause of this though seems to be a flag not being set for the compilation of RCL. #ifdef RCL_COMMAND_LINE_ENABLED

@coalman321
Copy link
Author

coalman321 commented Oct 15, 2021

Tracking the resolution of this compilation issue with the RCLCPP group. ros2/rclcpp#1803

@JanStaschulat
Copy link

JanStaschulat commented Oct 15, 2021

I have also documented first steps for a C++ wrapper around RCLC here: PR ros2/rclc#199 and issue ros2/rclc#126

@JanStaschulat
Copy link

JanStaschulat commented Oct 15, 2021

Hi @coalman321 cc @pablogs9

Yes, you need to define RCL_COMMAND_LINE_ENABLED
https://github.com/micro-ROS/rcl/blob/ae6b2abc43f71ad9e7d1fce6da76d0e2cc88578a/rcl/include/rcl/context.h#L117
Could you check with some print/echo, that the variable is indeed properly configured when compiling micro-ros/rcl?

Tracking the resolution of this compilation issue with the RCLCPP group. ros2/rclcpp#1803

This is an issue with the micro-ros/rcl package not the ros2/rcl package:
https://github.com/ros2/rcl/blob/72cee8a9fbda40304ff4f8e7c5d12b6ce6410f8d/rcl/include/rcl/context.h#L117
The file ros2/rcl/include/rcl/context.h does not have the RCL_COMMAND_LINE_ENABLED. but the micro-ros/rcl fork does:
https://github.com/micro-ROS/rcl/blob/ae6b2abc43f71ad9e7d1fce6da76d0e2cc88578a/rcl/include/rcl/context.h#L117

@JanStaschulat
Copy link

ROS 2 Galactic requires C++17 compiler:
https://www.ros.org/reps/rep-2000.html#galactic-geochelone-may-2021-may-2022

Minimum language requirements:
  - C++17
  - Python 3.6 

Does your compiler support that?

@coalman321
Copy link
Author

coalman321 commented Oct 15, 2021

ROS 2 Galactic requires C++17 compiler:

The compiler is an armv7 build of GCC, version 7.3.0, which does support the C++ 17 standard. I am not 100% sure on the python 3.6 support, though I believe the answer is yes. The armv7 version is not currently installed on the build machine though.

Could you check with some print/echo, that the variable is indeed properly configured when compiling micro-ros/rcl?

I am working on checking the cmake variable now
I tested, and the variable is being set during build. It was tested via editing the following code in the CMakelists.txt

if(RCL_COMMAND_LINE_ENABLED)
  ament_target_dependencies(${PROJECT_NAME}
    "rcl_yaml_param_parser"
  )
  message("RCL is commandline enabled")
endif()

Colcon does show the message during compile of RCL.

Starting >>> rcl
--- stderr: rcl                                 
CMake Warning at /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rcutils/cmake/ament_cmake_export_libraries-extras.cmake:116 (message):
 Package 'rcutils' exports library 'dl' which couldn't be found
Call Stack (most recent call first):
 /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rcutils/cmake/rcutilsConfig.cmake:41 (include)
 /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rosidl_runtime_c/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
 /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rosidl_runtime_c/cmake/rosidl_runtime_cConfig.cmake:41 (include)
 /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/builtin_interfaces/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
 /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/builtin_interfaces/cmake/builtin_interfacesConfig.cmake:41 (include)
 /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rcl_interfaces/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
 /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rcl_interfaces/cmake/rcl_interfacesConfig.cmake:41 (include)
 CMakeLists.txt:10 (find_package)


CMake Warning (dev) at CMakeLists.txt:78 (add_library):
 ADD_LIBRARY called with SHARED option but the target platform does not
 support dynamic linking.  Building a STATIC library instead.  This may lead
 to problems.
This warning is for project developers.  Use -Wno-dev to suppress it.

RCL is commandline enabled

@coalman321
Copy link
Author

ROS 2 Galactic requires C++17 compiler: https://www.ros.org/reps/rep-2000.html#galactic-geochelone-may-2021-may-2022

Minimum language requirements:
  - C++17
  - Python 3.6 

Does your compiler support that?

Per this comment, I have some new findings. This weekend, I was able to cross compile up to rclcpp using a stripped down ros installation for the platform. I am still curious about the application for micro_ros, and am willing to re-try the compilation process with both a valid sysroot, and toolchain setup. Although, in this situation, I doubt that it will solve the missing compile flags being an issue, as it was not an issue for normal micro_ros builds.

`

@pablogs9 pablogs9 added the question Further information is requested label Jan 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants