Distributed System in ROS2

When you run your ROS2 application, there are often requirements to run ROS2 nodes in different locations on different machines. Since ROS2 uses in the abstract DDS layer for communication between the nodes, we can arrange the communication very easily.
In order to understand fully the architecture of ROS2 I recommend you to be familiar with this article.

For the purpose of this article, I communicated the Nvidia Jetson Xavier and my PC. Both units run ROS2 Foxy. I connected the RGB camera to Jetso and ran a publisher (in C++ or Python). The PC runs subscribers (C++ or Python) to subscribe to the topic and displays a video stream.

The source code is available on my GitHub.

The architecture can be depicted as follows.

Distributed ROS2 nodes

Below I include, the ROS2 command to run your application according to the above figure. The application can be run on the same machine or other configurations without changes.

The most important however is to run the application (publisher — subscriber on the same ROS_DOMAIN_ID. In order to align you have to export the same ID.

//check the ROS environment (both on publisher and subscriber)

env | grep ROS

// you should recive similar

ROS_VERSION=2
ROS_PYTHON_VERSION=3
ROS_DOMAIN_ID=5
ROS_LOCALHOST_ONLY=0
ROS_DISTRO=foxy

// as you can see I adjhsted DOMAIN to 5

//run 

export ROS_DOMAIN_ID=5

If you run your application in C++,

mkdir -p open_dev_ws/src
cd open_dev_ws/src

//created ROS package
ros2 pkg create dev_opencv --dependencies rclcpp std_msgs sensor_msgs cv_bridge image_transport OpenCV

Modify your CMakeList.txt (see also my Github). Here I display CMakeList.txt to build publisher and subscriber. You have to choose your machine type.

add_executable(pub_cam src/pub_cam.cpp)
ament_target_dependencies(pub_cam rclcpp std_msgs sensor_msgs cv_bridge image_transport OpenCV)

add_executable(sub_cam src/sub_cam.cpp)
ament_target_dependencies(sub_cam rclcpp std_msgs sensor_msgs cv_bridge image_transport OpenCV)

install(TARGETS
   pub_cam
   sub_cam
   DESTINATION lib/${PROJECT_NAME}
 )

Copy the pub_cam.cpp and sub_cam.cpp (from my repository) to /dev_opencv/src.

cd open_dev_ws
colcon build
source ./install/setup.bash

// for publiser run
 ros2 run dev_opencv pub_cam

// for subscriber
 ros2 run dev_opencv sub_cam

If you run your application in Python,

mkdir -p /dev_opencv_py/src
cd dev_opencv_py/src


//create ROS package
ros2 pkg create --build-type ament_python dev_opencv_py --dependencies rclpy image_transport cv_bridge sensor_msgs std_msgs opencv2

Copy Python scripts to dev_opencv_py and modify setup.py,

    entry_points={
        'console_scripts': [
          'cam_pub = opencv_tools.cam_pub:main',
          'cam_sub = opencv_tools.cam_sub:main',
        ],
    },

In package.xml change

from
<depend>opencv2</depend>
to  
<depend>opencv-python</depend>

Run following command,

cd dev_open_py
colcon build
source ./install/setup.bash

// for publiser run
 ros2 run dev_opencv_py cam_pub

// for subscriber
 ros2 run dev_opencv_py cam_sub

Thank you for reading.