Skip to content

Commit 1335171

Browse files
quic-zhaoyuanquic-zhaoyuan
authored
Merge pull request #4 from zycczy/readme
update readme
2 parents 08adb06 + 70b5cc0 commit 1335171

File tree

1 file changed

+196
-72
lines changed

1 file changed

+196
-72
lines changed

‎README.md‎

Lines changed: 196 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -1,99 +1,223 @@
11
# QRB ROS Benchmark
22

3-
qrb_ros_benchmark is a ros package that extends the functionality of opensource [ROS2 Benchmark](https://github.com/NVIDIA-ISAAC-ROS/ros2_benchmark) package.
4-
53
## Overview
64

7-
qrb_ros_benchmark is a ros package that extends the functionality of opensource ros2_benchmark package. qrb_ros_benchmark supports testing the performance of ROS nodes after accelerating by the dmabuf_transport / qrb_ros_transport function. [QRB ROS Transport](https://github.com/qualcomm-qrb-ros/qrb_ros_transport), it leverages type adaption and intra process communication to optimize message formats and dramatically accelerate communication between participating nodes.
8-
5+
[QRB ROS Benchmark](https://github.com/qualcomm-qrb-ros/qrb_ros_benchmark) is a benchmarking tool designed for evaluating performance of ROS components on Qualcomm robotics platforms. It provides reusable components for benchmarking various message types and ROS nodes, with a focus on zero-copy transport mechanisms.
96

10-
## Build
7+
This package builds on [ros2_benchmark](https://github.com/ros-acceleration/ros2_benchmark) and extends it with specialized components for benchmarking QRB ROS transport types, including:
118

12-
Currently, we only support use QCLINUX to build
9+
- QRB transport types (Image, IMU, PointCloud2)
10+
- DMABuf transport types (Image, PointCloud2)
11+
- Standard ROS message types
1312

14-
1. Setup environments follow this document 's [Set up the cross-compile environment.](https://docs.qualcomm.com/bundle/publicresource/topics/80-65220-2/develop-your-first-application_6.html?product=1601111740013072&facet=Qualcomm%20Intelligent%20Robotics%20(QIRP)%20Product%20SDK&state=releasecandidate) part
13+
## Getting Started
1514

16-
2. Create `ros_ws` directory in `<qirp_decompressed_workspace>/qirp-sdk/`
15+
### Build
1716

18-
3. Clone this repository under `<qirp_decompressed_workspace>/qirp-sdk/ros_ws`
19-
```bash
20-
git clone https://github.com/qualcomm-qrb-ros/lib_mem_dmabuf.git
21-
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_imu.git
22-
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_transport.git
23-
git clone https://github.com/qualcomm-qrb-ros/dmabuf_transport.git
24-
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_tensor_list_msgs.git
25-
git clone https://github.com/NVIDIA-ISAAC-ROS/ros2_benchmark.git
26-
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_benchmark.git
27-
```
17+
For the Qualcomm QCLinux platform, we provide two ways to build this package:
2818

29-
4. Add dependencies in ros2_benhcmark/ros2_benchmark/CMakeLists.txt
30-
```bash
31-
find_package(rosbag2_compression REQUIRED)
32-
```
19+
<details>
20+
<summary>On-Device Compilation with Docker</summary>
3321

34-
5. Modify line 58 of the `/qirp-sdk/toolchain/install_dir/sysroots/armv8-2a-qcom-linux/usr/share/rosbag2_compression_zstd/cmake/export_rosbag2_compression_zstdExport.cmake` file to:
35-
```bash
36-
INTERFACE_LINK_LIBRARIES "rcpputils::rcpputils;rosbag2_compression::rosbag2_compression;${_IMPORT_PREFIX}/lib/libzstd.so"
37-
```
22+
1. Set up the QCLinux Docker environment following the [QRB ROS Docker Setup](https://github.com/qualcomm-qrb-ros/qrb_ros_docker?tab=readme-ov-file#quickstart).
3823

39-
6. Build this project
40-
```bash
41-
export AMENT_PREFIX_PATH="${OECORE_TARGET_SYSROOT}/usr;${OECORE_NATIVE_SYSROOT}/usr"
42-
export PYTHONPATH=${PYTHONPATH}:${OECORE_TARGET_SYSROOT}/usr/lib/python3.10/site-packages
24+
2. Clone and build the source code:
4325

44-
colcon build --merge-install --cmake-args \
45-
-DPython3_ROOT_DIR=${OECORE_TARGET_SYSROOT}/usr \
46-
-DPython3_NumPy_INCLUDE_DIR=${OECORE_TARGET_SYSROOT}/usr/lib/python3.10/site-packages/numpy/core/include \
47-
-DPYTHON_SOABI=cpython-310-aarch64-linux-gnu -DCMAKE_STAGING_PREFIX=$(pwd)/install \
48-
-DCMAKE_PREFIX_PATH=$(pwd)/install/share \
49-
-DBUILD_TESTING=OFF
50-
```
26+
```bash
27+
cd ~/qrb_ros_ws/src/qrb_ros_docker/scripts
28+
bash docker_run.sh
29+
30+
# Clone this repository and dependencies
31+
git clone https://github.com/qualcomm-qrb-ros/ros2_benchmark.git
32+
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_transport.git
33+
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_benchmark.git
34+
git clone https://github.com/qualcomm-qrb-ros/lib_mem_dmabuf.git
35+
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_imu.git
36+
git clone https://github.com/qualcomm-qrb-ros/dmabuf_transport.git
37+
38+
# Build packages
39+
colcon build --packages-select qrb_ros_benchmark
40+
```
5141

52-
7. Push to the device & Install
53-
```bash
54-
cd `<qirp_decompressed_workspace>/qirp-sdk/ros_ws/install`
55-
tar czvf qrb_ros_benchmark.tar.gz lib share
56-
scp qrb_ros_benchmark.tar.gz root@[ip-addr]:/opt/
57-
ssh root@[ip-addr]
58-
(ssh) tar -zxf /opt/qrb_ros_benchmark.tar.gz -C /opt/qcom/qirp-sdk/usr/
59-
```
42+
</details>
6043

61-
## Run
44+
<details>
45+
<summary>Cross Compilation with QIRP SDK</summary>
6246

63-
This package supports running it directly from the command with test script.
47+
1. Set up the QIRP SDK environment: Refer to [QRB ROS Documents: Getting Started](https://qualcomm-qrb-ros.github.io/main/getting_started/environment_setup.html).
6448

65-
a.Run with command
49+
2. Create a workspace and clone the source code:
6650

67-
1. Source this file to set up the environment on your device:
6851
```bash
69-
ssh root@[ip-addr]
70-
(ssh) export HOME=/opt
71-
(ssh) source /opt/qcom/qirp-sdk/qirp-setup.sh
72-
(ssh) export ROS_DOMAIN_ID=xx
73-
(ssh) source /usr/bin/ros_setup.bash
52+
mkdir -p <qirp_decompressed_workspace>/qirp-sdk/ros_ws
53+
cd <qirp_decompressed_workspace>/qirp-sdk/ros_ws
54+
55+
git clone https://github.com/qualcomm-qrb-ros/ros2_benchmark.git
56+
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_transport.git
57+
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_benchmark.git
58+
git clone https://github.com/qualcomm-qrb-ros/lib_mem_dmabuf.git
59+
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_imu.git
60+
git clone https://github.com/qualcomm-qrb-ros/dmabuf_transport.git
7461
```
7562

76-
2. Use this command to run this package
63+
3. Build the source code with QIRP SDK:
64+
7765
```bash
78-
(ssh) launch_test xx/qrb_ros_imu_benchmark.py
66+
colcon build --merge-install --packages-select qrb_ros_benchmark --cmake-args \
67+
-DPYTHON_EXECUTABLE=${OECORE_NATIVE_SYSROOT}/usr/bin/python3 \
68+
-DPython3_NumPy_INCLUDE_DIR=${OECORE_NATIVE_SYSROOT}/usr/lib/python3.12/site-packages/numpy/core/include \
69+
-DPYTHON_SOABI=cpython-312-aarch64-linux-gnu \
70+
-DCMAKE_MAKE_PROGRAM=/usr/bin/make \
71+
-DBUILD_TESTING=OFF
7972
```
8073

81-
82-
## Acceleration Performance Test
83-
84-
This package is based on ROS2 Benchmark, and additional support to test the [QRB ROS Transport](https://github.com/qualcomm-qrb-ros/qrb_ros_transport) optimize message formats and accelerate communication between participating nodes.
85-
86-
## Packages
87-
88-
Will update in the future.
89-
90-
91-
## Contributions
92-
93-
Thanks for your interest in contributing to qrb_ros_benchmark! Please read our [Contributions Page](CONTRIBUTING.md) for more information on contributing features or bug fixes. We look forward to your participation!
74+
</details>
75+
76+
### Usage
77+
78+
This package provides two main components:
79+
80+
1. `QrbPlaybackNode`: Manages message playback from buffers with various message types
81+
2. `QrbMonitorNode`: Records timing information for performance benchmarking
82+
83+
#### Creating a benchmark script
84+
85+
Here's an example of how to use these components in a benchmark script:
86+
87+
```python
88+
from launch_ros.actions import ComposableNodeContainer
89+
from launch_ros.descriptions import ComposableNode
90+
91+
from ros2_benchmark import ROS2BenchmarkConfig, ROS2BenchmarkTest
92+
from ros2_benchmark import MonitorPerformanceCalculatorsInfo
93+
from ros2_benchmark import BasicPerformanceCalculator, BenchmarkMode
94+
95+
def launch_setup(container_prefix, container_sigterm_timeout):
96+
"""Generate launch description for benchmarking a node."""
97+
98+
# Configure the playback node
99+
playback_node = ComposableNode(
100+
name='QrbPlaybackNode',
101+
namespace=TestBenchmarkNode.generate_namespace(),
102+
package='qrb_ros_benchmark',
103+
plugin='qrb_ros::benchmark::QrbPlaybackNode',
104+
parameters=[{
105+
'data_formats': [
106+
'qrb_ros/transport/type/Image'
107+
],
108+
}],
109+
remappings=[
110+
('buffer/input0', '/input_topic'),
111+
('input0', '/output_topic')
112+
]
113+
)
114+
115+
# Configure the node to benchmark
116+
node_under_test = ComposableNode(
117+
name='NodeUnderTest',
118+
namespace=TestBenchmarkNode.generate_namespace(),
119+
package='your_package',
120+
plugin='your_namespace::YourNodePlugin',
121+
parameters=[{
122+
# Your node parameters
123+
}],
124+
remappings=[
125+
('input', '/output_topic'),
126+
('output', '/result_topic')
127+
]
128+
)
129+
130+
# Configure the monitor node
131+
monitor_node = ComposableNode(
132+
name='QrbMonitorNode',
133+
namespace=TestBenchmarkNode.generate_namespace(),
134+
package='qrb_ros_benchmark',
135+
plugin='qrb_ros::benchmark::QrbMonitorNode',
136+
parameters=[{
137+
'monitor_data_format': 'qrb_ros/transport/type/Image',
138+
}],
139+
remappings=[
140+
('output', '/result_topic')
141+
]
142+
)
143+
144+
composable_node_container = ComposableNodeContainer(
145+
name='container',
146+
namespace=TestBenchmarkNode.generate_namespace(),
147+
package='rclcpp_components',
148+
executable='component_container_mt',
149+
prefix=container_prefix,
150+
sigterm_timeout=container_sigterm_timeout,
151+
composable_node_descriptions=[
152+
playback_node,
153+
node_under_test,
154+
monitor_node,
155+
],
156+
output='screen'
157+
)
158+
159+
return [composable_node_container]
160+
161+
def generate_test_description():
162+
return TestBenchmarkNode.generate_test_description_with_nsys(launch_setup)
163+
164+
class TestBenchmarkNode(ROS2BenchmarkTest):
165+
"""Performance benchmark test for your node."""
166+
167+
# Custom configurations
168+
config = ROS2BenchmarkConfig(
169+
benchmark_name='Your Node Benchmark',
170+
benchmark_duration=5,
171+
test_iterations=5,
172+
# Other configuration parameters as needed
173+
)
174+
175+
def test_benchmark(self):
176+
self.run_benchmark()
177+
```
178+
179+
For more complete examples, check the included benchmark scripts in the `/scripts` directory.
180+
181+
## Supported Message Types
182+
183+
The benchmark framework supports the following message types:
184+
185+
### QRB Transport Types
186+
- `qrb_ros/transport/type/Image`
187+
- `qrb_ros/transport/type/Imu`
188+
- `qrb_ros/transport/type/PointCloud2`
189+
190+
### DMABuf Transport Types
191+
- `dmabuf_transport/type/Image`
192+
- `dmabuf_transport/type/PointCloud2`
193+
194+
### ROS Message Types
195+
- `sensor_msgs::msg::Image`
196+
- `sensor_msgs::msg::CompressedImage`
197+
- `qrb_ros_tensor_list_msgs::msg::TensorList`
198+
199+
## Benchmark Results
200+
201+
Benchmark results are generated in JSON format and stored in the `results` directory. These files include performance metrics such as:
202+
203+
- Frame rates (mean playback and processing)
204+
- Latency statistics (min, max, mean)
205+
- Jitter statistics (min, max, mean, std dev)
206+
- CPU utilization metrics
207+
208+
## Contributing
209+
210+
We welcome contributions to QRB ROS Benchmark! Whether they are features, issues, documentation, guides, or anything else, please refer to our [contribution guidelines](./CONTRIBUTING.md) and [code of conduct](./CODE-OF-CONDUCT.md).
211+
212+
- Bug report: If you see an error message or encounter failures, please create a [bug report](../../issues)
213+
- Feature Request: If you have an idea for an improvement, please submit a [feature request](../../issues)
214+
215+
## Authors
216+
217+
* **Zhaoyuan Cheng** - *Maintainer* - [@quic-zhaoyuan](https://github.com/quic-zhaoyuan)
218+
219+
See also the list of [contributors](https://github.com/qualcomm-qrb-ros/qrb_ros_benchmark/contributors) who participated in this project.
94220
95221
## License
96222
97-
qrb_ros_benchmark is licensed under the BSD-3-clause "New" or "Revised" License.
98-
99-
Check out the [LICENSE](LICENSE) for more details.
223+
Project is licensed under the [BSD-3-Clause License](https://spdx.org/licenses/BSD-3-Clause.html). See [LICENSE](./LICENSE) for the full license text.

0 commit comments

Comments
 (0)