Multi-threading to handle image pub/sub

[lucasw]

Publishing and subscribing to large images takes a lot of cpu time ( https://answers.ros.org/question/312964/ros2-megapixel-image-pubsub-cpu-usage-is-very-high/ ), so would like to put those operations in a separate thread and pass images back and forth between them.

One thread for all image publishing and subscribing, or one thread for all publishing and a second for subscribing. Or many threads for all of those?

If the ros executor is to be used this needs to be wrapped in a separate rclcpp::Node, and create a separate thread within the main imgui_ros node thread to run spin() on it. That is a good approach because the nested executor could be swapped for the multi-threaded one if performance turns out to be much better - it could be the cpu usage goes away if every publish and subscribe gets a thread of its own.

Queue objects (deques) with strings for topics and Image messages can be used. If a topic is new create a publisher or subscriber for it. An update function within imgui_ros has a map to all the other objects that need to be updated when a new message arrives.

https://answers.ros.org/question/313838/multithreadedexecutor-thread-affinity-for-certain-callbacks/

[lucasw]

Large point clouds or mesh objects may deserve the same treatment, but images first.

[lucasw]

With a single 256x256 I don't see any change in performance of running pub in a separate thread, or same as imgui_ros thread, or not publishing at all.
It looks like the main thread (need to figure out which is which, there are many threads named 'imgui_ros_node').

Bump up the resolution and see if this changes.

At 1024x1024 with no publishing the cpu usage is 19%, 20% with same thread image publication, and different thread.

The cpu usage jumps to 45% when subscribing to the same image- didn't need to separate out publishing, try subscribing.

That cpu usage is already split into different threads, so maybe additional thread won't change that:

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                          
24580 lucasw    20   0 1592756 150104  40296 S 29.7  0.9   0:15.16 imgui_ros_node                   
24593 lucasw    20   0 1592756 150104  40296 S  7.7  0.9   0:02.69 imgui_ros_node                   
24592 lucasw    20   0 1592756 150104  40296 S  4.3  0.9   0:01.49 imgui_ros_node                   
24604 lucasw    20   0 1592756 150104  40296 S  2.3  0.9   0:01.48 imgui_ros_node                   
24589 lucasw    20   0 1592756 150104  40296 S  1.0  0.9   0:00.46 imgui_ros_node                   
24609 lucasw    20   0 1592756 150104  40296 S  0.3  0.9   0:00.15 imgui_ros_node                   
24588 lucasw    20   0 1592756 150104  40296 S  0.0  0.9   0:00.00 imgui_ros_node

[lucasw]

Putting the subscriber in a separate Node & thread doesn't look to improve overall cpu use:

31192 lucasw    20   0 1580376 142640  44152 S 29.0  0.9   0:17.12 imgui_ros_node                   
31205 lucasw    20   0 1580376 142640  44152 S  6.3  0.9   0:03.56 imgui_ros_node                   
31216 lucasw    20   0 1580376 142640  44152 S  4.3  0.9   0:02.50 imgui_ros_node                   
31215 lucasw    20   0 1580376 142640  44152 S  4.0  0.9   0:02.02 imgui_ros_node                   
31212 lucasw    20   0 1580376 142640  44152 S  0.7  0.9   0:00.58 imgui_ros_node                   
31200 lucasw    20   0 1580376 142640  44152 S  0.0  0.9   0:00.00 imgui_ros_node

[lucasw]

Make a ros1 ros2 apples apples comparison in https://github.com/lucasw/image_manip - have a publisher with configurable size and rate and a subscriber that does nothing but measure the rate of incoming messages.

[lucasw]

Don't necessarily need to revert these changes, they don't improve anything, but they could make it easier to swap out ros2 message pub/sub for an alternate manual passing of shared_ptrs around.