10 posts tagged with "ros"

ROS2 is widely used for building robotic systems with sensors like cameras, LiDAR, and IMUs. While it's great for communication (e.g., publishing and subscribing to topics), it lacks a built-in solution for storing large amounts of unstructured data, such as images.

Bag files are commonly used to store data in ROS2, but they aren't a good fit for long-term storage or real-time streaming. They're mainly meant for recording and replaying mission data or episodes, not for managing large volumes of unstructured data.

Addressing this challenge, this blog post will guide you through setting up ROS2 with ReductStore a high-performance storage and streaming solution optimized for unstructured, time-series data.

We will focus specifically on image data, but if you are interested in a more general overview you can read How to Store and Manage Robotic Data which covers the challenges and strategies for storing and managing robotic data in general.

For the full code example, we will be using the reduct-ros-example repository, which provides a complete implementation of the concepts discussed in this article.

A rosbag recording is ultimately written to files, and a single file cannot grow forever. Something has to close it at some point, so rosbag2 makes you pick: ros2 bag record -a -b 100000 closes one every 100 kilobytes, or -d 9000 closes one every 9000 seconds. Either way, a recording that runs for hours comes out as a directory of many small files instead of one huge file that becomes awkward to work with.

Splitting a recording into five minute bags seems like a simple solution. The problem starts when you need to turn them back into one file.

A rosbag2 user reported that merging 300 GB of split recordings required roughly 600 GB of disk capacity. The original 300 GB stays on disk while another 300 GB is written out as the merged bag.

Splitting solves one problem. Merging creates another, and it is easy to underestimate.

Robotics teams today wrestle with data that grows faster than their infrastructure. Every robot generates streams of images, sensor readings, logs, and events in different formats. These data piles are fragmented, expensive to move, and slow to analyze. Teams often rely on generic cloud tools that are not built for robotics. They charge way too much per gigabyte (when it should cost little per terabyte), hide the raw data behind proprietary APIs, and make it hard for robots (and developers) to access or use their own data.

ReductStore introduces a new category: a database purpose built for robotics data pipelines. It is open, efficient, and developer friendly. It lets teams store, query, and manage any time series of unstructured data directly from robots to the cloud.