ROS2 Bag to CSV Converter

A Python tool to convert ROS2 bag files to CSV format, with support for various message types including PointCloud2 data. This tool can process single bags or batch process multiple bags in a directory.

Features

• Multiple Storage Formats: Supports both MCAP and SQLite3 bag formats
• Automatic Format Detection: Automatically detects the storage format based on file extensions
• PointCloud2 Support: Special handling for PointCloud2 messages with proper field parsing
• Batch Processing: Process multiple bag files in a directory at once
• Flexible Output: Flattens nested message structures into CSV columns
• Array Handling: Intelligent handling of arrays and nested structures
• Topic Information: View detailed information about topics and message counts

Requirements

• Python 3.6+
• ROS2 (tested with ROS2 Jazzy)
• Required Python packages (see requirements.txt)

Installation

1. Clone the repository:

   git clone https://github.com/GTEC-UDC/ros2_bag_to_csv.git
   cd ros2_bag_to_csv

2. Install dependencies:

   pip install -r requirements.txt

3. Make sure ROS2 is installed and sourced:

   source /opt/ros/jazzy/setup.bash  # or your ROS2 distribution

Usage

Basic Usage

Show Bag Information

python ros2_bag_to_csv.py my_bag_folder --info

Convert Specific Topics

python ros2_bag_to_csv.py my_bag_folder --csv /sensor/imu /sensor/camera

Batch Processing

Process multiple bag files in a directory:

python ros2_bag_to_csv.py --batch /path/to/bags/directory --csv /sensor/imu /sensor/camera

Examples

1. Get information about a bag:

   python ros2_bag_to_csv.py rosbag2_2024_01_15-10_30_45 --info

2. Convert IMU and camera topics:

   python ros2_bag_to_csv.py rosbag2_2024_01_15-10_30_45 --csv /sensor/imu /sensor/camera/image_raw

3. Batch process multiple bags:

   python ros2_bag_to_csv.py --batch ./experiment_data/ --csv /sensor/imu /sensor/lidar

Output Format

CSV Structure

• Timestamp: ROS2 timestamp (nanoseconds)
• Flattened Fields: Message fields are flattened using dot notation (e.g., header.stamp.sec)
• Array Handling: Arrays are either:
  • Indexed (e.g., position[0], position[1], position[2])
  • Named (for PointField arrays using field names)
  • Summarized (for large data arrays)

Example CSV Output

For an IMU message:

timestamp,header.stamp.sec,header.stamp.nanosec,header.frame_id,angular_velocity.x,angular_velocity.y,angular_velocity.z,linear_acceleration.x,linear_acceleration.y,linear_acceleration.z
1642248645123456789,1642248645,123456789,imu_link,0.01,-0.02,0.05,9.81,0.1,-0.05

Special Features

PointCloud2 Support

The tool provides special handling for PointCloud2 messages:

• Parses binary point data using field definitions
• Creates one row per point in the CSV
• Includes header information and point cloud metadata
• Handles different data types (FLOAT32, UINT8, etc.)

Array Processing

• Simple Arrays: Converted to indexed columns
• Complex Arrays: May create multiple CSV rows
• Large Arrays: Truncated with summary information
• PointField Arrays: Use field names instead of indices

File Organization

• Single bag: Creates a folder named after the bag directory
• Batch mode: Creates a folder structure preserving bag names
• Topic names converted to valid filenames (e.g., /sensor/imu → sensor-imu.csv)

Supported Message Types

The tool supports any ROS2 message type, with special optimizations for:

• sensor_msgs/PointCloud2
• sensor_msgs/Imu
• geometry_msgs/*
• std_msgs/*
• Custom message types

Error Handling

• Graceful handling of missing topics
• Automatic storage format detection
• Informative error messages
• Continues processing other bags if one fails (in batch mode)

Command Line Options

usage: ros2_bag_to_csv.py [-h] [--batch BATCH_DIR] (--info | --csv TOPIC [TOPIC ...]) [bag_directory]

Convert ROS2 bag files to CSV format

positional arguments:
  bag_directory         Path to the ROS2 bag directory

optional arguments:
  -h, --help            show this help message and exit
  --batch BATCH_DIR     Batch mode: process all bags in the specified directory
  --info                Show information about topics in the bag
  --csv TOPIC [TOPIC ...]
                        Convert specified topics to CSV files

Technical Details

Storage Format Detection

The tool automatically detects the storage format by checking for:

1. .mcap files (MCAP format)
2. .db3 files (SQLite3 format)
3. Defaults to MCAP for newer ROS2 distributions

Message Processing Pipeline

1. Deserialization: Convert binary data to ROS2 message objects
2. Dictionary Conversion: Transform to ordered dictionary structure
3. Flattening: Convert nested structures to flat key-value pairs
4. Array Processing: Handle arrays based on content type
5. CSV Writing: Output to CSV with proper field ordering

License

MIT License

Copyright (c) 2025 Group of Electronic Technology and Communications. University of A Coruna.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE