I use ROS a lot in both my teaching and research to control various robots. ROS Bag files are wonderful, they provide an easy way to record information from an experiment. I’ve certainly run into issues where recording a bag file would interfere with the system I am trying to collect data from, for instance when trying to record a stream of images from a camera on a machine with a SD card drive (or through a virtual machine). I’ve even run into some ros packages (not written by me!) that would crash when recording some of their topics. In general, however, I think it is a great tool that makes it easy to record and play back data from an experiment.
The problem with bag files is getting data out to use in some other environment. I often want to get CSV (comma separated value files) out that include some subset of the topics recorded in the bag file. I then import these into other programs (e.g. matlab) to plot or analyze the data. If there is just one topic you are interested in, you can use the -p flag of rostopic echo to at least print out that single value in a nice way. But as far as I know, there isn’t a good built-in way to print out values from a bunch of different topics in a nice, machine (or matlab) readable format. There is, of course, a good reason for this. If you want to plot data from 3 different topics, they could all have very different rates. One may be coming at 5Hz and the other two at 30Hz. Which rate do you use? And even for the two at the same rate, there could be jitter that causes messages A and B to arrive in the order ABBAABB… How do you determine when to print one “line” of data? So I completely understand why there isn’t an easy way to do this since getting the data out will be very dependent on the types and frequency of the data. But most of the time, just printing a line with all the most recently received data every time A is received makes sense.
I have used a couple of different approaches to get data into usable formats from ROS bags. First, I have written ROS nodes that subscribe to the topics I’m interested in and then print them to a file at a given frequency. This works well, but in C++ (and even Python) the code to do this gets rather lengthy. I also have used the C++ API to read through the bag files, but this is also relatively inflexible. Recently, I decided to use the Python bag file API and while it is faster to write the Python code, it still results in a lot of repeated code.
Instead, I decided to write a python script that takes as command line arguments the topics and fields that you wanted printed in CSV format. The idea is that whenever a message from the first topic is received, all of the fields will be printed in CSV format. Here is the script:
#!/usr/bin/env python
import roslib; roslib.load_manifest('rosbag')
import rosbag
import sys
import os
if len(sys.argv) < 3:
print 'Please specify the bag file to parse followed by topics'
print 'For example:'
print ' ', sys.argv[0], 'test.bag', '/wirelessPowerRx/amps.data','/wirelessPowerRx/volts5.header.stamp'
print
print """
Note that these fields are output together on a single line each
time the first field is received.
"""
exit(1)
#First arg is the bag to look at
bagfile = sys.argv[1]
#The topic names
topics = []
#The fields of each topic to print out, indexed by topic name
topicFields = {}
for i in range(2,len(sys.argv)):
#Split the input into the topic (left of the first .) and field to look at (right of first .)
splitVals = sys.argv[i].split('.',1)
topics.append(splitVals[0])
topicFields[splitVals[0]] = splitVals[1]
print splitVals
lastValue = {}
#Print the values found in latValue in a comma separated format
def printValuesCSV():
print(lastValue[topics[0]]),
sys.stdout.softspace=False
for i in range(1,len(topics)):
sys.stdout.write(', ')
#sys.stdout.write('%.2f'%(lastValue[topics[i]]))
print(lastValue[topics[i]]),
sys.stdout.softspace=False
print
print 'Working directory: ' + os.getcwd()
print 'Bag file: ' + bagfile
print 'Outputting the following topics and fields: '
print topicFields
#Init the lastValue dictionary to zeros
for topic in topics:
lastValue[topic] = 0.0
#Go through the bag file
bag = rosbag.Bag(bagfile)
for topic, msg, t in bag.read_messages(topics=topics):
lastValue[topic] = eval('msg.' + topicFields[topic])
#Every time we see an instance of the first element, print out everything
if topic == topics[0]:
printValuesCSV()
bag.close();
You can use it by running:
parsebag.pl bagfile.bag /topic/name.data /topic2/name.header.stamp
The first argument is the bag file to look at, followed by the topics. The first topic is the topic that will be used to print out the CSV line. Each time that topic is received, a line will be printed. Each topic has the format:
/topic/name.data
Where “data” is the field in the message that you want to print. You can also print sub-fields in the message, for instance:
/topic2/name.header.stamp
which will print the time stamp in the header field. There are certainly a few other scripts out there like this, but when I was looking for something to do this, all of the others were rather complicated. This hasn’t been tested too extensively, so let me know in the comments if you find any problems.
