utexas-bwi / bwi_lab Goto Github PK

for background, see: utexas-bwi/bwi_common#41

The basic approach envisioned is to create a general facility for running selected scripts on each robot under the bwilab account. The upload script will be one of them.

We will create unique public and private ssh keys for the local bwilab account on each robot, storing their public keys on nixons-head. I will write a setuid("bwilab") C program that will only run a set of scripts we maintain.

That will make it easy to upload logs without requiring a login, and also to notify the server when a robot becomes active or inactive.

It currently uses an explicit list to distinguish Python scripts from bash scripts. If we put them in separate source directories, they could be installed using wild cards.

The default bag prefix is bwi_. Those bags will be uploaded to the server automatically.

But we want to be able to save additional topics to different bag files that can be uploaded manually using the same tools.

see: utexas-bwi/bwi_common#42

I think we should consider always requiring the -p parameter. Otherwise people will often forget and end up using bwi by default. When running the scripts by hand, that's frequently not what is wanted.

If we do this, we could also make a similar change to the bwi_logging record node. I'm not sure that's really necessary, because the launch file always passes that parameter explicitly.

@pato: comments? suggestions?

Want to be able to keep track of when a robot shuts down so that there is no stale information on the DNS server.

Blocked by #5

Traceback:

  time: 0:00:00  distance: 0.00  filename: bwi_2016-12-01-16-51-43.bag
Traceback (most recent call last):
  File "/usr/local/lib/bwi_local/distance_traveled", line 26, in <module>
    t, d, f = distance_traveled(filename)
  File "/usr/local/lib/python2.7/dist-packages/bwi_local/bagfile.py", line 71, in distance_traveled
    for topic, msg, t in bag.read_messages(topics=['odom_1hz']):
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbag/bag.py", line 2331, in read_messages
    yield self.seek_and_read_message_data_record((entry.chunk_pos, entry.offset), raw)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbag/bag.py", line 2436, in seek_and_read_message_data_record
    self.decompressed_chunk = bz2.decompress(compressed_chunk)
IOError: invalid data stream

It looks like the file only had one odom_1hz message. The script needs at least two.

Empty bags also throw an exception:

Traceback (most recent call last):
  File "/usr/local/lib/bwi_local/distance_traveled", line 26, in <module>
    t, d, f = distance_traveled(filename)
  File "/usr/local/lib/python2.7/dist-packages/bwi_local/bagfile.py", line 61, in distance_traveled
    bag = rosbag.Bag(filename)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbag/bag.py", line 173, in __init__
    self._open(f, mode, allow_unindexed)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbag/bag.py", line 1096, in _open
    if   mode == 'r': self._open_read(f, allow_unindexed)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbag/bag.py", line 1124, in _open_read
    self._version = self._read_version()
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbag/bag.py", line 1234, in _read_version
    raise ROSBagException('empty file')
rosbag.bag.ROSBagException: empty file

Either transfer the existing smallDNS that I wrote,
or build a new file-based DNS system where robots log into master server and in a specified folder update a file (named based on the hostname) with the IP address

I was trying to upload some bags from pickles that had been collected by @shiqizhang6. I was logged in as joq, and the ~szhang/.ros/bwi/bwi_logging/compressed subdirectory had no group write permissions (although the parent directory did).

Sadly, the script not only failed, but it "successfully" deleted the bags afterwards.

I had some trouble Wednesday with some of the uploads. If the compression succeeds but the rsync fails, the log files are left in a state that the upload script no longer handles correctly. It just says there is nothing to upload.

Since the rsync is really the only part requiring bwilab authorization, I'd like to divide the work into separate compression and rsync scripts, perhaps with another convenience script that runs one and then the other. That will make it easier to clean up after a previous incomplete upload. It also significantly reduces the amount of processing done under bwilab authorization.

@pato: comments? (I intend to merge your updates before making this change.)