strawlab / flydra Goto Github PK
View Code? Open in Web Editor NEWlive, low-latency markerless multi-camera 3D animal tracking system
License: Apache License 2.0
live, low-latency markerless multi-camera 3D animal tracking system
License: Apache License 2.0
This would be beneficial for all flydra/humdra users and seems like it would help get Steve up and running with the various data formats the flydra currently produces. Furthermore, it would support the goal of allowing us (the biologists) to focus on gathering the data and doing the experiments, and quite possibly encourage computer vision and machine learning people, for example, to help us analyze our data. Currently flydra emits realtime output in .h5 files with both the 2D data and rough 3D data and .ufmf movie files, which are then analyzed and cleaned up through a series of further scripts to produce more final data including things like Kalman smoothed 3D position and orientation trajectories. Documenting the intermediate and final file formats (and potentially what the various processing steps do) would be a really useful goal.
As observed in 20150522_191410.mainbrain.h5
, there seems to be a bug in which mainbrain can stop saving a file without flushing all (presumably currently tracked) 3D objects.
NotImplementedError: no mirroring implemented for skewed cameras
MCSC often produces calibration with the cameras pointed outwards, solving this would speed up the calibration process
mainbrain.h5 files causing such inverted calibrations and right calibrations are stored in: strawscience/Max/forAndrew/camera_orientation/
Continuation of #24, to update on the issue of trials in mainbrain warping in time, possible fixes at analysis level for old files and possible fixes when writing.
This code is a mixture of ad-hoc script and reusable module wannabe to check for "continuity" problems and generate reports to assess the extent of the problem in our files and suggest test cases. Some of it parses outputs from flydra's check continuity script. Most important is this function, which we could adapt to check stuff at least in flydra, if not also during the auto-pipeline, combine and others. We also generate a couple of handy dataframes.
Current logs live here:
/mnt/strawscience/santi/mainbrain-holy-logs
There there are some h5 files with data aggregated in pandas dataframes.
I think we have at least two kinds of discontinuities:
20131205_174729
, one of Lisa's DCN experiments, is in /mnt/strawscience/santi/mainbrain-holy-logs/mainbrain-full-logs
. If we need a fix, we could just drop repeated frames when resmoothing (if we agree that won't hurt much the estimates). These are some summary stats:Max number of jumps in an obj_id: 196
Max number of missing frames in an obj_id: 0
Max number of exceeding obs in an obj_id: 413
Seen jump sizes: [-3.0, -2.0, -1.0, 0.0]
I haven't really made sure that there are not other types of errors.
When using image_based_orientation to save movies it looks like the 'morphed' image (row 1, column 4) is the same as the 'abs_diff' image (row 1, column 3).
Also, when using the parameter --final-thresh (at least for the hummingbirds) this doesn't have an affect unless it's cranked up to >100. The default value is 7, which doesn't seem to have much of an effect.
@etiennecampione was unable to calibrate flycube8 using 0.6.8 as there were no points present in the h5 file, despite them being clearly visible in the image view. rolling back to 0.6.7 and not changing any other settings fixed the issue.
I don't think this is related to #30 as commenting out the logging code (both the service proxy on the camnode side, and the service on the mainbrain side) does not fix the bug.
this is in contrast with @maxlklaxl who has made several calibrations with 0.6.8 (although that was on ros-hydro)
When running flydra_analysis versions 0.7.6 through 0.7.9 (from source) or 0.7.10 (from pypi) on python 2.7 all give me the following error when running flydra_kalmanize:
(test_flydra_py27) buzz@tornado:~/src$ flydra_kalmanize -r /home/buzz/DATA/20201123_braid_calibration/data/20201130_132722.braidz.h5.recal/ /home/buzz/DATA/20201123_braid_calibration/data/20201130_132722.braidz.h5
Traceback (most recent call last):
File "/home/buzz/test_flydra_py27/bin/flydra_kalmanize", line 5, in
from flydra_analysis.kalmanize import main
File "/home/buzz/test_flydra_py27/local/lib/python2.7/site-packages/flydra_analysis/kalmanize.py", line 5, in
import flydra_core.reconstruct
File "/home/buzz/test_flydra_py27/local/lib/python2.7/site-packages/flydra_core/reconstruct.py", line 18, in
import flydra_core._pmat_jacobian_water as _pmat_jacobian_water # in pyrex/C for speed
File "flydra_core/_pmat_jacobian_water.pyx", line 1, in init flydra_core._pmat_jacobian_water
File "flydra_core/_refraction.pyx", line 1, in init flydra_core._refraction
ImportError: /home/buzz/test_flydra_py27/local/lib/python2.7/site-packages/flydra_core/_Roots3And4.so: undefined symbol: IsZero
When attempting to run on python 3.6 (using either the pypi version, or the dev branch of flydra), all give me this python 3 related error:
(test_flydra_py36) buzz@tornado:~/src$ flydra_kalmanize -r /home/buzz/DATA/20201123_braid_calibration/data/20201130_132722.braidz.h5.recal/ /home/buzz/DATA/20201123_braid_calibration/data/20201130_132722.braidz.h5
/home/buzz/test_flydra_py36/lib/python3.6/site-packages/flydra_analysis/kalmanize.py:432: UserWarning: dynamic model not specified. using "EKF mamarama, units: mm"
"dynamic model not specified. " 'using "%s"' % dynamic_model_name
/home/buzz/test_flydra_py36/lib/python3.6/site-packages/flydra_core/reconstruct.py:1205: UserWarning: No minimum eccentricity specified, using default
warnings.warn("No minimum eccentricity specified, using default")
/home/buzz/test_flydra_py36/lib/python3.6/site-packages/flydra_analysis/kalmanize.py:473: UserWarning: unable to determine reconstructor source filename for 'normal files'
"filename for %r" % reconstructor.cal_source_type
Traceback (most recent call last):
File "/home/buzz/test_flydra_py36/bin/flydra_kalmanize", line 8, in
sys.exit(main())
File "/home/buzz/test_flydra_py36/lib/python3.6/site-packages/flydra_analysis/kalmanize.py", line 1161, in main
kalmanize(*args, **kwargs)
File "/home/buzz/test_flydra_py36/lib/python3.6/site-packages/flydra_analysis/kalmanize.py", line 478, in kalmanize
frames_per_second = get_fps(results)
File "/home/buzz/test_flydra_py36/lib/python3.6/site-packages/flydra_analysis/analysis/result_utils.py", line 515, in get_fps
parsed = read_textlog_header(results, fail_on_error=fail_on_error)
File "/home/buzz/test_flydra_py36/lib/python3.6/site-packages/flydra_analysis/analysis/result_utils.py", line 438, in read_textlog_header
infostr = textlog1["message"].tostring().strip("\x00")
TypeError: a bytes-like object is required, not 'str'
Is there a way to get the python 2.7 version working?
I created the following launch file:
<launch>
<rosparam command="load" file="$(find freemoovr_arena)/launch/flydra.yaml" />
<node name="AVT-02-2020C-07056" pkg="ros_flydra" type="camnode2-pylon" args="--camera-name=AVT-02-2020C-07056 --http-server-addr=127.0.0.1:4011 --ros-tracking-start-delay=30"> </node>
<node name="AVT-02-2020C-07058" pkg="ros_flydra" type="camnode2-pylon" args="--camera-name=AVT-02-2020C-07058 --http-server-addr=127.0.0.1:4011 --ros-tracking-start-delay=30">
</node>
</launch>
which results in the following error:
flightlab@ubuntu:~$ roslaunch /home/flightlab/ros/flydra-kinetic/src/ros_flydra/launch/prosilica1.launch
... logging to /home/flightlab/.ros/log/f1867084-65f2-11e8-b48b-000c2967883e/roslaunch-ubuntu-95368.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.
Traceback (most recent call last):
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/__init__.py", line 306, in main p.start()
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/parent.py", line 268, in start self._start_infrastructure()
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/parent.py", line 217, in _start_infrastructure self._load_config()
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/parent.py", line 132, in _load_config roslaunch_strs=self.roslaunch_strs, verbose=self.verbose)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/config.py", line 451, in load_config_default loader.load(f, config, verbose=verbose)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 748, in load self._load_launch(launch, ros_config, is_core=core, filename=filename, argv=argv, verbose=verbose)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 720, in _load_launch self._recurse_load(ros_config, launch.childNodes, self.root_context, None, is_core, verbose)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 680, in _recurse_load self._rosparam_tag(tag, context, ros_config, verbose=verbose)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 95, in call return f(*args, **kwds)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 228, in _rosparam_tag cmd, ns, file, param, subst_value = self.opt_attrs(tag, context, (XmlLoader.ROSPARAM_OPT_ATTRS))
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 202, in opt_attrs return [self.resolve_args(tag_value(tag,a), context) for a in attrs]
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/xmlloader.py", line 183, in resolve_args return substitution_args.resolve_args(args, context=context.resolve_dict, resolve_anon=self.resolve_anon)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/substitution_args.py", line 370, in resolve_args resolved = _resolve_args(resolved, context, resolve_anon, commands)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/substitution_args.py", line 383, in _resolve_args resolved = commands[command](resolved, a, args, context)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/substitution_args.py", line 151, in _find source_path_to_packages=source_path_to_packages)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/roslaunch/substitution_args.py", line 197, in _find_executable full_path = _get_executable_path(rp.get_path(args[0]), path)
File "/usr/lib/python2.7/dist-packages/rospkg/rospack.py", line 203, in get_path raise ResourceNotFound(name, ros_paths=self._ros_paths)
ResourceNotFound: freemoovr_arena
ROS path [0]=/opt/ros/kinetic/share/ros
ROS path [1]=/home/flightlab/ros/flydra-kinetic/src
ROS path [2]=/opt/ros/kinetic/share
ufmf documents (the originals) are not viewed as a datanode and thus there is no auto-linking to the child nodes. For example, see this document in the Altshuler lab database.
no reason not to
Hi Andrew,
This isn't a real show-stopper, but I figured you'd like to know:
If one of the 4 cameras is missing when running Data2smoothed with
--time-data, this is the error:
cheers,
paolo
flydra_analysis_data2smoothed
3-12/90_0/DATA20100312_140356.3D_orientation.h5
--min-ori-quality-required=0 --time-data=3-12/DATA20100312_140356.h5
--dest-file=3-12/90_0/DATA20100312_140356.mat
traceback (most recent call last):
File "/usr/bin/flydra_analysis_data2smoothed", line 9, in
load_entry_point('flydra==0.4.69', 'console_scripts','flydra_analysis_data2smoothed')()
File "/usr/lib/pymodules/python2.6/flydra/a2/data2smoothed.py", line 250, in main
*_kwargs)
File "/usr/lib/pymodules/python2.6/flydra/a2/data2smoothed.py", line 125, in convert
mainbrain_timestamp = remote_timestamp_gain[remote_hostname] + offset[remote_hostname] # find mainbrain timestamp
KeyError: 'cam3'
Closing remaining open files:
3-12/90_0/DATA20100312_140356.3D_orientation.h5... done
3-12/DATA20100312_140356.h5... done
flightlab@desktop1:~/Desktop/repeatability2$ flydra_analysis_data2smoothed
3-12/90_0/DATA20100312_140356.3D_orientation.h5
--min-ori-quality-required=0 --time-data=3-12/DATA20100312_140356.h5
--dest-file=3-12/90_0/DATA20100312_140356.mat
There is an error when attempting to save hdf5 files.
Here are the steps:
Flydra launches with 8 cameras fine, except for the following warning:
/home/beelab/ros/flydra-kinetic/src/ros_flydra/nodes/main_brain:54: SyntaxWarning: The publisher should be created with an explicit keyword argument 'queue_size'. Please see http://wiki.ros.org/rospy/Overview/Publishers%20and%20Subscribers for more information.
self.pub = rospy.Publisher("%s/tracking" % cam_id, sensor_msgs.msg.Image, subscriber_listener=self)
Cameras syncs okay:
beelab@beelab:~/ros/flydra-kinetic/src/ros_flydra$ roscd ros_flydra; python scripts/simple-console.py --sync
sync cameras
done
But when trying to start saving hdf5:
beelab@beelab:~/ros/flydra-kinetic/src/ros_flydra$ roscd ros_flydra; python scripts/simple-console.py --start-saving-hdf5
the following error is produced:
[ERROR] [1537378936.478427]: Error processing request: image cannot be None
['Traceback (most recent call last):\n', ' File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 625, in _handle_request\n response = convert_return_to_response(self.handler(request), self.response_class)\n', ' File "/usr/lib/python2.7/dist-packages/flydra_core/MainBrain.py", line 676, in _ros_generic_service_dispatch\n result = getattr(self, calledfunction)(**kwargs)\n', ' File "/usr/lib/python2.7/dist-packages/flydra_core/MainBrain.py", line 1226, in start_saving_data\n raise ValueError(\'image cannot be None\')\n', 'ValueError: image cannot be None\n']
and in the other command line window:
beelab@beelab:~/ros/flydra-kinetic/src/ros_flydra$ roscd ros_flydra; python scripts/simple-console.py --start-saving-hdf5
start saving HDF5
Traceback (most recent call last):
File "scripts/simple-console.py", line 100, in <module>
start_saving_hdf5()
File "scripts/simple-console.py", line 34, in start_saving_hdf5
xx.call()
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 515, in call
responses = transport.receive_once()
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 727, in receive_once
p.read_messages(b, msg_queue, sock)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 353, in read_messages
self._read_ok_byte(b, sock)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 336, in _read_ok_byte
raise ServiceException("service [%s] responded with an error: %s"%(self.resolved_name, str))
rospy.service.ServiceException: service [/flydra_mainbrain/start_saving_data] responded with an error: error processing request: image cannot be None
This error mistifies me:
File "/data/work/scm/snp_env/src/flydra/flydra-sphinx-docs/ext/numpydoc.py", line 137, in get_directive
raise RuntimeError("No directive named '%s' found" % name)
RuntimeError: No directive named 'function' found
complete log from sphinx:
# Sphinx version: 1.0.5
# Python version: 2.6.1
# Docutils version: 0.7 release
# Jinja2 version: 2.5.5
Traceback (most recent call last):
File "/Library/Python/2.6/site-packages/sphinx/cmdline.py", line 172, in main
warningiserror, tags)
File "/Library/Python/2.6/site-packages/sphinx/application.py", line 116, in __init__
self.setup_extension(extension)
File "/Library/Python/2.6/site-packages/sphinx/application.py", line 259, in setup_extension
mod.setup(self)
File "/data/work/scm/snp_env/src/flydra/flydra-sphinx-docs/ext/numpydoc.py", line 120, in setup
app.add_directive('np-' + name, wrap_mangling_directive(name, objtype))
File "/data/work/scm/snp_env/src/flydra/flydra-sphinx-docs/ext/numpydoc.py", line 140, in wrap_mangling_directive
base_directive = get_directive(base_directive_name)
File "/data/work/scm/snp_env/src/flydra/flydra-sphinx-docs/ext/numpydoc.py", line 137, in get_directive
raise RuntimeError("No directive named '%s' found" % name)
RuntimeError: No directive named 'function' found
The h5 file seems to have have 'PST' as the timezone for me, but this is not recognized by pytz. Instead, use 'US/Pacific', which does work? This is more of a bug with pytz ('EST' and 'MST' both work, so 'PST' should), but perhaps we should be robust to that?
The previous analysis steps have --obj-only parameters but montage-ufmfs doesn't. If you try to input a set of frames into montage-ufmfs that have objects that to not have 3-D orientations it doesn't work. To get around this you can analyze the entire h5 file, but this is very time consuming if you only want to get a video of a short sequence.
Currently, to save realtime 3D data to disk, we rely on either A) a call to stop saving data or B) a tracked object being killed. However if the mainbrain crashes hard, this can mean long trajectories that are currently being tracked get completely lost. Therefore, we should implement periodically flushing the currently tracked objects to disk without requiring them to be killed.
For the humdra project, we've been saving out .ufmf ("micro .fmf") videos as standard operating procedure -- while we can do online 3D position tracking, the orientation tracking depends on image analysis that is not implemented for online usage and consequently we need to save image regions around the birds to disk. (Therefore, the hard requirement for realtime analysis in this case is limited to identifying the appropriate 2D image regions that are then saved to disk.)
The present issue is that we've discovered that in some cases, the low level online realtime 2D feature extraction is falling victim to an assumption made for Drosophila that does not hold true for hummingbirds. Specifically, we track only a single 2D feature coming from a given NxN pixel square of image space. With flies, multiple animals only transiently co-localize in a given small patch, and when considering views from multiple angles, this is almost never a problem in practice for estimating 3D positions, even of multiple flies. With hummingbirds, however, we are discovering that this is a real issue (exacerbated by other factors that have pushed us to increase the size of the NxN pixel local region).
The good news is that we do have all relevant information on disk - even though we didn't do the low-level 2D feature extraction correctly the first time around, the .ufmf videos (and the .h5 file with time sync information, amongst other things) have all the required information to completely re-extract this information. What is required is to develop the capability the re-play these videos to the relevant flydra realtime image processing code (with new parameters that minimize or eliminate the hummingbird-related issues). I've made one or two attempts in the past to get this working, but never managed to drive anything to completion. I anticipate that the effort will require two steps. The first is to get the appropriately synchronized data from the .ufmf files saved from multiple cameras available from processing. This is more-or-less already implemented in the flydra.a2.ufmf_tools.iterate_frames()
function. The second step is to feed these data into a suitably modified program that runs the same image analysis as camnode. Because the full-frame images that contribute to making the full background model are also saved to the .ufmf file, but not every full-size frame that arrives, there will be differences from the realtime system in terms of how to deal with updating the background model. I think the most useful way to implement this would be to extend the camnode program to handle this scenario rather than to re-implement a purely-offline low-level 2D tracker program. Nevertheless, the camnode program is quite ugly and I would be sympathetic to the creation of a new program for doing the job. (There are remnants of my attempts to do this still within the camnode program, which may help or hinder the do-it-in-camnode option.)
A) We can use this to add automated testing of the basic fly & bird 2D tracking code. This would go into the automated test suite that gets run nightly (see http://flydra.bb.astraw.com/waterfall ) and would ensure that new errors don't creep unnoticed into the code we all depend on. This would especially be facilitated if we did implement the replay solution within camnode itself.
B) It would allow us to trivially save out the relevant video data in a raw form that wold be suitable for sharing with others (e.g. computer vision people) who might want to develop new tracking algorithms and not depend on the whole creaking mass of crufty software we call flydra.
One further suggestion: a re-write of camnode to A) make it more ROS-centric and B) easier to extend (as suggested above) and maintain, would be to re-write the whole thing. I would suggest using ROS nodelets to handle the sharing of the realtime image acquisition, processing, saving, and streaming across different threads, and this would mean switching to C++ as the main implementation language. That would quite likely also mean a serious reduction in the complexity of the Python->C->Python conversions that happen in order to call the image acquisition and processing libraries mostly providing C interfaces.
This document (in the Altshuler lab database) shows a case where no images were saved despite requesting it. Also, no error messages were saved.
I am trying to run a camnode launch file. The launch looks like so:
<launch>
<node name="Cam1" pkg="camiface_ros" type="camnode">
<param name="device_guid" type="str" value="AVT-02-2020C-06767" />
</node>
<node name="Cam2" pkg="camiface_ros" type="camnode">
<param name="device_guid" type="str" value="AVT-02-2020C-07056" />
</node>
</launch>
I also tried using underscores.
And this is what the log looks like:
^[[0m[ INFO] [1531769131.179236474]: host timestamps OFF^[[0m
^[[33m[ WARN] [1531769132.255169176]: Selected camera AVT-02-2020C-07056 (-1) not found^[[0m
When running flydra in self-discovery mode, the cameras are found and detected as:
beelab@beelab:~$ rosrun ros_flydra camnode
[INFO] [1531768934.973439]: ROS name: /flydra_camnode
[INFO] [1531768935.003848]: choosing camera guid: Allied Vision Technologies-02-2020C-07056 (cam_id: 0)
[INFO] [1531768935.004234]: choosing camera guid: Allied Vision Technologies-02-2020C-07058 (cam_id: 1)
[INFO] [1531768935.004501]: choosing camera guid: Prosilica-02-2020C-06767 (cam_id: 2)
This Flydra instance is a fresh installation on July 16th.
Thank you!
tired it 2 times on the fishtrax machine, both times the same error
the .h5 file is 13G
(PY)strawlab@fishtrax:~/FLYDRA$ flydra_analysis_export_flydra_hdf5 20140719_152830.mainbrain.h5
STAGE 1: finding timestamps
opening file 20140719_152830.mainbrain.h5...
caching raw 2D data... done
(cached index of 145054462 frame values of dtype int64)
hostname time_gain time_offset
'localhost' 1.0 -0.00697165842925
caching Kalman obj_ids...
finding unique obj_ids...
(found 57384)
(will export 57384)
finding 2d data for each obj_id...
skipping frame 403 (obj 1): no data2d_distorted data
skipping frame 404 (obj 2): no data2d_distorted data
skipping frame 405 (obj 3): no data2d_distorted data
skipping frame 4439 (obj 47): no data2d_distorted data
skipping frame 4621 (obj 52): no data2d_distorted data
/home/strawlab/src/flydra/flydra/a2/data2smoothed.py:168: UserWarning: no host fishtrax in timestamp data. making up data.
'data.'%remote_hostname)
STAGE 2: running Kalman smoothing operation
detected file loaded with dynamic model "EKF fishbowl40"
for smoothing, will use dynamic model "fishbowl40"
/home/strawlab/src/flydra/flydra/a2/core_analysis.py:1467: UserWarning: passing data_file as string to core_analysis.CachingAnalyzer.load_data()
warnings.warn('passing data_file as string to '
/home/strawlab/src/flydra/flydra/a2/core_analysis.py:555: RuntimeWarning: invalid value encountered in true_divide
velocity_direction = velocity/speed[:,np.newaxis]
/home/strawlab/src/flydra/flydra/a2/data2smoothed.py:243: UserWarning: algebra error smoothing obj_id 17414, skipping.
warnings.warn('algebra error smoothing obj_id %d, skipping.'%(obj_id,))
/home/strawlab/src/flydra/flydra/a2/data2smoothed.py:243: UserWarning: algebra error smoothing obj_id 47242, skipping.
warnings.warn('algebra error smoothing obj_id %d, skipping.'%(obj_id,))
[obj_id: 57432] 2.07 obj_ids /s100%|##########################################################|Time: 7:41:40
Traceback (most recent call last):
File "/home/strawlab/PY/bin/flydra_analysis_export_flydra_hdf5", line 9, in
load_entry_point('flydra==0.5.9', 'console_scripts', 'flydra_analysis_export_flydra_hdf5')()
File "/home/strawlab/src/flydra/flydra/a2/data2smoothed.py", line 280, in export_flydra_hdf5
main(hdf5_only=True)
File "/home/strawlab/src/flydra/flydra/a2/data2smoothed.py", line 377, in main
**kwargs)
File "/home/strawlab/src/flydra/flydra/a2/data2smoothed.py", line 262, in convert
recarray = numpy.rec.array(allrows)
File "/usr/lib/python2.7/dist-packages/numpy/core/records.py", line 790, in array
new = new.copy()
MemoryError
Closing remaining open files: /home/strawlab/FLYDRA/20140719_152830.mainbrain.kh5-smoothcache... done 20140719_152830.mainbrain.h5... done
There is a warning 'roiradius hard-coded to %d: could be set from 3D tracking' which is exactly true. This looks like an extra 2 or 3 lines of code could really help here.
I think there may be bugs with the scale and the units of the various dynamic models and calibrations. We should really write some unit tests to find them and eliminate them.
To enable use with ROS systems, the flydra_camera_nodes should publish their images to the ROS network.
When montage-ufmf finishes running it exits with this error message:
Closing remaining open files: /home/flightlab/Desktop/1/1/DATA20100218_142912.3D_orientation.kh5-smoothcache... done 1/DATA20100218_142912.3D_orientation.h5... done
Exception AttributeError: AttributeError("'NoneType' object has no attribute 'ref'",) in <bound method CachingAnalyzer.del of <flydra.a2.core_analysis.CachingAnalyzer instance at 0x4adcb48>> ignored
Exception TypeError: 'must be type, not None' in ignored
But the video turns out fine so this is not really an urgent issue.
Starting main_brain.launch works, camnode runs and detects four cameras, but camera synchronization fails in the following way:
roscd ros_flydra; python scripts/simple-console.py --sync
sync cameras
Traceback (most recent call last):
File "scripts/simple-console.py", line 86, in <module>
sync_cameras()
File "scripts/simple-console.py", line 16, in sync_cameras
xx.call()
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 495, in call
service_uri = self._get_service_uri(request)
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 463, in _get_service_uri
raise ServiceException("service [%s] unavailable"%self.resolved_name)
rospy.service.ServiceException: service [/flydra_mainbrain/do_synchronization] unavailable
The flydra camera node class should support the normal ROS camera info stuff. See this http://wiki.ros.org/camera_info_manager_py
@etiennecampione writes: "Hereafter is the error message I almost always get (let's say 95% of the time) while calibrating a Flycube. Can you indicate me what is going on? It's a really annoying and time-consuming problem".
[strawlab@flycube8 flycube8](master=)$ roslaunch ./flycube_only_flydra_calib.launch --screen
... logging to /home/strawlab/.ros/log/7433d1b0-cf1a-11e5-b183-bcee7bdac428/roslaunch-flycube8-5300.log
[...]
Exception in thread LogMessageSender:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/dist-packages/flydra/camnode.py", line 1041, in run
self._main_brain.log_message(*log_message)
File "/usr/lib/python2.7/dist-packages/flydra/camnode.py", line 197, in log_message
self._log_message(req)
File "/opt/ros/ros.electric.boost1.46/ros_comm/clients/rospy/src/rospy/impl/tcpros_service.py", line 430, in __call__
return self.call(*args, **kwds)
File "/opt/ros/ros.electric.boost1.46/ros_comm/clients/rospy/src/rospy/impl/tcpros_service.py", line 506, in call
transport.send_message(request, self.seq)
File "/opt/ros/ros.electric.boost1.46/ros_comm/clients/rospy/src/rospy/impl/tcpros_base.py", line 590, in send_message
self.write_buff.truncate(0)
AttributeError: 'NoneType' object has no attribute 'truncate'
[INFO] [WallTime: 1455014842.446372] setting camera property expected_trigger_framerate=100.0
[FATAL] [WallTime: 1455014842.446997] ERROR: thread LogMessageSender died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.497085] ERROR: thread save_cam.mainloop died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.537913] ERROR: thread save_small.mainloop died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.578685] ERROR: thread cam_processor.mainloop died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.619367] ERROR: thread save_cam.mainloop died unexpectedly. Quitting
[INFO] [WallTime: 1455014842.628120] triggerbox_client: waiting for clockmodel estimate
[FATAL] [WallTime: 1455014842.629951] ERROR: thread save_small.mainloop died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.630143] ERROR: thread cam_processor.mainloop died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.630313] ERROR: thread LogMessageSender died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.630480] ERROR: thread save_cam.mainloop died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.630644] ERROR: thread save_small.mainloop died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.630809] ERROR: thread cam_processor.mainloop died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.631007] ERROR: thread LogMessageSender died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.631231] ERROR: thread save_cam.mainloop died unexpectedly. Quitting
[FATAL] [WallTime: 1455014842.631436] ERROR: thread save_small.mainloop died unexpectedly. Quitting
[INFO] [WallTime: 1455014843.128506] triggerbox_client: waiting for clockmodel estimate
[flydra_camera_node-4] process has died [pid 5341, exit code 1].
log files: /home/strawlab/.ros/log/7433d1b0-cf1a-11e5-b183-bcee7bdac428/flydra_camera_node-4*.log
How does the transport die? Is it possible that the logging service (and therefore the transport) is closed in one camera LogMessageThread thread while there are still messages to log in some other camera thread, and so we would need to add sync code? Or could it just be a network problem? Surely @astraw will know better about this.
I'm saving the full logs of that run here; they include a few other stacktraces.
/mnt/strawscience/santi/strawlab/calibration-transport-not-open-logs
I just hit this:
Exception in thread CoordinateProcessor:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/dist-packages/flydra/main_brain/coordinate_receiver.py", line 418, in run
self.process_data( incoming_2d_data )
File "/usr/lib/python2.7/dist-packages/flydra/main_brain/coordinate_receiver.py", line 555, in process_data
deferred_2d_data,
File "/usr/lib/python2.7/dist-packages/flydra/main_brain/coordinate_receiver.py", line 706, in _process_parsed_data
debug2=self.debug_level.isSet())
File "/usr/lib/python2.7/dist-packages/flydra/kalman/flydra_tracker.py", line 117, in calculate_a_posteriori_estimates
debug1=debug2) for tro in self.live_tracked_objects ]
File "_flydra_tracked_object.pyx", line 365, in _flydra_tracked_object.TrackedObject.calculate_a_posteriori_estimate (src/_flydra_tracked_object.c:5965)
File "_flydra_tracked_object.pyx", line 472, in _flydra_tracked_object.TrackedObject._filter_data (src/_flydra_tracked_object.c:10111)
File "_flydra_tracked_object.pyx", line 515, in _flydra_tracked_object.TrackedObject._filter_data (src/_flydra_tracked_object.c:8011)
File "/usr/lib/python2.7/dist-packages/flydra/reconstruct.py", line 1646, in find2d
distorted=distorted )
File "/usr/lib/python2.7/dist-packages/flydra/water.py", line 48, in view_points_in_water
height, r[i], depth[i]))
File "_refraction.pyx", line 41, in _refraction.find_fastest_path_fermat (src/_refraction.c:746)
File "_refraction.pyx", line 63, in _refraction.find_fastest_path_fermat (src/_refraction.c:618)
File "_Roots3And4.pyx", line 24, in _Roots3And4.real_root_less_than (src/_Roots3And4.c:651)
ValueError: more than one valid root found
This might be a issue because I am using ros hydro, changing:
nodename = nodename[1:].split(roslib.names.SEP)[-1]
to:
nodename = nodename[1:].split(rospy.names.SEP)[-1]
Seems to fix the problem.
The following command only plots the first episode:
flydra_analysis_plot_summary --stim-xml=fanout.xml --interactive -k DATA20101130_233000.kalmanized.h5
(This fanout.xml file has two <single_episode> elements for this .h5 file.) Of course, plotting without the fanout.xml file shows all the data.
Hi flydra users,
I am facing a problem after running the "roslaunch ros_flydra main_brain.launch" and then "rosrun ros_flydra camnode". In the main_brain terminal it shows a warning "[WARN] [1594270086.445616]: timestamps off by more than 5 msec -- synchronization error". Is that just a warning or creating any issues?
There is a race when multiple camnodes try to register at once and the port increment counter in mainbrain gets confused.
trace from camnode:
Traceback (most recent call last):
File "/home/strawlab/ros-flydra.electric.boost1.46/bin/flydra_camera_node", line 9, in
load_entry_point('flydra==0.4.69-git', 'console_scripts', 'flydra_camera_node')()
File "/home/strawlab/flydra.git/flydra/camnode.py", line 2870, in main
parse_args_and_run()
File "/home/strawlab/flydra.git/flydra/camnode.py", line 2953, in parse_args_and_run
benchmark=benchmark,
File "/home/strawlab/flydra.git/flydra/camnode.py", line 2315, in init
camnode_ros_name = rospy.get_name())
File "/usr/lib/pymodules/python2.7/Pyro/core.py", line 381, in call
return self.__send(self.__name, args, kwargs)
File "/usr/lib/pymodules/python2.7/Pyro/core.py", line 456, in _invokePYRO
return self.adapter.remoteInvocation(name, Pyro.constants.RIF_VarargsAndKeywords, vargs, kargs)
File "/usr/lib/pymodules/python2.7/Pyro/protocol.py", line 457, in remoteInvocation
return self._remoteInvocation(method, flags, *args)
File "/usr/lib/pymodules/python2.7/Pyro/protocol.py", line 532, in _remoteInvocation
answer.raiseEx()
File "/usr/lib/pymodules/python2.7/Pyro/errors.py", line 72, in raiseEx
raise self.excObj
socket.error: [Errno 98] Address already in use
trace from mainbrain
REGISTER NEW CAMERA Basler_21029386 on noise.flycave @ ros node /flydra_camera_node_noise
Exception in thread CoordinateProcessor thread:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
self.run()
File "/home/strawlab/flydra.git/flydra/MainBrain.py", line 884, in run
predicted_framenumber = n_frames_skipped + self.last_framenumbers_skip[cam_idx] + 1
IndexError: list index out of range
Hi,
@etiennecampione is having some troubles when trying to generate the reprojection error plots.
Traceback (most recent call last):
File "/home/auto-data-webserver/.virtualenvs/flycave_webserver/bin/plot-historical-reproj", line 5, in <module>
pkg_resources.run_script('ros-electric-flycave==3.3', 'plot-historical-reproj')
File "/home/auto-data-webserver/.virtualenvs/flycave_webserver/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 499, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/home/auto-data-webserver/.virtualenvs/flycave_webserver/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 1235, in run_script
execfile(script_filename, namespace, namespace)
File "/home/auto-data-webserver/.virtualenvs/flycave_webserver/lib/python2.7/site-packages/ros_electric_flycave-3.3-py2.7.egg/EGG-INFO/scripts/plot-historical-reproj", line 190, in <module>
data,limit = get_data(args.assay,start_date,args.uuid,args.limit)
File "/home/auto-data-webserver/.virtualenvs/flycave_webserver/lib/python2.7/site-packages/ros_electric_flycave-3.3-py2.7.egg/EGG-INFO/scripts/plot-historical-reproj", line 130, in get_data
show_progress=False).get_reprojection_distance()
File "/home/auto-data-webserver/.virtualenvs/flycave_webserver/local/lib/python2.7/site-packages/ros_electric_flycave-3.3-py2.7.egg/autodata/files.py", line 294, in get_reprojection_distance
for i,s in store['cameras'].iterrows():
File "/usr/lib/python2.7/dist-packages/pandas/io/pytables.py", line 416, in __getitem__
return self.get(key)
File "/usr/lib/python2.7/dist-packages/pandas/io/pytables.py", line 619, in get
raise KeyError('No object named %s in the file' % key)
KeyError: 'No object named cameras in the file'
Closing remaining open files:/mnt/strawscience/data/auto_pipeline/raw_archive/by_uuid/6c8accbc347311e58634d850e6c4a608/20150727_172257.repro_errors.h5...done
The cameras dataset is not there anymore (I guess we could just fake it, if necessary). It seems to be happening in all recent experiments. Are we missing something?
The docs at https://code.astraw.com/flydra-doc/calibration.html are out of date: the calibration code at https://github.com/strawlab/MultiCamSelfCal can now run just fine using Octave. Also, the configuration is much simplified and can reside in simple files alongside the calibration data.
It would be good to have a way to check 2D orientations in pseudo real time. By pseudo I mean that while the program is running you can run an analysis script and generate a graph of the 2D position tracking of what has happened up to that point in time. If you want to take this a step further, it would be great to try to implement a script that has a graphical output of what is going on in actual real time. This is a humdra-specific project to the extent that 2D orientation is estimated differently for humdra (sampling rate 5x that of wingbeat rate) versus flydra (sampling rate 0.5x that of wingbeat rate), but a majority of the legwork should be the shared.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.