Coder Social home page Coder Social logo

pyulog's Introduction

pyulog

This repository contains a python package to parse ULog files and scripts to convert and display them. ULog is a self-describing logging format which is documented here.

The provided command line scripts are:

  • ulog_info: display information from an ULog file.
  • ulog_messages: display logged messages from an ULog file.
  • ulog_params: extract parameters from an ULog file.
  • ulog2csv: convert ULog to CSV files.
  • ulog2kml: convert ULog to KML files.

Installation

Installation with package manager:

pip install pyulog

Installation from source:

python setup.py build install

Development

To install the code in a format so that it can be easily edited use the following command (this will install the package as a link to the repo):

pip install -e .

Testing

pytest test

or

python setup.py test

Code Checking

pylint pyulog/*.py

Command Line Scripts

All scripts are installed as system-wide applications (i.e. they be called on the command line without specifying Python or a system path), and support the -h flag for getting usage instructions.

The sections below show the usage syntax and sample output (from test/sample.ulg):

Display information from an ULog file (ulog_info)

Usage:

usage: ulog_info [-h] [-v] file.ulg

Display information from an ULog file

positional arguments:
  file.ulg       ULog input file

optional arguments:
  -h, --help     show this help message and exit
  -v, --verbose  Verbose output

Example output:

$ ulog_info sample.ulg
Logging start time: 0:01:52, duration: 0:01:08
Dropouts: count: 4, total duration: 0.1 s, max: 62 ms, mean: 29 ms
Info Messages:
 sys_name: PX4
 time_ref_utc: 0
 ver_hw: AUAV_X21
 ver_sw: fd483321a5cf50ead91164356d15aa474643aa73

Name (multi id, message size in bytes)    number of data points, total bytes
 actuator_controls_0 (0, 48)                 3269     156912
 actuator_outputs (0, 76)                    1311      99636
 commander_state (0, 9)                       678       6102
 control_state (0, 122)                      3268     398696
 cpuload (0, 16)                               69       1104
 ekf2_innovations (0, 140)                   3271     457940
 estimator_status (0, 309)                   1311     405099
 sensor_combined (0, 72)                    17070    1229040
 sensor_preflight (0, 16)                   17072     273152
 telemetry_status (0, 36)                      70       2520
 vehicle_attitude (0, 36)                    6461     232596
 vehicle_attitude_setpoint (0, 55)           3272     179960
 vehicle_local_position (0, 123)              678      83394
 vehicle_rates_setpoint (0, 24)              6448     154752
 vehicle_status (0, 45)                       294      13230

Display logged messages from an ULog file (ulog_messages)

Usage:

usage: ulog_messages [-h] file.ulg

Display logged messages from an ULog file

positional arguments:
  file.ulg    ULog input file

optional arguments:
  -h, --help  show this help message and exit

Example output:

ubuntu@ubuntu:~/github/pyulog/test$ ulog_messages sample.ulg
0:02:38 ERROR: [sensors] no barometer found on /dev/baro0 (2)
0:02:42 ERROR: [sensors] no barometer found on /dev/baro0 (2)
0:02:51 ERROR: [sensors] no barometer found on /dev/baro0 (2)
0:02:56 ERROR: [sensors] no barometer found on /dev/baro0 (2)

Extract parameters from an ULog file (ulog_params)

Usage:

usage: ulog_params [-h] [-d DELIMITER] [-i] [-o] file.ulg [params.txt]

Extract parameters from an ULog file

positional arguments:
  file.ulg              ULog input file
  params.txt            Output filename (default=stdout)

optional arguments:
  -h, --help            show this help message and exit
  -d DELIMITER, --delimiter DELIMITER
                        Use delimiter in CSV (default is ',')
  -i, --initial         Only extract initial parameters
  -o, --octave          Use Octave format

Example output (to console):

ubuntu@ubuntu:~/github/pyulog/test$ ulog_params sample.ulg
ATT_ACC_COMP,1
ATT_BIAS_MAX,0.0500000007451
ATT_EXT_HDG_M,0
...
VT_OPT_RECOV_EN,0
VT_TYPE,0
VT_WV_LND_EN,0
VT_WV_LTR_EN,0
VT_WV_YAWR_SCL,0.15000000596

Convert ULog to CSV files (ulog2csv)

Usage:

usage: ulog2csv [-h] [-m MESSAGES] [-d DELIMITER] [-o DIR] file.ulg

Convert ULog to CSV

positional arguments:
  file.ulg              ULog input file

optional arguments:
  -h, --help            show this help message and exit
  -m MESSAGES, --messages MESSAGES
                        Only consider given messages. Must be a comma-
                        separated list of names, like
                        'sensor_combined,vehicle_gps_position'
  -d DELIMITER, --delimiter DELIMITER
                        Use delimiter in CSV (default is ',')
  -o DIR, --output DIR  Output directory (default is same as input file)

Convert ULog to KML files (ulog2kml)

Note The simplekml module must be installed on your computer. If not already present, you can install it with:

pip install simplekml

Usage:

usage: ulog2kml [-h] [-o OUTPUT_FILENAME] [--topic TOPIC_NAME]
                [--camera-trigger CAMERA_TRIGGER]
                file.ulg

Convert ULog to KML

positional arguments:
  file.ulg              ULog input file

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT_FILENAME, --output OUTPUT_FILENAME
                        output filename
  --topic TOPIC_NAME    topic name with position data
                        (default=vehicle_gps_position)
  --camera-trigger CAMERA_TRIGGER
                        Camera trigger topic name (e.g. camera_capture)

Convert ULog to rosbag files (ulog2rosbag)

Note You need a ROS environment with px4_msgs built and sourced.

Usage:

usage: ulog2rosbag [-h] [-m MESSAGES] file.ulg result.bag

Convert ULog to rosbag

positional arguments:
  file.ulg              ULog input file
  result.ulg            rosbag output file

optional arguments:
  -h, --help            show this help message and exit
  -m MESSAGES, --messages MESSAGES
                        Only consider given messages. Must be a comma-
                        separated list of names, like
                        'sensor_combined,vehicle_gps_position'

Migrate/setup the database for use with the DatabaseULog class (ulog_migratedb)

Warning This command must be run whenever the schema changes, otherwise DatabaseULog won't function.

Warning Even if you store logs in the database, you should keep the original .ulg files. Otherwise you may lose your data.

Usage:

usage: ulog_migratedb [-h] [-d DB_PATH] [-n] [-s SQL_DIR] [-f]

Setup the database for DatabaseULog.

optional arguments:
  -h, --help            show this help message and exit
  -d DB_PATH, --database DB_PATH
                        Path to the database file
  -n, --noop            Only print results, do not execute migration scripts.
  -s SQL_DIR, --sql SQL_DIR
                        Directory with migration SQL files
  -f, --force           Run the migration script even if the database is not
                        created with this script.

Example output (to console):

ubuntu@ubuntu:~/github/pyulog$ ulog_migratedb
Using migration files in /home/ubuntu/github/pyulog/pyulog/sql.
Database file pyulog.sqlite3 not found, creating it from scratch.
Current schema version: 0 (database) and 1 (code).
Executing /home/ubuntu/github/pyulog/pyulog/sql/pyulog.1.sql.
Migration done.

pyulog's People

Contributors

binsquare avatar bkueng avatar cclauss avatar dakejahl avatar dayjaby avatar devbharat avatar hamishwillee avatar hermankolden avatar jgoppert avatar johannesbrand avatar kd0aij avatar ken-voly avatar kimbongsik avatar marcinolokk avatar mhkabir avatar ndepal avatar nicolasm0 avatar roman-dvorak avatar tayyabkhalil-313 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pyulog's Issues

Attitude transformation base: body / inertial frame

@bkueng @jgoppert We should update the routine here to allow different references:

def _add_roll_pitch_yaw_to_message(self, message_name, field_name_suffix = ''):

@MaEtUgR Our flight logs show that the rate tracking is excellent but the angle tracking is off, depending on the vehicle yaw. I think there is something incorrect in the way we report roll and pitch set points currently. Example from this log:

http://logs.px4.io/plot_app?log=e55ed4c2-09db-416b-b622-98c3a18c948e

The rate tracking is excellent the angle tracking is horrible. The vehicle would have flown away given those tracking errors, so its just the reporting, not the actual flight performance which is wrong.

screen shot 2016-12-30 at 12 47 53

screen shot 2016-12-30 at 12 51 57

If you correlate it with the yaw plot it becomes obvious that there is a missing transformation from inertial to body here. And it is interesting that the yaw angle tracking seems to be off but that's probably not what happened. This is using #6213. Is the removal of the attitude setpoint publications maybe uncovering this?

pyulog PX4 log file read error when using ulog2csv

Dear pyulog user group.

On running ulog2csv in Windows 10, I receive the current error log.

C:\Flights 23-06-21\Pixhawk>ulog2csv log001.ulg
Traceback (most recent call last):
File "C:\Users\eg3023\AppData\Local\Programs\Python\Python38\Scripts\ulog2csv-script.py", line 11, in
load_entry_point('pyulog==0.7.0', 'console_scripts', 'ulog2csv')()
File "c:\users\eg3023\appdata\local\programs\python\python38\lib\site-packages\pyulog\ulog2csv.py", line 42, in main
convert_ulog2csv(args.filename, args.messages, args.output, args.delimiter, args.ignore)
File "c:\users\eg3023\appdata\local\programs\python\python38\lib\site-packages\pyulog\ulog2csv.py", line 59, in convert_ulog2csv
ulog = ULog(ulog_file_name, msg_filter, disable_str_exceptions)
File "c:\users\eg3023\appdata\local\programs\python\python38\lib\site-packages\pyulog\core.py", line 132, in init
self._load_file(log_file, message_name_filter_list)
File "c:\users\eg3023\appdata\local\programs\python\python38\lib\site-packages\pyulog\core.py", line 493, in _load_file
self._read_file_data(message_name_filter_list)
File "c:\users\eg3023\appdata\local\programs\python\python38\lib\site-packages\pyulog\core.py", line 664, in _read_file_data
msg_add_logged = self._MessageAddLogged(data, header,
File "c:\users\eg3023\appdata\local\programs\python\python38\lib\site-packages\pyulog\core.py", line 387, in init
self.message_name = ULog.parse_string(data[3:])
File "c:\users\eg3023\appdata\local\programs\python\python38\lib\site-packages\pyulog\core.py", line 91, in parse_string
ret = _parse_string(cstr)
File "c:\users\eg3023\appdata\local\programs\python\python38\lib\site-packages\pyulog\core.py", line 18, in _parse_string
return str(cstr, 'utf-8', errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 0: invalid start byte

Could you please advise if the data can be read into csv or recovered with the pyulog code. The px4 log file is found here: https://1drv.ms/u/s!AqvNv7Mai6RqhbUm0EkdBxk5MYOQCg?e=AtUwU3

The online px4 flight reviewer is flagging that the file is too large to read.

Many thanks, Nick (Cranfield Uni)

String decoding errors on python3

Currently python3 implementation of Ulog decodes to ascii and throws exception on decoding errors. That can be an issue in case there is a corrupt byte in the stream or non-ascii bytes. What would be your thought on switching to utf-8, and setting str() error handling to ignore?

Shift GPS data to make log location anonymous

There are quite a lot of scenarios where log data needs to be made somewhat anonymous prior to being able to being shared. After some conversations with @dagar, this can be a bit more tricky than initially thought since everything from gps, global position, home position, mission items, setpoints to even mag declination give a pretty clear indication of the location of the flight. There are just cases where substantial amounts of flight data can be used for analysis but often reporting where those flight took place isn't allowable, for whatever reason.

Initial thoughts would be to create the ability to have a random offset of all location data and then hold that offset for all coordinates in the log, that way everything is related and will still follow course. for analysis. This is obviously very preliminary but here to get the conversation started.

unable to load px4 logs

I am running offboard missions and generating logs using the current master of px4/Firmware (nuttx_px4fmu-v2_default config). However I am unable to load those logs into pyFlightAnalysis as pyulog (v0.6.0) fails to load the files.

Traceback (most recent call last):
  File "ulog_test.py", line 6, in <module>
    log_data = ULog(log_file_name).data_list
  File "/usr/local/lib/python2.7/dist-packages/pyulog/core.py", line 108, in __init__
    self._load_file(log_file, message_name_filter_list)
  File "/usr/local/lib/python2.7/dist-packages/pyulog/core.py", line 428, in _load_file
    self._read_file_header()
  File "/usr/local/lib/python2.7/dist-packages/pyulog/core.py", line 450, in _read_file_header
    raise Exception("Invalid file format (Failed to parse header)")
Exception: Invalid file format (Failed to parse header)

pyulog represents `char[x]` as `x` columns of individual chars

pyulog represents char[x] fields as x number of str fields. It seems less surprising to represent them all as one string instead of having to have a postprocess that concats them later. In the docs it's described as a char[127] which would more intuitively map to a string than 127 char values each with their own key (or in this case column).

I've shown an example here but as far as I can tell it impacts all fields that are char[x].

image

# A logging message, output with PX4_{WARN,ERR,INFO}

uint64 timestamp		# time since system start (microseconds)

uint8 severity # log level (same as in the linux kernel, starting with 0)
char[127] text

uint8 ORB_QUEUE_LENGTH = 4

https://docs.px4.io/main/en/msg_docs/LogMessage.html

Feature request - Add Timestamp when parameters changes during flight

When parameter is being change during flight there is only list of the values, but not their timestamp of being changed.
For PID tuning and other peruses its important to know the time.
I seen that the 'changed_parameters' array has a timestamp value, but its not correct when I have tried to print it.

Adding ulog_params arg to print the timestamp can help alot

ulog2csv creating multiple files

Hey, I am working on a program to take a ulg file and then output specific user requested columns of data. I wrote a component that takes in a large, sparse (often the only data on a row is time), csv file and parses through it and pulls out the user requested data and puts it into a new csv (still sparse). I had originally been using PlotJuggler to convert the ulog files to a csv (in that single but sparse format) and then running my program to get the specific data out of that large CSV file.

I hoped to automate this process by using pyulog to convert the .ulg files into a single csv document however pyulog splits the data into many 53 (in my case) csv files based on how often the data is collected. Is there any way I can get pyulog to put all of the information into a single csv instead? Alternatively, is there any way to get pyulog to just pull specific columns of data?

px4.add_roll_pitch_yaw doesn't work with DatabaseULog.get_dataset

When I call px4.add_roll_pitch_yaw(), I expect e.g. 'vehicle_attitude.roll' to be retrievable from uog.get_dataset('vehicle_attitude'). However, since the former modifies the ULog object directly and the latter gets the dataset from the DB, the 'roll' field is not found.

ModuleNotFoundError: No module named 'pyulog.libevents_parse'

I am trying to run ulog_messages -h got the follwoing error

Traceback (most recent call last):
  File "/home/mubashir/.local/bin/ulog_messages", line 11, in <module>
    load_entry_point('pyulog', 'console_scripts', 'ulog_messages')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/mubashir/pyulog/pyulog/pyulog/messages.py", line 9, in <module>
    from .px4_events import PX4Events
  File "/home/mubashir/pyulog/pyulog/pyulog/px4_events.py", line 7, in <module>
    from .libevents_parse.parser import Parser
ModuleNotFoundError: No module named 'pyulog.libevents_parse'

Logging messages with a 'Tag'

The current MSG_TYPE_LOGGING packet holds severity, timestamp and a generic message string. It'd be helpful if we could extend this to add a 'tag' string to signify the source of the generated message (instead of encoding it into the message itself). Then ulog could parse messages from each source into a dictionary with the tag as the key. Alternatively, we could create a new message type with the tag included. Which would be a preferred option?

Parameter change wrong timestamp

Hi,

I found a bug which makes it impossible to use the ulog.changed_parameters data. There seems to be a bug in the implementation of the last timestamp value, so that the timestamp always stays at the original timestamp value. This can be verified since ulog.start_timestamp and ulog.last_timestamp are equal. Therefore ulog.changed_parameters[0][0] is the same value and the data can not be used.

Would be awesome if this couldbe fixed.
Best
Vincenz

ulog_params doesn't show parameter values

Hi everyone! I have this weird issue: When I use ulog_params, I get only the parameter names, not parameter values.

$ ulog_params mylogfile.ulg
ASPD_BETA_GATE
ASPD_BETA_NOISE
ASPD_DO_CHECKS
ASPD_FALLBACK
ASPD_FS_INNOV
ASPD_FS_INTEG
ASPD_FS_T1
ASPD_FS_T2
ASPD_PRIMARY
ASPD_SCALE

I'm using pip-installed version 0.8.0:

$ pip install --upgrade pyulog
Defaulting to user installation because normal site-packages is not writeable
Collecting pyulog
  Downloading pyulog-0.8.0.tar.gz (36 kB)
Requirement already satisfied, skipping upgrade: numpy in /home/george/.local/lib/python2.7/site-packages (from pyulog) (1.16.5)
Building wheels for collected packages: pyulog
  Building wheel for pyulog (setup.py) ... done
  Created wheel for pyulog: filename=pyulog-0.8.0-py2-none-any.whl size=22879 sha256=bc03f68916ee7ff70a232e0e5bead97fa4d4c0c86f58e7a4b1c2f51e75ed6d73
  Stored in directory: /home/george/.cache/pip/wheels/58/cb/81/e3e83fba56877bceafc1a6dd4c77f6bc659bde5a445e6e6e96
Successfully built pyulog
Installing collected packages: pyulog
Successfully installed pyulog-0.8.0
WARNING: You are using pip version 20.0.2; however, version 20.1.1 is available.
You should consider upgrading via the '/usr/bin/python2 -m pip install --upgrade pip' command.

Description/Units of the various outputs

I noticed that several of the output csv's from pyulog do not contain units for the different outputted values, which has made some of these files somewhat difficult to use.
My main concern is with the altitude contained in the vehicle_global_position csv, where 3 altitudes are given: alt, alt_ellipsoid, and terrain_alt.
-Does PX4 use WGS84 for its ellipsoid? Is that the ellipsoid being referenced?
-Is alt referring to MSL?
-Where does the terrain_alt come from?
-Are these parameters each in units of meters or feet?
Thanks in advance!

KeyError: 'uint8_t[1169]' after `ulog.write_ulog(...)`

I have this snipped of code.

from pyulog import ULog
ulog = ULog("12_43_07.ulg")
ulog.write_ulog("/tmp/a.ulg")`

The file 12_43_07.ulg is a PX4 log generated using the code from the main branch of PX4

After executing the code above, the following exception is thrown and only partial data is saved in /tmp/a.ulg:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[3], line 1
----> 1 ulog.write_ulog("/tmp/a.ulg")

File ~/programming/px4/Anonymize_ULog_Data/pyulog/pyulog/core.py:248, in ULog.write_ulog(self, path)
    246 self._write_format_messages(ulog_file)
    247 self._write_info_messages(ulog_file)
--> 248 self._write_info_multiple_message(ulog_file)
    249 self._write_initial_parameters(ulog_file)
    250 self._write_default_parameters(ulog_file)

File ~/programming/px4/Anonymize_ULog_Data/pyulog/pyulog/core.py:303, in ULog._write_info_multiple_message(self, file)
    301 continued = False
    302 for value in value_set:
--> 303     data = self._make_info_message_data(key, value, value_type, continued)
    304     header = struct.pack('<HB', len(data), self.MSG_TYPE_INFO_MULTIPLE)
    306     file.write(header)

File ~/programming/px4/Anonymize_ULog_Data/pyulog/pyulog/core.py:360, in ULog._make_info_message_data(self, key, value, value_type, continued)
    356     data.extend(value_bytes)
    357 # elif value_type.startswith('uint8_t['):
    358 #     data.extend(value)
    359 else:
--> 360     code = self._UNPACK_TYPES[value_type][0]
    361     data.extend(struct.pack('<' + code, value))
    363 return data

KeyError: 'uint8_t[1169]'

CI broken after update

@bkueng I think this commit:
8301e51

Broke potentially PX4 CI. This might be because the PX4 usage is not correct, but we need to figure out how to fix it.

@AndreasAntener FYI

[ROSTEST]-----------------------------------------------------------------------

[px4.rosunit-box/test_mission][ERROR]-------------------------------------------
'PX4MessageDict' object has no attribute 'resample_and_concat'
 File "/usr/lib/python2.7/unittest/case.py", line 331, in run
   testMethod()
 File "/job/catkin/src/px4/integrationtests/python_src/px4_it/mavros/mission_test.py", line 372, in test_mission
   data = px4tools.ulog.read_ulog(last_log).resample_and_concat(0.1)

ulog2kml fails with missing module simplekml

Library is missing dependency (all other scripts work). This is on Ubuntu 16.04 LTS, Python 2.7 default, installed using either method.

ulog2kml -h
Traceback (most recent call last):
  File "/usr/local/bin/ulog2kml", line 7, in <module>
    from pyulog.ulog2kml import main
  File "/usr/local/lib/python2.7/dist-packages/pyulog/ulog2kml.py", line 9, in <module>
    import simplekml # pylint: disable=import-error
ImportError: No module named simplekml

log the IMU and PWM outputs with timestamp in pixhawk 4 mini

Hi I am research student ,
actually to find the inertial tensor I need IMU gyro and PWM outputs values with timestamp for this how can I boost the logging data to 400HZ. The log which is generated in QGC is limited to 50HZ but I need these values at which rate the pix hawk generates the signal. so can anyone help to log the IMU gyro and PWM outputs at its maximum rate . so that it is easy for me to find the inertial tensor

Using Python script instead of command line to parse ulog files

I want to be able to parse .ulg files using python script instead of a command line, how do I go about that? I'm used to using python script instead of command prompt in developing and running my code. I have tried a number of things but not getting it, please let me know what to do.

Request Script to Unroll sensor_gyro_fifo and sensor_accel_fifo

The sensor_gyro_fifo and sensor_accel_fifo messages in PX4 are hard to analyze due to the combined nature of the message. Each message can contain up to 32 datapoints with a dt and scale.

I propose we create a script to take a log, unroll the fifo messages, and save a new log file with the additional unrolled messages.

It appears this is not possible in PlotJuggler with just LUA scripting.

@dagar

pyulog not extractting vehicle_air_data

Hi

I trying to convert ulog data I have to csv using pyulog.
In a few u log files, if I use flight review tool, I can see the barometric data being plotted. Here is the link to the logs - https://logs.px4.io/plot_app?log=b852b2d8-93e1-4ae2-b204-6ffa89453f86
https://drive.google.com/file/d/19dI-8yOWR5siHObcn2UmUTob-soISPu5/view?usp=sharing

But when I try to convert it to csv using pyulog I dont see a file associated with vehicle_air_data.
For some other files it works fine.

Not sure what the issue might be since flight review is able to plot it.
Any suggestions?
Thanks!

Add data corruption handling

From slack discussion here

In sdlog2 (ancient I know) logging format their used to be these identifier bytes xA3 x95 before start of every valid packet, so if their was corruption in previous logged packet you could seek for these bytes in the file stream and recover parsing again. It seems that ulog packets start directly with 3 bytes of MSG_TYPE and MSG_SIZE. How does one recover if while parsing a file you encounter a MAG_TYPE byte that isn't supposed to be there?
You can't just look of a valid MSG_TYPE byte in the stream following the corrupt byte, it might randomly occur in the payload packets and you'd endup reading incorrect bytes for MSG_SIZE

not able to fin binary installed

Hi all, sorry I am sure this is a dumb question;
I use ubuntu 18.04 and I installed the package as indicated. the result is this:

user1@user1-VirtualBox:~$ python3.7 -m pip install pyulog
Collecting pyulog
Using cached https://files.pythonhosted.org/packages/f4/f8/00495d753b987fe228f3eb1b81c32a7bab5339627f52e30a4b887af7bdb0/pyulog-0.9.0-py3-none-any.whl
Collecting numpy (from pyulog)
Using cached https://files.pythonhosted.org/packages/b0/27/86c9c52a335695fabc79514a196abd2d703ff8b089ba3207f069b63115ce/numpy-1.21.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl
Installing collected packages: numpy, pyulog
Successfully installed numpy-1.21.3 pyulog-0.9.0

Now I don't know where to find the binary of ulog2cvs. This is needed for me because I want to use a Matlab tool called Ulog Viewer which requires this:

% REPLACE this with the location of your installed ulog2csv binary
handles.ulog2csv = '???';

Sorry again, hope you can help me

ulog2kml not entered as entry point in setup.py

Users are not able to use ulog2kml to convert in the command line if they installed using pip.

The cause is due to this line : 'ulog2kml=pyulog.ulog2kml:main' is missing from the setup.py file under entry_points.

Name mapping from sdlog2 format

I've coded px4tools to look for specific field names when using sdlog2. Now that the fields are renamed in ulog, it is a pain. It would be nice if there was an sdlog2 compatibility csv export.

Also, Is there a way to dump the csv into one file without listing all of the messages? The sdlog2 csv converter would dump it all into one file which I could then read directly into pandas for analysis in python.

Decode error on pyulog streamed over wifi

Here are the log files I colleted, some of them work, but the large one doesn't even after updating to master:

https://drive.google.com/open?id=0B3wIEs1gcVahNlVydy0zazZ5bU0

ulog_info 001-2016-12-02-13-21-25-513.ulg 
Traceback (most recent call last):
  File "/home/jgoppert/anaconda3/bin/ulog_info", line 11, in <module>
    sys.exit(main())
  File "/home/jgoppert/anaconda3/lib/python3.5/site-packages/pyulog/info.py", line 25, in main
    ulog = ULog(ulog_file_name)
  File "/home/jgoppert/anaconda3/lib/python3.5/site-packages/pyulog/core.py", line 265, in __init__
    self.loadFile(file_name, message_name_filter_list)
  File "/home/jgoppert/anaconda3/lib/python3.5/site-packages/pyulog/core.py", line 276, in loadFile
    self.read_file_data(message_name_filter_list)
  File "/home/jgoppert/anaconda3/lib/python3.5/site-packages/pyulog/core.py", line 328, in read_file_data
    self.message_formats)
  File "/home/jgoppert/anaconda3/lib/python3.5/site-packages/pyulog/core.py", line 164, in __init__
    self.message_name = parseString(data[3:])
  File "/home/jgoppert/anaconda3/lib/python3.5/site-packages/pyulog/core.py", line 14, in parseString
    return str(cstr, 'ascii')

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.