Coder Social home page Coder Social logo

ebroecker / canmatrix Goto Github PK

View Code? Open in Web Editor NEW
873.0 873.0 381.0 7.02 MB

Converting Can (Controller Area Network) Database Formats .arxml .dbc .dbf .kcd ...

License: BSD 2-Clause "Simplified" License

Python 100.00% Shell 0.01%
arxml can canbus compare convert dbc dbf dissector fibex json kcd modify python sym xlsx

canmatrix's People

Contributors

4cylinder avatar altendky avatar annguyen0 avatar arcola-kankan avatar bombilee avatar cdbrkfxrpt avatar christiansandberg avatar cimadure avatar coccyx00 avatar danielhrisca avatar ebroecker avatar ezfreak avatar funth0mas avatar granquet avatar jazi007 avatar juliengrv avatar martytemple avatar pempem98 avatar pierreluctg avatar rfrowe avatar russkel avatar seneda avatar shevonar avatar snideto avatar sou1hacker avatar tainnok avatar timschneider avatar welbouri avatar werux avatar xrowe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

canmatrix's Issues

AutoSAR version targeted in conversions?? -- Support for AUTOSAR4 Export

I'm performing python convert.py -vv x.dbc x.arxml but the resulting file doesn't contain a schema header. So I'm trying to guess the version and paste in a header in order to load it into AE, but either the file fails schema validation or simply fails to load and errors out.

Should a schema header do the trick and, if so, what version would I use? If not do you have other ideas what my problems would be? Could indentation also be an issue?

Broken frame names converting .arxml -> .dbc

Hello,
I am contacting you here because Stefan Obermeier told me to do so.
I am trying to convert a .arxml file to .dbc.
The conversion runs without an error (.exe and python version).
When i am looking into the .dbc afterwards, all ("Botschaften") frame names look like this:
"XDIS_1exfe32hc85lboe430gpg0si4"
The signal and ECU names seems to be correct.
The mapping of Rx Frames and signals to the ECUs are also looking correct.
The "Tx frames" and "mapped Tx signals" lists are empty for each ECU.
Also the Sender isn't shown for all Frames and Signals.
I hope you can help me with this.
Thank you a lot.
Regards Florian Müller

Below some information out of the .arxml Header´:
<?xml version="1.0" encoding="UTF-8"?> <AUTOSAR xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://autosar.org/3.2.2" xsi:schemaLocation="http://autosar.org/3.2.2 AUTOSAR.xsd" S="0A73F475379076439F1BC44F44D64CDEA6E47BFC" T="2015-11-04T07:40:38"> <ADMIN-DATA> <SDGS> <SDG GID="ExportInfo"> <SD GID="ExportedWithXDISVersion">5.3.1</SD> ... <SD GID="CopyrightInfo">Copyright (c) 2015 Daimler AG</SD>

canconvert dbc=>xlsx error (v0.3)

After "pip install canmatrix" within a cygwin installation I get this error message now by performing canconvert xxx.dbc xxx.xlsx:
Traceback (most recent call last):
File "/usr/bin/canconvert", line 11, in
sys.exit(main())
File "/usr/lib/python2.7/site-packages/canmatrix/convert.py", line 173, in main
convert(infile, outfileName, *_cmdlineOptions.dict)
File "/usr/lib/python2.7/site-packages/canmatrix/convert.py", line 93, in convert
ex.exportXlsx(db, outfile, *_options)
File "/usr/lib/python2.7/site-packages/canmatrix/exportxlsx.py", line 312, in exportXlsx
sigHash["%02d" % int(sig.getMsbStartbitReverse()) + sig._name] = sig
AttributeError: 'Signal' object has no attribute 'getMsbStartbitReverse'
Exception Exception: Exception('Exception caught in workbook destructor. Explicit close() may be required for workbook.',) in <bound method Workbook.del of <xlsxwriter.workbook.Workbook object at 0xff494a8c>> ignored

canconvert xxx.dbc xxx.xls is working fine with the same dbc.
The "old" version from last year was working fine for the conversion from dbc to xlsx with the same dbc input file

Error message converting dbc to Excel w and w/o --dbcCharset=utf8

With latest Github sync I'm not able to convert dbc to xls/xlsx anymore!

Traceback (most recent call last):
File "convert.py", line 122, in
sys.exit(main())
File "convert.py", line 119, in main
charset=args.dbcCharset, comment_charset=args.commentCharset)
AttributeError: 'list' object has no attribute 'dbcCharset'

Feature Request: Grouping of Signals in xls format

For better usability it would be appreciated if all CAN signals of a CAN messages are grouped in the .xls file. So at the end you can hide/unhide or collapse/uncollapse the signals and only all CAN messages are shown/highlighted in Excel

Error converting from arxml (to anything) using dev-0.6

INFO - convert - Importing ASR.arxml ...
DEBUG - arxml - Read arxml ...
DEBUG - arxml -  Done

DEBUG - arxml - Build arTree ...
DEBUG - arxml -  Done

DEBUG - arxml - DEBUG 0 frames in arxml...
DEBUG - arxml - DEBUG 181 can-frame-triggering in arxml...
DEBUG - arxml - DEBUG 0 SIGNAL-TO-PDU-MAPPINGS in arxml...
DEBUG - arxml - DEBUG 808 I-SIGNAL-TO-I-PDU-MAPPING in arxml...
DEBUG - arxml - Busname: CAN_2_Cluster
DEBUG - arxml -  Speed: 500
DEBUG - arxml -  Speed: 500
DEBUG - arxml - 181 frames found in arxml
...
...
Traceback (most recent call last):
  File "/usr/bin/canconvert", line 9, in <module>
    load_entry_point('canmatrix==0.5', 'console_scripts', 'canconvert')()
  File "build/bdist.cygwin-2.4.1-i686/egg/canmatrix/convert.py", line 294, in main
  File "build/bdist.cygwin-2.4.1-i686/egg/canmatrix/convert.py", line 41, in convert
  File "build/bdist.cygwin-2.4.1-i686/egg/canmatrix/formats.py", line 54, in loadp
  File "build/bdist.cygwin-2.4.1-i686/egg/canmatrix/formats.py", line 63, in load
  File "build/bdist.cygwin-2.4.1-i686/egg/canmatrix/arxml.py", line 1558, in load
IndexError: list index out of range

cancompare xxx.dbc (old) xxx.dbc (new) - error

cancompare (latest master from 3rd of Feb 2016):
cancompare xxx.dbc xxx.dbc

Traceback (most recent call last):
File "/usr/bin/cancompare", line 9, in
load_entry_point('canmatrix==0.3', 'console_scripts', 'cancompare')()
File "build/bdist.cygwin-2.3.1-i686/egg/canmatrix/compare.py", line 356, in main
File "build/bdist.cygwin-2.3.1-i686/egg/canmatrix/compare.py", line 64, in compareDb
File "build/bdist.cygwin-2.3.1-i686/egg/canmatrix/compare.py", line 206, in compareFrame
File "build/bdist.cygwin-2.3.1-i686/egg/canmatrix/compare.py", line 268, in compareSignal
TypeError: %d format: a number is required, not unicode

converting dbc <=> xls <=> dbc

converting a dbc to xls and vice versa back to dbc again without changing anything I get this fault:

Traceback (most recent call last):
File "convert.py", line 65, in
dbs["default"] = im.importXls(infile)
File "/cygdrive/v/CANtools/canmatrix/library/importxls.py", line 217, in importXls
newSig._factor = float(factor)
ValueError: could not convert string to float: degree

Issue converting arxml -> dbc

Build arTree ...
Traceback (most recent call last):
File "/usr/bin/canconvert", line 9, in
load_entry_point('canmatrix==0.2', 'console_scripts', 'canconvert')()
File "build/bdist.cygwin-2.3.1-i686/egg/canmatrix/convert.py", line 145, in main
File "build/bdist.cygwin-2.3.1-i686/egg/canmatrix/convert.py", line 50, in convert
File "build/bdist.cygwin-2.3.1-i686/egg/canmatrix/importarxml.py", line 356, in importArxml
File "build/bdist.cygwin-2.3.1-i686/egg/canmatrix/autosarhelper.py", line 47, in arParseTree
TypeError: 'NoneType' object is not iterable

Im pretty sure this is a problem with the arxml file. Is there somedebug code I can add or uncomment to help identify where in the arxml file the issue is?

Default value exception when converting `.sym`->`.dbc`

I'm doing a little work on the tests starting with just generating all outputs for each provided input file type. When it gets to .sym->.dbf I get an issue with a missing default value for a define. My gut feeling is that this is an issue with the export and should be handled there, but I'm not certain. Since you are much more familiar, what do you think?

Here's the error in the context of a direct manual call to `convert.py`.

altendky@tp:/epc/t/109/canmatrix$ git log -n 1
commit 46e7a32616c6865eef254a59d1bedfe70f066d73
Author: eduard <[email protected]>
Date:   Mon Nov 23 21:48:42 2015 +0100

    small documentation update (hopefully no german comments left)
altendky@tp:/epc/t/109/canmatrix$ ./convert.py AFE_CAN_ID247_FACTORY.sym AFE_CAN_ID247_FACTORY.dbf
Importing AFE_CAN_ID247_FACTORY.sym ... 
done

Exporting AFE_CAN_ID247_FACTORY.dbf ... 

26 Frames found
Traceback (most recent call last):
  File "./convert.py", line 93, in <module>
    ex.exportDbf(db, outfile)
  File "/epc/t/109/canmatrix/library/exportdbf.py", line 150, in exportDbf
    f.write('"' + type + '",' + define._definition.encode(dbfExportEncoding,'replace').replace(' ',',') + ',' + define._defaultValue + '\n')
TypeError: cannot concatenate 'str' and 'NoneType' objects
altendky@tp:/epc/t/109/canmatrix$ 

Decoding signed vs unsigned inputs

It would be very helpful to have another column in the output csv file indicating signed or unsigned for every signal.

For example:

A) SG_ my_signal : 16|16@1- (1,0) [-600|1000] "m" Vector__XXX
B) SG_ my_signal_2 : 0|5@1+ (1,0) [0|31] "m" Vector__XXX

In A), the "-" that follows "@1" indicates that this signal is signed. Likewise in B), the "+" that follows the @1 indicates that this signal is unsigned.

dbc import error (endswith() does not work with str)

File "convert.py", line 35, in convert
    dbs[""] = im.importDbc(infile, dbcCharset,  dbcCommentCharset)
  File [...]/canmatrix/library/importdbc.py", line 80, in importDbc
    if l.endswith('";'):

TypeError: endswith first arg must be bytes or a tuple of bytes, not str

Possible solution: conversion to bytestring?
if l.endswith('";'): --> if l.endswith(b'";'):
(in lines 59, 69, 79)

arxml->dbc KeyError: 'GenMsgStartDelayTime'

canconvert -v source.arxml destination.dbc
INFO - convert - Importing source.arxml ...
INFO - convert - done

INFO - convert - Exporting destination.dbc ...
INFO - convert - DIAGNOSTICS
INFO - convert - 706 Frames found
Traceback (most recent call last):
File "C:\Program Files (x86)\Python35-32\Scripts\canconvert-script.py", line 9, in
load_entry_point('canmatrix==0.5', 'console_scripts', 'canconvert')()
File "C:\Program Files (x86)\Python35-32\lib\site-packages\canmatrix-0.5-py3.5.egg\canmatrix\convert.py", line 293, in main
File "C:\Program Files (x86)\Python35-32\lib\site-packages\canmatrix-0.5-py3.5.egg\canmatrix\convert.py", line 139, in convert
File "C:\Program Files (x86)\Python35-32\lib\site-packages\canmatrix-0.5-py3.5.egg\canmatrix\formats.py", line 102, in dumpp
File "C:\Program Files (x86)\Python35-32\lib\site-packages\canmatrix-0.5-py3.5.egg\canmatrix\formats.py", line 78, in dump
File "C:\Program Files (x86)\Python35-32\lib\site-packages\canmatrix-0.5-py3.5.egg\canmatrix\dbc.py", line 288, in dump
KeyError: 'GenMsgStartDelayTime'

Question: Converting from Vector .dbc to .json I only get a subset of fields.

I got the tool installed and working running well but it's only converting a subset of the fields in my .dbc file. I wanted 'Comments', 'PGN', and 'priority' among others that are missing.. Am I missing some options or does it only mine specific fields

The 4 images show all the interesting fields in the .dbc file then I show the output of the .json

From the CANdb++ edit I get data...
c1

c2

c3

c4

All that I get out of the converter is:
{ "id": "0x18d0fefe", "name": "CabIlluminationMessage", "signals": { "0": { "bit_length": 8, "factor": "0.4", "name": "IlluminationBrightnessPercent", "offset": "0" }, "8": { "bit_length": 8, "factor": "0.4", "name": "SwitchBacklightIlluminPercent", "offset": "0" } } }

External access to matrix objects

I would like to use the imported CanMatrix instance programmatically but currently I need to use the private attributes prefixed with _. Is there a plan to have a fairly stable API?

Either the attributes could be renamed, get/set functions could be implemented or property functions be added.

Missing byteorder conversion for multiplexed signals

Hello together.
As I am working with canmatrix from time to time I figured out a DBC file importing problem. I would like to share my finding:

The conversion of the MSB-startbit while DBC-import is not done for multiplexed signals.
tempSig.setMsbStartbit(int(temp.group(2))) from importdbc.py:113 is missing for multiplexed signals a few lines below.

Thank you for providing and maintaining this great collection of python code which really enlightens the work with CAN databases.

Greetings
Andi

startbit of motorola coded signal may be inconsistent

The startbit of motorola coded signals are coded a bit strange in 'dbc'.
Canmatrix internally uses this (dbc-like) startbit.
Sym: Im/exportsym converts this correct.
semms that kcd and dbf currently is converted wrong:

But I'm not sure about other arxml.
In excel it would be much better to use a 'better readable' startbit.

==> Conclusion: Strange dbc startbits should be handled in dbc-modules and not in sym, dbf, kcd; internal representation should be human readable and not dbc-like.

EDIT:
kcd does need converted startbits:
https://github.com/julietkilo/CANBabel/blob/master/src/main/java/com/github/canbabel/canio/dbc/DbcReader.java:
/**
* Calculates the least significant bit offset for big endian byte order
* @param msb Most significant bit offset
* @param length signal length in bit
* @return lsb Least significant bit offset
/
public static int bigEndianLeastSignificantBitOffset(int msb, int length){
int lsb;
int pos;
int cpos;
int bytes;
pos = 7 - (msb % 8) + (length - 1);
if (pos < 8){
/
msb pass a byte order */
lsb = msb - length + 1;
} else {
cpos = 7 - (pos % 8);
bytes = pos / 8;
lsb = cpos + (bytes * 8) + (msb/8) * 8;
}
return lsb;
}

EDIT2:
dbf does also some conversion (https://github.com/rbei-etas/busmaster/blob/master/Sources/Format%20Converter/DBC2DBFConverterLibrary/DBFSignal.cpp):

if(SIG_DF_MOTOROLA == m_ucDataFormat)
{
//The following is done because if signal type is of MOTOROLA
//THe MSB is stored and will be in Big Endian format.
//Get the LSB
/In CANoe if we view the bits as a 8x8 Matrix, for Motorala signals
the numbering will be in the reverse order with in the each row. So to get the
proper MSB we need to again swap the bit position with in the row
/
//i.e here we need to get the left half of the bits to the right and right
//to the left with in the same row
int ntemp;
if(uiStartBit == 0)
{
ntemp = 0;
}
else
{
ntemp = uiStartBit % 4; //get the bit position with its the row.
}
//swap left bits to the right and right bits to the left.
if(uiStartBit % 8 <= 3)
{
uiStartBit = uiStartBit + 7 - ntemp * 2;
}
else
{
uiStartBit = uiStartBit - 1 - ntemp * 2;
}
uiStartBit = uiStartBit + m_ucLength-1;
//Invert the bit position (Converting to little endian format)
uiStartBitX = (CDBFConverter::ucMsg_DLC*8)-1 - uiStartBit;
}

Hints of removing some signal attributes?

Hi
I need to remove some signal attributes, but can not figure out how to do this.
Any hint is much appreciated :)

See below for my noobish try :)
unwanted_attributes is list of ...... unwanted attributes

def replace_unwanted_signal_attributes(frame,replace):
    unwanted_attributes = ['A','B','C']
    for signal in frame._signals:
        for attribute in signal._name, signal._attributes:
            for elem in attribute:
                if elem in unwanted_attributes:
                    attribute[elem] = replace
    return frame

Per mux value comments in .sym are lost

While canmatrix only creates a single mux signal for all possible values the .sym files can have separate comments associated with each mux value since the mux signal is repeated in the file.

diff --git a/canmatrix/importsym.py b/canmatrix/importsym.py
index d906e84..e77efd4 100644
--- a/canmatrix/importsym.py
+++ b/canmatrix/importsym.py
@@ -211,7 +211,7 @@ def importSym(filename, **options):
                 if tmpMux == "Mux":
                     signal = frame.signalByName(frameName + "_MUX")
                     if signal == None:
-                        signal = Signal(frameName + "_MUX", 
+                        signal = Signal(frameName + "_MUX",
                                           startBit = startBit, 
                                           signalSize = signalLength,
                                           is_little_endian=intel, 
@@ -228,7 +228,9 @@ def importSym(filename, **options):
                         if intel == 0:
                             #motorola set/convert startbit
                             signal.setStartbit(startBit)
+                        signal.comments = {}
                         frame.addSignal(signal)
+                    signal.comments[int(multiplexor)] = comment
 
                 else:
  #                   signal = Signal(sigName, startBit, signalLength, intel, is_signed, factor, offset, min, max, unit, "", multiplexor)

I haven't really been touching canmatrix because it's been working well for me for quite awhile so I have to ask... Where should I be updating to and where would a commit like this go?

Cheers,
-kyle

dbc export: string attributes should be quoted

as described in #64 exporting string attributes to dbc seems to we wrong:

correct BA_ "signame" SG_ 1234567890 ECU "StringAttribute"

currently wrong exportted BA_ "signame" SG_ 1234567890 ECU StringAttribute

Export KCD uses the wrong endianess

Looking through the source code to try to figure out why my .kcd files were producing different results than when I converted to other formats, I came across this bug.

if signal._is_little_endian == 0:
        sig.set('endianess',"little")

Obviously, if signal._is_little_endian == 0 the signal is big endian and should not be set as little endian.
The fix appears to be just changing it to

if signal._is_little_endian == 1:'
        sig.set('endianess',"little")

I'll try to submit a pull request, too but if someone else just wants to roll this into the next commit that might be easier.

Edit:
The problem appears to be the same in the importkcd.py file as well. I suggest changing the code from:

is_little_endian = True
if 'endianess' in signal.attrib:
    if signal.get('endianess') == 'little':
        is_little_endian = False

to:

is_little_endian = False
if 'endianess' in signal.attrib:
    if signal.get('endianess') == 'little':
        is_little_endian = True

Export I-SIGNAL-TO-I-PDU-MAPPING

Hi,

I am attempting to convert a large arxml file to dbc, but no dbc file is created although 524 I-SIGNAL-TO-I-PDU-MAPPINGs are found.

INFO - convert - Importing my.arxml ... 
DEBUG - importarxml - Read arxml ...
DEBUG - importarxml -  Done

DEBUG - importarxml - Build arTree ...
DEBUG - importarxml -  Done

DEBUG - importarxml - DEBUG 0 frames in arxml...
DEBUG - importarxml - DEBUG 0 can-frame-triggering in arxml...
DEBUG - importarxml - DEBUG 0 SIGNAL-TO-PDU-MAPPINGS in arxml...
DEBUG - importarxml - DEBUG 524 I-SIGNAL-TO-I-PDU-MAPPING in arxml...
INFO - convert - done

INFO - convert - Exporting my.dbc ... 
INFO - convert - done

Is that to be expected? How can I get a clean list of signals?

Question: What do you guys use for encoding decoding can data frames?

Hi all,

I was wondering what you guys use for encoding/decoding the 8 bytes into signals and visa versa? I have seen this library: https://github.com/openxc/bitfield-c but I cannot get it to work properly for all different DBC formats (little/big endian, signed unsigned etc).

For python, I use https://pypi.python.org/pypi/bitarray/0.8.1 which works fine. However, my implementation is too slow if I have to handle 2000 frames * x signals per second.

I'm interested in what kind of libraries you guys use, especially the C, C++ libs?

Thanks!

Setuptools

I have started adjusting for Setuptools. The most significant change is to rename 'library' to 'canmatrix'. Not really a huge deal but it does break existing YAML files since they have the Python objects in them such as !!python/object:library.canmatrix.Frame. This can 'easily' be recursively fixed for all YAML files via:

find . -name '*.yaml' -exec sed -i 's/object:library.canmatrix/object:canmatrix.canmatrix/g' {} \;

But, I figured I should mention this sooner than later to see how much concern there is.

add a feature to delete unused ´defines´

add method to delete all defines which are not needed.
add also command line option for it.

This would be interesting in combination with deleting unwanted attributes (see also #50)

Use integers for value table keys

Signal.multiplex is an integer but the multiplex signal's values keys are strings. These should be consistent and I recommend choosing integer (unless maybe there could be float keys?).

commit 7386a982bafdc53e1de8df7bde83101c18cc81ac
Author: Kyle Altendorf <[email protected]>
Date:   Tue May 16 16:28:35 2017 -0400

    Normalize value tables to have integer keys

diff --git a/canmatrix/canmatrix.py b/canmatrix/canmatrix.py
index cbff60b..84b9c7e 100644
--- a/canmatrix/canmatrix.py
+++ b/canmatrix/canmatrix.py
@@ -163,6 +163,10 @@ class BoardUnitList(object):
         return len(self._list)
 
 
+def normalizeValueTable(table):
+    return {int(k): v for k, v in table.items()}
+
+
 class Signal(object):
     """
     contains on Signal of canmatrix-object
@@ -258,7 +262,7 @@ class Signal(object):
 
     @values.setter
     def values(self, valueTable):
-        self._values = valueTable
+        self._values = normalizeValueTable(valueTable)
 
     @property
     def comment(self):
@@ -838,7 +842,7 @@ class CanMatrix(object):
         return iter(self._fl)
 
     def addValueTable(self, name, valueTable):
-        self._valueTables[name] = valueTable
+        self._valueTables[name] = normalizeValueTable(valueTable)
 
     def addAttribute(self, attribute, value):
         """

DBC converter is not idempotent

it seems to easy to create a matrix that changes when converting it to some format and then back.

Consider the following file 1.json:

{
    "messages": [
        {
            "id": 111,
            "is_extended_frame": false,
            "name": "foo",
            "signals": [
                {
                    "name": "bar",
                    "start_bit": 3,
                    "bit_length": 16,
                    "is_big_endian": true,
                    "is_signed": true,
                    "offset": 0.0,
                    "factor": 1.0
                }
            ]
        }
    ]
}

Now running the following:

canconvert 1.json 2.dbc
canconvert 2.dbc 3.json
canconvert 3.json 4.dbc
canconvert 4.dbc 5.json

will move the startbit around:

  • 1.json: start_bit=3
  • 3.json: start_bit=19
  • 5.json: start_bit=35

i would expect that the data stays the same when converting between two formats.

Python3 support

I'm working on Python3 support and needed somewhere to put a few questions.

@ebroecker, while looking at changes in the test result files while adjusting for Python3 I noticed that some of the .yaml test reference files have !!python/unicode in front of the strings. I noticed because the Python3 stuff wasn't writing that. I'm guessing we don't want references to Python in our YAML files but wanted to confirm with you.

I also had issues where some of the items were being output in different orders than before which is annoying for running the diff as the test. Specifically, some of the def's here. Am I correct that these would be OK in any order so I can sort the list prior to writing it to the file?

@nexulm, I am trying to retain the dual-encoding option Eduard added for you but it would be really helpful if I could get the reference file. Since we have a basic regression test script now it would also be nice to be able to include such a reference file in the tests. My email is in my profile, or we can arrange another transfer method.

Use Canmatrix as a library

Hi, I am currently writing a Python GTK+ 3 front end for Canmatrix, as I can't find any project which does something like that already. The idea is to call a Canmatrix function with a file path and recieve a Can Matrix Object:

def getCanMatrixObject(infile):
    canMatrixObject = CanMatrix()
    # do stuff
    return canMatrixObject

In the same way I would like to call a Canmatrix function with a Can Matrix Object and a file path for the export.

def exportCanMatrixObject(canMatrixObject, outfile):
    # do export

Then I can use the object in the GUI to visualise the data and let the user e.g. delete signals. What would be needed to achieve this? With a little help I would be more than happy to do the changes myself and submit a patch.

Greetings, Dennis

Export encoding vs. comment encoding?

I am working over in altendky/canmatrix with a couple short term goals.

In the context of the Python3 situation, encode() now returns bytes rather than a string which write() doesn't like. I noticed that if there were only a single encoding specified then it could simply be specified via open() rather than encoding each line individually. But, there are two encodings specified.

In what situation would it make sense to encode the content and comments differently and does that really result in a valid file?

Thanks for any explanation you can provide.

Recalc DLC

A feature to recalc DLC would be nice.

Maybe also something like recalc max dlc which should calc the dlc and compare with current dlc and store the bigger one in canmatrix.

Converting a dbc file with chinese characters aren't working

18 Frames found
Traceback (most recent call last):
File "convert.py", line 71, in
db = ex.exportXls(db, outfile)
File "/cygdrive/v/canmatrix/library/exportxls.py", line 288, in exportXls
writeFrame(frame, worksheet, row, framestyle)
File "/cygdrive/v/canmatrix/library/exportxls.py", line 49, in writeFrame
worksheet.write(row, 0, label = "%3Xh" % frame._Id, style=mystyle)
File "/usr/lib/python2.7/site-packages/xlwt/Worksheet.py", line 1088, in write
self.row(r).write(c, label, style)
File "/usr/lib/python2.7/site-packages/xlwt/Worksheet.py", line 1136, in row
self.__rows[indx] = self.Row(indx, self)
File "/usr/lib/python2.7/site-packages/xlwt/Row.py", line 43, in init
raise ValueError("row index was %r, not allowed by .xls format" % rowx)
ValueError: row index was 65536, not allowed by .xls format

dbc does not support zero-lenght-signals

While converting from other formats, it can happen that a zero-length signal appears in dbc.
Candb++ does not support this (compare to #23).

suggestion: add some clean-up-method to canmatrix such that zero-length-signals can optional (command line) be deleted before export.

export to file-like objects

would it be possible to have the exporters/importers take filelike objects (aka streams) instead of filenames?

e.g. I would like to get a JSON representation of the database, without having to create a temporary file and then read that file back into my application.
instead i'd like to do something like:

import canmatrix.exportjson
import io
f = io.StringIO()
canmatrix.exportjson.exportJson(db, f)
j = f.getvalue()

(though of course in reality i would like to really be able to get a meaningful representation of the database in Python; this is related to #57 ; but i still think that it is a good idea to not have external files (references via filenames) as the sole interface)

Retaining BA_ and BA_DEF_ in conversions

I like starting with your xlsx format because it's a lot cleaner with Excel than mucking directly with the dbc files in a text editor. Unfortunately, in the conversions between dbc and xlsx, the definitions such as BA_ and BA_DEF_, etc. are lost.

Is there any way to retain that information? Or at least would it be possible to have an option that pulls them in from a specified dbc perhaps (e.g., canconvert --defs=reference.dbc source.xlsx target.dbc)??

Notify of invalid named parameters

As I started into issue 39 I first updated the canmatrix I was using (normally stick with a single revision until I need to change something) and ran into some changes I needed to make to my application. In making the changes I mistyped a few of the named parameters that are now handled via **kwargs. I didn't get any notification like a normally named argument would provide and ended up down various wrong paths before I found my mistake. So, back to the title, it would be nice to throw an exception on invalid function parameters. As I did this example I also converted it to a much more compact and automated form that may or may not interest you.

https://gist.github.com/altendky/b2e4ad619ec738b462d49f1b1082201b

If you like this, I might convert the various places you have the new **kwargs form, or I might not get to it for a bit. Time will tell. :]

It's a bit less legible for the multiplex case but since that's one out of 12 parameters or so, I think it may be worthwhile.

What is a CAN database

I came across this library while searching for something else. It wasnt immediately obvious what this package does.
I assume CAN is Controller Area Network.
I think it would be helpful to explain CAN in the README.

Quoted strings when importing

A coworker happened across canmatrix the other day and asked if I'd heard of it... :] and he also had a .sym file that broke the importer. Basically, there doesn't seem to be any awareness of quoted strings in the few places this exposed trouble. Commas in enumeration items and spaces in variable names. I asked real quick over on #python and the recommendation for a lightweight (and correspondingly not-so-robust) solution was tea.shutil.split().

>>> mystring = 'a="1", b="2, 0", c="3"'
>>> mystring.split(',')
['a="1"', ' b="2', ' 0"', ' c="3"']
>>> [s.rstrip(', ') for s in tea.shutil.split(mystring)]
['a=1', 'b=2, 0', 'c=3']
>>> [(s[0], s[1]) for s in [s.rstrip(', ').split('=') for s in tea.shutil.split(mystring)]]
[('a', '1'), ('b', '2, 0'), ('c', '3')]

Has this already been dealt with elsewhere? Any opinions on how locally or globally you might like to have this addressed? I assume that there are big parsing/grammar libraries but I'm not sure either of us want to get into that.

dbc export error

while exporting dbc I get following error.
Seems to be some type missmatch

File "convert.py", line 68, in convert
ex.exportDbc(db, outfile, dbcCharset, dbcCommentCharset)
File "somepath\canmatrix\library\exportdbc.py", line 129, in exportDb
c
for (type,define) in sorted(list(db._globalDefines.items()), key=lambda x: i
nt(x[0])):
File "somepath\canmatrix\library\exportdbc.py", line 129, in

for (type,define) in sorted(list(db._globalDefines.items()), key=lambda x: i

nt(x[0])):
File "c:\Python27\lib\site-packages\future\types\newint.py", line 76, in new
__
return super(newint, cls).__new
(cls, val)
ValueError: invalid literal for long() with base 10: 'BusType'

python2 support

it seems that the python2 support is somewhat broken with current canmatrix.

given the file foo.dbc:

BO_ 262 foo: 8 Vector__XXX
 SG_ bar : 40|8@1+ (0.2,0) [0|0] "%" Vector__XXX

i can run canconvert under python3 just fine:

$ python /usr/bin/canconvert /tmp/foo.dbc foo3_.json
$

but doing the same with python2 fails:

$ python /usr/bin/canconvert /tmp/foo.dbc foo3_.json
[...]
Traceback (most recent call last):
  File "/usr/bin/canconvert", line 11, in <module>
    load_entry_point('canmatrix==0.4', 'console_scripts', 'canconvert')()
  File "/usr/lib/python2.7/dist-packages/canmatrix/convert.py", line 220, in main
    convert(infile, outfileName, **cmdlineOptions.__dict__)
  File "/usr/lib/python2.7/dist-packages/canmatrix/convert.py", line 40, in convert
    dbs = im.importany(infile, **options)
  File "/usr/lib/python2.7/dist-packages/canmatrix/importany.py", line 37, in importany
    dbs[key] = im.importDbc(infile, **options)
  File "/usr/lib/python2.7/dist-packages/canmatrix/importdbc.py", line 113, in importDbc
    receiver = list(map(str.strip, temp.group(11).split(',')))
TypeError: descriptor 'strip' requires a 'str' object but received a 'unicode'
$

should i spend time to fix this or is py2 officially unsupported (which is ok for me, i just want to know)?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.