christoph2 / pya2l

ASAM ASAP2 Library for Python

pyA2L

pyA2L is an ASAM MCD-2MC processing library written in Python.

ASAM MCD-2MC, also known as ASAP2, is a non-XML file format for defining calibration parameters, measureable variables, and communication interface specific parameters, widely used in automotive applications.

ASAP2 is typically used together with CCP (CAN Calibration Protocol) or XCP (Universal Calibration Protocol).

Supported Versions: 1.6

Installation

• Via pip (Currently only Windows and MacOS):

  $ pip install pya2ldb

  IMPORTANT: Package-name is pya2ldb NOT pya2l!!!

• From Github:

  • Clone / fork / download pyA2Ldb repository.
  • Make sure you have a working Java installation on your system, like AdoptOpenJDK or OpenJDK.
  • Download and install ANTLR 4.9.3:
    • curl -O -C - -L https://www.antlr.org/download/antlr-4.11.1-complete.jar
    • Add ANTLR to your CLASSPATH environment variable, e.g.: export CLASSPATH=$CLASSPATH:~/jars/antlr-4.9.3-complete.jar (you may put this in your .bashrc, .zshrc, ...).
  • Run setup-script: python setup.py develop

Getting Started

---

pyA2L is part of pySART (Simplified AUTOSAR-Toolkit for Python).

pya2l's Issues

Documentation for using the A2L packages

Please share the documentation for using the pya2l package. also if possible please provide few examples for how to use it for merging the a2l files.
Thanks.

Token recognition error

I'm trying to parse an a2l file using pyA2L library.

Here's a snippet:

  /begin MOD_PAR "D915E_N46A610"
    VERSION "[REDACTED]"
    ADDR_EPK 0x1C21F0
    EPK "[READACTED]"
    CUSTOMER_NO "5"
    USER "GS-ES/EAD1-Richard"
    PHONE_NO ""
    CPU_TYPE "GoldenOak"
    ECU "[REDACTED]"
  /end

When using db.import_a2l to parse a file containg the snippet above I get a bunch parsing errors (line 880 is CUSTOMER_NO):

[INFO (pya2l.Preprocessor)]: Pre-processing 'definitions.a2l'[UTF-8-SIG]
[INFO (pya2l.DB)]: Parsing pre-processed data ...
definitions.a2l::line 880:17 extraneous input '5' expecting {'VERSION', 'ADDR_EPK', 'CPU_TYPE', 'CUSTOMER', 'CUSTOMER_NO', 'ECU', 'ECU_CALIBRATION_OFFSET', 'EPK', 'NO_OF_INTERFACES', 'PHONE_NO', 'SUPPLIER', 'SYSTEM_CONSTANT', 'USER', '/begin', '/end'}
definitions.a2l::line 881:12 token recognition error at: '-E'
definitions.a2l::line 881:15 token recognition error at: '/E'
definitions.a2l::line 881:20 token recognition error at: '-R'

Is there a problem with the library or do I need to fix my a2l file?

Configure A2L parser from JSON file.

tool cannot create db when long names are having multiple quotes

the issue

line 53940:95 mismatched input '"aktiv"' expecting {'ASCII', 'CURVE', 'MAP', 'CUBOID', 'CUBE_4', 'CUBE_5', 'VAL_BLK', 'VALUE'}
line 53952:5 mismatched input 'CHARACTERISTIC' expecting 'MODULE'

when i checked the a2l file the long name was

/begin CHARACTERISTIC

DestfRst_C
"Abschaltung  ""aktiv"" zu ""nicht aktiv"""
VALUE
0x91C6000
Wu8
1.00000000000
compu_VERB
0.00
1.00000000000

FORMAT "%1.11"

/end CHARACTERISTIC

Generate a2l file

hello，
We want to generate intermediate a2l based on source code or configuration files, rather than modifying or merging based on a2l files. Can this tool be implemented? If it can be implemented, do you have any usage examples or explanatory documents? thanks

Suggestion: Remove autogenerated code

This project currently includes a lot of ANTLR4-generated code under the py3 package. This creates unnecessary code churn when upgrading ANTRL. It may also hide bugs if CI-servers regenerate these files as part of the build process, which AppVeyor does.

I propose that the generation of parsers and lexers be moved to build-time. This would create an build-time dependency on ANTLR. It would not affect existing unit tests, since AppVeyor already does this and Travis is broken anyway.

@christoph2 If you think this sounds like a way forward, I can prepare a pull request.

Parse an xml/arxml to create intermidiate A2L

Hello Christoph,

I have 2 queries about using the tool,

1. I am actually trying to convert an arxml to an intermediate A2L file. Do you think this library can support this requirement?

2. Can there be documentation of the APIs provided in the library, to understand how each API can be used or certain(variety) use-cases so as to cover all the usage? [ I did come across the documentation however isn't exhaustive ]

Message type for asap version

Hello! We were wondering if this should be a warning or an error?

pyA2L/pya2l/a2l_listener.py

Line 338 in ca54775

self.error("ASAP2 Version '{}.{}' may not parsed correctly.".format(versionNo, upgradeNo))

We say warning because we don't stop or as far as I can see and since it's something that can happen, not which has happened?

Logger shall output filename and linenumber.

Parse xcp_on_can

Hi thanks for you pyA2l.

is it possible to read the xcp_on_can daq entries of my a2l file? i have the following structure in the file and would like an xcp_on_can table in the database with the following information.

/begin XCP_ON_CAN
0x0100 /* XCP on CAN version /
CAN_ID_BROADCAST 0x0100 / Broadcast /
CAN_ID_MASTER 0x0200 / CMD/STIM /
CAN_ID_SLAVE 0x0300 / RES/ERR/EV/SERV/DAQ /
BAUDRATE 500000 / BAUDRATE /
/begin DAQ_LIST_CAN_ID
0x0000 / for DAQ_LIST 0 /
FIXED 0x310
/end DAQ_LIST_CAN_ID
/begin DAQ_LIST_CAN_ID
0x0001 / for DAQ_LIST 1 /
FIXED 0x320
/end DAQ_LIST_CAN_ID
/begin DAQ_LIST_CAN_ID
0x0002 / for DAQ_LIST 2 */
FIXED 0x330
/end DAQ_LIST_CAN_ID
/end XCP_ON_CAN

Greetings

Extremely slow performance when running in test

Fact:
When running this lib using robot-framework tests, the pya2l.DB.import_a2l() slows down drastically.

Info:
general info

• I have a couple of CAN and XCP tests. These tests interact with a hardware which receives and sends CAN signals.
• Between the master(PC) and slave(target device) there is a 3rd hardware: neoVI Fire 2. The communication goes thru this device.
• The robot tests are placed in separate directories(got 5 tests, each of them inside separate directory)
• The tests ran with an automation test tool
• If the corespondig tests are placed inside one directory and it is run using -m robot, the result/problem is the same.

test info

• I have 4 robot tests which checks/ sends CAN signals to the hardware.
• I have 2 XCP tests(one of them is pure XCP, the second one uses XCP and CAN, meaning it uses python-can lib. to open bus and it uses pyxcp to open XCP connection)

test run info

• When running the 2 XCP related tests, it runs without a problem.
• When running only CAN related tests, it runs without a problem.
• When running 3 CAN related tests and the 2 XCP related tests, then at the second XCP related test it takes a lot of time for pya2l.DB.import_a2l() to import a2l file/create a database.
• When running 2 CAN tests, then 2 XCP tests, it runs without a problem.

Other info

• Running open_create() and/or using the remove_existing=True option in import_a2l() produces the same result(slowness)

conclusion

• It seems, that the more CAN/XCP is ran on a single "shot", at once, one after another, then the process of importing/creating DB becomes extremely slow.

Create dynamic AML parser.

A2L parsing error due to the preprocessing the multipline comments

In the preprocessing of multiple line comments as following lines, all this COMPU_METHOD will be removed. It will cause the error in the lexcer paring. so either

1. don't preprocess the comments, because the lexcer will first channel hidden to remove the comments
2. keep the text after / when / and */ at the same line.

/begin COMPU_METHOD
/* Name of CompuMethod / SWC_JctTempEstm_CM_single
/ Long identifier / "Q = V"
/ Conversion Type / RAT_FUNC
/ Format / "%8.6"
/ Units / ""
/ Coefficients */ COEFFS 0 1 0 0 0 1
/end COMPU_METHOD

/begin COMPU_METHOD
/end COMPU_METHOD

Use pyA2L for export to excel

I'm trying to use pyA2L for exporting the A2L file into excel file. Is there an API for that?
Or could you help me with a sample implementation?

A2l.g4 is not match the code in py3 folder

I'm trying to generate the python file using Antlr4, but seems there is some difference which I generated in py3 folder, would you please check?
And what is the a2l.py for, looks like generated by Antlr4, but not in py3 folder?

python setup.py install failed

python 3:

C:\python3\python>python D:\01_Project\asap2\pyA2L\setup.py install
running install
running bdist_egg
running egg_info
writing pya2l.egg-info\PKG-INFO
writing dependency_links to pya2l.egg-info\dependency_links.txt
writing requirements to pya2l.egg-info\requires.txt
writing top-level names to pya2l.egg-info\top_level.txt
warning: manifest_maker: standard file 'setup.py' not found

**_error: package directory 'pya2l' does not exist_**

Autodetection of a2l file encoding is not accurate

=============================
if encoding is not None:
warnings.warn("Don't use parameter encoding anymore -- file encoding is autodetected now.", DeprecationWarning, stacklevel = 2)

encoding = detect_encoding(self._a2lfn)

The implication is that the 'encoding=' parameter has been deprecated, however the actual behaviour of the code is to override it. I have an a2l file which I believe is Windows-1252 (or might be ISO-8859-1) generated by a vector tool. However the auto-detection does not work and pya2ldb is unable to parse the file without generating an error.

Is it possible to reinstate the encoding parameter. My understanding is that, in general it is not possible to autodetect text encoding reliably.

Implement A2L include mechanism.

A2L Parsing error when COMPU_METHOD name has square brackets in middle

When the conversion name for a dataitem has square-brackets in the middle of it, the parser seems to assume that that's the end of the name.
When I read the conversion of a dataitem like this, it only includes everything up to the closing square-bracket.
Snippets:

From the A2L file:
/begin MEASUREMENT ***.******_12 "" UWORD ***.******[12]_******* 0 0.0 0.0 65535.0 ECU_ADDRESS 0x******** DISPLAY_IDENTIFIER ***.******[12] /end MEASUREMENT

An error message from when the parser was running (after calling db.import_a2l(...))
**************.a2l::line ****:** extraneous input '_*******' expecting {INT, HEX}

Taking signals from A2L file

Hello! I have a question for you about using A2L file.
Is possible to import A2L file in Python code for controling signals while ECU is connected?
I was working with CAN communication and using cantools for import, control and decode signals from DBC file, now, I am facing with XCP and I need to know whether the similar possible with A2L file.
If your answer is yes, please provide me some example how to do this. If not, please let me know.
Best regards and thank you in advance!

ModuleNotFoundError: No module named 'pya2l.py3'

Hi @christoph2,

Since there is no information related to installation, after cloning the repo I used the usual command to setup python package from souce:
python setup.py install

Looks like all went well. When I tried to test the package with the built-in test you have @ pyA2L/pya2l/tests/testA2LAcceptance.py

It gives me a ModuleNotFoundError: No module named 'pya2l.py3'

Below is the log,
$ py test.py Traceback (most recent call last): File "test.py", line 617, in <module> parser.parseFromString(TEST_A2L) File "D:\Python\36\lib\site-packages\pya2l-0.1.0-py3.6.egg\pya2l\a2lparser.py", line 265, in parseFromString File "D:\Python\36\lib\site-packages\pya2l-0.1.0-py3.6.egg\pya2l\a2lparser.py", line 268, in parse File "D:\Python\36\lib\site-packages\pya2l-0.1.0-py3.6.egg\pya2l\aml.py", line 63, in __init__ File "D:\Python\36\lib\site-packages\pya2l-0.1.0-py3.6.egg\pya2l\aml.py", line 69, in _load File "D:\Python\36\lib\importlib\__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 978, in _gcd_import File "<frozen importlib._bootstrap>", line 961, in _find_and_load File "<frozen importlib._bootstrap>", line 936, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 978, in _gcd_import File "<frozen importlib._bootstrap>", line 961, in _find_and_load File "<frozen importlib._bootstrap>", line 948, in _find_and_load_unlocked ModuleNotFoundError: No module named 'pya2l.py3'

Am I doing something wrong with the setup? Your response will be highly appreciated.

Parser fails on multidimensional arrays

DATA='''/begin CHARACTERISTIC TEST[0][0].TEST
         "TEST[FL,0].TEST"
         VALUE
         0x0003237C
         _HELLO
         30
         _TEST
         -18
         12
         DISPLAY_IDENTIFIER TEST[FL_0].TEST
         FORMAT "%3.1"
      /end CHARACTERISTIC'''

parser = ParserWrapper('a2l', 'characteristic', A2LListener, debug = False)
session = parser.parseFromString(DATA)

Fails with:
line 1:29 mismatched input '[' expecting STRING

Probably caused by:

partialIdentifier:
    i = IDENT (a = arraySpecifier)?
    ;

in a2l.g4