Econolite Cobalt EOS about atspm

[shanejohnson-atspm]

Very interesting, thank you.
UDOT doesn't have EOS controllers yet. However, the decoder works with Cobalt controllers on the ASC3 and 2070 platforms.
As far as I know, UDOT does not currently have plans to add an EOS-specific decoder.

[Derek-Lehrke]

MnDOT did deploy its first ATC intersection with a Cobalt running EOS so I will be trying to get it working. My thought is to just copy the "NewDecodedandImportASCLogs" and run it separately.

I tried a couple of quick changes "**.dat*" and a new encoder with "*.datZ" and it did do any thing. I haven't dug into it too much but will over the next couple of weeks.

I added an example file if anyone feel like looking at it. I added the ".log" so github would load it

ECON_10.69.157.16_2018_05_10_1200.dat.log

[wgortney08]

If you need the EOS decoder, you can contact me at [email protected]<mailto:[email protected]> I have been the test guinea pig for EOS for a few years now. Thank you, Terry Gortney, P.E. Senior Engineer - Signal Systems Charlotte Department of Transportation

…

Sent from my iPhone On May 10, 2018, at 3:24 PM, Derek-Lehrke <[email protected]<mailto:[email protected]>> wrote: EXTERNAL EMAIL: This email originated from the Internet. Do not click any images, links or open any attachments unless you recognize and trust the sender and know the content is safe. MnDOT did deploy its first ATC intersection with a Cobalt running EOS so I will be trying to get it working. My thought is to just copy the "NewDecodedandImportASCLogs" and run it separately. I tried a couple of quick changes "**.dat*" and a new encoder with "*.datZ" and it did do any thing. I haven't dug into it too much but will over the next couple of weeks. [image]<https://user-images.githubusercontent.com/34098526/39889469-96890152-545d-11e8-8071-0d996e926c71.png> I added an example file if anyone feel like looking at it. I added the ".log" so github would load it ECON_10.69.157.16_2018_05_10_1200.dat.log<https://github.com/udotdevelopment/ATSPM/files/1992928/ECON_10.69.157.16_2018_05_10_1200.dat.log> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#38 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AYfzNWTbWtS_oOpZd7jDNd9vm3Gk_A37ks5txJPYgaJpZM4T6Qfr>.

[Chris-Sobie]

We are running into this issue in Albuquerque. We have the new Cobalt EOS decoder, and it's able to decode both .dat and .datz files. My thought was to swap it out with the existing "NEWDecodeandImportASC3Logs", but that didn't work. The decoder we received came with 1 .exe file and 2 .dll files. I'm relatively new to development so if anyone can lay out the steps to get the EOS firmware working, I would greatly appreciate it.

Thanks!

Chris Sobie
[email protected]

[shanejohnson-atspm]

"NEWDecodeandImportASC3Logs" is NEW because we wrote the translator directly in the program. That avoids the need to turn .dat files into .csv files and read the .csv files into the database. It reads the .dat files and converts them in memory, then imports the records to the database directly. UDOT has 1000+ asc3 controllers, so it was well worth our time to make that effort.

If we have to use external tools, like the EOS decoder, then the steps are similar to all of the "decode and import" programs we have created for the other controller types (and you should probably look at those for examples).

Roughly, here are the steps:
for-each through the directories that hold the log files.
for-each through the log files and use the EOS decoder to create a .csv file
Go back through all of the .csv files, add the signalID to each row (usually we get that from the directory name) This will create a complete event record.
Bulk-import the records into the database.
Delete the files (both the .dat and the .csv)

[Chris-Sobie]

Hi,
We are working on the .exe.config file for setting up the EOS decoder for ATSPM. We are using the following code to test the .config file but are having trouble getting the files to translate successfully. Here is the code we are currently using with "LogPath", "DecoderPath","LogsPath" and "CSVOutPath" pointing to local folder on the server to test the process. It would be great if someone can double check the code for any obvious omissions:

C:\Users\Administrator\Desktop\TestTranslator\ True C:\EOS\EOS\HighResTranslator\ False 1000 C:\Users\Administrator\Desktop\TestTranslator\ C:\DecodedSiemensLogs\ True False 10 Controller_Event_Log 2010-01-01 5000 10

[shanejohnson-atspm]

Chris,

Changing the .config file for an existing decoder program probably is not going to work. Most likely, you are going to have to write a new program. using one of the existing decoder programs as a pattern will probably help.

[Chris-Sobie]

Shane, I appreciate the email! Thank you. Christopher Sobie, EI SR Engineering Designer Lee Engineering, LLC [phone] phone [phone] mobile [email protected]<mailto:[email protected]> From: Shane Johnson [mailto:[email protected]] Sent: Thursday, June 21, 2018 11:34 AM To: udotdevelopment/ATSPM <[email protected]> Cc: Chris Sobie <[email protected]>; Comment <[email protected]> Subject: Re: [udotdevelopment/ATSPM] Econolite Cobalt EOS (#38) Chris, Changing the .config file for an existing decoder program probably is not going to work. Most likely, you are going to have to write a new program. using one of the existing decoder programs as a pattern will probably help. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#38 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AlH7bw5TFYUEERTACAEVgoyvGIoJ9mV8ks5t-9kRgaJpZM4T6Qfr>.

[gmonk]

The EOS datZ format is nothing more than a zlib comressed dat file. So if you take the dat file and run it through binwalk or zlib_flate on a linux workstation you will end up with a dat file that can be decoded

binwalk -eM ECON_172.22.251.14_2018_08_16_1700.datZ you will have a folder with _ECON_172.22.251.14_2018_08_16_1700.datZ inside two file 0.zlib and 0 the 0 file is the dat file
or using zlib_flate

zlib-flate -uncompress < ECON_172.22.251.14_2018_08_16_1700.datZ > exctracted.dat this will be the dat file you need

[udotdevelopment]

We are meeting with Traffop on Monday to review their changes to the asc3 decoder. They have adjusted it to allow for the new format.

[mandrewbot3k]

Just an FYI, our new Rackmount Cobalts are shipped with SW version 32.66.10, which also creates datZ files. I believe 32.65 does not, we are going to jump back in version to verify. My last 4 rackmount Cobalts I have received are running 32.66.10 though.

The interesting behavior though is it does save .dat files still, but it's intermittent behavior. I'm only bench testing so it's hard to tell what is really going on. I suspect that it only creates the dat when there is something actually logged. See below.

I'm very new to ATSPM and am utilizing the raspberry pi data logging method to do a pilot program here, so maybe I'm missing something.

[Traffop]

We have added support for decompressing the datZ files into the original decoder file in the last pull request. We use slightly different code in production therefore we recommend testing these changes before pushing it into your production environment.

[gault00]

As mandrewbot3k noted above, I also encountered a shelf-mounted ASC/3 today with firmware version 2.66.10 that is creating datZ files, so it's not limited to EOS software. My experience was the controller compressed the file at the end of the interval, and the file currently being recorded is in regular dat format. If the UDOT method of turning off and on the logger is used to not capture the same file twice, then it's possible the controller firmware isn't compressing the dat file when the logger is stopped. I haven't been using it that way.

[dbrandesky]

I just tested the "NEWDecodeAndImportASC3Logs.exe" package that comes with SPM 4.2.0, and can confirm that it works with .datZ files from an ASC/3 running 2.66.30 software. I'll give Eos a shot sometime soon, but I would expect success there as well.

[mandrewbot3k]

I just wanted to add in regards to the hi-res MOE logging on the ASC/3 platform, to make sure you are not running 32.66.10 with one minute hi-res MOE enabled (MM-1-6-1).

Upgrade to 32.66.20 or using a different interval should work. When set to one-minute, the controller may freeze up. I found this in a benchtop setup when my ethernet was disconnected, It appeared to not to buffer correctly when the logs were not being downloaded, despite circular being enabled. However, the release notes for 32.66.20 state this issue to be corrected, though I have not verified yet personally.

Earlier versions (such as 32.65.40) with logging enabled via SNMP or 32.66.20+ through the Logging menu (MM-1-6-1) should work fine otherwise.

We are running Rackmount Cobalts for reference. Cheers.

[dbrandesky]

The NEWDecodeAndImportASC3Logs.exe did not work with Eos log files, though the FTP uploader executable faithfully retrieves the files. I'm recalling now that there are some differences in the MOE data format for Eos, so that may take some dev work. If I have the opportunity I'll see if I can figure something out, but I am far from a Windows ASP.NET or JS developer so I may not be much help!

[shawngotfredson]

Which version of the FTP program should I be using with the newer Cobalts? The one I am using works fine with older ASC3 firmware but it is not able to delete files off the controller it downloads the .datz files.

[dbrandesky]

@shawngotfredson I don't know of any issues deleting files on Cobalts, and I don't know of different versions of the FTP application for Windows. If you haven't already, you might try connecting to the controller using the actual command-line FTP client and see if you can manually delete .datZ files.

[shawngotfredson]

I am able to delete the .datz files with a Windows FTP client. It is when I am trying to run FTPfromAllControllers that I get an issue. I am running ASC3 32.67.20.

It appears the directory structure may have changed as well on the newer firmware. I have to use /econolite/set1 vs just /set1. If I use just /set1 with the new firmware I get an error saying directory not found.

There is a chance I am using the wrong version of FTPfromAllControllers or I need to wait till development catches up. OR I maybe missing something simple. I have the FTPfromAllControllers running with my older firmware so no worries.