orolia2s / oscillatord Goto Github PK

Tried to run the oscillatord v3.3.9 on CLS Time Card to configure the new u-blox module, however it comes up with permission error. Please refer to the attached log file.
oscillatord config ublox fail log.txt

@jachenilleau @leoleovich

Oscillatord 3.4.5 is segfaulting

After updating to Oscillatord 3.5 we can no longer get the "disciplining" parameters for CLS time card. We check if one of the sub parameters "ready_for_holdover" is true or false and based on the result we start the holdover test or continue disciplining the Time Card.
Even though we set "disciplining: false" in oscillatord config file, can the "ready_for_holdover" parameter be added regardless of the config file parameters?

We constantly observe a problem with oscillatord - it show huge offset just before entering Holdover, which creates phase offset and provides wrong timestamp on ptp device. You can find this behaviour in logs, and the diff is visible in ts2phc logs
oscillatord.log.txt
ts2phc.log.txt

Hi guys,
We noticed an issue when a GNSS antenna signal is instantly lost (disconnected or died) we update the number of satellites, but not the survey in error which stays on the previous level forever, giving an impression that everything is fine:

06:22:11 DEBUG GNSS data: Fix no fix (1), Fix ok: False, satellites num 0, survey in error: 1.07, antenna status: 4......

Ideally, the survey in should be updated to reflect the issue.
Thank you!

It takes a while to start monitoring socket after oscillatord restart and until it's up we see "connection refused" which is easy to confuse with oscillatord being down.
Please make sure we have monitoring socket up at an earlier stage.

We see a situation when oscillatord doesn't report any errors, but at the same time /dev/ptp2 returns time in 1970

In 3.4.4 these values are missing when mRO-50 is initializing. We have to have these values populated, even if these are just defaults.
Please always keep compatibility with the tools relying on the API https://github.com/facebook/time/blob/main/oscillatord/monitoring.go#L174-L189

• Condition for oscillatord to be initialized properly are too strict, with no pps gnss we are locked in a loop until we have one, so (for instance), as a side effect, oscillatord main loop never start and some of the features aren't working properly, here we noticed that gnss is not connected to shared data exposed by monitoring-socket so ubx data are fetched but monitoring keeps sending dummy/default values.

• We need to segregate what's about disciplining, which may indeed stay blocked in init without good conditions, and what's about basic daemon feature (here, monitoring and exposing system status properly), which should always be working even with poor conditions.

To fix GNSS case, a good start could be modifying/moving/deleting the loop (do ... while ...) line 386 in oscillatord.c and adapt the code consequently, there are probably other similar issues in the code.

In case, we wait gnss data for init, the software can't react to SIGNAL:

12:09:46 DEBUG Phasemeter: GNSS, ts 1679051385507904405
12:09:46 DEBUG Phasemeter: INT , ts 1679051386000000000
12:09:46 DEBUG Phasemeter: phase_error: -492095595ns
12:09:47 DEBUG UBX-TIM-SVIN: dur: 814, meanX 420386803, meanZ 16332503, meanZ 477796615, meanV 348072, obs 790, valid 0, active 1
12:09:47 DEBUG GNSS data: Fix 3D (5), Fix ok: True, satellites num 32, survey in error: 0.59, antenna status: 2, valid 1, time 1679051387, leapm_seconds 18, leap_notify 0, lsChange 0, timeToLsChange -40993787, lsSet: True, QErr(n) 0, qErr(n-1) 0
12:09:47 WARN  Could not tai time from gnss, please check GNSS Configuration if this message keeps appearing more than 25 minutes

Probably in init state we can't kill the soft properly otherwise init can last forever whitout proper gnss conditions orwhitout mro lockness

Finally there is a timeout but maybe we could do better
[email protected]: State 'stop-sigterm' timed out. Killing.

listen_inet_socket() uses inet_addr() to resolve string to sockaddr_in. This limits us to use IPv4 addresses only. This is hard limitation in 2023, let's remove it and use proper interfaces like getadrinfo()

Currently, after a restart of oscillatord we enter uncalibrated (52) clock class and then transition to holdover (7) while fine disciplining is in progress. We should transition to calibrating (13) instead.

It seems like art_integration_in_server_test version using serial interface to mRO50 is currently broken. Here is the observation summary:

Jul 26 06:15:15 srv oscillatord[117392]: 06:15:15 #033[33mWARN #033[0m Error parsing gnss-cable-delay value

Missing option should act as 0 IMHO.
Currently setting to 0 doesn't send an update (I had to set 1).

Right now it takes up to 12 minutes to reconfigure GNSS receiver and get first signal from it. But actually we need no reconfigure it once after host power up. It would be great if oscillatord could check GNSS receiver configuration at start up and apply only difference to it. It will greatly reduce the speed up time in case of software restart

The track for the issue when mRO50 is drifting too much and oscillatord filters out all the points instead of disciplining atomic clock.