Once timezone is set it cannot be cleared + antiquated pytz about meross_lan

[krahabb]

Thank you @jaytea33 ,
the 'empty' TimeZone not updating is 'by design' but obviously in need of rethinking.
The idea was: TimeZone 'blanked' in configuration -> don't configure it or touch in any way
Now this is ok when you start blank but if you want to go blank in the device there's no option to do that. I'll refactor this a bit

As per pytz I remember choosing that over dateutil because it seemed more fit at the time but I'll check and port my code to dateutil anyway based on your report since I know for sure (some users issues) pytz might not be there and I don't expose any dependency in my manifest so my timezone code works only when pytz is present failing gracefully if not.

[jaytea33]

Thanks krahabb, looking forward to the eventual update (no rush, I realize you do this on your own personal time). I'll leave my environment as-is currently so I can confirm the issue is fixed once it's implemented.

Also, regarding pytz, I missed it before but it seems "dateutil" has been removed as well and that "zoneinfo" should be used instead. The bottom of the link above shows the following:

"...Python 3.9 includes upgraded timezone handling and we should use that instead. With the help of Nick and Paul python-dateutil has been removed again and zoneinfo is used instead"

[krahabb]

Yeah, I've read that too after your comments. I had the time to look back at my code and tried to port it to zoneinfo but there's a 'special' info in pytz I'm using and that's why I decided for it in the first place..let's see
I too would prefer to 'align' and don't rely on an external library but that feature I'm using from pytz is not clearly available in zoneinfo (even dateutil for the matter lacks it)

[jaytea33]

Hi Krahabb,

Thanks for the quick work on this. Unfortunately the device is still reaching out via Port 123, I'm starting to wonder if the MSG100 requires it. The timezone does now appear blank though (debug log attached)

msg100-1632248450.csv

Here are the blocked NTP requests from my router's logs:

DST=72.30.35.88 LEN=76 TOS=0x00 PREC=0x00 TTL=254 ID=15119 PROTO=UDP SPT=61910 DPT=123 LEN=56
DST=203.107.6.88 LEN=76 TOS=0x00 PREC=0x00 TTL=254 ID=15117 PROTO=UDP SPT=61910 DPT=123 LEN=56
DST=204.11.201.12 LEN=76 TOS=0x00 PREC=0x00 TTL=254 ID=15111 PROTO=UDP SPT=61910 DPT=123 LEN=56
DST=38.229.58.9 LEN=76 TOS=0x00 PREC=0x00 TTL=254 ID=15109 PROTO=UDP SPT=61910 DPT=123 LEN=56
DST=129.250.35.250 LEN=76 TOS=0x00 PREC=0x00 TTL=254 ID=15121 PROTO=UDP SPT=61910 DPT=123 LEN=56

Note that I have tried rebooting the router, rebooting the AP, reloading the integration, rebooting entire HA host, and killing power to the device via my home's power panel, all with the same result. If you have any suggestions, please let me know, but I've re-allowed outbound UDP port 123 for now again so that it doesn't spam NTP requests.

Update: I read somewhere that the device needs to connect to NTP on bootup, so I tried killing power to the device and turning back on, letting the 1 NTP packet through and then reblocking Port 123. After 1 hour, same result - NTP spam. I then switched the Timezone to America/Los_Angeles and did the same thing - let the first NTP packet through, reblocked, and after 1 hour, NTP spam again.

Here is the log result with the America/Los_Angeles timezone on bootup:

2021/09/21 - 14:19:31 mqtt GETACK Appliance.System.All {"all": {"system": {"hardware": {"type": "msg100", "subType": "us", "version": "3.5.0", "chipType": "mt7682", "uuid": "", "macAddress": ""}, "firmware": {"version": "3.2.3", "compileTime": "2021/06/16 15:35:05 GMT +08:00", "encrypt": 1, "wifiMac": "", "innerIp": "", "server": "", "port": "", "userId": ""}, "time": {"timestamp": 3, "timezone": "America/Los_Angeles", "timeRule": [[1615741200, -25200, 1], [1636304400, -28800, 0]]}, "online": {"status": 0}}, "digest": {"togglex": [{"channel": 0, "onoff": 1, "lmTime": 0}], "triggerx": [], "timerx": [], "garageDoor": [{"channel": 0, "open": 0, "lmTime": 1632242233}]}}}
2021/09/21 - 14:19:31 auto LOG WARNING MerossDevice(UUID Redacted) has incorrect timestamp
2021/09/21 - 14:19:31 auto LOG DEBUG MerossDevice(UUID Redacted) timedelta = 1632259168
2021/09/21 - 14:19:31 mqtt PUSH Appliance.System.Clock {"clock": {"timestamp": 1632259171}}
2021/09/21 - 14:19:32 mqtt PUSH Appliance.System.Report {"report": [{"type": "1", "value": "0", "timestamp": 1632259171}]}
2021/09/21 - 14:19:32 mqtt PUSH Appliance.GarageDoor.State {"state": [{"channel": 0, "open": 0, "lmTime": 1632242233}], "reason": {"bootup": {"timestamp": 1632259171}}}

Man this device is stubborn! Potential solution ideas:

1. DNS rule to redirect NTP requests (see albertogeniola/MerossIot#142)
2. Somehow have the addon push "Appliance.System.Clock {"clock": {"timestamp":" once every hour so that the device can receive its precious friggin timestamp, leveraging local NTP (i.e. Chrony)
3. Remove the Default Gateway from the device so that it only communicates on layer 2 and hope maintains the IP that's currently set as a sort of pseud-static config (dangerous, but worst come to worst, factory reset). If the Default Gateway can't be removed, somehow set it to an unused IP outside of the DHCP scope so that it can't reach out to layer 3. If this is possible, I'm not sure how it would affect the device and may halt booting.
4. Put the device on a VLAN without internet access and hope it doesn't complain

[krahabb]

I guess we cannot set a 'definitive' rule/behaviour on how these devices work.
Personally, in my initial testings, after reading albertogeniola/MerossIot#142, I thought I 'had' to let NTP flow through my firewall and that's what I did. Then, after a while, by error, I understood some devices were booting and working almost correctly even when blocked on a full insulated LAN.

I've never checked what they are doing at the network level since ...they were working good anyway and I was not really interested in a full 'regression' especially since, like I said, every device/firmware might introduce changes.

Also tbh there a lot of devices beside these Meross, trying to 'spam' (it's not really spamming: it's just trying to connect to some server, sometime, someway). As far as they're blocked, they're not 'overloading' the network or the router, and the device keeps working I don't think that's an issue per se

Like @albertogeniola suggests, it would be ideal to configure the router to 'intercept/redirect' these NTP requests to a local NTP service (the router itself) and maybe there are routers able to easily be configured these way. I have a FritzBox and, even though I expected it was able to implement a sort of automatic filtering-redirection of NTP it is not so I'm left with some manual tricks which I never managed to implement...

I would 'attach' to your list of possible fixes also these 'tricks':

• customize the router routing table so to direct those external NTP servers IP to a local machine (not sure if it would work without also 'masquerading' these IP's)
• better: if the Meross are 'resolving' standard NTP server names before actually trying to connect to them (I only see logs for NTP so I don't know if the DST addr is fixed in firmware or resolved by DNS before connecting), you can likely configure your LAN to provide basic DNS (it is almost always a good idea though for a lot of reasons) and configure it to 'redirect' those public server names to (again) a locally provided NTP service (like the router itself). This is also the idea suggested by @albertogeniola and I find it the most elegant and functional: just a bit of network configuration with your own DNS server

[jaytea33]

I was able to redirect "pool.ntp.org" on my edgerouter 4 by setting a DNS rule to point to my HA server running Chrony and I'd agree that's probably the easiest workaround. Problem is Chrony by default uses "pool.ntp.org" so it would essentially be a loopback if left that way, so I changed Chrony itself to use "time.windows.com" and now the MSG100 is no longer desperately reaching for NTP. I do realize it doesn't affect the device or cause issues but it realllly tries to reach out and while I have the router set to Drop rather than Reject for performance, I just hate seeing a device I've attempted to block all internet access to try to reach outside the LAN lol. It won't be a great fix for everyone but for me, this DNS rule suffices for my needs.

[krahabb]

I just hate seeing a device I've attempted to block all internet access to try to reach outside the LAN lol.

I totally agree. I too would like to have everything smooth and clean but...sometimes it is just not possible (with a reasonable effort)
I think this 'dns redirect' is quite a nice trick and it could come to be useful for many other behaviours too (think of ip cameras which also use NTP: I have some on a private LAN and their time is random...still have to fix them since I don't managed yet to setup my private dns)
Thank you for confirming this is a viable scenario and keep up!

[Joshndroid]

I also noticed that the devices spam the ntp logs in the dns server. I have done as suggested and basically rerouted the ntp to a local domain running a slighly different pool... curious on why it needs to hit it so many times though