Unable to communicate with Edge modules (in EFLOW) from the host OS about iotedge

[nyanzebra]

@bhjertaas would you mind providing a support bundle? The command is iotedge support-bundle.

From what I understand, there is no issue retrieving the metrics via curl from EFLOW vm side, but when trying from host machine side it does not work. Is this correct?

Additionally, is this a recent issue? Or something that has been happening?

[bhjertaas]

Sorry for the delay, here is a support bundle.

support_bundle_2024_02_25_14_47_17_UTC.zip

You asked: From what I understand, there is no issue retrieving the metrics via curl from EFLOW vm side, but when trying from host machine side it does not work. Is this correct? ==> Yes correct.

This has been happening for a while. We have, as mentioned, 7 test-PCs that are set to reboot and we've been seeing this problem quite regularly. If you setup a PC with EFLOW, that reboots every 2 hours say, you should be able to reproduce this behaviour within two days I estimate. (of course you need something on Windows that makes HTTP calls to a module endpoint to detect when it happens).

According to our health-reporting system, non-communication problem to this particular test-PC-6, started Feb 25th 12:15 CET.

The IP at the time was 172.18.164.255 as provided by get-eflowVmAddr.

curl http://172.18.164.255:9602/metrics
curl: (28) Failed to connect to 172.18.164.255 port 9602 after 21047 ms: Couldn't connect to server

I've had a look at the bundle and can't find much. It is unfortunate that logs for the modules themselves are "too large". See my other issue on that. But I doubt this networking problem is caused by module wrongdoing anyway.

[nyanzebra]

@jagadishmurugan while edge team is investigating the support bundle is there anything to recommend to @bhjertaas to validate EFLOW to host networking. Just want to make sure that all looks correct.

[konichi3]

@jagadishmurugan @nyanzebra Can you follow up and share your finding?

[konichi3]

@jagadishmurugan @nyanzebra Any updates on this?

[nyanzebra]

@bhjertaas sorry for delay, this got lost amongst other issues I have been looking at. The support bundle provided seems to have failed to get logs Error grabbing logs: log message is too large (973735539 > 1000000). Would it be possible to run a smaller window of time support bundle? For example support-bundle --since 6h?

[bhjertaas]

I have tried to make a support-bundle with smaller time frame, but the "log message is too large" problem remains.

Until the underlying Moby bug is fixed, the recommended Local logging driver is not a viable option. Which log driver do you recommend instead on Edge devices? I'm just thinking that perhaps we will have to sort that out first, before we can get clarity on this issue.