Monitor intermittently doesn't report readings about air_quality_monitor

[henare]

I increased the debug output and it didn't really help. All you could see was that the device couldn't connect to the APIs. It would say something like:

PM10:  10.55
PM2.5: 5.95
----
Temperature: 18.90⸮C
Humidity: 77.00%
----
Creating data string:
WLAN signal strength: -63 dBm
----
## Sending to luftdaten.info (SDS): 
Start connecting to api.luftdaten.info
connection failed
## Sending to luftdaten.info (DHT): 
Start connecting to api.luftdaten.info
connection failed
## Sending to madavi.de: 
Start connecting to api-rrd.madavi.de
connection failed
Time for sending data: 17153570

On a hunch I decided to see if it was just a problem connecting to those APIs. I set up RequestBin locally and added it as the custom API to my esp8266. I saw the following:

PM10:  12.73
PM2.5: 7.60
----
Temperature: 18.90⸮C
Humidity: 77.80%
----
Creating data string:
WLAN signal strength: -63 dBm
----
## Sending to luftdaten.info (SDS): 
Start connecting to api.luftdaten.info
connection failed
## Sending to luftdaten.info (DHT): 
Start connecting to api.luftdaten.info
connection failed
## Sending to madavi.de: 
Start connecting to api-rrd.madavi.de
connection failed
## Sending to custom api: 
Start connecting to 192.168.0.6
Requesting URL: /z4k0rqz4
818835
{"esp8266id": "818835", "software_version": "NRZ-2018-103", "sensordatavalues":[{"value_type":"SDS_P1","value":"12.73"},{"value_type":"SDS_P2","value":"7.60"},{"value_type":"temperature","value":"18.90"},{"value_type":"humidity","value":"77.80"},{"value_type":"samples","value":"580606"},{"value_type":"min_micro","value":"247"},{"value_type":"max_micro","value":"26623"},{"value_type":"signal","value":"-63"}]}

closing connection
----


End connecting to 192.168.0.6
Time for sending data: 18507031

Ah ha! So that narrows it down to:

a) A problem with the DNS resolution on the device (the custom API is using an IP address). This is easily testable by using a DNS address instead of IP for the custom API
b) A problem with the device accessing the internet (the custom API is on the local subnet). This seems implausible since the device can access the network and other devices can get beyond the router.
c) The upstream APIs have a problem (we can use our custom ones but they're failing). This should be verifiable by looking at other devices - do they also have periods of outages?

[henare]

I changed the IP address of the custom API endpoint to 192.168.0.6.xip.io so that it would do DNS resolution and it's started to fail when the device has it's intermittent outage.

This suggests it could be a problem with DNS resolution. But that doesn't make much sense. Everything else on the network can resolve properly.

I've switched it back to double-check this.

[henare]

I've updated the issue title as when there is one of these outages, I see the monitor failing over serial on my computer, but I can still access the device's internal webserver via my machine (so it's on the network/WiFi).

[henare]

OK this is weird. I just saw it fail to connect to luftdaten.info API but not the madavi.de API, or my local custom API:

PM10 (sec.) : 46.00
PM2.5 (sec.): 40.20
0 - 170 - aa - 170 .
1 - 192 - c0 - 192 .
2 - 148 - 94 - 148 .
3 - 1 - 1 - 1 .
4 - 210 - d2 - 210 .
5 - 1 - 1 - 1 .
6 - 216 - d8 - 216 .
7 - 160 - a0 - 160 .
8 - 224 - e0 - 224 .
Checksum is: 224 - should: 224
9 - 171 - ab - 171 .
PM10 (sec.) : 46.60
PM2.5 (sec.): 40.40
PM10:  45.70
PM2.5: 40.00
----
End reading SDS011
Call sensorDHT
Start reading DHT11/22
Temperature: 19.90⸮C
Humidity: 78.70%
----
End reading DHT11/22
Creating data string:
WLAN signal strength: -63 dBm
----
## Sending to luftdaten.info (SDS): 
Start connecting to api.luftdaten.info
connection failed
## Sending to luftdaten.info (DHT): 
Start connecting to api.luftdaten.info
connection failed
## Sending to madavi.de: 
Start connecting to api-rrd.madavi.de
Requesting URL: /data.php
818835
{"software_version": "NRZ-2018-103", "sensordatavalues":[{"value_type":"SDS_P1","value":"45.70"},{"value_type":"SDS_P2","value":"40.00"},{"value_type":"temperature","value":"19.90"},{"value_type":"humidity","value":"78.70"},{"value_type":"samples","value":"585774"},{"value_type":"min_micro","value":"238"},{"value_type":"max_micro","value":"519213"},{"value_type":"signal","value":"-63"}]}

closing connection
----


End connecting to api-rrd.madavi.de
## Sending to custom api: 
Start connecting to 192.168.0.6
Requesting URL: /z4k0rqz4
818835
{"esp8266id": "818835", "software_version": "NRZ-2018-103", "sensordatavalues":[{"value_type":"SDS_P1","value":"45.70"},{"value_type":"SDS_P2","value":"40.00"},{"value_type":"temperature","value":"19.90"},{"value_type":"humidity","value":"78.70"},{"value_type":"samples","value":"585774"},{"value_type":"min_micro","value":"238"},{"value_type":"max_micro","value":"519213"},{"value_type":"signal","value":"-63"}]}
HTTP/1.1 200 OK
Server: gunicorn/19.8.1
Date: Wed, 04 Jul 2018 08:27:40 GMT
Connection: close
Content-Type: text/html; charset=utf-8
Content-Length: 3
Sponsored-By: https://www.runscope.com

ok

closing connection
----


End connecting to 192.168.0.6
Time for sending data: 12578529
Call sensorSDS
Start reading SDS011
End reading SDS011

Ah, it's because it's actually down right now: https://downforeveryoneorjustme.com/api.luftdaten.info

[henare]

Overnight I set the device's custom API to be a RequestBin on the public internet. The results were plenty of outages, and all (that I could see) included the custom API.

This is not good and I'm not quite sure where to go from here. It's probably worth isolating my local network as the problem by connecting the device to another network. I could use my mobile phone as an AP over night and see what happens.

I've also checked out some other sensors and it does look like other sensors exhibit this problem. It might be time to try and contact the upstream project to see if they know anything about it and potential solutions.

[henare]

I've updated the issue title as when there is one of these outages, I see the monitor failing over serial on my computer, but I can still access the device's internal webserver via my machine (so it's on the network/WiFi).

Right now it's doing the intermittently not reporting thing:

But it's definitely not connected to WiFi, according to my access point. I can't ping it either, of course.

I think it's still worth connecting it to another network, like my phone's AP, but the problem is clearly with the device ☹️

[henare]

A wonderfully helpful person on the upstream project suggested the power supply might be the issue.

At around 2018-07-11 22:00 GMT I changed the power supply to a dedicated new one I just bought (previously I was just using a hokey old phone charger as the power supply). Let's see what happens 🤞

[henare]

Changing the power supply didn't work:

And coming to think of it, I've observed the problem when the device was connected to my laptop for debugging (and therefore using the power from the laptop). In other words, I've already used 3 different power supplies.

I guess there's a small chance it could be the cable. I've put another dodgy cable on it in case (it was Fri 13 Jul 01:22:13 UTC 2018 when I did this change).

[henare]

I've finally had some time to do the separate network test. The device is now (Tue 17 Jul 06:59:33 UTC 2018) connected via my phone's hotspot. I'll check in the morning to see if we've had any change.

[henare]

Progress! 🚂 I can now confirm it's the interaction of the device and something on my home network.

You can see the before above and here's the after:

As you can see, I ran the device on my phone's hotspot for almost 24 hours and had no outages at all during that time.

The OP on opendata-stuttgart/sensors-software#200 also said changing to another network fixed their problem so I strongly suspect there's something about the device that has an intermittent problem when connecting to some WiFi networks (like me and the OP both have at home).

My first thoughts are some fancy WiFi modes or bands. First up I'm going to set my home AP to 54 Mbps only. I think this means only use 802.11g. Then we can try bands or something.

[henare]

We're back to outages so that didn't help:

I've just done a firmware update on my router at Thu 19 Jul 00:40:21 UTC 2018.

After doing the update I noticed that the router's DNS resolver didn't come up. Could it be that that is flakey? I can't easily test that because the crappy router doesn't allow me to change the DNS server that's configured via DHCP 😕

[henare]

Then we can try bands or something.

My phone was set to 2.4 Ghz and that's all my router does so it's definitely not that.

There are a maximum of 2 nearby networks and they're on different channels so it's not that either.

As per my previous comment I'm starting to think it's higher up the stack, potentially that DNS problem.

[wendybacon1]

Interested to know how this is going? have these monitors been tried yet in Australia in a different location?

[henare]

This problem solved itself for me when I got a new WiFi access point.