(ESP32) Ethernet support about ems-esp32

[bbqkees]

I had a look at this a while ago because every once in a (long) while I get a question about Ethernet support for the Gateway.

I believe the ESP-IDF supports both the LAN8720 and the TI TLK110.
The TLK110 will be discontinued and it's an expensive chip. The LAN8720 is more common.
There are several options for the LAN8720 with those Chinese Ethernet proto boards but some require reworking the PCB.
Most LAN8720 board are clones of the Waveshare board.
Aside from the two above there is also the very cheap ENC28J60 and the Wiznet W5500.

Feature could be nice to have in the end, but I don't think there is massive demand for it (yet).

[crocchietta]

For me, WiFi is a last resort. I prefer to go first through wired connections.
So the support of an Ethernet connection is very good news. And you will most likely be faster than me to implement it!

[glitter-ball]

PoE as another powering option then... 🤣

[bbqkees]

@glitter-ball POE is fine for devices that are not electrically connected to anything else but with EMS-ESP it's also connected to a bus so you need additional galvanic isolation.
But it can be done if you really want. Just take the circuit board out of the Gateway or use an EMS interface board and connect it to a board like this: https://www.olimex.com/Products/IoT/ESP32/ESP32-POE-ISO/.

[glitter-ball]

I suggested PoE as a joke, but actually there might be some people who would use it, especially if it avoided having to get a USB power supply connected and powered. Isolation would be important.

I've got no Ethernet near my boiler, though, so WiFi it has to be.

[crocchietta]

@proddy : do you have an idea of ​​a date for the availability of this option?

[proddy]

@crocchietta soon I hope. The code is done, I just don't have a working ESP32 with Ethernet board to test it on. My olimex ESP32-EVB broke

[proddy]

First implementation done in branch https://github.com/proddy/EMS-ESP/tree/fb-632-refactorvalues

Supporting two profiles for now, the common Waveshare LAN8720 PHY and Espressif's Ethernet board's with a TLK119 PHY.

Let me know how you get on.

[crocchietta]

I only have one ENC28J60 available. I will try to find something else (olimex ESP32-EVB 50€) but difficult to get interesting prices in France.

[bbqkees]

Just get a LAN8720 board from Ali for like 3 Euro. Needs a bit of modification though.

[proddy]

Note @crocchietta that I've coded this only for ESP32. From what I've read people suggest to stay away from W5100 and W5500 ethernet shields from China. And ENC28J60 chips are even more unreliable with poor quality regulators and TCP/IP buffer overflows.

[proddy]

At some point also might consider adding support for IP101 - espressif/arduino-esp32#3554 (comment)

[crocchietta]

I bought this LAN8720 module for testing. : https://www.ebay.fr/itm/Module-LAN8720-Ethernet-RMII-LAN-10-100-Network-3-3V-RJ45-IoT-PIC-RPI-STM-ESP/323278761945?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2057872.m2749.l2649

What pins should I use on the esp32?

[proddy]

google it. try a sketch and see if it works before going to EMS-ESP. You can configure any pins you want. A good starter is https://easyeda.com/Antongsm/esp32-lan8720. Important is the gpio0 for the clock. Default mdc is 23 and mdio is 18. It's not plug&play so you'll need to do some work. There are plenty of examples on the web to follow.

[crocchietta]

Hello,
Finally I ordered an EMS-EVB card. Is there a particular configuration to do at the level of EMS-ESP?
Is it just the branch https://github.com/proddy/EMS-ESP/tree/fb-632-refactorvalues ​​that implements ethernet?

[bbqkees]

If I am correct the default configuration for Ethernet in EMS-ESP V3 is right for the Olimex board(s).
LAN-8720, ETH_CLOCK_GPIO0_IN, ETH_POWER_PIN = -1, ETH_ADDR = 0, ETH_MDC_PIN = 23, ETH_MDIO_PIN= 18.

Strap your pc because EMS-ESP with Ethernet is lightning fast.

[proddy]

Choose 'Profile 1' from "Network Settings/Ethernet Options" in the WebUI which works with all Olimex ESP32-EVBs . Note it's still experimental so let us know if you find weird things.

[crocchietta]

ok, thanks.
I try to flash the card but I can't :

python3 esptool.py --chip esp32 --port "/dev/ttyUSB0" --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x1000 bootloader_dio_40m.bin 0x8000 partitions.bin 0xe000 boot_app0.bin 0x10000 EMS-ESP-3_0_0b3-esp32.bin 
esptool.py v2.6
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP32D0WDQ5 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
MAC: c4:dd:57:9c:c9:30
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...

A fatal error occurred: Timed out waiting for packet content

Do you have an idea ?

[crocchietta]

When I connect the card to the computer's usb port, the red and yellow leds stay on.

[crocchietta]

It's okay, I succeeded by removing --baud 921600 --before default_reset --after hard_reset

[crocchietta]

After the card was flashed and restarted, it worked once. I have configured ethernet. The card was responding to ping via ethernet but nothing was displayed in the firefox browser. I re-flashed the card and now I can't see the EMS-ESP through wifi.
How do you go about using this ESP32-EVB board?

[proddy]

connect it to USB and see what comes in via the Serial console. It should give you an idea what is happening.

[crocchietta]

Finally the card is flashed by modifying the speed: --baud 115200
There are errors when starting the card but this does not prevent the operation of EMS-ESP

rst:0x10 (RTCWDT_RTC_RESET),boot:0xb (HSPI_FLASH_BOOT)
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
ets Jul 29 2019 12:21:46

rst:0x10 (RTCWDT_RTC_RESET),boot:0xb (HSPI_FLASH_BOOT)
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
ets Jul 29 2019 12:21:46

rst:0x10 (RTCWDT_RTC_RESET),boot:0xb (HSPI_FLASH_BOOT)
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
ets Jul 29 2019 12:21:46

rst:0x10 (RTCWDT_RTC_RESET),boot:0xb (HSPI_FLASH_BOOT)
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
ets Jul 29 2019 12:21:46

rst:0x10 (RTCWDT_RTC_RESET),boot:0xb (HSPI_FLASH_BOOT)
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
invalid header: 0x00000000
ets Jul 29 2019 12:21:46

rst:0x10 (RTCWDT_RTC_RESET),boot:0x1b (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5828
entry 0x400806ac
E (241) psram: PSRAM ID read error: 0xffffffff


��┌��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─┐
��│ EMS-ESP version 3.0.0b3              ��│
��│ https://github.com/proddy/EMS-ESP    ��│
��│                                      ��│
��│ type help to show available commands ��│
��└��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─┘

ems-esp:/$ 000+00:00:00.000 I 0: [system] System ems-esp booted (EMS-ESP version 3.0.0b3)
000+00:00:00.000 I 1: [syslog] Log level set to OFF
000+00:00:00.000 I 2: [system] External multi-functional button not detected
000+00:00:00.000 I 3: [emsesp] Ethernet Started
000+00:00:00.000 I 4: [emsesp] EMS Device library loaded with 77 records
ems-esp:/$ 000+00:00:02.157 I 5: [emsesp] Ethernet Connected with IP=192.168.10.80, speed 100 Mbps
ems-esp:/$

How to remove these errors?

Maybe this ticket could be reopened because it might interest other people, right?

[proddy]

I'm testing now on my ESP32-EVB board.

[proddy]

I don't get those errors.

I'm using esptool.py --chip esp32 --port "COM6" --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x1000 bootloader_dio_40m.bin 0x8000 partitions.bin 0xe000 boot_app0.bin 0x10000 firmware.bin to upload at baud 921600.

I did read https://www.olimex.com/forum/index.php?topic=6118.0 so I'll update the doc and mention that some drivers need to use a lower baud when updating.

[proddy]

https://emsesp.github.io/docs/#/Uploading-firmware?id=via-a-usb-cable

[crocchietta]

I forgot to indicate that I was working under Linux CentOS 7.

[ 2657.909637] usb 2-2: new full-speed USB device number 6 using xhci_hcd
[ 2658.033578] usb 2-2: New USB device found, idVendor=1a86, idProduct=7523, bcdDevice= 2.64
[ 2658.033597] usb 2-2: New USB device strings: Mfr=0, Product=2, SerialNumber=0
[ 2658.033600] usb 2-2: Product: USB Serial
[ 2658.073628] usbcore: registered new interface driver ch341
[ 2658.073710] usbserial: USB Serial support registered for ch341-uart
[ 2658.073740] ch341 2-2:1.0: ch341-uart converter detected
[ 2658.074818] usb 2-2: ch341-uart converter now attached to ttyUSB0

I tried with a baud of 460800 and it seems to work

sudo python3 esptool.py --chip esp32 --port "/dev/ttyUSB0" --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x1000 bootloader_dio_40m.bin 0x8000 partitions.bin 0xe000 boot_app0.bin 0x10000 EMS-ESP-3_0_0b3-esp32.bin 
esptool.py v2.6
Serial port /dev/ttyUSB0
Connecting........____
Chip is ESP32D0WDQ5 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
MAC: c4:dd:57:9c:c9:30
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 15872 bytes to 10319...
Wrote 15872 bytes (10319 compressed) at 0x00001000 in 0.2 seconds (effective 541.4 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 129...
Wrote 3072 bytes (129 compressed) at 0x00008000 in 0.0 seconds (effective 4061.3 kbit/s)...
Hash of data verified.
Compressed 8192 bytes to 47...
Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 15728.0 kbit/s)...
Hash of data verified.
Compressed 1688272 bytes to 1048909...
Wrote 1688272 bytes (1048909 compressed) at 0x00010000 in 26.0 seconds (effective 518.5 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

rst:0x1 (POWERON_RESET),boot:0x1b (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5828
entry 0x400806ac
E (241) psram: PSRAM ID read error: 0xffffffff


��┌��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��┐
��│ EMS-ESP version 3.0.0b3              ��│
��│ https://github.com/proddy/EMS-ESP    ��│
��│                                      ��│
��│ type help to show available commands ��│
��└��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��─��┘

ems-esp:/$ 000+00:00:00.000 I 0: [system] System ems-esp booted (EMS-ESP version 3.0.0b3)
000+00:00:00.000 I 1: [syslog] Log level set to OFF
000+00:00:00.000 I 2: [system] External multi-functional button not detected
000+00:00:00.000 I 3: [emsesp] Ethernet Started
000+00:00:00.000 I 4: [emsesp] EMS Device library loaded with 77 records
ems-esp:/$ 000+00:00:02.200 I 5: [emsesp] Ethernet Connected with IP=192.168.10.80, speed 100 Mbps

[crocchietta]

Quick question: why is the ethernet configuration kept even after reflashing the card?

Another question: with the ethernet configuration enabled, is the wifi off (for less power consumption)?

Finally, in the ethernet configuration, the gateway address is mandatory although you don't have to use one.
It would be nice to make this field optional.

Otherwise it works pretty well and the interface is really very fast.

Thank you for adding Ethernet support.

[proddy]

EMS-ESP stores all its configured in JSON files in the File System (LITTLEFS). It's in a separate partition so not touched when you flash the firmware which only uploads the application to the app partition. This is standard behaviour in ArduinoIDE and PlatformIO.

I still haven't found the most optimal way to handle the Wifi when Ethernet is detected. Right now it's a bit of a dirty hack where I disable the Wifi and AP forever by changing the user settings. This is because EMS-ESP can only have one end-point (IP address). I was discussing this earlier with BBQKees. I better solution is to automatically re-enable the AP if the Ethernet is not found on boot so there's always a fail-safe method to get back into EMS-ESP and re-configure it, if you for example used the wrong Ethernet profie.

The gateway address being optional - good spot. I'll look into this.

thanks for the feedback

[crocchietta]

Thank you for these explanations.

It would be nice if there was the possibility of completely deactivating the wifi, via a button in the interface. And with the possibility of reactivating the wifi and/or modifying the ethernet configuration with the serial console.

But maybe the ESP32 doesn't dynamically enable or disable wifi, I don't know.

Otherwise I still have an error when booting the card
E (241) psram: PSRAM ID read error: 0xffffffff

An idea ?

[proddy]

I'll look into those suggestions. thanks.

E (241) psram: PSRAM ID read error: 0xffffffff
This is a bug in the Arduino ESP32 library and has been fixed in later versions (IDF 4.2, we're on 3.1). It's just a warning. Means your ESP32 dev board doesn't have any external PSRAM (memory chip).

[crocchietta]

ok, thanks.
the GPIOs that I used my first ESP32, GPIO 25 for Tx and 26 for Rx no longer work (loss of the ethernet connection).
Which GPIO are you using with this ESP32-EVB board?

[crocchietta]

that's ok, the default configuration (gpio 25 and 3) works fine.

[proddy]

ok good.

[proddy]

@crocchietta I'm making some changes to how the ethernet settings are stored, so please don't update to any latest Dev builds just yet. It'll break your config. I'll let you know when it's completed

[crocchietta]

@proddy : ok, thanks.

[proddy]

built and ready to test. Now WiFi and Ethernet work together. Use a blank SSID to disable Wifi. If the Ethernet drops it will start the AP captive portal.