walmis / blackmagic-espidf

Blackmagic Wireless SWD Debug probe hosted on esp-idf SDK (for ESP8266) with UART on Telnet port and HTTP using xterm.js

Introduction

blackmagic-espidf is a project which aims to support debugging SWD and JTAG targets over wifi by integrating blackmagic probe firmware to the espressif IDF platform for esp8266.

Features

• NEW Automatic Attach to target on connect to GDB server
• NEW Live Expressions support in STM32CubeIDE
• NEW Non-Stop GDB protocol and multiple GDB connections (beta)
• All the debug features and supported targets of the blackmagic firmware:
  • Targets ARM Cortex-M and Cortex-A based microcontrollers.
  • Connects to the target processor using the JTAG or Serial Wire Debug (SWD) interface.
  • Provides full debugging functionality, including: watchpoints, flash memory breakpoints, memory and register examination, flash memory programming, etc.
  • Load your application into the target Flash memory or RAM.
  • Single step through your program.
  • Run your program in real-time and halt on demand.
  • Examine and modify CPU registers and memory.
  • Obtain a call stack backtrace.
  • Set up to 6 hardware assisted breakpoints.
  • Set up to 4 hardware assisted read, write or access watchpoints.
  • Set unlimited software breakpoints when executing your application from RAM.
• Implements the GDB extended remote debugging protocol for seamless integration with the GNU debugger and other GNU development tools.
• GDB server on TCP port 2022
• Serial port server on TCP port 23
• Serial port over websocket on embedded http server (powered by xterm.js) @ http://192.168.4.1
• OTA updates over tftp
• Platform/BMP debug messages terminal over http://192.168.4.1/debug.html

Live Expressions in STMCubeIDE

STMCubeIDE settings

Supported Targets:

Requirements

esp8266 module with >= 2MB flash. Default configuration is set for 4MB flash for OTA updates. It's possible to configure for other flash sizes. see make menuconfig

By disabling OTA it should work on 1MB devices.

GPIO defaults for esp8266

GPIO0 - SWDIO

GPIO2 - SWCLK

TX0 - UART TXD

RX0 - UART RXD

Serial terminal

Connecting to serial terminal can be done using socat:

socat tcp:192.168.4.1:23,crlf -,echo=0,raw,crlf

Building

Grab the toolchain from https://github.com/espressif/ESP8266_RTOS_SDK#developing-with-the-esp8266_rtos_sdk and add it to $PATH.

git clone --recursive https://github.com/walmis/blackmagic-espidf.git
cd blackmagic-espidf
make menuconfig # optional, if you want to change some settings
make
make flash # this will flash using esptool.py over serial connection

Station Mode Configuration

To use the ESP8266 in Station mode, in the Blackmagic configuration section:

• Configure Station mode
• Specify the SSID and password you wish to connect to. NOTE: The SSID is case sensitive
• (optional) Specify a hostname to make it easier to connect to the probe.

Development/Debug Configuration

When working on blackmagic-espidf it is frequently desirable to continue to use the ESP8266 UART for debugging. To achieve this you can disable Monitor target UART in the Blackmagic configuration section.

In this mode you will be unable to use the ESP UART to monitor the target and connecting the ESP UART to the target may result in undefined behavior since the debug messages will be sent to the target.

OTA Flashing

If the firmware is already on the esp8266 device, it is possible to flash using tftp. Make sure you have tftp-hpa package installed then run:

make tftpflash

Buy me a coffee

If you find this project useful, consider buying me a coffee :-)

Don't forget to support Blackmagic developers !

blackmagic-espidf's Issues

Compiling with Station configuration results in compiler error

I used make menuconfig to reconfigure the project to connect to my home wifi rather than operating as an AP. With this configuration the compilation fails.

blackmagic-espidf/main/platform.c:471:3: error: implicit declaration of function 'wifi_init_softap' [-Werror=implicit-function-declaration]
   wifi_init_softap();

I believe there is a mismatch between the Wifi options configured by menuconfig and what is expected in platform.c.
e.g. configuring for station mode results in the following in sdkconfig:

# CONFIG_ESP_WIFI_IS_SOFTAP is not set
CONFIG_ESP_WIFI_IS_STATION=y
# CONFIG_ESP_WIFI_MODE_AP is not set

Configuring for AP mode results in:

CONFIG_ESP_WIFI_IS_SOFTAP=y
# CONFIG_ESP_WIFI_IS_STATION is not set
CONFIG_ESP_WIFI_MODE_AP=y

Looking in platform.c, it is looking for CONFIG_ESP_WIFI_MODE_AP and CONFIG_ESP_WIFI_MODE_STA starting at line 382. In AP mode, the correct function gets included by virtue of the MODE_AP configuration setting, rather than the IS_SOFTAP setting.

At line 468, however, an #if / #else is used:

#if CONFIG_ESP_WIFI_MODE_STA
  wifi_init_sta();
#else
  wifi_init_softap();
#endif

which results in this always calling wifi_init_softap since there is no code that defines CONFIG_ESP_WIFI_MODE_STA in any configuration that I can find.

Changing the section at 468 to:

#if CONFIG_ESP_WIFI_IS_STATION
  wifi_init_sta();
#else
  wifi_init_softap();
#endif

allows you to compile in station mode successfully although I'm unable to connect due to a MAC address problem that I will submit a different issue for.

Getting a WDT Reset Loop

Hi!

I tried the BMP using a prebuilt binary I found online and it worked great. Currently I'm trying to build the newest version with Live Expressions.

I can build the firmware and flash it to a NodeMCU (ESP8266-12E) and I always get into a boot loop caused by a watchdog reset.

On the UART I get:
rst cause: 4, boot mode:(3,6)
wdt reset
load 0x60000014, len 8, room 16
tail 8

As a build system I'm using a fresh Ubuntu 20.04.2 VM with the xtensa toolchain version 8.4.0 (2020r3).

Hope someone can help.
Best regards!

Attempting to connect as a Station results in a MAC address error

This may be a false error if my approach in #3 is not valid, but after implementing that fix I am unable to connect to my wifi network. The system serial output is:

I (315) system_api: Base MAC address is not set, read default base MAC address from EFUSE
I (319) system_api: Base MAC address is not set, read default base MAC address from EFUSE
�phy_version: 1155.0, 6cb3053, Nov 11 2019, 17:31:08, RTOS new
I (469) phy_init: phy ver: 1155_0
I (470) reset_reason: RTC reset 2 wakeup 0 store 4, reason is 2
Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC      : 0x40222e8b  PS      : 0x00000033  A0      : 0x40222e88  A1      : 0x3fffba40  
A2      : 0x00000001  A3      : 0x3ffea750  A4      : 0x00000020  A5      : 0xfffffffe  
A6      : 0x00000000 �������������������������������p����������������������������������I (307) system_api: Base MAC address is not set, read default base MAC address from EFUSE
I (311) system_api: Base MAC address is not set, read default base MAC address from EFUSE
�phy_version: 1155.0, 6cb3053, Nov 11 2019, 17:31:08, RTOS new
I (461) phy_init: phy ver: 1155_0
I (462) reset_reason: RTC reset 2 wakeup 0 store 4, reason is 2
Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC      : 0x40222e8b  PS      : 0x00000033  A0      : 0x40222e88  A1      : 0x3fffba40  
A2      : 0x00000001  A3      : 0x3ffea750  A4      : 0x00000020  A5      : 0xfffffffe  
A6      : 0x00000000  A7      : 0x7fffffff  A8      : 0x00000000  A9      : 0x00000000  
A10     : 0x00000000  A11     : 0x00000059  A12     : 0x00000000  A13     : 0x3fffba4c  
A14     : 0x00000000  A15     : 0x3fffba8c  SAR     : 0x00000000  EXCCAUSE: 0x0000001c  

Backtrace: 0x40222e8b:0x3fffba40 0x4022a870:0x3fffba80 0x4022abb1:0x3fffbaa0 0x4024b4f0:0x3fffbae0 0x4024b699:0x3fffbaf0 0x4024e143:0x3fffbb30 0x4022d02c:0x3fffbb40 0x40222e88:0x3fffbb40 

I'm digging in here and will update this if I find a solution.

Problem starting debugging in CubeIde

Problem starting debugging in CubeIde(Version: 1.14.0 Build: 19471_20231121_1200 (UTC))
I successfully compiled and uploaded the latest version of blackmagic-espidf, but when entering debug mode I get an error "Could not verify ST device! Please verify that the latest version of the GDB-server is used for the connection."

[Websocket] Connection established
I (150958) GDB_client: Started task 53 this:0x40109854 tlsp:0x3fffb69c mowner:0
I (150961) gdb_main_loop: cur_target=0 last_target=0x401090c8

I (150963) GDB: Scanning SWD
I (150963) gdb_target_destroy_callback: gdb_target_destroy_callback 0x401090c8 cur_target=0 last_target=0x401090c8

W (150965) BMP:W: scan_multidrop: false 
I (150966) BMP:I: DP DPIDR 0x1ba01477 (v1 rev0) designer 0x43b partno 0xba 
I (151171) BMP:I: RESET_SEQ failed 
I (151172) BMP:I: AP   0: IDR=14770011 CFG=00000000 BASE=e00ff003 CSW=23000040
I (151173) BMP:I:  (AHB-AP var1 rev1) 
I (151174) BMP:I: Halt via DHCSR: success 01030003 after 0ms 
I (151176) BMP:I: ROM: Table BASE=0xe00ff000 SYSMEM=0x00000001, Manufacturer  20 Partno 410 
I (151178) BMP:I: 0 0xe000e000: Generic IP component - Cortex-M3 SCS (System Control Space) (PIDR = 0x00000004001bb000  DEVTYPE = 0x00 ARCHID = 0x0000) 
I (151180) BMP:I: -> cortexm_probe 
I (151181) BMP:I: CPUID 0x411fc231 (M3 var 1 rev 1) 
I (151186) BMP:I: 1 0xe0001000: Generic IP component - Cortex-M3 DWT (Data Watchpoint and Trace) (PIDR = 0x00000004001bb002  DEVTYPE = 0x00 ARCHID = 0x0000) 
I (151188) BMP:I: 2 0xe0002000: Generic IP component - Cortex-M3 FBP (Flash Patch and Breakpoint) (PIDR = 0x00000004000bb003  DEVTYPE = 0x00 ARCHID = 0x0000) 
I (151191) BMP:I: 3 0xe0000000: Generic IP component - Cortex-M3 ITM (Instrumentation Trace Module) (PIDR = 0x00000004001bb001  DEVTYPE = 0x00 ARCHID = 0x0000) 
I (151193) BMP:I: 4 0xe0040000: Debug component - Cortex-M3 TPIU (Trace Port Interface Unit) (PIDR = 0x00000004001bb923  DEVTYPE = 0x11 ARCHID = 0x0000) 
I (151199) BMP:I: 5 0xe0041000: Debug component - Cortex-M3 ETM (Embedded Trace) (PIDR = 0x00000004101bb924  DEVTYPE = 0x13 ARCHID = 0x0000) 
I (151201) BMP:I: ROM: Table END 
I (151203) GDB: Found 1
I (151214) GDB: *** Unsupported packet: vMustReplyEmpty 
I (151267) GDB: *** Unsupported packet: qTStatus 
W (151271) ?: halted 4
I (151287) GDB: *** Unsupported packet: qOffsets 
I (151309) GDB: *** Unsupported packet: qSymbol:: 
I (151319) CMD: ReadAPEx 0x0 0xF8
I (151329) GDB_client: destroy 53
I (151331) GDB_client: delete 0x40109854

Unable to use it

[edit, solved] my issue was not having python2 installed. Somehow make compiles even though it was throwing an error during compilation of the bootloader.

Hi there, I've been trying to use this port to no avail.
Things I've done:

• Compile seems to go according to plan, make ends with:

[...compilation steps cut out...]
CC build/wpa_supplicant/src/fast_crypto_ops.o
CC build/wpa_supplicant/port/os_xtensa.o
AR build/wpa_supplicant/libwpa_supplicant.a
Generating esp8266.project.ld
LD build/blackmagic.elf
esptool.py v2.4.0
To flash all build output, run 'make flash' or:
python /home/marko/git/blackmagic-espidf/ESP8266_RTOS_SDK/components/esptool_py/esptool/esptool.py --chip esp8266 --port /dev/ttyUSB0 --baud 921600 --before default_reset --after soft_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size 4MB 0xd000 /home/marko/git/blackmagic-espidf/build/ota_data_initial.bin 0x0000 /home/marko/git/blackmagic-espidf/build/bootloader/bootloader.bin 0x10000 /home/marko/git/blackmagic-espidf/build/blackmagic.bin 0x8000 /home/marko/git/blackmagic-espidf/build/partitions_two_ota.bin

• After that I run make flash, which goes something like:

Toolchain path: /home/marko/esp/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc
Toolchain version: crosstool-ng-1.22.0-100-ge567ec7
Compiler version: 5.2.0
Python requirements from /home/marko/git/blackmagic-espidf/ESP8266_RTOS_SDK/requirements.txt are satisfied.
Flashing binaries to serial port /dev/ttyUSB0 (app at offset 0x10000)...
esptool.py v2.4.0
Connecting....
Chip is ESP8266EX
Features: WiFi
MAC: 4c:11:ae:0a:ca:7b
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Compressed 8192 bytes to 31...
Wrote 8192 bytes (31 compressed) at 0x0000d000 in 0.0 seconds (effective 32817.3 kbit/s)...
Hash of data verified.
Compressed 16 bytes to 24...
Wrote 16 bytes (24 compressed) at 0x00000000 in 0.0 seconds (effective 65.3 kbit/s)...
Hash of data verified.
Compressed 499696 bytes to 364577...
Wrote 499696 bytes (364577 compressed) at 0x00010000 in 4.6 seconds (effective 870.8 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 106...
Wrote 3072 bytes (106 compressed) at 0x00008000 in 0.0 seconds (effective 8203.0 kbit/s)...
Hash of data verified.

Leaving...
Soft resetting...

• And then my board starts blinking the LED quite erratically. Opening its port on a terminal (@74800 baud, the baudrate during booting) delivers the following:

 ets Jan  8 2013,rst cause:2, boot mode:(3,7)
load 0x60000014, len 8, room 16
tail 8

Which, to my understanding, indicates that the ESP is in a reset loop.

Any help is appreciated. This project is a great idea but I can't make it work and don't know what I'm doing wrong 😭

compiler error because `CONFIG_ESP_WIFI_IS_STATION` is not defined

For some reason CONFIG_ESP_WIFI_IS_STATION is not set in sdkconfig

CONFIG_ESPTOOLPY_MONITOR_BAUD_OTHER_VAL=74
CONFIG_ESPTOOLPY_MONITOR_BAUD=230400
CONFIG_ESP_WIFI_IS_SOFTAP=y
# CONFIG_ESP_WIFI_IS_STATION is not set   <--
CONFIG_ESP_WIFI_MODE_AP=y
CONFIG_ESP_WIFI_SSID="auto"
CONFIG_ESP_WIFI_PASSWORD="helloworld"
CONFIG_MAX_STA_CONN=4

And this causes a compilation error here

blackmagic-espidf/main/platform.c

Line 453 in 2f58e68

if(strcmp(CONFIG_ESP_WIFI_SSID, "auto") == 0 || CONFIG_ESP_WIFI_IS_STATION) {

Faster SWD by using SPI port

Don't know if you've seen this. https://www.pcbway.com/blog/technology/OpenOCD_on_Raspberry_Pi__Better_with_SWD_on_SPI.html
It's for the OpenOCD codebase but the underlying SWD 'bus' is the same so the same should be possible using an ESP8266 instead of an raspberry pi. It might be quite some work to get up and running so I understand if it won't be implemented.
Just wanted to let you know if you needed an interesting challenge ;-)

Name (some?) usable ESP module

Please name a/some available ESP modules that can be used.

Pre-compiled binaries

Any chance you could upload pre-compiled binaries?
I'm having problems getting the compile environment set up correctly so I can't build currently.
I already have an older version of the code compiled and downloaded on en ESP8266 using this guide.
https://www.visualmicro.com/page/Create-a-WiFi-Black-Magic-GDB-Probe-from-ESP8266.aspx
So now I'd just like to upload a newer version to try out the latest features like live debugging etc.
Any chance you could do this?

Ps. I realize that there are multiple configurations options but just a binary with default values would be great.

Error when compiling the bootloader.bin

This is what I got:

CC build/bootloader/esp8266/source/esp_fast_boot.o
AR build/bootloader/esp8266/libesp8266.a
LD build/bootloader/bootloader.elf
esptool.py v2.4.0
Traceback (most recent call last):
  File "/home/me/blackmagic-espidf/ESP8266_RTOS_SDK/components/esptool_py/esptool/esptool.py", line 3034, in <module>
    _main()
  File "/home/me/blackmagic-espidf/ESP8266_RTOS_SDK/components/esptool_py/esptool/esptool.py", line 3027, in _main
    main()
  File "/home/me/blackmagic-espidf/ESP8266_RTOS_SDK/components/esptool_py/esptool/esptool.py", line 2800, in main
    operation_func(args)
  File "/home/me/blackmagic-espidf/ESP8266_RTOS_SDK/components/esptool_py/esptool/esptool.py", line 2369, in elf2image
    image.save(args.output)
  File "/home/me/blackmagic-espidf/ESP8266_RTOS_SDK/components/esptool_py/esptool/esptool.py", line 1380, in save
    checksum = self.save_segment(f, segment, checksum)
  File "/home/me/blackmagic-espidf/ESP8266_RTOS_SDK/components/esptool_py/esptool/esptool.py", line 1290, in save_segment
    f.write(segment.data)
TypeError: a bytes-like object is required, not 'str'
make[1]: *** [/home/me/blackmagic-espidf/ESP8266_RTOS_SDK/components/esptool_py/Makefile.projbuild:72: /home/me/blackmagic-espidf/build/bootloader/bootloader.bin] Error 1
make: *** [/home/me/blackmagic-espidf/ESP8266_RTOS_SDK/components/bootloader/Makefile.projbuild:40: /home/me/blackmagic-espidf/build/bootloader/bootloader.bin] Error 2

If we inspect the bootloader.bin well notice that it is 16 bytes, that is wrong, don't use it!

A workaround

• Type make again it will compile the rest of the firmware
• overwrite the bad bootloader.bin with a good one you can get from the binaries you can find in this page: https://www.visualmicro.com/page/Create-a-WiFi-Black-Magic-GDB-Probe-from-ESP8266.aspx

P.D: Many thanks for this awesome project, this tool is fantastic!!