ESP8266 issue about sdfat

[greiman]

I updated github with this change

#if defined(ESP8266)
inline void SysCall::yield() {
  // Avoid ESP8266 bug
  delay(0);
}
#elif defined(ARDUINO)

[greiman]

Let me know when you find the problem. I really depend on users to support ESP8266.

I ran the bench example on a Adafruit HUZZAH ESP8266 Breakout and it will write and read a 5 MB file.

Type is FAT32
Card size: 32.01 GB (GB = 1E9 bytes)

Manufacturer ID: 0X1B
OEM ID: SM
Product: 00000
Version: 1.0
Serial number: 0X725D8465
Manufacturing date: 10/2015

File size 5 MB
Buffer size 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
445.84,32564,989,1147
441.20,33692,988,1159

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
972.51,1395,519,525
972.13,2198,518,525

[luc-github]

you mean I provide you a test sketch to reproduce issue ?

[greiman]

No you must find the problem since I don't want to dig through your code and my examples work.

I tried the VolumeFreeSpace example and it also works.

Type any character to start

First call to freeClusterCount scans the FAT.

freeClusterCount() call time: 4996441 micros
freeClusters: 976358
freeSpace: 31993.299 MB (MB = 1,000,000 bytes)

[luc-github]

Yes it is what I wrote VolumeFreeSpace works on your Git version but not on https://github.com/tannewt/SdFat fork.
Open small file with your git works
Open big file with your git generate the issue mentioned above
Use Volume free space with your git works

but using https://github.com/tannewt/SdFat which is based on your git before last big changes
Open small file works
Open big file works
Use Volume free space do not works

I did not asked you to dig in the my code I wrote :

Anything I can do to help to debug this issue ?

I was looking for advice to find the problem - seems not possible so I won't bother you anymore - no problem

[greiman]

Why did you expect me to debug the tannewt fork? It is almost two years old.

[luc-github]

No but you did several changes since and you may know which could lead me to search - anyway please do not bother - I can handle it

[luc-github]

I solved my issue by changing ::yield(); by delay(0); in SysCall.h
https://github.com/greiman/SdFat/blob/master/SdFat/src/FatLib/SysCall.h#L54-L58

#if defined(ARDUINO)
inline void SysCall::yield() {
  // Use the external Arduino yield() function.
  //::yield();
  delay(0);
}

now big files can also be read without generating an error and no side effects - issue solved

[greiman]

Strange, must be a bug in the ESP8266 you are using. yield() works for me.

See this:

https://github.com/esp8266/Arduino/blob/master/doc/reference.md

There is also a yield() function which is equivalent to delay(0). The delayMicroseconds function, on the other hand, does not yield to other tasks, so using it for delays more than 20 milliseconds is not recommended.

I use yield() since it also works for schedulers in other systems. Also yield() is a weak function so if you replace it with your own version you can do other scheduler stuff.

[greiman]

I tested bench with this version of SysCall::yield() and it works.

inline void SysCall::yield() {
  // Use the external Arduino yield() function.
  ::yield();
}

If I comment out ::yield(); it crashes so something you are using replaces yield or there is a bug in your version of ESP8266.

I used yield instead of delay(0) since it allow more flexible scheduling.

[luc-github]

I use https://github.com/esp8266/Arduino, 2.3.0 and git version - I did not change anything in core version

[greiman]

That's the version I am using.

You don't need to change anything in the core version to have yield replaced. If some other code/library you are using defines yield, it replaces the core yield. I assume you know what a weak function is.

You may still have a problem. Some SD cards have long latencies so another yield call may be needed in SdFat.

[greiman]

I ran this test and the WDT timeout is about 2100 ms so no SD card should cause a problem.

void setup() {
  Serial.begin(9600);
  uint32_t ms = 0;
  while(1) {
    Serial.println(ms);
    uint32_t m = millis();
    while (millis() < (m + ms)) {}
    delay(500);
    ms += 100;
  }
}
void loop() {}

Output at crash

2000
2100

Soft WDT reset

If you wish, you can set this to less than 10,000 microseconds by editing SdFatConfig.h in the current SdFat.

// If Particle device or ESP8266 call yield.
#define WDT_YIELD_TIME_MICROS 100000

[luc-github]

Yes I know weak function but SdFat is the only library I am using, no other external code - I have no idea where the issue come from - I just shared what issue I experienced, asked for advice, and shared how I solved, that is all.

[luc-github]

Using optimistic_yield(10000); instead of delay(0); works also for me

like set in 2.30 for embedded SD: esp8266/Arduino@9172033

[greiman]

That means something is likely wrong with the weak function linking.

optimistic_yield calls yield.

Here is the code for all the yield functions.

extern "C" void __yield() {
    if (cont_can_yield(&g_cont)) {
        esp_schedule();
        esp_yield();
    }
    else {
        panic();
    }
}

extern "C" void yield(void) __attribute__ ((weak, alias("__yield")));

extern "C" void optimistic_yield(uint32_t interval_us) {
    if (cont_can_yield(&g_cont) &&
        (system_get_time() - g_micros_at_task_start) > interval_us)
    {
        yield();
    }
}

I have seen something like this before with Arduino. I may just call delay(0) for ESP8266. Sad since the reason for a weak yield is so you can implement a custom scheduler.

delay(0) is the same as __yield() which should be the same as yield().

One more thing. I found a version of yield() in some ESP8266 test code that would cause the problem if it were to replace the weak version.

It's in this file.

esp8266\hardware\esp8266\2.3.0\tests\host\common\Arduino.cpp

Here is the function:

extern "C" void yield()
{
}

[greiman]

Everything above is wrong. I just noticed that the correct yield is called but __yield calls panic in your code.

Panic C:\Users\user\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0\cores\esp8266\core_esp8266_main.cpp:98 __yield

It's not a WDT problem.

delay(0) is the same as yield() except it doesn't call panic, it just returns if this is not true.

cont_can_yield(&g_cont)

More terrible code int ESP8266. So it is a bug in ESP8266.

optimistic_yield also just returns.

[luc-github]

wow !!! thanks so actually delay() and optimistic_yield hide the problem but issue is elsewhere

[luc-github]

should this reported to ESP8266 github ? as issue is on their side ?

Thanks for you help on this - I learn something today

[greiman]

I think it should be since there are many ported libraries that use yield() in the ESP8266 software.

I develop for more than 10 platforms and stuff like this is killing my time but reporting and following up takes even more time.

Recently I spent two days finding a bug in the Particle Electron SPI driver.

So I wish you would report it.

[luc-github]

I have started to share the problem on gitter, as seems other people do not have the issue which is weird