igrr / mkspiffs

Tool to build and unpack SPIFFS images

mkspiffs

Tool to build and unpack SPIFFS images.

Usage

   mkspiffs  {-c <pack_dir>|-u <dest_dir>|-l|-i} [-d <0-5>] [-b <number>]
             [-p <number>] [-s <number>] [--] [--version] [-h]
             <image_file>


Where: 

   -c <pack_dir>,  --create <pack_dir>
     (OR required)  create spiffs image from a directory
         -- OR --
   -u <dest_dir>,  --unpack <dest_dir>
     (OR required)  unpack spiffs image to a directory
         -- OR --
   -l,  --list
     (OR required)  list files in spiffs image
         -- OR --
   -i,  --visualize
     (OR required)  visualize spiffs image


   -d <0-5>,  --debug <0-5>
     Debug level. 0 means no debug output.

   -b <number>,  --block <number>
     fs block size, in bytes

   -p <number>,  --page <number>
     fs page size, in bytes

   -s <number>,  --size <number>
     fs image size, in bytes

   --,  --ignore_rest
     Ignores the rest of the labeled arguments following this flag.

   --version
     Displays version information and exits.

   -h,  --help
     Displays usage information and exits.

   <image_file>
     (required)  spiffs image file

Build

You need gcc (≥4.8) or clang(≥600.0.57), and make. On Windows, use MinGW.

Run:

$ git submodule update --init
$ make dist

SPIFFS configuration

Some SPIFFS options which are set at mkspiffs build time affect the format of the generated filesystem image. Make sure such options are set to the same values when builing mkspiffs and when building the application which uses SPIFFS.

These options include:

• SPIFFS_OBJ_NAME_LEN
• SPIFFS_OBJ_META_LEN
• SPIFFS_USE_MAGIC
• SPIFFS_USE_MAGIC_LENGTH
• SPIFFS_ALIGNED_OBJECT_INDEX_TABLES
• possibly others

To see the default values of these options, check include/spiffs_config.h file in this repository.

To override some options at build time, pass extra CPPFLAGS to make. You can also set BUILD_CONFIG_NAME variable to distinguish the built binary:

$ make clean
$ make dist CPPFLAGS="-DSPIFFS_OBJ_META_LEN=4" BUILD_CONFIG_NAME=-custom

To check which options were set when building mkspiffs, use --version command:

$ mkspiffs --version
mkspiffs ver. 0.2.2
Build configuration name: custom
SPIFFS ver. 0.3.7-5-gf5e26c4
Extra build flags: -DSPIFFS_OBJ_META_LEN=4
SPIFFS configuration:
  SPIFFS_OBJ_NAME_LEN: 32
  SPIFFS_OBJ_META_LEN: 4
  SPIFFS_USE_MAGIC: 1
  SPIFFS_USE_MAGIC_LENGTH: 1
  SPIFFS_ALIGNED_OBJECT_INDEX_TABLES: 0

License

MIT

To do

• Add more debug output and print SPIFFS debug output
• Error handling
• Code cleanup

mkspiffs's Issues

Mount failed error with option -c

Trying mkspiffs-0.2.2-esp-idf-linux64

Always reports: SPIFFS mount failed with error: -10025

It seems to build the image OK and can list (-i & -l) OK.

~/src/esp/mkspiffs/mkspiffs -b 4096 -p 256 -s 0xe1000 -c

I've also compiled from git - it's called "generic" which behaves the same.

libc++abi.dylib: terminating with uncaught exception of type std::length_error: vector

On macOS I’m attempt to uploadfs using PlatformIO. I run pio run -t uploadfs after creating the data directory (containing a JSON file) and after successfully uploading my program to a Wemos D1 Mini Pro board (ESP8266).

I an exception when I run this. From the verbose output I see pio running the below mkspiffs command.

"mkspiffs" -c data -p 256 -b 8192 -s -1069056 .pioenvs/d1_mini_pro/spiffs.bin
libc++abi.dylib: terminating with uncaught exception of type std::length_error: vector
*** [.pioenvs/d1_mini_pro/spiffs.bin] Error -6

Any idea what might be wrong or how to troubleshoot this? I’m using PlatformIO version 3.5.2rc4.

mkspiffs制作的根文件烧录进去看不到分区表？

您好，我利用mkspiffs工具，执行下面两个步骤，把生成的spiffs.bin烧录到flash.

mkspiffs -c ./src_folder -b 4096 -p 256 -s 0x100000 spiffs.bin
python esptool.py --chip esp32 --port /dev/ttyUSB1 --baud 115200 write_flash -z 0x110000 spiffs.bin

然后利用：make monitor
发现看不到spiffs分区，请问这是什么原因啊？

SPIFFS implementation for ESP32?

Is there a SPIFFS implementation for the ESP32 which is compatible with the mkspiffs tool found here?

[feature request]Modify SPIFFS_OBJ_META_LEN for ESP32 and add in version

Hi to add time on SPIFFS :
espressif/esp-idf#1061

It need to modify SPIFFS_OBJ_META_LEN which means different mkspiffs between ESP8266 / ESP32

So I would like to suggest to add the target in version to know which SPIFFS is for which board
like for ESP32:

C:\Users\user\Desktop>mkspiffs.exe --version

mkspiffs.exe  version: 0.2.1 ESP32

and for ESP8266/8285

mkspiffs.exe  version: 0.2.1 ESP82XX

It is something possible ?

Space replacement in BUILD_CONFIG_STR does not work for multiple parameters

Needs the global flag adding to the sed regex.

FreeBSD

any chance of modifying the makefile for target : freebsd..

I've tried editing it, but I'm hopeless with Makefiles.

:)

MKSPIFFS Images cannot be mounted -- corrupted

The current version of MKSPIFFS produces images that are incompatible with current SPIFFS. Specifically I am flashing a SPIFFS image to a ESP8266 running the Arduino cores code. With images produced by MKSPIFFS the filesystem will not mount.

What is happening is that spiffs_obj_lu_scan is returning SPIFFS_ERR_NOT_A_FS when called from SPIFFS_mount. The tool is creating filesystems with too many unerased blocks.

If I try to mount the fs with SPIFFS built with SPIFFS_USE_MAGIC set to zero then the mount will succeed but reading data from the files will fail. Reads will return zero data.

SPIFFSImpl: allocating 512+240+1400=2152 bytes
SPIFFSImpl: mounting fs @eb000, size=10000, block=1000, page=100
SPIFFS_mount sz:0 logpgsz:256 logblksz:0 perasz:0 addr:00000000 fdsz:240 cachesz:1400
SPIFFSImpl: mount rc=-10025

Replacing 'diff --strip-trailing-cr'

I am converting most of my toolchains to work in docker containers and the linux distro of choice for me is alpine because of it's small size (4mb base image). However, since it uses busy box for the core components there are usually some workaround required to get things to work.

While trying to get this to work in an alpine container I found that diff doesnt support the --strip-trailing-cr parameter. Do you see any difficulties in replacing this with a sed-command and run diff afterwards?

Getting error (-10010) while trying to pack a simple gzipped webpage

Hello,

I am trying to pack a webpage into an image file, so that i can upload it to my esp32. This is what i try to run: mkspiffs -c ./web -b 4096 -p 256 -s 524288 web.img -d 5
As a result i get "SPIFFS_write error(-10010): unknown" for every file but the index.html. (it's the smallest file from the bunch, i don't know if the issue has anything to do with file size though)

I have tried running the command with mkspiffs 0.2.1 64bit linux, 32bit linux and 0.2.0 64 bit. No luck in any of the versions.

This is a screenshot of what i am getting: https://i.imgur.com/1FJw6hP.png

SPIFFS_write error(-10001): File system is full

Hi,
I have a esp32-device with a webpage on it. There is a html script (from the example) for uploading and saving files to flash and it works fine, but it is not very useful for production. I tried to create an spiffs image of web with mkspiffs and it said "SPIFFS_write error(-10001): File system is full". Instead of 0x220000 bytes it requires 0x260000. What might be a problem?..

Unable to read file from spiffs partition

I wanted to use SPIFFS for storing AWS certs but I had trouble with it so I started finding what was wrong.
so I made a spiffs.bin image with mkspiffs and flashed with esptool.py and when I try to read files it shows nothing or lot of question marks.
here I have posted my first attempt to figure out where I was facing the trouble..

but then I tried given spiffs example(example works fine even if I open file and write some thing and read back it works) but if I try to read a file flashed by esptool I get nothing or ?????.

like this example I modified to read file from spiffs which is written by esptool.

    FILE* f;
    // Open file for reading
    ESP_LOGI(TAG, "Reading file");
    f = fopen("/spiffs/test.txt", "r");
    if (f == NULL) {
        ESP_LOGE(TAG, "Failed to open file for reading");
        return;
    }
    char line[64];
    fgets(line, sizeof(line), f);
    fclose(f); 

Output:

I (241) heap_init: At 40088828 len 000177D8 (93 KiB): IRAM
I (247) cpu_start: Pro cpu start user code
I (265) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (267) example: Initializing SPIFFS
I (277) example: Partition size: total: 52961, used: 6275
I (277) example: Reading file
I (277) example: Read from file: ''
I (277) example: SPIFFS unmounted

Am I using mkspiffs correctly ?

Not sure I'm doing right using mkspiffs...

Any help appreciated.

I'm trying to use mkspiffs (0.2.3) to create an image .bin file for Arduino ESP8266 Sketch Data to be uploaded on a WeMos D1 R1 Arduino Development board (4MB Flash) with selected FlashSize: 4M (FS: 1M OTA:~1019KB)

The /data folder only contains files of total size ~50kB and uploading this sketch data from the Arduino IDE works fine.

I built the spiffs.bin file with the following parameters:

mkspiffs -p 256 -b 8192 -s 0x0FB000 -c data/ spiffs.bin

Two things appear strange to me:

1. When I update the board using ESPhttpUpdate from this spiffs.bin file, it fails (with error "Not enough space")

2. If I try to list the content of the generated .bin file with mkspiffs -l spiffs.bin I get the following result:
   >mkspiffs -l spiffs.bin
   SPIFFS mount failed with error: -10025
   error: failed to mount image

(Same result with mkspiffs -i spiffs.bin to visualize the image)

What am I doing wrong ?

Note: If I upload the Sketch Data folder from the Arduino IDE "ESP8266 Sketch Data Upload" I get the following output:
[SPIFFS] data : C:\Users...\data
[SPIFFS] size : 1000
[SPIFFS] page : 256
[SPIFFS] block : 8192
/ColorPicker.html
/config.json
/DisplayOptions.html
/index.html
/style.css
/TimezoneSelection.html
[SPIFFS] upload : C:\Users...spiffs.bin
[SPIFFS] address : 0x300000
[SPIFFS] reset : --before default_reset --after hard_reset
[SPIFFS] port : COM5
[SPIFFS] speed : 921600
[SPIFFS] python : C:\Users...\python3.exe
[SPIFFS] uploader : C:\Users...\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.7.4\tools\upload.py

esptool.py v2.8
Serial port COM5
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 24:a1:60:30:2c:8d
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 1024000 bytes to 17074...
Wrote 1024000 bytes (17074 compressed) at 0x00300000 in 4.2 seconds (effective 1929.4 kbit/s)...
Hash of data verified.

Corrupt/unusable image created without "-b" option

I tried to get mkspiffs to work on my Linux system, based off of the 0.2.2 / 0.3.7 release of the code. While images were created, I tried to use the options that the Arduino plugin used for creation of the spiffs image:

mkspiffs -c testdir -p 256 -b 8192 -s 1028096 spiffs.img

(default settings for a Wemos D1 R1 w/ 1M spiffs)

While the 0.2.0 version of mkspiffs creates an image without complaining when using these options, and the image is perfectly fine on the device itself, 0.2.2 complains about the -b option:

error: Image size should be a multiple of block size

Problem is, not using the option causes the image to be corrupt or at least unusable on the ESP, only a small part of the files are readable.

Not sure if either the "error" or the "should" is correct on that message, but at least there is some problem here that needs to be further analyzed ...

mkspiffs

@igrr
I asked on arduino-esp8266 how to make custom SPIFFS Size (see).
I followed Markus kind reply and all seems to be fine for me (new linker script/boards.txt entries) until I'm really trying to generate my new spiffs image for 1M/160KB SPIFFS target

1st point when creating SPIFFS from Arduino IDE if there is a problem (full FS) we do not see error on IDE, looks like IDE says all is fine, resulting truncated files on SPIFFS (makes sense since we're full) but took me some time to understand why my web app wasn't working anymore because I did not had any error message from IDE;about SPIFFS error -)

So I tried to generate SPIFFS from command line and here something I don't understand, it says :
SPIFFS_write error(-10001): File system is full. but calculation looks like there is enough place, here the screenshot

Total size is 124331 bytes, in worst case with 4096 block has we've got 6 files it could add 6*4096 = 24576 bytes so 148907 bytes which is less than 163840 defined on FS size ?

Do I'm doing something wrong ?

I've tested with 256K SPIFFS and here sketch result from HTTP query to my web app, as you can see total used returned by SPIFFS is 127006 so far below my 160K (163840) FS passed to mkspiffs

latest Arduino ESP32 core + latest 0.2.3 & AsyncWebserver no static files are served

HI,
I don't get SPIFFS to run. I can upload without errors, but with AsyncWebserver no static SPIFFS file are served. Any ideas ? The same code (without partition) worked well with Wemos D1,R1 mini

Name, Type, SubType, Offset, Size, Flags

nvs, data, nvs, 0x009000,0x005000,
factory, app, factory, 0x010000,0x0C0000,
eeprom, data, 0x99, 0x0D0000,0x001000,
spiffs, data, spiffs, 0x0D1000,0x320000,

Please add git submodule instructions to Readme :)

Ran into an issue where ./build_all_configs.sh wouldn't run. Found this thread
espressif/esp-idf#990 and quickly realized that adding git submodule update --init to this repo's Readme will help other newbies.

Thank you for writing this awesome tool

SPIFFS created files with leading / -- can this be avoided with additional parameter?

Hi, I'm using SPIFFS with Simba framework, under PlatformIO, and it calls mkspiffs to make the SPIFFS image for me just fine. But I noticed that all the files it creates begin with leading '/' which is a little odd under Simba framework as Simba mounts SPIFFS under /fs.

We end up having to refer to SPIFFS files in this manner:

$ filesystems/fs/read /fs//config.ini
[network]
ssid=ATOMIC_IOT
psk=*****************

Notice the double-slash that is needed to make this work under Simba. Perhaps if we could add a switch to mkspiffs that avoids the pre-pending of the leading slash that would be awesome.

Suggestion: SPIFFS image from a ZIP file

This is a suggestion rather than an issue ....

What would be great is to create a SPIFFS image from the content of a ZIP file. The value in this would be that I could create a ZIP of what my file system contains and then as part of my build process, the tool could create the image from the ZIP. The ZIP then becomes a source of "truth" of what is in my file system and can be managed as a unit.

Compatibility with esp-open-rtos mkspiffs

Hello, I have a project that I just need to create a new file system to upload with my firmware every time I upload it. The thing is that my VM with esp-open-rtos isn't working well anymore and I wanted to upgrade to windows since it's my host OS. The thing is that comparing the binaries generated by both mkspiffs I noticed that there are some differences that may affect the functionality of the code, since it may not be able to read the file system.

Unfortunately I won't be able to test this for a few days, maybe weeks. Has anyone changes host OS me? are those difference significant or not? My firmware uses esp-open-sdk that uses spiffs version 0.3.6

Fails to compile in OSX Mojave

I'm unable to compile without errors

ld: warning: The i386 architecture is deprecated for macOS (remove from the Xcode build setting: ARCHS)
ld: warning: ignoring file /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/lib/libSystem.tbd, missing required architecture i386 in file /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/lib/libSystem.tbd
ld: warning: ignoring file /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/lib/libc++.tbd, missing required architecture i386 in file /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/lib/libc++.tbd
Undefined symbols for architecture i386

It seems i386 has now been removed in OSX - removing it from the Makefile builds successfully.

mkspiffs doesn't pack directories recursively

Would be usefull for web pages which are usually arranged into a few directories like js/, img/, css/.

ESP822 Arduino

Hi Igrr,

sorry for the post here, but i have to tell you something. And i didn't find any other way to contact you. No mail address and private message @ esp8266.com didn't work too. And i don't use facebook, twitter or so.
And i think its better to post it here and not in the esp8266 project.

My time is currently very limited because I am at the end of my studies.
And therefore i have no time for the esp8266 arudino project in the next months.
And sorry for my absence in the past.

You make a great work with the esp8266 project, but you can remove me from the collaborator list.
I would like to continue, but i lack the time.

Best regards
pgollor

Make error windows - help please

$ make dist
mkdir -p spiffs_t
cp spiffs/src/.h spiffs_t/
cp spiffs/src/.c spiffs_t/
rm -rf spiffs_t/.git
rm -f spiffs_t/.DS_Store
ls -1 spiffs_t > out.list0
touch spiffs_t/.DS_Store
mkdir -p spiffs_t/.git
touch spiffs_t/.git/foo
./mkspiffs -c spiffs_t -s 0x100000 -p 512 -b 0x2000 out.spiffs_t | sort | sed s/^\/// > out.list1
./mkspiffs -u spiffs_u -s 0x100000 -p 512 -b 0x2000 out.spiffs_t | sort | sed s/^\/// > out.list_u
./mkspiffs -l -s 0x100000 -p 512 -b 0x2000 out.spiffs_t | cut -f 2 | sort | sed s/^\/// > out.list2
diff --strip-trailing-cr out.list0 out.list1
1,7d0
< spiffs.h
< spiffs_cache.c
< spiffs_check.c
< spiffs_gc.c
< spiffs_hydrogen.c
< spiffs_nucleus.c
< spiffs_nucleus.h
make: *** [Makefile:130: test] Error 1

File exclusion in mkspiffs

Hello @igrr,

I started using mkspiffs (as part of PlatformIO) recently and hit a limitation in that mkspiffs doesn't seem to support file/folder exclusions other than a few hardcoded ones (.DS_Store, .git, etc). I went ahead and added support for adding exclusions (e.g.: -x *.h, to exclude all files with the .h extension). If you think this could be useful, I'd be happy to share my changes.

Let me know.

Thanks!

SPIFFS_ERR_NOT_A_FS

I'm trying to use mkspiffs to build a spiffs image to put some files onto ESP32, I must be doing something wrong, because mkspiffs can't read it's own image.

tMac:ESP tim$ git clone [email protected]:igrr/mkspiffs.git
Cloning into 'mkspiffs'...
remote: Counting objects: 307, done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 307 (delta 16), reused 22 (delta 9), pack-reused 274
Receiving objects: 100% (307/307), 210.27 KiB | 893.00 KiB/s, done.
Resolving deltas: 100% (164/164), done.
tMac:ESP tim$ cd mkspiffs
tMac:mkspiffs tim$ git submodule init
Submodule 'spiffs' (https://github.com/pellepl/spiffs.git) registered for path 'spiffs'
tMac:mkspiffs tim$ git submodule update --recursive
Cloning into '/Users/tim/ESP/mkspiffs/spiffs'...
Submodule path 'spiffs': checked out 'f5e26c4e933189593a71c6b82cda381a7b21e41c'
tMac:mkspiffs tim$ make
c++ -std=gnu++11 -Os -Wall -mmacosx-version-min=10.7 -arch i386 -arch x86_64 -stdlib=libc++  -Itclap -Iinclude -Ispiffs/src -I. -D VERSION=\"0.2.3-1-g011c110\" -D SPIFFS_VERSION=\"0.3.7-5-gf5e26c4\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__   -c -o main.o main.cpp
cc -std=gnu99 -Os -Wall -mmacosx-version-min=10.7 -arch i386 -arch x86_64  -Itclap -Iinclude -Ispiffs/src -I. -D VERSION=\"0.2.3-1-g011c110\" -D SPIFFS_VERSION=\"0.3.7-5-gf5e26c4\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__   -c -o spiffs/src/spiffs_cache.o spiffs/src/spiffs_cache.c
cc -std=gnu99 -Os -Wall -mmacosx-version-min=10.7 -arch i386 -arch x86_64  -Itclap -Iinclude -Ispiffs/src -I. -D VERSION=\"0.2.3-1-g011c110\" -D SPIFFS_VERSION=\"0.3.7-5-gf5e26c4\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__   -c -o spiffs/src/spiffs_check.o spiffs/src/spiffs_check.c
cc -std=gnu99 -Os -Wall -mmacosx-version-min=10.7 -arch i386 -arch x86_64  -Itclap -Iinclude -Ispiffs/src -I. -D VERSION=\"0.2.3-1-g011c110\" -D SPIFFS_VERSION=\"0.3.7-5-gf5e26c4\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__   -c -o spiffs/src/spiffs_gc.o spiffs/src/spiffs_gc.c
cc -std=gnu99 -Os -Wall -mmacosx-version-min=10.7 -arch i386 -arch x86_64  -Itclap -Iinclude -Ispiffs/src -I. -D VERSION=\"0.2.3-1-g011c110\" -D SPIFFS_VERSION=\"0.3.7-5-gf5e26c4\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__   -c -o spiffs/src/spiffs_hydrogen.o spiffs/src/spiffs_hydrogen.c
cc -std=gnu99 -Os -Wall -mmacosx-version-min=10.7 -arch i386 -arch x86_64  -Itclap -Iinclude -Ispiffs/src -I. -D VERSION=\"0.2.3-1-g011c110\" -D SPIFFS_VERSION=\"0.3.7-5-gf5e26c4\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__   -c -o spiffs/src/spiffs_nucleus.o spiffs/src/spiffs_nucleus.c
c++ main.o spiffs/src/spiffs_cache.o spiffs/src/spiffs_check.o spiffs/src/spiffs_gc.o spiffs/src/spiffs_hydrogen.o spiffs/src/spiffs_nucleus.o -o mkspiffs -mmacosx-version-min=10.7 -arch i386 -arch x86_64 -stdlib=libc++ 
strip mkspiffs
tMac:mkspiffs tim$ ./mkspiffs  -c include -b 8192 -p 256 -s 1048576 ./spiffs_image.img
/spiffs_config.h
tMac:mkspiffs tim$ ls -la spiffs_image.img 
-rw-r--r--  1 tim  staff   1.0M 12 Apr 15:52 spiffs_image.img
tMac:mkspiffs tim$ ./mkspiffs  -l ./spiffs_image.img
SPIFFS mount failed with error: -10025
error: failed to mount image
tMac:mkspiffs tim$ grep -rn "\-10025" ./ --include *.h
./spiffs/src/spiffs.h:43:#define SPIFFS_ERR_NOT_A_FS             -10025

I'm seeing the same error on the ESP32 since I updated esp-idf the other day. I suspect that theres a regression in spiffs itself, unless I'm missing something ^^^^ ?

mkspiffs - > Command not found

I cloned it, i did the make now i do not understand how to run the function.

The windows build is failing

As highlighted by Travis CI, windows build is failing. Can you please fix this or provide a workaround?

A fatal error occurred: Timed out waiting for packet header

Hi @igrr ,

i am using mkspiffs to create image before flashing to the external spi flash but having timed out issue during flashing.
i am using windbond 16MB external flash added to VSPI in esp32.

mkspiffs.exe -c data --size 0xFFA000 --page 256 --block 4096 ext-spiffs.bin
esptool.py --chip esp32 --port com6 --baud 115200 write_flash 0x0 ext-spiffs.bin --flash_mode dio --flash_size 16MB --spi-connection 18,19,23,21,5

esptool.py v3.1-dev
Serial port com6
Connecting........_
Chip is ESP32-D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 24:6f:24:c3:a5:e5
Uploading stub...
Running stub...
Stub running...
Configuring SPI flash mode...
Configuring flash size...
Compressed 16752640 bytes to 354021...
Writing at 0x00010000... (22 %)
A fatal error occurred: Timed out waiting for packet header

wondering what was not correct here. Please help and thank you in advance!

mkspiffs make dist error message: cant find spiffs.h plus solution

I tried ~/esp32/esp-idf/components/mkspiffs-master $ git submodule update --init
I didn't notice anything happening, though. Next I ran make dist
~/esp32/esp-idf/components/mkspiffs-master $ make dist
got error message spiffs.h not found.

There is a subdirectory under mkspiffs-master called spiffs but it is empty. I searched for spiffs.h on my hard drive, and found the likely one in ~/esp32/esp-idf/components/spiffs/spiffs/src/spiffs.h along with several other files. I copied these 7 files into mkspiffs-master/spiffs/src. When I ran make dist, it looks like it worked.

ESP8266 can successfully mount spiffs and open the files that spiffs updates, but the read file fails, possix read() returns -1

git clone https://github.com/igrr/mkspiffs.git
cd mkspiffs
git submodule update --init
make dist BUILD_CONFIG_NAME="-esp-idf" CPPFLAGS="-DSPIFFS_OBJ_META_LEN=4"
./mkspiffs/mkspiffs -c spiffs_py/spiffs_root/ -b 4096 -p 256 -s 0x200000 spiffs.bin

uodate the file system use esptools.py:
python esptool.py --chip esp8266 --port /dev/ttyUSB0 --baud 115200 write_flash -z 0x90000 spiffs.bin

and then,uart print:
example: Initializing SPIFFS
example: Partition size: total: 1920401, used: 2761
example: Reading file
SPIFFS: Failed to update mtime (-10021)
example: Open the file success, fd: [3]
example: Read from ret: [Bug: I read 20 bytes of file, and read() return: -1]
example: Read from file: [Bug:Here I want to print some text of file,but read_buffer is NULL...]
example: SPIFFS unmounted

error when trying to make the project

Dear Mr. Grokhotkov,

When I tried to make this project, I got an error saying that the file “spiffs.h” could not be found. I can I use the one contained in the ESP-IDF?

Best regards

mkspiffs tool does not unpack image files ...

I'm guessing not many people have tried to use this utility other than to create files ...

I have tried using the mkspiffs tool (the one packaged here in releases ... as well as the ones packages with the Arduino/Esp8266 core (v2.5.2 - v2.6.3)) to unpack the file I just created (using the arduion ide toolchain) while the spiffs image works fine after uploading it to the device ... I'm not able to see all the files (only shows the first 3 files) when I use the utility to either --list or --unpack the files ... and 'YES' i'm using the same version of the tool to both create and (try) to unpack ...

I have tried with both windows 10, linux64 & linux32 versions

Has anyone else used this utility to 'unpack'?

Windows help

Windows 10;
Installation run ok (recommend to add comment on zip and diff gnu utils dependency for windows users)
trying to run mkspiffs --version shows that msys-2.0.dll is missing...

mkspiffs not working with the last espressif/esp-idf version

hi,

i use the last espressif/esp-idf with the spiffs (spiffs @ f5e26c4).
I try to make spiffs image with the tools (i tested all versions)

I (51) boot: ## Label Usage Type ST Offset Length
I (95) boot: 5 storage Unknown data 01 82 002f0000 000f8000

./mkspiffs -c ./data -b 8192 -p 256 -s 0xFB000 ./spiffs.bin
esptool.py --port /dev/ttyUSB0 -b 921600 write_flash 0x002f0000 ./spiffs.bin

result:
I (353) spiffs: Initializing SPIFFS
W (383) SPIFFS: mount failed, -10025. formatting...

option -s doesn't work

If specify partition size with -s an image is built which cannot be read. With -i it prints total and used as 0 and 255. The -l option causes seg fault.

It builds image OK when -s not specified but size is too small.

It's version V3.0V3.0-dev-806-gde750e99

Linux system GCC version 6.3.

(Sorry about brevity, men dug up my internet connection and I have to use my phone 😡.)

image comes up empty

Dear Mr. Grokhotkov,

I use the mkspiffs software to create images containing text files. The images are then flashed onto an ESP32. If the mkspiffs software is running on macOS (10.12.6) everything works perfectly. If the software is running on Ubuntu 17.10 however, there are no errors either(neither during image creation, nor when the ESP32 uses them), but the text files come up empty. Do you have any idea, what the problem could be? Thanks for your help in advance.

Best regards

mkspiffs partition seems to be ReadOnly when mounted into an ESP

After mounting a partition made with mkspiffs, all files are readable, but opening a new file, either in "a" or "w" mode is failing.
Also, trying to open existing one in "a" mode will not fail but the content is never written.

Field Updatable SPIFFS

Hi.
I have a project where I am writing the initial code and loading it on to a ESP32 which has 4MB of Flash.
It will read image files from the SPIFFS and display them on a screen. The people that will use this will have an app that I will write on their PC. This app will get them choose image files and I will then convert them in to a SPIFFS file with mkspiffs and then upload it to the ESP32 using the esptool.exe
I will set the maximum limit on the SPIFFS file of 2mb.

What I am stuck on is the parameters needed to create and save an SPIFFS file, and load it up to the ESP32 in a format that can be read by the loaded Arduino program.
I have included the batch file that I use to create and upload the SPIFFS image.
I am also using Arduino 1.8.8 with the the following loaded settings (image)

Can you suggest what I am doing wrong, and how to fix it. I know with the ESP32 I can use OTA,Wifi,Web etc.. But I would prefer a direct connect for ease.
Many thanks for any help.
Dave

Load_SPIFFS.txt

Minimize the size of the SPIFFS image?

Could it be possible to generate an image with only the blocks subsequent from the beginning which actually contains data and skip all the following blocks with only FF in them?

I guess it could then be possible to flash this small image and just erase the following blocks within the file system address range (if a range erase function is available in esptool)

Hereby a lot of upload time could be gained for a 3 MB file system...

BR
/Peter

SPIFFS_write error(-10001): File system is full.

Compiled sketch from Arduino IDE for ESP-01 (1m) with Generic ESP8266 Module and Flash Size 1M (192K SPIFFS).

mkspiffs -c data/ firmware.bin

result:

SPIFFS_write error(-10001): File system is full

During 1st file. I tried changed to 256K with same error. Works from IDE using ESP8266 Sketch Data Upload plugin.

if FileName length 64 char

Hi
can you prepare additional version of mkspiffs for 64 chars max length of FileName ?

Unable to unpack image created by spiffsgen

Hello,

I'm creating spiffs image this way:
spiffsgen.py 40960 --page-size 256 --block-size 4096 mypackdir image.bin
then unpacking with mkspiffs:
mkspiffs.exe -b 4096 -p 256 -s 40960 -u mydestdir image.bin
and getting random numbers in extracted files.

The same issue is with the images downloaded from the chip.
ESP-IDF version is 4.1.
mkspiffs.exe correctly unpacks images created by himself.
e.g. mkspiffs.exe -b 4096 -p 256 -s 40960 -c mypackdir image.bin
The resulting images created by mkspiffs and spiffsgen are different.
I tried several versions from /Releases with the same results.

Please help, maybe I missed some crucial parameter? Thanks.

SPIFFS Newbe question

Ok,
So I've managed to build the spiffs image using mkspiffs, I could not find how to flash it to ESP32...
(I'm using esp-idf framework...).

mkspiffs fails with wear levelling V2

I am using arduino-esp32. In 1.0.2-rc1, SPIFFS partitions created with mkspiffs fail to mount.

Extract fails when first entry depth is over 2

Problem

Let's say I have a SPIFFS which returns the following with a mkspiffs -l call:

44300   /ui/fonts/material-icons.woff2
14696   /ui/fonts/roboto-300.woff2
14748   /ui/fonts/roboto-500.woff2
14600   /ui/fonts/roboto-400.woff2
1406    /ui/favicon.ico
498     /ui/index.html
28294   /ui/main.0bd010c7.js.gz
1418    /ui/manifest.0bd010c7.js
73767   /ui/vendors.0bd010c7.js.gz
3       /config.json

Unpacking such a SPIFFS image fails with a Can not create directory!!! error.

Analysis

After creating the root folder, the program tries to create /ui/fonts/ with a single dirCreate call.

Correction

I see two solutions :

• (Better approach) Recursively creates directories by cutting the path to pieces.
• Ensure files are inserted before directories when building SPIFFS.

aarch64

At this moment you can compile mkspiffs for aarch64, but it's not included in the arduino board manager package.
Could you add a build for aarch64 and add this build to https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_dev_index.json ? Thanks.

unpacking file

Hi,
do you have an implementation to unpack all, or one specific file from the hole spiffs binary image?

I load the hole spiffs image from the esp8266-01 (1M) with this command:
esptool.py --port /dev/ttyUSB0 read_flash 0x6b000 0x10000 /tmp/foo.bin

And i want to get the jason file from inside the image.
The -l option works fine for printing the file list.

Thank you.
Best regards pgollor

0.3.7 produces corrupt images

Tested with latest Arduino where 0.3.7 is the library. Files are existing but have 0 bytes.

Any idea?

Windows build error

I get the following error when trying to build:

$ mingw32-make dist
find: paths must precede expression: `./spiffs/LICENSE'
find: possible unquoted pattern after predicate `-path'?
g++ -std=gnu++11 -Os -Wall   -Itclap -Iinclude -Ispiffs/src -I. -D VERSION=\"0.2.3-6-g983970e\" -D SPIFFS_VERSION=\"0.3.7-5-gf5e26c4\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__   -c -o main.o main.cpp
cc -std=gnu99 -Os -Wall -mno-ms-bitfields  -Itclap -Iinclude -Ispiffs/src -I. -D VERSION=\"0.2.3-6-g983970e\" -D SPIFFS_VERSION=\"0.3.7-5-gf5e26c4\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__   -c -o spiffs/src/spiffs_cache.o spiffs/src/spiffs_cache.c
<builtin>: recipe for target 'spiffs/src/spiffs_cache.o' failed
process_begin: CreateProcess(NULL, cc -std=gnu99 -Os -Wall -mno-ms-bitfields -Itclap -Iinclude -Ispiffs/src -I. -D VERSION=\"0.2.3-6-g983970e\" -D SPIFFS_VERSION=\"0.3.7-5-gf5e26c4\" -D BUILD_CONFIG=\"\" -D BUILD_CONFIG_NAME=\"-generic\" -D __NO_INLINE__ -c -o spiffs/src/spiffs_cache.o spiffs/src/spiffs_cache.c, ...) failed.
make (e=2): The system cannot find the file specified.
mingw32-make: *** [spiffs/src/spiffs_cache.o] Error 2

What is my next move here?