Pi Zero 2w 32bit build of xnnpack failing about onnxstream

[vitoplantamura]

hi, Building 32-bit XNNPACK/OnnxStream is something I've never done and never seen here in the issues, so this is uncharted territory... Yes, unfortunately it is not possible to use a prebuilt version of XNNPACK, as OnnxStream requires exactly that specific commit referenced in the "git checkout" command. I would try building the latest version of XNNPACK in the master branch (i.e. without running my "git checkout" command), in order to see if this is a problem that has been resolved. Vito

[BranchEnterprise]

I installed a prebuilt XNNPACK from 2020, but thats as far as I went. it became clear that i would need to clone the xnnpack git repo anyway so i could set up the XNNPACK path flag as you instructed. the main conflict for installing the newer XNNPACK prebuild was needing libc6 > 3.4. 32 bit bookworm fullfills this requirement.

Ill give it a go with a fresh bookworm 32 bit install and report back. it may help some who expect to use an OS with working remote desktop/gui . also, raspis main site is still recommending legacy 32bit bullseye within their imaging software for models pi zero, 1,2 and 3 (due to software conflicts).installing non-lite 64 bit on pi02 is almost unuseably slow (i tried). having some success with XNNPACK prebuilds gives me hope. should the ONNXSTREAM build also fail on 32 bit bookworm, i'll make it known. ill reuse the instructions you posted, including the git checkout.

edit -->attempted to build XNNPACK without checkout (legacy bullseye 32 bit os) gave errors like "
CMake Error at CMakeLists.txt:588 (ADD_LIBRARY):
No SOURCES given to target: microkernels-all
CMake Error at CMakeLists.txt:603 (ADD_LIBRARY):
No SOURCES given to target: memory
CMake Error at CMakeLists.txt:608 (ADD_LIBRARY):
No SOURCES given to target: operator-utils
CMake Error at CMakeLists.txt:604 (ADD_LIBRARY):
No SOURCES given to target: microkernel-utils

CMake Generate step failed. Build files cannot be regenerated correctly.
gmake: *** No targets specified and no makefile found. Stop."

it seems weird like i messed something up. i merely copied your instructions without the git checkout line.

can't wait! much thanks

[BranchEnterprise]

hello again!

i've successfully built XNNPACK (with your git checkout) and ONNXSTREAM on 32 bit bookworm OS non-lite (the one with integrated desktop). i have done this without sudo apt ugrade for now. i am looking to try rebuilding after sudo apt upgrade later. However, I am trying to run SD 1.5 using your weights. your readme directed me here: https://github.com/vitoplantamura/OnnxStream/releases/download/v0.1/StableDiffusion-OnnxStream-Windows-x64-with-weights.rar. it seems to be a windows specific file, but i've attempted to extract it nonetheless. extraction yielded many errors.

i am simply trying to gather the setup you used for the 512x512 SD 1.5 images.you've succesfully generated images on the rpi02. "...The third image was generated by my RPI Zero 2 in about 3 hours 1.5 hours (using the MAX_SPEED option when compiling)". i seem to be missing necessary files on extraction (attempted with 7z-full and archiver):
ERROR: Unsupported Method : SD/vae_decoder_qu8/onnx_3A__3A_Mul_5F_882.bin
ERROR: Unsupported Method : SD/vae_decoder_qu8/onnx_3A__3A_Mul_5F_890.bin
ERROR: Unsupported Method : SD/vae_decoder_qu8/onnx_3A__3A_Reshape_5F_251.bin
ERROR: Unsupported Method : SD/vae_decoder_qu8/onnx_3A__3A_Reshape_5F_825.bin
ERROR: Unsupported Method : SD/vae_decoder_qu8/onnx_3A__3A_Reshape_5F_838.bin
ERROR: Unsupported Method : SD/vae_decoder_qu8/range_data.txt
Sub items Errors: 1053
Archives with Errors: 1
Sub items Errors: 1053

this is my attempt to run a 3-step stable diffusion example:

wourldd@wourldd:/OnnxStream/src/build $ ls
CMakeCache.txt CMakeFiles cmake_install.cmake Makefile sd text_encoder_fp32 tokenizer unet_fp16 vae_decoder_fp16 vae_decoder_qu8
wourldd@wourldd:/OnnxStream/src/build $ ./sd --rpi --steps 3 --output result1.p ng
----------------[start]------------------
positive_prompt: a photo of an astronaut riding a horse on mars
negative_prompt: ugly, blurry
output_png_path: result1.png
steps: 3
seed: 721920
----------------[prompt]------------------
WARNING: The merges.txt file is missing from the tokenizer folder.
Running without byte pair encoding results in subpar tokenization.
The file can be downloaded here:
https://huggingface.co/AeroX2/stable-diffusion-xl-turbo-1.0-onnxstream/blob/main /sdxl_tokenizer/merges.txt
Token: "a"
Warning token: "a" was ignored
Token: "photo"
Warning token: "photo" was ignored
Token: "of"
Warning token: "of" was ignored
Token: "an"
Warning token: "an" was ignored
Token: "astronaut"
Warning token: "astronaut" was ignored
Token: "riding"
Warning token: "riding" was ignored
Token: "a"
Warning token: "a" was ignored
Token: "horse"
Warning token: "horse" was ignored
Token: "on"
Warning token: "on" was ignored
Token: "mars"
Warning token: "mars" was ignored
Token: ""
Warning token: "" was ignored
=== ERROR === read_file: invalid size of file.

am i missing a crucial step or set of files here? even adding the merges.txt as instructed in the warning runs, but with no image generation: ...Warning token: "m" was ignored
Token: "a"
Warning token: "a" was ignored
Token: "r"
Warning token: "r" was ignored
Token: "s"
Warning token: "s" was ignored
=== ERROR === read_file: invalid size of file.

[vitoplantamura]

awesome, it seems to work! The error you are getting now is because the archive was not extracted correctly. Please note that this is a RAR file, not a ZIP file. Vito

[BranchEnterprise]

thanks for getting back so soon. i extracted the weights on a windows os and transferred to pi via ftp--now running an example
--rpi-lowmem --steps 3 with swap just in case. So its known, regular --rpi hung on diffusion for 1hr+ with no swap. i cancelled the process before it finished.

I should know within a few hours if it has successfully generated an image. let me know if i should close this thereafter. I also planned on a build AFTER updating via sudo apt update to see if it breaks. i would first flash a fresh 32 bit bookworm install. that would take a few hours to setup.

followup on results for 3 step generation ---> succesfull, taking 26.7668 minutes for a --steps 3 image --rpi-lowmem (pi zero 2w bookworm 32bit full, no full system library upgrade) .

[BranchEnterprise]

now confirming that a fully updated, fresh install (sudo apt-get update, sudo apt-ge upgrade) compiles both XNNPACK and ONNXSTREAM also. the OS is bookworm 32 bit -- Release date: March 15th 2024

[vitoplantamura]

Thanks for your feedback! Vito