Comments (3)
ER-EPR
commented on June 3, 2024
I tested tinydream with latest image, still not working. The backend is not compiled into the binary. I looked into the Dockerfile. And I found the tinydream Go Tag was not set at the second build step (set at the beginning of the Dockerfile, but was overridden later). I tried to set tinydream Go Tag in the Dockerfile and build it myself. But it failed to build the tinydream backend.
from localai.
M0Rf30
commented on June 3, 2024
Need to build localai with GO_TAGS="tinydream"
Apply the model with
curl http://localhost:8080/models/apply -H "Content-Type: application/json" -d '{
"id": "model-gallery@tinydream"
}'And use it with:
curl localhost:8080/v1/images/generations -H "Content-Type: application/json" -d '{
"prompt": "floating hair, portrait, ((loli)), ((one girl)), cute face, hidden hands, asymmetrical bangs, beautiful detailed eyes, eye shadow, hair ornament, ribbons, bowties, buttons, pleated skirt, (((masterpiece))), ((best quality)), colorful|((part of the head)), ((((mutated hands and fingers)))), deformed, blurry, bad anatomy, disfigured, poorly drawn face, mutation, mutated, extra limb, ugly, poorly drawn hands, missing limb, blurry, floating limbs, disconnected limbs, malformed hands, blur, out of focus, long neck, long body, Octane renderer, lowres, bad anatomy, bad hands, text",
"mode": 2, "seed":9000,
"size": "256x256",
"n":2,
"model": "tinydream",
"backend": "tinydream"
}'
here a generated output with name
/var/lib/local-ai/generated/images/tinydream_qid4zt654tzh_15_9000.png
Some DEBUG log
mar 16 22:06:05 unknown local-ai[106967]: 10:06PM INF Loading model 'tinydream_assets' with backend tinydream
mar 16 22:06:05 unknown local-ai[106967]: 10:06PM DBG Model already loaded in memory: tinydream_assets
mar 16 22:06:14 unknown local-ai[106967]: 10:06PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 3 took 26128.534912 Milliseconds.
mar 16 22:06:40 unknown local-ai[106967]: 10:06PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 4 took 26149.714111 Milliseconds.
mar 16 22:07:06 unknown local-ai[106967]: 10:07PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 5 took 26035.990967 Milliseconds.
mar 16 22:07:30 unknown local-ai[106967]: 10:07PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 6 took 23695.460938 Milliseconds.
^[[2~mar 16 22:07:49 unknown local-ai[106967]: 10:07PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 7 took 19015.408936 Milliseconds.
mar 16 22:08:08 unknown local-ai[106967]: 10:08PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 8 took 19120.135010 Milliseconds.
mar 16 22:08:27 unknown local-ai[106967]: 10:08PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 9 took 19012.836914 Milliseconds.
mar 16 22:08:46 unknown local-ai[106967]: 10:08PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 10 took 19028.675049 Milliseconds.
mar 16 22:09:07 unknown local-ai[106967]: 10:09PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 11 took 21369.666992 Milliseconds.
mar 16 22:09:26 unknown local-ai[106967]: 10:09PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 12 took 19163.957031 Milliseconds.
mar 16 22:09:50 unknown local-ai[106967]: 10:09PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 13 took 23933.158936 Milliseconds.
mar 16 22:10:14 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 14 took 24049.186035 Milliseconds.
mar 16 22:10:14 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout Denoising via (UNetModel FP16) took 344412.925049 milliseconds to complete.
mar 16 22:10:14 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Starting Variational Auto-Encoder (VAE) model (AutoencoderKL FP16) to decode images from latent representations...
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout Variational Auto-Encoder (VAE) took 42761.839111 milliseconds to complete.
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: 512x512 inference output took 388459.645996 milliseconds to complete.
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Inference Completed-]: Output image: '/var/lib/local-ai/generated/images/tinydream_qid4zt654tzh_15_9000.png' saved to disk.Output Image location: /var/lib/local-ai/generated/images
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM INF Loading model 'tinydream_assets' with backend tinydream
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG Model already loaded in memory: tinydream_assets
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout /var/lib/local-ai/generated/images/b644286375331.png
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout Tiny Dream 1.7.5: Stable Diffusion Inference (ncnn) in C++ - Copyright (C) PixLab | Symisc Systems(https://pixlab.io/ - https://symisc.net/). All rights reserved.
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Starting Stable Diffusion Inference with the following configuration:
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout Positive Prompt: floating hair, portrait, ((loli)), ((one girl)), cute face, hidden hands, asymmetrical bangs, beautiful detailed eyes, eye shadow, hair ornament, ribbons, bowties, buttons, pleated skirt, (((masterpiece))), ((best quality)), colorful
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout Negative Prompt: ((part of the head)), ((((mutated hands and fingers)))), deformed, blurry, bad anatomy, disfigured, poorly drawn face, mutation, mutated, extra limb, ugly, poorly drawn hands, missing limb, blurry, floating limbs, disconnected limbs, malformed hands, blur, out of focus, long neck, long body, Octane renderer, lowres, bad anatomy, bad hands, text
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout Inference Steps: 15
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout Current Seed Generation: 9000
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Token Embedding - Building Tokens Table from Vocabulary
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Token Embedding - Finished Building (49408) Tokens Table
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Token Embedding - Using Frozen CLIP Text Encoder for encoding prompts...
mar 16 22:10:57 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout Frozen CLIP Text Encoder took 103.312012 milliseconds to complete.
mar 16 22:10:58 unknown local-ai[106967]: 10:10PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Starting UNet Model (UNetModel FP16) to denoise the encoded image latent...
mar 16 22:11:20 unknown local-ai[106967]: 10:11PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 0 took 21429.835938 Milliseconds.
mar 16 22:11:39 unknown local-ai[106967]: 10:11PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 1 took 19083.134033 Milliseconds.
mar 16 22:12:01 unknown local-ai[106967]: 10:12PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 2 took 21443.739014 Milliseconds.
mar 16 22:12:25 unknown local-ai[106967]: 10:12PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 3 took 24364.732910 Milliseconds.
mar 16 22:12:51 unknown local-ai[106967]: 10:12PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 4 took 25878.584229 Milliseconds.
mar 16 22:13:17 unknown local-ai[106967]: 10:13PM DBG GRPC(tinydream_assets-127.0.0.1:37499): stdout [-Info-]: Denoiser step: 5 took 26136.026855 Milliseconds.
extra note
to use 4k upscaler put a height or width value > 512
Tested right now on 2.10.0
from localai.
ER-EPR
commented on June 3, 2024
Hi, @M0Rf30 thanks for the instruction, but I still can't build the tinydream backend. Lots of unused parameter warnings
warning: ignoring '#pragma omp parallel' [-Wunknown-pragmas]
1323.9 255 | #pragma omp parallel for num_threads(opt.num_threads)
and the final error that terminate the build process:
1412.2 make[2]: Leaving directory '/build/sources/go-tiny-dream/ncnn/build'
1412.2 g++ -I./ncnn/build/install/include -I. -I./tiny-dream -funsafe-math-optimizations -Ofast -flto=auto -funroll-all-loops -pipe -march=native -std=c++17 -Wextra -lstdc++ -pthread -flto -fopt-info-vec-optimized -fPIC tinydream.cpp -o tinydream.o -c
1413.7 In file included from ./tiny-dream/tinydream.hpp:75,
1413.7 from tinydream.cpp:6:
1413.7 ./tiny-dream/stb_image_write.h: In function 'int stbi_write_bmp_to_func(void (*)(void*, void*, int), void*, int, int, int, const void*)':
1413.7 ./tiny-dream/stb_image_write.h:514:32: warning: missing initializer for member 'stbi__write_context::context' [-Wmissing-field-initializers]
1413.7 514 | stbi__write_context s = { 0 };
1413.7 | ^
1413.7 ./tiny-dream/stb_image_write.h:514:32: warning: missing initializer for member 'stbi__write_context::buffer' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h:514:32: warning: missing initializer for member 'stbi__write_context::buf_used' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h: In function 'int stbi_write_bmp(const char*, int, int, int, const void*)':
1413.7 ./tiny-dream/stb_image_write.h:522:32: warning: missing initializer for member 'stbi__write_context::context' [-Wmissing-field-initializers]
1413.7 522 | stbi__write_context s = { 0 };
1413.7 | ^
1413.7 ./tiny-dream/stb_image_write.h:522:32: warning: missing initializer for member 'stbi__write_context::buffer' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h:522:32: warning: missing initializer for member 'stbi__write_context::buf_used' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h: In function 'int stbi_write_tga_to_func(void (*)(void*, void*, int), void*, int, int, int, const void*)':
1413.7 ./tiny-dream/stb_image_write.h:613:32: warning: missing initializer for member 'stbi__write_context::context' [-Wmissing-field-initializers]
1413.7 613 | stbi__write_context s = { 0 };
1413.7 | ^
1413.7 ./tiny-dream/stb_image_write.h:613:32: warning: missing initializer for member 'stbi__write_context::buffer' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h:613:32: warning: missing initializer for member 'stbi__write_context::buf_used' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h: In function 'int stbi_write_tga(const char*, int, int, int, const void*)':
1413.7 ./tiny-dream/stb_image_write.h:621:32: warning: missing initializer for member 'stbi__write_context::context' [-Wmissing-field-initializers]
1413.7 621 | stbi__write_context s = { 0 };
1413.7 | ^
1413.7 ./tiny-dream/stb_image_write.h:621:32: warning: missing initializer for member 'stbi__write_context::buffer' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h:621:32: warning: missing initializer for member 'stbi__write_context::buf_used' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h: In function 'int stbi_write_hdr_to_func(void (*)(void*, void*, int), void*, int, int, int, const float*)':
1413.7 ./tiny-dream/stb_image_write.h:789:32: warning: missing initializer for member 'stbi__write_context::context' [-Wmissing-field-initializers]
1413.7 789 | stbi__write_context s = { 0 };
1413.7 | ^
1413.7 ./tiny-dream/stb_image_write.h:789:32: warning: missing initializer for member 'stbi__write_context::buffer' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h:789:32: warning: missing initializer for member 'stbi__write_context::buf_used' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h: In function 'int stbi_write_hdr(const char*, int, int, int, const float*)':
1413.7 ./tiny-dream/stb_image_write.h:796:32: warning: missing initializer for member 'stbi__write_context::context' [-Wmissing-field-initializers]
1413.7 796 | stbi__write_context s = { 0 };
1413.7 | ^
1413.7 ./tiny-dream/stb_image_write.h:796:32: warning: missing initializer for member 'stbi__write_context::buffer' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h:796:32: warning: missing initializer for member 'stbi__write_context::buf_used' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h: In function 'int stbi_write_jpg_to_func(void (*)(void*, void*, int), void*, int, int, int, const void*, int)':
1413.7 ./tiny-dream/stb_image_write.h:1609:32: warning: missing initializer for member 'stbi__write_context::context' [-Wmissing-field-initializers]
1413.7 1609 | stbi__write_context s = { 0 };
1413.7 | ^
1413.7 ./tiny-dream/stb_image_write.h:1609:32: warning: missing initializer for member 'stbi__write_context::buffer' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h:1609:32: warning: missing initializer for member 'stbi__write_context::buf_used' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h: In function 'int stbi_write_jpg(const char*, int, int, int, const void*, int)':
1413.7 ./tiny-dream/stb_image_write.h:1618:32: warning: missing initializer for member 'stbi__write_context::context' [-Wmissing-field-initializers]
1413.7 1618 | stbi__write_context s = { 0 };
1413.7 | ^
1413.7 ./tiny-dream/stb_image_write.h:1618:32: warning: missing initializer for member 'stbi__write_context::buffer' [-Wmissing-field-initializers]
1413.7 ./tiny-dream/stb_image_write.h:1618:32: warning: missing initializer for member 'stbi__write_context::buf_used' [-Wmissing-field-initializers]
1413.8 In file included from tinydream.cpp:6:
1413.8 ./tiny-dream/tinydream.hpp: In member function 'constexpr void tinyDream::_log(const string&) const':
1413.8 ./tiny-dream/tinydream.hpp:89:17: error: call to non-'constexpr' function 'std::function<_Res(_ArgTypes ...)>::operator bool() const [with _Res = void; _ArgTypes = {const char*, int, void*}]'
1413.8 89 | if (xLog) {
1413.8 | ^
1413.8 In file included from /usr/include/c++/11/functional:59,
1413.8 from /usr/include/c++/11/pstl/glue_algorithm_defs.h:13,
1413.8 from /usr/include/c++/11/algorithm:74,
1413.8 from ./tiny-dream/tinydream.hpp:49,
1413.8 from tinydream.cpp:6:
1413.8 /usr/include/c++/11/bits/std_function.h:572:16: note: 'std::function<_Res(_ArgTypes ...)>::operator bool() const [with _Res = void; _ArgTypes = {const char*, int, void*}]' declared here
1413.8 572 | explicit operator bool() const noexcept
1413.8 | ^~~~~~~~
1413.8 In file included from tinydream.cpp:6:
1413.8 ./tiny-dream/tinydream.hpp: In member function 'ncnn::Mat tinyDream::getTextEmbedding(ncnn::Net&, std::string)':
1413.8 ./tiny-dream/tinydream.hpp:491:50: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<std::vector<int> >::size_type' {aka 'long unsigned int'} [-Wsign-compare]
1413.8 491 | for (auto it_tokenized = 0; it_tokenized < tokenized.size(); it_tokenized++)
1413.8 | ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
1414.2 tinydream.cpp: In function 'void logCallback(const char*, int, void*)':
1414.2 tinydream.cpp:20:14: warning: ignoring return value of 'ssize_t write(int, const void*, size_t)' declared with attribute 'warn_unused_result' [-Wunused-result]
1414.2 20 | write(STDOUT_FILENO, static_cast<const void*>(zLogMsg), static_cast<size_t>(msgLen));
1414.2 | ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1414.2 make[1]: *** [Makefile:25: tinydream.o] Error 1
1414.2 make[1]: Leaving directory '/build/sources/go-tiny-dream'
1414.2 make: *** [Makefile:201: sources/go-tiny-dream/libtinydream.a] Error 2
------
Dockerfile:114
--------------------
112 |
113 | # Rebuild with defaults backends
114 | >>> RUN gcc --version && make build
115 |
116 | RUN if [ ! -d "/build/sources/go-piper/piper-phonemize/pi/lib/" ]; then \
--------------------
ERROR: failed to solve: process "/bin/sh -c gcc --version && make build" did not complete successfully: exit code: 2
I tried to add Go Tag tinydream to the dockerfile in the repository and build it. But got the error messages above. Here is the part of the dockerfile I changed:
###################################
###################################
FROM requirements-${IMAGE_TYPE} as builder
ARG GO_TAGS="stablediffusion tinydream tts"
ARG GRPC_BACKENDS
ARG BUILD_GRPC=true
ARG MAKEFLAGS
ENV GRPC_BACKENDS=${GRPC_BACKENDS}
ENV GO_TAGS=${GO_TAGS}
ENV MAKEFLAGS=${MAKEFLAGS}
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
ENV NVIDIA_REQUIRE_CUDA="cuda>=${CUDA_MAJOR_VERSION}.0"
ENV NVIDIA_VISIBLE_DEVICES=all
What should I do now? I don't understand the error message.
from localai.
Related Issues (20)
- “runtime error: invalid memory address or nil pointer dereference” happened,when i send test prompt to cublas-cuda12-core HOT 1
- Cant change DTYPE inside VLLM settings
- feature request: add docker tags "latest-ffmpeg-core" and "latest-ffmpeg" HOT 1
- document use of api_key / API_KEY HOT 4
- localai:v2.10.1-cublas-cuda12-core ”Fails: grpc process not found” HOT 4
- Not correctly accounting for Line Endings in tmpl files.
- LocalAI sends empty chunk to chatbot_ui and closes stream HOT 3
- single binary HOT 3
- [AIO] NVIDIA Detection on WSL2
- [AIO] WSL detection for NVIDIA VRAM size is not working
- Dockerhub images referenced in the documentation don't exist HOT 3
- feat: Retrieval
- gpt4all backend doesn't respect gpu_layers config HOT 2
- ci: latest image tags
- default front-page
- "error":{"code":500,"message":"rpc error: code = Unknown desc = unimplemented","type":""}} HOT 7
- 'response_format' field in OpenAI image creation request does not match OpenAI API spec HOT 1
- can we build non avx cpu aio images? HOT 1
- llamacpp chat/completions response unrelated to prompt on cpu local deploy HOT 1
- feat(llama-cpp): support token count HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from localai.