Coder Social home page Coder Social logo

capsize-games / airunner Goto Github PK

View Code? Open in Web Editor NEW
192.0 5.0 16.0 10.17 MB

Stable Diffusion and LLMs offline on your own hardware

Home Page: https://capsizegames.itch.io/ai-runner

License: GNU General Public License v3.0

Python 99.18% Dockerfile 0.40% Shell 0.31% Batchfile 0.10%
ai ai-art art asset-generator draw drawing drawing-app illustration python stable-diffusion deep-learning image-generation img2img torch upscaling text-to-image text-to-speech vision-processing kandinsky multimodal

airunner's Introduction

Banner Discord Windows Build Linux Build PyPi GitHub GitHub last commit GitHub issues GitHub closed issues GitHub pull requests GitHub closed pull requests


AI RUNNER

v3.0.0.dev

AI Runner can be compiled with pyinstaller however the current version is not yet stable (check the releases for stable versions).

Version 3.0 is a major upgrade which aims to move the application out of prototype stage and into a more stable and user-friendly state.

It comes with a new UI, new features, and a more robust codebase, security updates, and a more streamlined installation process and much more.

We plan to release future distributions via Snap on Linux.

See Privacy and Security for more information.


Stable Diffusion on your own hardware

img_3.png


โญ Features

AI Runner is a multi-modal AI interface which allows you to run open-source large language models and AI image generators on your own hardware.

Feature Description
๐Ÿ—ฃ๏ธ Communication
โœ… Voice-based chatbot conversations Have conversations with a chatbot using your voice
โœ… Text-to-speech Convert text to spoken audio
โœ… Speech-to-text Convert spoken audio to text
โœ… Vision-to-text Extract text from images
โœ… Text generation with LLMs Generate text using large language models
๐ŸŽจ Image Generation
โœ… Stable Diffusion (all versions) Generate images using Stable Diffusion
๐Ÿ”œ Kandinsky Generate images using Kandinsky
โœ… Near Real-Time Drawing Draw and generate images in near real-time
โœ… Text to Image (aka TextToImage / Txt2Img) Generate images from textual descriptions
โœ… Image to Image (aka ImageToImage / Image2Image) Generate images based on input images
๐Ÿ–ผ๏ธ Image Manipulation
โœ… Inpaint and Outpaint Modify parts of an image while maintaining context
โœ… Pix2Pix Transform images from one domain to another
โœ… Depth to Image (aka DepthToImage / Depth2Img) Generate images from depth maps
โœ… Controlnet Control image generation with additional input
โœ… LoRA Efficiently fine-tune models with LoRA
โœ… Textual Embeddings Use textual embeddings for image generation control
๐Ÿ”œ Upscale with GFPGAN Use textual embeddings for image generation control
๐Ÿ”ง Utility
โœ… Run offline, locally Run on your own hardware without internet
โœ… Fast generation Generate images in ~2 seconds (RTX 2080s)
โœ… Run multiple models at once Utilize multiple models simultaneously
โœ… Drawing tools Built-in tools for drawing and image manipulation
โœ… Image filters Apply various filters to images
โœ… Dark mode Comfortable viewing experience in low-light environments
โœ… Infinite scrolling canvas Seamlessly scroll through generated images
โœ… NSFW filter toggle Help control the visibility of NSFW content
โœ… NSFW guardrails Help prevent generation of harmful content
โœ… Standard Stable Diffusion settings Easily adjust standard Stable Diffusion parameters
โœ… Fast load time, responsive interface Enjoy a smooth and responsive user experience
โœ… Pure python No reliance on a webserver, pure python implementation

๐Ÿ’ป System Requirements

Minimum system requirements

  • Cuda capable GPU
  • 6gb of RAM
  • 6gb of disc space to install AI Runner

Recommended system specs

  • RTX 2080s or higher
  • 32gb of RAM
  • 100gb disc space

๐Ÿ”ง Installation

Linux

  1. Open your file explorer and navigate to the directory containing the install.sh script
  2. Open the terminal using the keyboard shortcut Ctrl + Alt + T
  3. Drag the install.sh script into the terminal and press Enter
  4. Follow the on-screen instructions

๐Ÿš€ Running AI Runner

Linux

  1. Open the terminal using the keyboard shortcut Ctrl + Alt + T
  2. Navigate to the directory containing the run.sh script (cd ~/airunner for example)
  3. Run the bin/run.sh script by typing ./bin/run.sh and pressing Enter
  4. AI Runner will start and you can begin using it after following the on-screen setup instructions

โœ๏ธ Using AI Runner

Instructions on how to use AI Runner can be found in the wiki


๐Ÿ’พ Compiling AI Runner

Clone this repository

git clone https://github.com/Capsize-Games/airunner.git
cd airunner

Build from source

pip install -e .
pip install pyinstaller
bash build.dev.sh

๐Ÿ”ฌ Unit tests

Run a specific test

python -m unittest src/airunner/tests/test_draggable_pixmap.py

Test coverage is currently low, but the existing tests can be run using the following command:

python -m unittest discover tests

Test coverage

Run tests with coverage tracking:

coverage run --source=src/airunner --omit=__init__.py,*/GFPGAN/*,*/data/*,*/tests/*,*_ui.py,*/enums.py,*/settings.py -m unittest discover src/airunner/tests

To see a report in the terminal, use:

coverage report

For a more detailed HTML report, run:

coverage html

View results in htmlcov/index.html.


Privacy and Security

Although AI Runner v3.0 is built with Huggingface libraries, we have taken care to strip the application of any telemetry or tracking features.

The main application itself is unable to access the internet, and we are working towards properly sandboxing certain features to ensure user privacy and security.

As this application evolves we will migrate away from the Huggingface libraries.

Internet access

The core application is incapable of access the internet. However there are two features which require internet access. These two features are the setup wizard and the model manager.

Each of these tools are isolated in their own application windows which are capable of directly accessing and downloading files on Huggingface.co and civitai.com (depending on the given URL). Any other URL will be blocked.

The Huggingface Hub library is not used to access these downloads.

For more information see src/security/no_internet_socket.py

Disc access

Write access for the transformers library has been disabled, preventing it from creating a huggingface cache directory at runtime.

The application itself may still access the disc for reading and writing, however we have restricted reads and writes to the user provided airunner directory (by default this is located at ~/.airunner).

All other attempts to access the disc are blocked and logged for your review.

For more information see src/security/restrict_os_access.py.

Huggingface Hub

The Huggingface Hub is installed so that Transformers, Diffusers and other Huggingface libraries will continue to function as expected, however it has been neutered to prevent it from accessing the internet.

The security measures taken for this library are as follows

  • Prevented from accessing the internet
  • Prevented from accessing the disc
  • All environment variables set for maximum security
  • All telemetry disabled

Planned security measures for Huggingface Libraries

We plant o remove the Huggingface libraries from the application in the future. Although the architecture is currently dependent on these libraries, we will migrate to a better solution in the future.

airunner's People

Contributors

dependabot[bot] avatar gitcodeboi1654 avatar w4ffl35 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

airunner's Issues

CivitAI downloader

  • Ability to paste a link and download a model from CivitAI
  • Embeddings download handling
  • LoRA download handling

error when exporting JPG files

error encountered when exporting to jpg format:

OSError: cannot write mode RGBA as JPEG

convert image to RGB prior to exporting JPEG.

ModuleNotFoundError: No module named 'diffusers'

Describe the bug
When trying to run from source (in a docker environment with ubuntu 22.04, which I'll share shortly) seems like aihandler is missing to declare a dependency on diffusers as I get the error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/aihandler/runner.py", line 1162, in generator_sample
    self._generate(data, image_var=image_var)
  File "/usr/local/lib/python3.10/dist-packages/aihandler/runner.py", line 1077, in _generate
    self._initialize()
  File "/usr/local/lib/python3.10/dist-packages/aihandler/runner.py", line 636, in _initialize
    self._load_model()
  File "/usr/local/lib/python3.10/dist-packages/aihandler/runner.py", line 486, in _load_model
    "scheduler": self.scheduler,
  File "/usr/local/lib/python3.10/dist-packages/aihandler/runner.py", line 166, in scheduler
    import diffusers
ModuleNotFoundError: No module named 'diffusers'

To Reproduce
TODO

Expected behavior
Use the app normally :)

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: Ubuntu 20.04 running a docker with ubuntu 22.04
  • Version main branch

Layer spacing

Layers should not have large gabs between them. As more layers are added the gaps close. Potential solution is using a vertical spacer in the template but since the layers are in a scroll frame that might not work.

Program crashed after installing lora, and also deleted my AI model

Describe the bug
I downloaded the Stable Diffusion 1.4 model and put it in a completely separate directory to AI runner. Did a couple of generations, then installed the Lora extension.

Next time I tried to start AI runner, it would run through the startup script, but die before the program launched. I was not able to capture the console log when it was dying, but I did see an orange square drawn around a segment of code before it died.

After multiple attempts at removing the AI Runner directory, and manually cleaning up any files I could fine under C:\User, I finally managed to get AI runner started again.

I am running on Windows 10, running AI Runner from one of my secondary NVME drives.

Failed to run airunner from pre-built version (from Ubuntu 20.04)

Describe the bug
I downloaded the latest version of airunner as a zip file from https://capsizegames.itch.io/ai-runner extracted it, executed airunner and it crashed with:

[18042] PyInstaller Bootloader 5.x
[18042] LOADER: executable is /home/sam/Downloads/ai-runner-ubuntu/airunner
[18042] LOADER: homepath is /home/sam/Downloads/ai-runner-ubuntu
[18042] LOADER: _MEIPASS2 is NULL
[18042] LOADER: archivename is /home/sam/Downloads/ai-runner-ubuntu/airunner
[18042] LOADER: Cookie found at offset 0x29E9FC0
[18042] LOADER: No need to extract files to run; setting up environment and restarting bootloader...
[18042] LOADER: LD_LIBRARY_PATH=/home/sam/Downloads/ai-runner-ubuntu
[18042] PyInstaller Bootloader 5.x
[18042] LOADER: executable is /home/sam/Downloads/ai-runner-ubuntu/airunner
[18042] LOADER: homepath is /home/sam/Downloads/ai-runner-ubuntu
[18042] LOADER: _MEIPASS2 is /home/sam/Downloads/ai-runner-ubuntu
[18042] LOADER: archivename is /home/sam/Downloads/ai-runner-ubuntu/airunner
[18042] LOADER: Cookie found at offset 0x29E9FC0
[18042] LOADER: Already in the child - running user's code.
[18042] LOADER: Python library: /home/sam/Downloads/ai-runner-ubuntu/libpython3.10.so.1.0
[18042] Error loading Python lib '/home/sam/Downloads/ai-runner-ubuntu/libpython3.10.so.1.0': dlopen: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.35' not found (required by /home/sam/Downloads/ai-runner-ubuntu/libpython3.10.so.1.0)

To Reproduce
Steps to reproduce the behavior:
Download ai-runner-ubuntu.zip (Version 1.8.17).
Unzip: unzip -d ai-runner-ubuntu ai-runner-ubuntu.zip
Execute: cd ai-runner-ubuntu && ./airunner

Expected behavior
I guess I expected the app to open!

Desktop (please complete the following information):

  • OS: Ubuntu 20.04
  • Browser: Irrelevant
  • Version: Version 1.8.17 of airunner

Additional context
I'll debug a bit, see if I can patch it up. Otherwise I'll try from source.

Great work @w4ffl35 , though!

Add txt2vid

  • add new tab for txt2vid
  • generate video and save in runner
  • display video in client

Line drawing deletion bug

steps to reproduce

  1. draw lines
  2. create line layer
  3. draw lines on new layer
  4. move top layer down
  5. delete both layers
Error caught was: 'NoneType' object has no attribute 'origin' - 51
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Traceback (most recent call last) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ in paintEvent:152                                                                                โ”‚
โ”‚                                                                                                  โ”‚
โ”‚ in draw:62                                                                                       โ”‚
โ”‚                                                                                                  โ”‚
โ”‚ in current_layer:33                                                                              โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
IndexError: list index out of range
QBackingStore::endPaint() called with active painter; did you forget to destroy it or call QPainter::end() on it?

Audio Generation

  • Add tab for audio generation
  • Generate and save in runner
  • Play audio in client

Layer color

The green layer color is ugly. Additionally the text is hard to see when the layer is selected.

fix the build

Currently the build process is cumbersome with two docker images (one for each os supported). No artifact is produced from the build (build script pushes to docker after building).
Ideally we would have 2 stages

  1. Build code and produce artifact
  2. Download artifact and deploy to itch.io

Currently the github action fails.

  1. Windows reports running out of disc space - splitting into two separate actions might prevent this error
  2. Linux is able to pull the docker container and build, but fails to upload the artifact

It may be that the artifact is too large and we must simply push to docker from within the image as was the case with v1.8.21


Thus far all efforts have resulted in a broken build. Next step is to

  1. Separate os actions (build Linux and win separately)
  2. Use butler from within docker rather than producing an artifact

This is similar to the previous setup, the difference being that we have a single docker image.

If windows continues to time out we may need to produce two docker containers after all.

add the pypi build files

Prior to adding the docker build workflow to github actions yesterday I had a pypi build action which would checkout the latest code, build it and push to pypi. I deleted it because the docker image required lots of releasing and deleting of the same version. Rather than constantly trying to push to pypi during this, I deleted the action file with the intent to restore it after getting docker finalized.

add windows build to cloud

Add the files required to build for windows on docker in the cloud.

These files reside in a separate repo and must be combined with airunner just as I did with the Ubuntu build files.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.