polyfloyd / shady

CLI tool to render GLSL shaders

Ohai! 👋

I'm polyfloyd, a software engineer by profession from The Netherlands 🇳🇱!

• 🧠 I mainly specialize in Rust and Go, but know a bunch of other programming languages as well
• ✨ Interested in computer graphics, signal processing and software reliability, to name a few things
• 📖 My blog is over here
• Mastodon
• ⢀⣢ Involved in running the local Hackerspace, Bitlair

shady's Issues

Help with led cube on Rpi

Hi. I'm new to the world of shaders etc and I'm trying to get shady working on my Rpi 4. I have installed Go on the pi:

pi@raspberrypi:~/rpi-rgb-led-matrix/examples-api-use $ go version
go version go1.16.6 linux/arm

and I'm now trying to work out how to use shady. I've tried this:

../../go/bin/shady -i example2.glsl -ofmt rgb24 -f 20

which complains as follows:

Could initialize engine: failed to call eglCreateContext

I'm not sure if I need OpenGL, and/or what to try next? Sorry, can you help?

Shadertoy buffer support?

Hi! Any plan to add Shadertoy BufferA-D support?

Auto-reload source files

To make debugging easier, source files should be reloaded when written to.

Cubemap Environment

For my LED-cube project it would be cool to be able to render cubemaps. Perhaps a kind of spherical/uv geometry would be easy to use.

Add the ability to set the shader version from source with #version (Enhancement)

Images are upside down

Somewhere along the way, the image is flipped over the X-axis.

Rendering to video files is very slow (buffering problem?), see explanation

This is sorta a long write but I had to explain my findings and a bit of a context

I'm a bit inexperienced with Golang but I dug on the code for a bit lurking for some slow // inefficient code or loops that were degrading performance a low then executing shady then piping its image contents from -ofmt rgb24 to the stdout and piping to FFmpeg / FFprobe.

At first glance I thought this was very strange because on a Python project of mine I draw stuff to a OpenGL surface then get those images from the GPU, pipe to FFmpeg that is being ran with ffmpeg = subprocess.Popen(command, stdout = subprocess.PIPE) when write stuff with ffmpeg.stdin.write(raw_image_data), this is basically if I sent to the stdout my raw arrays then piping to FFmpeg like python script.py | ffmpeg -f rawvideo -i - .... I get about 30 frames per second with 1080p images being piped.

On your Golang code I eventually found the RGB24Format Encode function and saw those two for loops, thought that those were the main issue with such slow speeds (only about 10 fps at 1080p), tried searching somewhere for getting the full array in a single variable then returning it, I got it to work but speeds were still slow.

That was when I remembered about buffering, saw your function openWriter on the file cmd/shady/main.go and it looks like it is an unbuffered writing operation, that means the Go presumably would have to wait until the operation was executed step by step, much like how HDD buffering works, you kinda get a "Promise" of writing from the hard disk, it doesn't happen instantaneously as opposed to (most) programming languages default stdout implementations (for avoiding racing conditions I assume) so stuff are written linearly.

I tried changing the io.Writer to some quick searches to something like bufio.NewWriterSize(os.Stdout, 1024*1024*16) in hope I'd get a 16 MB buffer so we let Go write the frames as fast as it can to the OS's shell buffer.

Unfortunately that doesn't look like the case and I made very few progress after that.

I have some vague ideas on where this can be happening, one of those are simply that getting images out of the GPU is VERY expensive, but I'm a bit skeptical about this since my code can do almost 1080p30 (I'm very much limited by Python's slow nature at this point) and that if on mpv video player I run mpv input.mkv --glsl-shader=./some.glsl -vg=gpu=w=1920:h=1080 -o video_with_shaders.mkv (will only work on Linux OS) it literally saturates my CPU (mostly because lots of information and the encoder asking compute power) and renders about realtime to an output file.

Can you check based on these ideas if by any means we're being limited by OpenGL itself or Golang when writing the images to stdout?

It could be as explained previously some buffering problem as I don't think Golang would be slow enough to not be able to milk lots of raw image data for FFmpeg to render.

Headless support

Shady currently requires a DISPLAY to be set. That's not really needed.

Having headless support would make it easier to run it on servers that don't have Xorg installed and running.

Video quality

Hello, I like Shady a lot, thanks for making the tool! I have a question about video encoding quality.

This is my command:

shady -i marble_exponent_warpy_2.glsl -ofmt rgb24 -g 1366x768 -f 60 \
  | ffmpeg -f rawvideo -pixel_format rgb24 -video_size 1366x768 \
    -framerate 60 -vb 20M -t 12 -i - example.mp4

This is the output I get. Apparently, bitrate setting (vb=20M) did not help.

This is the code I am using.

Unable to install shady on Raspberry Pi (kinect.go:216:9: undefined: unsafe.Slice)

Hello @polyfloyd

Thank you for sharing this amazing project.

Unfortunaly i'm unable to install the project on my Raspberry Pi 4.

I figured out the needed packages with a kind of try and error procedure.
Now i stuck at an "undefined: unsafe.Slice" error while installing.

root@raspberrypi:~# go get -u -v github.com/polyfloyd/shady/cmd/shady
github.com/polyfloyd/shady (download)
github.com/fsnotify/fsnotify (download)
get "golang.org/x/sys/unix": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at //golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
golang.org/x/sys (download)
get "golang.org/x/sys/internal/unsafeheader": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at //golang.org/x/sys/internal/unsafeheader?go-get=1
get "golang.org/x/sys/internal/unsafeheader": verifying non-authoritative meta tag
github.com/go-gl/gl (download)
github.com/go-gl/glfw (download)
github.com/mjibson/go-dsp (download)
github.com/tarm/serial (download)
github.com/polyfloyd/shady/shadertoy/kinect
# github.com/polyfloyd/shady/shadertoy/kinect
go/src/github.com/polyfloyd/shady/shadertoy/kinect/kinect.go:216:9: undefined: unsafe.Slice
go/src/github.com/polyfloyd/shady/shadertoy/kinect/kinect.go:230:11: undefined: unsafe.Slice

dpgkg -l Shows me following freenect library:

ii  libfreenect-dev:armhf          1:0.5.3-1                           armhf        library for accessing Kinect device -- development files
ii  libfreenect0.5:armhf           1:0.5.3-1                           armhf        library for accessing Kinect device

I'm really new to golang. May you have a tip for me how to solve this Problem.

Thanks in advance! 😊 ✌🏻

QUESTION: Ideal hardware to perform headless renders with Shady?

I'd like to purchase a new headless Linux box with a 3D accelerator to speed up some Shady renders. I plan to ssh or chrome remote desktop into the machine and have it run (very cooly) in my basement. The workflow would be something like: ssh into the box, run a shell script to render with Shady, come back in a n hour, and finally pick up the mp4s post-render to use them elsewhere.

For a modest budget (best rendering speedup per dollar) on this new machine, could you suggest the ideal...

platform:

• intel
• amd
• arm64

linux distro

• debian
• ubuntu
• other?

3d video card:

• NVIDIA GEFORCE RTX 3080
• AMD Radeon

Graphics Drivers:

• open source
• proprietary binaries, like nVidia provides for Ubuntu

...to speed up Shady renders in this kind of headless workflow:

export EGL_PLATFORM=surfaceless

shady \
  -i gravity.glsl \
  -ofmt rgb24 \
  -g 3840x2160 \
  -f 60 \
| ffmpeg \
  -f rawvideo -pixel_format rgb24 -video_size 3840x2160 \
  -framerate 60 -t 3600 -i - out.mp4

If it is at all possible to just plug an external GPU (eGPU) into a Raspberry Pi 4B to massively speed up a Shady render... this would my preferred hardware platform!

Any advice would be greatly appreciated! @polyfloyd

V/R,
@589290

Shadertoy support

https://www.shadertoy.com/browse

Output flips vertically every frame when using buffer and video loaders

I'm trying to apply a shader that uses a buffer to my webcam in real time, but I'm having quite a few issues getting it to work. When I run the command:

./shady -i /hdd/Downloads/shaders/video-feedback-flow/image.glsl -ofmt rgb24 -g 640x480 -f 60 | ffplay -f rawvideo -pixel_format rgb24 -video_size 640x480 -framerate 60 -

everything works almost as expected, but at some point along the line, the image gets flipped vertically every frame, which makes the output pretty much unusable. I tried using a video file as input instead of my webcam just in case it was a problem with that, but I got the same result. I also tested outputting to a file with

./shady -i /hdd/Downloads/shaders/video-feedback-flow/image.glsl -ofmt rgb24 -g 640x480 -f 60 -n 300 -o /hdd/Downloads/shaders/video-feedback-flow/test.bin

and playing it back with ffplay, but this had the same result. Finally I tried rendering to an mp4 with

./shady -i /hdd/Downloads/shaders/video-feedback-flow/image.glsl -ofmt rgb24 -g 640x480 -f 60 | ffmpeg -f rawvideo -pixel_format rgb24 -video_size 640x480 -framerate 60 -t 5 -i - /hdd/Downloads/shaders/video-feedback-flow/test2.mp4

but nothing changed that time either. As far as I can tell, this only seems to happen when a shader is using both the video loader and the buffer loader, and each one works fine when used without the other. I'm not particularly knowledgeable about this type of stuff so there's a decent chance I just made a mistake somewhere, but I couldn't figure it out so hopefully someone else will be able to track down the problem.

If it would be helpful, I could attach the files for the shaders I used and/or an example recording of the problem, but to keep this short I'm leaving them off for now. Let me know if you need more information.

Unable to install shady

Really appreciate your work with Shady and making it run on the led-cube. I'm trying to install shady on my mac and when I try to install shady, it gives me the following error:

Command: go get -u -v github.com/polyfloyd/shady/cmd/shady

github.com/polyfloyd/shady (download)
github.com/fsnotify/fsnotify (download)
get "golang.org/x/sys/unix": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at //golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
golang.org/x/sys (download)
get "golang.org/x/sys/internal/unsafeheader": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at //golang.org/x/sys/internal/unsafeheader?go-get=1
get "golang.org/x/sys/internal/unsafeheader": verifying non-authoritative meta tag
github.com/go-gl/gl (download)
github.com/go-gl/glfw (download)
github.com/mjibson/go-dsp (download)
github.com/tarm/serial (download)
github.com/polyfloyd/shady/egl

pkg-config --cflags -- egl

pkg-config: exec: "pkg-config": executable file not found in $PATH

Multiple input files

For modularity, it would be neat to be able to specify multiple input files via repeated -i flags.

Unable to install on macOS (13.4.1)

After instaling pkg-config, mesa, glw and freeglut with MacPorts, I still get this:

Package egl was not found in the pkg-config search path.
Perhaps you should add the directory containing `egl.pc'
to the PKG_CONFIG_PATH environment variable
No package 'egl' found
pkg-config: exit status 1```

(QOL) Mapping shader as a sampler2D texture buffer not implemented for OnScreenEngine, does work on others

First of all,

• I have read on the README.md that NOTE: Buffer support is not very well tested, your mileage may vary.

.. however I really need this (sorta).

Context

I'm coding a sorta "video editor" / layers compositor for music visualization and having layers is essential to this task, not only I can separate the elements but apply effects on each and individually.

Issue / reproducing

I tried manually to set up two shaders with almost the same code called layer1.glsl and layer2.glsl:

void mainImage(out vec4 fragColor, in vec2 fragCoord) {
    vec2 uv = fragCoord.xy / iResolution.xy;
    fragColor = vec4(0.0, 0.0, uv.y, 1);
}

The only difference was that layer2.glsl returns fragColor = vec4(uv.x, 0.0, 0.0, 1); so I could mix the two with some textureLod method and expect a purple at top right corder, black bottom left, red bottom right and blue top left by adding each other's vec4 pixel_color.

However either by defining on the layer2.glsl file the map buffer with:

#pragma map layer1=buffer:/path/to/layer1.glsl;1280x720

Or by giving to the CLI: ./shady -i layer2.glsl -g 1280x720 -map "layer1=buffer:/path/to/layer1.glsl;1280x720"

It just didn't work, outputting the following error:

Error reloading environment: failed to call eglCreateContext, edit: which ended up being a bug on the code where the GL version wasn't being passed properly to the OnScreenEngine render method.

What I tried

See next comments, edited this one for consistency

Error compiling

Hi !
I'm trying to test your shady but I'm getting errors when compiling:

`
pi@raspberrypi:~ $ go get -u github.com/polyfloyd/shady/cmd/shady

github.com/polyfloyd/shady/renderer

go/src/github.com/polyfloyd/shady/renderer/renderer.go:296:40: undefined: errors.Is
go/src/github.com/polyfloyd/shady/renderer/renderer.go:451:41: undefined: errors.Is
`

I've installed all the dependencies it said it needed, but I can't go further ...
Using latest rasbian buster (debian_version 10.4)

I really wanted to try those shaders . Thank you for the instructions and all the work !