I have a 289GB folder of 960x720 that I want to upscale to 1920x1080.

I am very interested in your program but I need asymmetric upsize to work (as I think most would.)
960x720 is quite common on older 4:3 animes and there's a group that wants to upscale and a group that says leave it alone.

I say upsize them all and most think that way. I'm trying to do my part by upscaling this MASSIVE hand drawn epic called Legend of the Galactic Heroes to 1080 with tons of picture correction I call the "NVIDIA Effect" by balancing color and upping the frame speed lowering brightness and white wash these old animes are famous for. NVIDIA also has a sharpening function and noise reduction all based on CUDA. Im INCREDIBLY excited about AI helping us upgrade these old films to modern style with what could become almost DIGITAL PERFECTION with enough AI "thought" put into keeping these upscales pure and beautiful with AI's help.

Let me know if you can make the changes necessary for my upscale to work. I would LOVE to try an AI guided hand in this upscale. Im not minimizing the amount of work it would take because I simply have no idea. If its too much work just say so, otherwise Id be happy to share my outcomes with you if you are interested in doing this with me. I see the program and understand it a little bit but the actual logic behind it is beyond me. (at least for the moment). But I intend to stare at it long enough the logic will hit me eventually.

Any instruction or help understanding the AI logic would be appreciated if you feel like sharing.
Its an amazingly short program.

Anyway. let me know UP or DOWN on the idea so if your not interested Ill move on and just use Spline36 to upscale this whopper.

When I upscale I also edit out the excess intro and exit music of the nearly 200 video files which makes for less data to seed in torrents and easy viewing without all the fast forwarding through the SAME BORING INTROS AND EXITS. and sometimes even soda and other drink ads. GOOD GOD. I edited almost 2000 minutes of this crap in one series I upscaled.

Any, I hope u get this and we can try this experiment to beat all the other upscale algorithms.

cheers mate
Randy Kuehn
[email protected]

First I'd like to thanks for your very good work.

I was surprised at the speed of vapoursynth tests you showed and want to perform the test myself, however as a fresher to mpv and vs-placebo I got some import error at the very beginning, due to vapoursynth's weak error indication system I can't clearly identify where the error occurs. So I would like to ask you if you can share your test script.

As a side note my vs script and error message is as follows:

import vapoursynth as vs
core = vs.core

clip = core.lsmas.LWLibavSource(r'test.hevc')
# format: YUV444P16, width = 1920px height= 1080px
clip = core.placebo.Shader(clip, width=clip.width*2, height=clip.height*2, 
shader=r'‪absolute\path\to\AiUpscale_Fast_2x_LineArt.glsl')
clip.set_output()

error message:

2021-05-30 00:47:35.143
Failed to evaluate the script:
Python exception: placebo.Shader: Failed reading shader file!

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 2244, in vapoursynth.vpy_evaluateScript
File "C:\Documents\Compress\test.vpy", line 7, in 
clip = core.placebo.Shader(clip, width=clip.width*2, height=clip.height*2,
File "src\cython\vapoursynth.pyx", line 2069, in vapoursynth.Function.__call__
vapoursynth.Error: placebo.Shader: Failed reading shader file!

Platform: latest vs-placebo/aiupscale release at May 30th, python3.8 on windows10.
thanks!

But I put “Shaders” and "AIUPscale.avsi" in somewhere like "\Desktop\Study\Filters\AviSynth AiUpscale\AiUpscale.avsi")
Parameters:
AiUpscale(Factor=1, Luma="HQ Sharp", Chroma="Lanczos", Mode="LineArt", CPlace="MPEG2", OutDepth=10)

I have copied the avisynth script, and the shaders folder, into my 64 bit plugins directory, but this is the result I get, when trying to invoke the script...

Script error: There is not function named 'ConvertToShader'.

Sorry, I do not see the HQ and VDSR glsl files in the release list. Is it too slow to use in the mpv? I want to test that in the mpv if possible.

Hello!
For some reason, the update can't load one of the pixel shaders. I'm certain I've updated the .avsi for this and the .dll for AviSynthShader. Please help, I'm at a loss.

Use MAD (Mean Absolute Deviation) pooling, it's more accurate than mean pooling with SSIM metric (I, honestly, would trust PSNR more than SSIM with mean pooling).
This is what I'm using for evaluation (using >2 scale levels doesn't make much of a difference):

import sys
from PIL import Image
import numpy as np
from scipy.ndimage import gaussian_filter

WEIGHTS = [0.0448]#, 0.2856, 0.3001, 0.2363, 0.1333]

def msssim(file1, file2):
    img1 = Image.open(file1).convert('RGB')
    img2 = Image.open(file2).convert('RGB')

    width, height = img1.size
    img1 = np.frombuffer(img1.tobytes(), dtype=np.uint8).reshape(height, width, 3) / 255
    img2 = np.frombuffer(img2.tobytes(), dtype=np.uint8).reshape(height, width, 3) / 255
    
    img1 = np.where(img1 > 0.04045, np.power((img1 + 0.055) / 1.055, 2.4),  img1 / 12.92)
    img2 = np.where(img2 > 0.04045, np.power((img2 + 0.055) / 1.055, 2.4),  img2 / 12.92)

    img1 = 0.2126 * img1[:,:,0] + 0.7152 * img1[:,:,1] + 0.0722 * img1[:,:,2]
    img2 = 0.2126 * img2[:,:,0] + 0.7152 * img2[:,:,1] + 0.0722 * img2[:,:,2]

    mssim = []
    for i in range(len(WEIGHTS)):
        mssim.append(ssim(pow(img1,1./2.2), pow(img2,1./2.2), i, i<len(WEIGHTS)-1))
        img1 = gaussian_filter(img1, 1.08, truncate=1.5)[::2,::2]
        img2 = gaussian_filter(img2, 1.08, truncate=1.5)[::2,::2]

    return np.sum(np.multiply(np.stack(mssim), WEIGHTS)) / np.sum(WEIGHTS)

def mad(x, l):
    return np.mean(np.absolute(x - np.power(np.mean(x), np.power(.5, l)))) # np.mean(np.absolute(x - np.mean(x if l==0 else np.sort(x, axis=None)[-int(x.size//1.5):])))

def ssim(L1, L2, lvl, cs_map):
    C1=(0.01)**2
    C2=(0.03)**2
    sd, t = 1.5, 3 #kernel radius = round(sd * truncate)

    mu1 = gaussian_filter(L1, sd, truncate=t)
    mu2 = gaussian_filter(L2, sd, truncate=t)
    mu1_sq = mu1 * mu1
    mu2_sq = mu2 * mu2
    mu1_mu2 = mu1 * mu2
    sigma1_sq = gaussian_filter(L1 * L1, sd, truncate=t) - mu1_sq
    sigma2_sq = gaussian_filter(L2 * L2, sd, truncate=t) - mu2_sq
    sigma12 = gaussian_filter(L1 * L2, sd, truncate=t) - mu1_mu2

    if cs_map:
        value = (2.0*sigma12 + C2)/(sigma1_sq + sigma2_sq + C2)
    else:
        value = ((2.0*mu1_mu2 + C1)*(2.0*sigma12 + C2))/((mu1_sq + mu2_sq + C1)*
                    (sigma1_sq + sigma2_sq + C2))

    return mad(value, lvl)

def main():
    for arg in sys.argv[2:]:
        score = msssim(sys.argv[1], arg)
        print(str(score) + "\t" + arg)

if __name__ == '__main__':
    main()

I am trying to use the AiUpscale plugin and have problems!

Software:
Windows 10 Pro - x86-64
Avisynth+ 3.7.0 (None Standard Directory - C:\Users\edit1\AviSynth+)
AviSynthShader 1.6.6
AviSynth AiUpscale v1.2.0

Error
Script error: There is no function named 'ConvertToShader'
(C:/Users/edit1/AviSynth+/plugins64+/AiUpscale.avsi, line 230)
(C:/Users/edit1/AviSynth+/plugins64+/AiUpscale.avsi, line 232)
(C:/Users/edit1/AviSynth+/plugins64+/AiUpscale.avsi, line 146)
(C:/Users/edit1/AviSynth+/plugins64+/AiUpscale.avsi, line 147)

Have I missed something?

Hi, testing some shaders including

`#glsl-shaders="~~home/shaders/AI_upscale/AiUpscale_Fast_2x_Photo.glsl"
#glsl-shaders="~~home/shaders/AI_upscale/AiUpscale_Fast_Sharp_2x_Photo.glsl"
#glsl-shaders="~~home/shaders/AI_upscale/AiUpscale_HQ_2x_Photo.glsl"
#glsl-shaders="~~home/shaders/AI_upscale/AiUpscale_HQ_3x_Photo.glsl"
#glsl-shaders="~~home/shaders/AI_upscale/AiUpscale_HQ_4x_Photo.glsl"
#glsl-shaders="~~home/shaders/AI_upscale/AiUpscale_HQ_Sharp_2x_Photo.glsl"
#glsl-shaders="~~home/shaders/AI_upscale/AiUpscale_HQ_Sharp_3x_Photo.glsl"

glsl-shaders="~~home/shaders/AI_upscale/AiUpscale_HQ_Sharp_4x_Photo.glsl"
#glsl-shaders="~~home/shaders/AI_upscale/AiUpscale_HQ_4x_LineArt.glsl"
#glsl-shaders="~~home/shaders/AI_upscale/AiUpscale_HQ_Sharp_4x_LineArt.glsl"
`

But when I play a 1080p video on a 4K HDR screen, I don't see any proof the shader loaded. The logs show them loading. So how can I test if they're working and upscaling? Am I missing something?

Thanks! An obvious newbie :)