djdefrag / realscaler

RealScaler - image/video AI upscaler app (Real-ESRGAN)

RealScaler - image/video AI upscaler app (Real-ESRGAN)

What is RealScaler?

RealScaler is a Windows app powered by RealESRGAN AI to enhance, upscale and de-noise photos and videos.

Other AI projects.🤓

• https://github.com/Djdefrag/QualityScaler / QualityScaler - image/video AI upscaler app
• https://github.com/Djdefrag/FluidFrames.RIFE / FluidFrames.RIFE - video AI interpolation app (RIFE-HDv3)

Credits.

Real-ESRGAN - https://github.com/xinntao/Real-ESRGAN

How is made. 🛠

RealScaler is completely written in Python, from backend to frontend. External packages are:

• AI -> torch / onnxruntime-directml
• GUI -> customtkinter
• Image/video -> OpenCV / moviepy
• Packaging -> nuitka

Requirements. 🤓

• Windows 11 / Windows 10
• RAM >= 8Gb
• Any Directx12 compatible GPU with >= 4GB VRAM

Features.

• Easy to use GUI
• Images and Videos upscale
• Multiple AI models
• Automatic image tiling and merging to avoid gpu VRAM limitation
• Resize image/video before AI upscaling
• Interpolation between the original and upscaled image/video
• Multiple Gpu support
• Compatible images - png, jpeg, bmp, webp, tif
• Compatible video - mp4, wemb, gif, mkv, flv, avi, mov, qt

Next steps. 🤫

• 1.X versions
  • Switch to Pytorch-directml to support all Directx12 compatible gpu (AMD, Intel, Nvidia)
  • New GUI with Windows 11 style
  • Include audio for upscaled video
  • Optimizing video frame resize and extraction speed
  • Multi GPU support (for pc with double GPU, integrated + dedicated)
  • Python 3.10 (expecting ~10% more performance)
• 2.X versions
  • New, completely redesigned graphical interface based on @customtkinter
  • Upscaling images and videos at once (currently it is possible to upscale images or single video)
  • Upscale multiple videos at once
  • Choose upscaled video extension
  • Interpolation between the original and upscaled image/video
  • More Interpolation levels (Low, Medium, High)
  • Show the remaining time to complete video upscaling
  • Support for SRVGGNetCompact AI architecture
  • Metadata extraction and application from original file to upscaled file (via exiftool)
• 3.X versions
  • New AI engine powered by onnxruntime-directml (https://pypi.org/project/onnxruntime-directml/)
  • Python 3.11 (~10% performance improvements)
  • Display images/videos upscaled resolution in the GUI
  • FFMPEG 6.1.1 (latest release)
  • Python 3.12
  • Video upscaling pause and restart

Some Examples.

Videos

• Original

• RealESRGANx4 - upscaled in 3 minutes and 23 seconds

RealESRGAN_plus_x4.mp4

• RealESR_Gx4 - upscaled in 57 seconds

RealESR_generalv3_x4.mp4

Images!

realscaler's Issues

Unable to select proper GPU

Hello,
I have 3 GPUs on my laptop
Integrated
built-in 2060
and 3060 in eGPU enclose.

I can only select integrated, I'd like to be able to switch to 3060, I dig some digging in the code I can only find
providers = [('DmlExecutionProvider', {'device_id': '0'})] for "high performance" but that just uses the Intel integrated.
Any idea?

Feature - Add ability to disable resize operation.

I'm often working with 480p DVD images and find that resizing doesn't benefit me in any way. Yet the resize operation takes a very long time.

As a user, I'd like the ability to turn that "feature" off. If this option already exists, I cannot locate it.

Used up ~1TB of space and crashed, no disk space left

I have quite a large file to process. 521 minutes @ 938531 frames, 30fps, filezie 10.5GB. I have a crap GPU (2gb VRAM).

Settings used: RealESR-Gx4, Full precision, Interpolation yes, .png, .mp4 output (input is .mp4), using my GPU GTX 1050, 2GB of vRAM, 50% resolution, 4 CPU count (have 12 but didn't want to slow down other things).

Anyways, went 3 days doing just fine. Was at 43% complete. It generated something like 1.4 million files in RealScaler_temp. In doing so, it ate up all space on the machine - 1TB worth. Then, it naturally crashed the upscale as well as other running apps.

How can one control the amount of files/diskspace used? I'm hardlimited to 1TB. Prefer not to push over 500GB. I would think, and hope, it could assemble and remove tons of files as it goes otherwise it seems to need 3-4TB just to do a 10GB file.

No Such File error

Upscale failed error

Upscale Failed

Feature - Add Encoding Quality Presets

In addition to scaling operations I'd like the ability to also select the encoding quality.

These can be a handful of presets for common devices.

Presets like:

• Streaming - Low
• Streaming - Medium
• Streaming - High
• Archival - Medium
• Archival - High
• Editing/Production - High
• Editing/Production - Highest

Alternatively:
Always use something like "Editing/Production - High" and inform the user to perform transcoding in another software.

Request: add a Face Enhancement option

Hello
I bought RealScaler 2.4 today and it works great.
Is there a possibility to add an option for Face Enhancement?
Like what GFPGAN do.

Trial Version

Hey, i would like to test this. Is there like a trial version or so to test it?

error while switching to high performace gpu option

Feature - Add ability to configure "batch size" or auto-detect better values

Currently, when using this tool, my RTX 3080 Ti is pretty under-utilized, especially with VRAM. This headroom indicates I could likely process more images per batch.

Given that I already supply VRAM/RAM information this could be auto-detected. However, a slider of reasonable values based on the RAM selection and video image size could also work.

Feature Request- Deinterlace Video

I propose a feature to detect if a video is interlaced, and if so, convert it to progressive. Ability is available with open source tools and just needs a GUI.

1. The tool Mediainfo can be set to "Text" and report if a video is interlaced or progressive. OR Maybe AVISynth can do it.

2. Then use a tool like AVISynth to deinterlace. Or FFMPERG or whatever.

Plans for MacOS version?

Hi there,

I'm currently using this tool on my Mac M1 with Windows 11 VM running. I bought it on itch.io, installed it in my VM but having mixed results.

Are there any plans for a native MacOS version?

Being able to restart a pending upscale

It would be amazing if we could pause and be able to continue an ongoing upscale process.