Patch: Support for "Long Paths" on Windows about ffmpeg-builds

[nihil-admirari]

long_paths_utf8.patch seems to work. @Blacklands, please test: https://github.com/nihil-admirari/FFmpeg-Builds/releases.

If everything works fine, I'll try to submit the patch to FFmpeg developers, which is quite a hassle considering that they do mailing list-driven development.

[pukkandan]

I don't mind adding this. But I don't see the patch source in any of the links you provided

[Blacklands]

Yes, it doesn't seem to be there. I couldn't find it, so far, unfortunately.
I was hoping it might be possible without the source, for people who are more familiar with the manifest stuff.

The comments on the FFmpeg ticket may have some more detail?
Otherwise, Patman86 seems to be the author of the patched builds, maybe they could be contacted about it?

[Blacklands]

here is some more information.
Apparently, the manifest needs to be embedded into the application when it is built. The Microsoft docs describe this.

I've used Resource Hacker (any hex editor works, too, just search for the text-string <assembly xmlns=) to check for the manifest in Patman86's patched FFmpeg version, and confirmed that it exists there.

<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
    <assemblyIdentity type="win32" name="FFMPEG" version="1.0.0.0"></assemblyIdentity>
    <application xmlns="urn:schemas-microsoft-com:asm.v3">
        <windowsSettings>
            <longPathAware xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">true</longPathAware>
            <activeCodePage xmlns="http://schemas.microsoft.com/SMI/2019/WindowsSettings">UTF-8</activeCodePage>
        </windowsSettings>
    </application>
</assembly>

This seems to enable the application to use the "updated" Windows API with Long Path support, if enabled on the target system.
The limitation is that this (of course) doesn't help for any cases where the application explicitly defines and uses the "normal" maximum length in its source code. (Apparently, the ancient MAX_PATH Windows macro is always 260.) That would need to be separately patched, and more work.
But it seems like adding the manifest would be easy to incorporate, and go most of the way already (since it seems to work fine for the projects using the patched FFmpeg?).

[nihil-admirari]

FFmpeg does use MAX_PATH in its sources:

• https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/avisynth.c#L575
• https://github.com/FFmpeg/FFmpeg/blob/master/compat/w32dlfcn.h#L45
• https://github.com/FFmpeg/FFmpeg/blob/master/fftools/cmdutils.c#L2077

Enabling long path support is not a simple manifest change.

However, if you have short paths to all of your AviSynth scripts, DLLs and preset files (which seem to be the things affected by MAX_PATH from looking at the sources), and you don't mind subtle potential problems down the road, you should be able to create an external ffmpeg.exe.manifest with the content you just described and put it alongside ffmpeg.exe.

Binaries from this repo don't have internal manifests at all, so there should be no problems, but if Windows refuses to load an external manifest, a little registry tweak may be necessary:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\SideBySide]
"PreferExternalManifest"=dword:00000001

It seems that GCC lacks an analogue of mt.exe; thus manifests are treated like ordinary resources. As a wild guess, placing inside fftools the files ffmpeg.manifest and ffmpeg.rc with the following content:

#include <windows.h>

CREATEPROCESS_MANIFEST_RESOURCE_ID RT_MANIFEST "ffmpeg.manifest"

should do the trick, since .rc files are already processed by the build system, but I haven't checked whether it actually works or not.

Considering that Patman builds are on MediaFire and not on GitHub, he most likely embedded manifests by hand.

[Blacklands]

FFmpeg does use MAX_PATH in its sources:
[...]
Enabling long path support is not a simple manifest change.

Yes, I know. I didn't know the exact source code locations where it does (thanks for providing those, and looking this up!), but it was said in multiple places that it does use MAX_PATH.
I don't know if the Patman patched version modifies those places in the source code, too (since I couldn't find the source code for that patched version), or if it simply isn't a problem in practice most of the time, so nobody bothered. 🤷

This repo is about providing a patched version, isn't it? I don't know how much work it would be changing these three places in the source code for the patch (I haven't dealt with the Windows API, yet).

Thanks for the other information! I'll give this a try soon. (I switched to using yt-dlp within WSL2, for now.)

Considering that Patman builds are on MediaFire and not on GitHub, he most likely embedded manifests by hand.

That does suggest that they didn't do any source code modifications, and just added the manifest, then.

[pukkandan]

@nihil-admirari If they don't accept the patch, feel free to push it here (or make PR)

[nihil-admirari]

I'll do, but we need at least one tester.

[pukkandan]

it works on win11. Might not on versions below 10

❯ md 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890

❯ .\ffmpeg.exe -i test.webm -c copy 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890.mkv
ffmpeg version N-105570-gfad5c6d743-20220213 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (crosstool-NG 1.24.0.498_5075e1f)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --enable-shared --disable-static --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20220213
  libavutil      57. 21.100 / 57. 21.100
  libavcodec     59. 20.100 / 59. 20.100
  libavformat    59. 17.101 / 59. 17.101
  libavdevice    59.  5.100 / 59.  5.100
  libavfilter     8. 27.100 /  8. 27.100
  libswscale      6.  5.100 /  6.  5.100
  libswresample   4.  4.100 /  4.  4.100
  libpostproc    56.  4.100 / 56.  4.100
Input #0, matroska,webm, from 'test.webm':
  Metadata:
    ENCODER         : Lavf59.17.101
  Duration: 00:00:09.86, start: -0.007000, bitrate: 917 kb/s
  Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv, bt709), 1920x1080, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 1k tbn (default)
    Metadata:
      DURATION        : 00:00:09.833000000
  Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 00:00:09.861000000
Output #0, matroska, to '1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890.mkv':
  Metadata:
    encoder         : Lavf59.17.101
  Stream #0:0(eng): Video: vp9 (Profile 0) (VP90 / 0x30395056), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 30 tbr, 1k tbn (default)
    Metadata:
      DURATION        : 00:00:09.833000000
  Stream #0:1(eng): Audio: opus ([255][255][255][255] / 0xFFFFFFFF), 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 00:00:09.861000000
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=  295 fps=0.0 q=-1.0 Lsize=    1104kB time=00:00:09.84 bitrate= 919.4kbits/s speed=1.39e+03x
video:963kB audio:135kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.557582%

[pukkandan]

Another trick is to convert the path into UNC path, which will work with unpatched ffmpeg, and with earlier versions of windows (though the file may be inaccessible to most programs)

❯ ffmpeg.exe -i test.webm -c copy \\?\D:\Trash\Downloads\bin\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890.mkv
ffmpeg version N-105465-gbedb4bac1c-20220206 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 11.2.0 (crosstool-NG 1.24.0.498_5075e1f)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-nonfree --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --enable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20220206
  libavutil      57. 19.100 / 57. 19.100
  libavcodec     59. 20.100 / 59. 20.100
  libavformat    59. 17.101 / 59. 17.101
  libavdevice    59.  5.100 / 59.  5.100
  libavfilter     8. 26.101 /  8. 26.101
  libswscale      6.  5.100 /  6.  5.100
  libswresample   4.  4.100 /  4.  4.100
  libpostproc    56.  4.100 / 56.  4.100
Input #0, matroska,webm, from 'test.webm':
  Metadata:
    ENCODER         : Lavf59.17.101
  Duration: 00:00:09.86, start: -0.007000, bitrate: 917 kb/s
  Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv, bt709), 1920x1080, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 1k tbn (default)
    Metadata:
      DURATION        : 00:00:09.833000000
  Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 00:00:09.861000000
File '\\?\D:\Trash\Downloads\bin\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890.mkv' already exists. Overwrite? [y/N] y
Output #0, matroska, to '\\?\D:\Trash\Downloads\bin\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890.mkv':
  Metadata:
    encoder         : Lavf59.17.101
  Stream #0:0(eng): Video: vp9 (Profile 0) (VP90 / 0x30395056), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 30 tbr, 1k tbn (default)
    Metadata:
      DURATION        : 00:00:09.833000000
  Stream #0:1(eng): Audio: opus ([255][255][255][255] / 0xFFFFFFFF), 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 00:00:09.861000000
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=  295 fps=0.0 q=-1.0 Lsize=    1104kB time=00:00:09.84 bitrate= 919.4kbits/s speed= 459x
video:963kB audio:135kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.557582%

[Blacklands]

it works on win11. Might not on versions below 10

I would assume it won't, since the Long Paths support only got introduced in Windows 10 (in a 2016 update) from what I read.
(Plus, it needs to be explicitly enabled in the registry or group policy editor in Windows 10, don't know if it's the default in 11 now.)

And I'll do some tests too as soon as I find some time.

[nihil-admirari]

Sent a patch to FFmpeg developers: https://ffmpeg.org/pipermail/ffmpeg-devel/2022-February/292947.html.

[pukkandan]

041caf0