eirteam / eirteam.ffmpeg

Plugin for godot that adds an FFmpeg-based decoder.

EIRTeam.FFmpeg

Note: The version of FFmpeg shipped with EIRTeam.FFmpeg allows loading of videos using the patent-encumbered h264 codec, check with your local laws to see if software patents are recognized in your country.

GDExtension Video Decoder library for Godot Engine >4.1, using the FFmpeg library.

Building

Currently building ffmpeg from source is supported on Linux and macOS hosts. And ffmpeg libraries for Android can be cross compiled on both hosts. Follow these steps:

• run git submodule update --init to fetch ffmpeg-kit sources.
• run make bootstrap to install building tools.
• run make gdextension PLATFORM=linux for Linux.
• run make gdextension PLATFORM=macos for macOS.
• run make gdextension PLATFORM=android for Android with default arch arm64-v8a.
• run make gdextension PLATFORM=android TARGET_ARCH=arm-v7a for Android with arch arm-v7a.

Documentation

The official documentation can be found here.

Supporting development

You can also support EIRTeam by donating on Patreon or purchasing Project Heartbeat.

eirteam.ffmpeg's Issues

Instructions for turning on Tracy?

just curious about how to do this? I think I have a couple of tricks that should speed things up, but I want to be sure and not just be guessing.

V1.1.0 video does not work on windows with latest stable godot (4.2.2)

The latest downloadable release does not work on windows in godot 4.2.2 and 4.2.1.

Audio works, but video results in a "blackscreen":


(sound is playing)

Using the previous release godot 4.2.2 fully works:

When building the newest godot from source v4.3.dev.custom_build [7529c0bec] the latest release works.

Will web versions be supported in the future

I encountered an issue exporting the project to the web

Loading AV1 videos causes a crash.

I tried it with this sample.

There are no error messages. It just crashes.

I use Windows 10 and Godot 4.2.1.

With the latest FFmpeg binary build, it works.

EDIT: I guess it makes sense that a stripped down version can't play many codecs. So this isn't really about AV1 support, but more about graceful error handling and better error messages.

Loop skips last frame

Godot: 4.3beta3
lib: Auto-Build 2024-06-07
Different behavior between ogv and ffmpeg. Ogv loop draws frames at the same frequency. Ffmpeg is very unstable. Minimal project with two videos:
project.zip

Also compatibility mode cause errors

There are some problems with compilation

Audio stuttering?

I have been getting inconsistent stuttering / skipping or audio playback issues.

I'm having a hard time narrowing the cause, at first I thought it was videos loaded from a spinning drive, but its happened on SSD videos as well.

Even worse It sometimes goes away and comes back when scrubbing.

Pulling out my hair a little bit, If I cant find a cause I may have to scrap my project...

Add camera support

Hi all, thanks for this wonderful project. One thing that I'm currently searching is a library that gets me a live video feed on all platforms. The current Godot CameraFeed implementation is Apple only and people are desperate for an universal solution, see godotengine/godot#46531. One sidenote is that ffmpeg 6 has now hw acceleration support for Android https://www.qt.io/blog/ffmpeg-android

Cheers!

Will android/arm64 be supported in the future?

I just want to use ffmpeg on quest2.

there is no plugin

there is not even a plugin script so you can't use the extension.

Could you please add support for MacOS ?

Thank you~~

compilation errors when using own ffmpeg libraries

for testing I'm trying to use my system ffmpeg. which currently is is the arch user repository package "ffmpeg-full" this is a kitchen sink build with everything in it. it's not what I plan to ship with but it would give me flexibility to try a few things
https://aur.archlinux.org/packages/ffmpeg-full

In file included from /run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.cpp:31:
/run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.h:56:16: error: 'AVHWDeviceType' was not declared in this scope
  56 |         Vector<AVHWDeviceType> hw_device_types;
     |                ^~~~~~~~~~~~~~
/run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.h:56:30: error: template argument 1 is invalid
  56 |         Vector<AVHWDeviceType> hw_device_types;
     |                              ^
/run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.h:59:16: error: 'AVHWDeviceType' was not declared in this scope
  59 |         Vector<AVHWDeviceType> get_supported_hw_device_types();
     |                ^~~~~~~~~~~~~~
/run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.h:59:30: error: template argument 1 is invalid
  59 |         Vector<AVHWDeviceType> get_supported_hw_device_types();
     |                              ^
In file included from /usr/include/libavutil/hwcontext.h:23,
                from /usr/include/libavcodec/codec.h:27,
                from /run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.cpp:32:
/usr/include/libavutil/frame.h:340:16: error: redefinition of 'struct AVFrame'
 340 | typedef struct AVFrame {
     |                ^~~~~~~
In file included from /run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.h:34:
/usr/local/include/libavcodec/avcodec.h:1099:16: note: previous definition of 'struct AVFrame'
1099 | typedef struct AVFrame {
     |                ^~~~~~~
/usr/include/libavutil/frame.h:808:3: error: conflicting declaration 'typedef int AVFrame'
 808 | } AVFrame;
     |   ^~~~~~~
/usr/local/include/libavcodec/avcodec.h:1498:3: note: previous declaration as 'typedef struct AVFrame AVFrame'
1498 | } AVFrame;
     |   ^~~~~~~
In file included from /usr/include/libavcodec/codec.h:33:
/usr/include/libavcodec/codec_id.h:49:6: error: multiple definition of 'enum AVCodecID'
  49 | enum AVCodecID {
     |      ^~~~~~~~~
/usr/local/include/libavcodec/avcodec.h:97:6: note: previous definition here
  97 | enum AVCodecID {
     |      ^~~~~~~~~
/usr/include/libavcodec/codec.h:179:16: error: redefinition of 'struct AVProfile'
 179 | typedef struct AVProfile {
     |                ^~~~~~~~~
/usr/local/include/libavcodec/avcodec.h:3203:16: note: previous definition of 'struct AVProfile'
3203 | typedef struct AVProfile {
     |                ^~~~~~~~~
/usr/include/libavcodec/codec.h:182:3: error: conflicting declaration 'typedef int AVProfile'
 182 | } AVProfile;
     |   ^~~~~~~~~
/usr/local/include/libavcodec/avcodec.h:3206:3: note: previous declaration as 'typedef struct AVProfile AVProfile'
3206 | } AVProfile;
     |   ^~~~~~~~~
/usr/include/libavcodec/codec.h:187:16: error: redefinition of 'struct AVCodec'
 187 | typedef struct AVCodec {
     |                ^~~~~~~
/usr/local/include/libavcodec/avcodec.h:3215:16: note: previous definition of 'struct AVCodec'
3215 | typedef struct AVCodec {
     |                ^~~~~~~
/usr/include/libavcodec/codec.h:238:3: error: conflicting declaration 'typedef int AVCodec'
 238 | } AVCodec;
     |   ^~~~~~~
/usr/local/include/libavcodec/avcodec.h:3305:3: note: previous declaration as 'typedef struct AVCodec AVCodec'
3305 | } AVCodec;
     |   ^~~~~~~
/usr/include/libavcodec/codec.h:257:16: error: ambiguating new declaration of 'const AVCodec* avcodec_find_decoder(AVCodecID)'
 257 | const AVCodec *avcodec_find_decoder(enum AVCodecID id);
     |                ^~~~~~~~~~~~~~~~~~~~
/usr/local/include/libavcodec/avcodec.h:3856:10: note: old declaration 'AVCodec* avcodec_find_decoder(AVCodecID)'
3856 | AVCodec *avcodec_find_decoder(enum AVCodecID id);
     |          ^~~~~~~~~~~~~~~~~~~~
/usr/include/libavcodec/codec.h:265:16: error: ambiguating new declaration of 'const AVCodec* avcodec_find_decoder_by_name(const char*)'
 265 | const AVCodec *avcodec_find_decoder_by_name(const char *name);
     |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/libavcodec/avcodec.h:3864:10: note: old declaration 'AVCodec* avcodec_find_decoder_by_name(const char*)'
3864 | AVCodec *avcodec_find_decoder_by_name(const char *name);
     |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/libavcodec/codec.h:273:16: error: ambiguating new declaration of 'const AVCodec* avcodec_find_encoder(AVCodecID)'
 273 | const AVCodec *avcodec_find_encoder(enum AVCodecID id);
     |                ^~~~~~~~~~~~~~~~~~~~
/usr/local/include/libavcodec/avcodec.h:4283:10: note: old declaration 'AVCodec* avcodec_find_encoder(AVCodecID)'
4283 | AVCodec *avcodec_find_encoder(enum AVCodecID id);
     |          ^~~~~~~~~~~~~~~~~~~~
/usr/include/libavcodec/codec.h:281:16: error: ambiguating new declaration of 'const AVCodec* avcodec_find_encoder_by_name(const char*)'
 281 | const AVCodec *avcodec_find_encoder_by_name(const char *name);
     |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/libavcodec/avcodec.h:4291:10: note: old declaration 'AVCodec* avcodec_find_encoder_by_name(const char*)'
4291 | AVCodec *avcodec_find_encoder_by_name(const char *name);
     |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.cpp:34:24: error: no declaration matches 'godot::Vector<AVHWDeviceType> FFmpegCodec::get_supported_hw_device_types()'
  34 | Vector<AVHWDeviceType> FFmpegCodec::get_supported_hw_device_types() {
     |                        ^~~~~~~~~~~
/run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.h:59:32: note: candidate is: 'int FFmpegCodec::get_supported_hw_device_types()'
  59 |         Vector<AVHWDeviceType> get_supported_hw_device_types();
     |                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.h:53:7: note: 'class FFmpegCodec' defined here
  53 | class FFmpegCodec : public RefCounted {
     |       ^~~~~~~~~~~
Compiling godot-cpp/gen/src/classes/web_rtc_peer_connection_extension.cpp ...
scons: *** [/run/media/sirus/aux/code/arachnid/godot/EIRTeam.FFmpeg/ffmpeg_codec.os] Error 1
scons: building terminated because of errors.
[Time elapsed: 00:04:04.017]

am I correct in assuming that I could just rename my so files to drop the minor versions so they match the ones it autodownloads (the major version already matches), put those in the third party directory then comment out the downloading and installing of ffmpeg in the SConstruct file and then build?

VideoStream not reaching the end

It appears that videos don't play to the end. I noticed the behavior on Godot 4.2.

When I add a 0.14s padding at the end, the videos play to the expected end of the original video. The timing may not be 100% accurate as I added the padding with shotcut, that doesn't allow steps more precise than from 0.01s.
If it helps, the videos are encoded with a 30ips framerate with a variable bitrate of 2Mb/s and 2 B images. The videos are created from a series of .png images.
Of course, videos encoded in Theora work fine without the padding.

scons: *** [build\addons\ffmpeg\win64\avcodec-60.dll] The system cannot find the file specified.

系统找不到指定文件（The system cannot find the file specified）

There is nothing under EIRTeam.FFmpeg\thirdparty\ffmpeg\bin

getting error when building for android.

after installing all the dependencies and implementing commands, it still throws an error.

build.log -

Is .mov supported?

When i try to play a .mov file it crashes. Is mov not supported or could it be a codec issue? I was trying to use codec ProRes4444 with alpha channel.

stream_position property support is missing ?

Hi @EIREXE , thanks for your work on this GDExtension. Ultimately I could run it on my machine. :)

But I found that the VideoStreamPlayer node cannot set 'stream_position' property in gdscript. The code just overlook this setting and always play the stream from the very beginning. As stream_position is a very very important property of the stream playback, will you please add this feature in your code?

Thanks!

Engine Freezes Indefinitely when Invalid File is Used for the Stream

If you provide a filetype that ffmpeg can't load as a video, upon calling play() on the VideoStreamPlayer, the engine will freeze indefinitely without any error message. Ideally this should just error out.

H.264 only playing audio

With this simple setup, when trying to play an H.264 video, it would only play the audio.

Here's the project without the video:
test_project.zip
console output:

Supported video codecs:
	decode h263
	decode mpeg4
	decode h264
	decode vp8
	decode vp9
	decode mp3
	decode aac
	decode vorbis
	  codec: libvorbis
	decode aac_latm
	decode opus
	  codec: libopus
Succesfully initialized video decoder:h264

Video metadata:

Version: Auto-Build 2024-06-07 02:57 as well as self built

OS: Arch Linux 6.6.33-1-lts

Only audio plays

Hi,
I'm unable to get it work correctly in godot 4.2.1, plugins says:

Succesfully initialized video decoder:h264
MAKE SHADER
CRECREATE OUTPUT TEXTURE

But only audio plays, am i missing something?
Thanks

Diagonal Skewed lines Issue

Just wanted to follow up on the issue from the other thread...

There are a few videos that I have that look diagonally skewed... every next line is shifter over to the right.
Here's what it looks like:

So in this picture, the 2 on the left are videos that are diagonally skewed... and the 2 on the right are fine.
There is an m4v and mp4 example for each:

Here is a link to an m4v that is doing this: https://drive.google.com/file/d/15JGGat5iTH9Gv5Vn9ph3AYgrOaAE292d/view?usp=sharing
if you want to inspect it

Request support for http address playback

Request support for http address playback