nyanmisaka / ffmpeg-rockchip Goto Github PK

FFmpeg with async and zero-copy Rockchip MPP & RGA support

License: Other

Makefile 1.30% C 91.01% C++ 0.18% Perl 0.04% Shell 0.08% Assembly 7.08% Verilog 0.01% Objective-C 0.15% Cuda 0.13% Metal 0.01% Roff 0.01% Awk 0.01% HTML 0.01% Ruby 0.01% Python 0.01%

multimedia arm arm64 decoder encoder ffmpeg filter librga linux mpp

ffmpeg-rockchip's Introduction

Alipay | 支付宝 欢迎捐赠！你的支持是我开发的动力

ffmpeg-rockchip's People

Contributors

Stargazers

Watchers

ffmpeg-rockchip's Issues

使用av_read_frame进行rtsp拉流阻塞的问题

当rtsp流设备以外断电后再恢复供电。av_read_frame会被永远阻塞。尽管我已经设置了stimeout / timeout 。这并没有效果
@nyanmisaka

Hello When I compile, I encounter an error.

libavfilter/rkrga_common.c: In function 'get_drm_afbc_format':
libavfilter/rkrga_common.c:154:38: error: 'DRM_FORMAT_INVALID' undeclared (first use in this function); did you mean 'DRM_FORMAT_MOD_INVALID'?
154 | default: return DRM_FORMAT_INVALID;
| ^~~~~~~~~~~~~~~~~~
| DRM_FORMAT_MOD_INVALID
libavfilter/rkrga_common.c:154:38: note: each undeclared identifier is reported only once for each function it appears in
CC libavfilter/setpts.o
libavfilter/rkrga_common.c: In function 'query_frame':
libavfilter/rkrga_common.c:554:29: error: 'DRM_FORMAT_INVALID' undeclared (first use in this function); did you mean 'DRM_FORMAT_MOD_INVALID'?
554 | if (drm_afbc_fmt == DRM_FORMAT_INVALID) {
| ^~~~~~~~~~~~~~~~~~
| DRM_FORMAT_MOD_INVALID
libavfilter/rkrga_common.c:562:13: error: implicit declaration of function 'DRM_FORMAT_MOD_ARM_AFBC'; did you mean 'DRM_FORMAT_MOD_ARM_TYPE_AFBC'? [-Werror=implicit-function-declaration]
562 | DRM_FORMAT_MOD_ARM_AFBC(AFBC_FORMAT_MOD_SPARSE | AFBC_FORMAT_MOD_BLOCK_SIZE_16x16);
| ^~~~~~~~~~~~~~~~~~~~~~~
| DRM_FORMAT_MOD_ARM_TYPE_AFBC
libavfilter/rkrga_common.c:562:37: error: 'AFBC_FORMAT_MOD_SPARSE' undeclared (first use in this function)
562 | DRM_FORMAT_MOD_ARM_AFBC(AFBC_FORMAT_MOD_SPARSE | AFBC_FORMAT_MOD_BLOCK_SIZE_16x16);
| ^~~~~~~~~~~~~~~~~~~~~~
libavfilter/rkrga_common.c:562:62: error: 'AFBC_FORMAT_MOD_BLOCK_SIZE_16x16' undeclared (first use in this function)
562 | DRM_FORMAT_MOD_ARM_AFBC(AFBC_FORMAT_MOD_SPARSE | AFBC_FORMAT_MOD_BLOCK_SIZE_16x16);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libavfilter/rkrga_common.c: In function 'get_drm_afbc_format':
libavfilter/rkrga_common.c:156:1: error: control reaches end of non-void function [-Werror=return-type]
156 | }

HW decoding HEVC w/Jellyfin doesn't work when subtitles enabled

(First, thanks for this project --- it's awesome, and the performance on my Orange Pi 5+ is fantastic with HW transcoding!)

I recently did a fresh pull on the docker image nyanmisaka/jellyfin:latest-rockchip and noticed that, with HW HEVC decode enabled, some videos will not show the video with the subtitles enabled (the subtitles will show, however it's just subtitles on a black screen). This happens on browser and android client. At least one video successfully worked on android client when the subtitles were displayed by the client and not embedded into the video stream, however.

My recollection is that this did not happen prior to pulling the docker image today; I'm pretty new to using docker so I'm not sure how to revert to the previous image to test this out.

Happy to assist however I can in debugging, and apologies if this is the wrong place for this.

Rendering Docs

I am giving the rendering wiki page update as a patch, i think gh does not support prs for wiki.

Basically documents how to get acceleration on mpv and kodi including afbc support as well.

rendering-docs.patch.tar.gz

depends on
#13
#4
75eca72 (afbc stride issue)

Preview is here:
https://github.com/hbiyik/ffmpeg-rockchip/wiki/Rendering

Help building ffplay

Hello,

after seeing your work here: jellyfin/jellyfin-ffmpeg#34
i decided to try building your ffmpeg fork, following the wiki.

However, when i try to use it using the following command: DISPLAY=:0 ffplay -codec:v hevc_rkmpp /etc/jellyfinlib/bbb_sunflower_2160p_60fps_normal.mp4

The video doesn't even open. I also tried to use vlc with this custom ffmpeg and the output is unreadable. I also tried transcoding video and reading it on an x86 pc and the video is corrupted.

While trying to read the original video with the your ffmpeg fork, FFPLAY gives me this error:

[hevc_mp4toannexb @ 0x7f382b4b50] No parameter sets in the extradata
[hevc_rkmpp @ 0x7f382dc220] Failed to get packet (code = -1094995529)
    Last message repeated 437 times
[hevc_rkmpp @ 0x7f382dc220] Failed to get packet (code = -1094995529)
    Last message repeated 1705 times
[hevc_rkmpp @ 0x7f382dc220] Failed to get packet (code = -1094995529)
    Last message repeated 1944 times

Hardware: CM3588 from friendlyelec
Uname: Linux CM3588 6.1.25 #123 SMP Thu Nov 16 11:25:37 CST 2023 aarch64 aarch64 aarch64 GNU/Linux
FFMPEG version: 668585f built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
Configuration configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-libdrm --enable-rkmpp --enable-rkrga

If you need further details please let me know, i've been looking into this for hours now and i can't find anything telling me what's wrong.

Thank you for your time.

Scaling affects color output

Hello, thank you for your effort on this 👍
I'm trying to do hardware scaling -vf realtime,scale_rkrga=w=1280:h=720:format=yuv420p,hwmap=mode=read,format=yuv420p -r 5 -f rawvideo but my image turns all weird. Without the w/h (-vf realtime,scale_rkrga=format=yuv420p,hwmap=mode=read,format=yuv420p -r 5 -f rawvideo) the video looks fine.

Full ffmpeg is

ffmpeg -threads 2 -hwaccel rkmpp -hwaccel_output_format drm_prime -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://127.0.0.1:8554/front -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/framsida@%Y%m%d%H%M%S%z.mp4 -vf realtime,scale_rkrga=w=1280:h=720:format=yuv420p,hwmap=mode=read,format=yuv420p -r 5 -f rawvideo -y pipe:

Input format is

Input #0, rtsp, from 'rtsp://....':
  Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 1920x1080, 15 fps, 30 tbr, 90k tbn
  Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_rkmpp) -> rawvideo (native))

Any pointers are highly appreciated 🙏

crashes - various errors

camera two

024-03-27 02:52:40.643412721  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :   Metadata:
2024-03-27 02:52:40.643416221  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :     title           : Media Presentation
2024-03-27 02:52:40.643420013  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :   Duration: N/A, start: 0.160000, bitrate: N/A
2024-03-27 02:52:40.643424971  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :   Stream #0:0: Video: hevc (Main), yuvj420p(pc), 640x360 [SAR 1:1 DAR 16:9], 10 fps, 12.50 tbr, 90k tbn
2024-03-27 02:52:40.643428179  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   : Stream mapping:
2024-03-27 02:52:40.643431971  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :   Stream #0:0 -> #0:0 (hevc (hevc_rkmpp) -> rawvideo (native))
2024-03-27 02:52:40.643435470  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   : Press [q] to stop, [?] for help
2024-03-27 02:52:40.643438970  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   : Output #0, rawvideo, to 'pipe:':
2024-03-27 02:52:40.643442179  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :   Metadata:
2024-03-27 02:52:40.643445678  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :     title           : Media Presentation
2024-03-27 02:52:40.643449178  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :     encoder         : Lavf60.16.100
2024-03-27 02:52:40.643454428  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :   Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(pc, progressive), 640x360 [SAR 1:1 DAR 16:9], q=2-31, 27648 kb/s, 10 fps, 10 tbn
2024-03-27 02:52:40.643457636  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :     Metadata:
2024-03-27 02:52:40.643461136  [2024-03-27 02:52:40] ffmpeg.camera_two.detect ERROR   :       encoder         : Lavc60.31.102 rawvideo

2024-03-27 02:52:20.682267528  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : frame=  268 fps=9.7 q=-0.0 size=   90112kB time=00:00:29.20 bitrate=25280.9kbits/s speed=1.05x    
2024-03-27 02:52:20.682451272  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : frame=  272 fps=9.6 q=-0.0 size=   91462kB time=00:00:29.70 bitrate=25227.6kbits/s speed=1.05x    
2024-03-27 02:52:20.682637350  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : frame=  279 fps=9.7 q=-0.0 size=   93825kB time=00:00:30.30 bitrate=25366.8kbits/s speed=1.05x    
2024-03-27 02:52:20.682821970  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : frame=  285 fps=9.6 q=-0.0 size=   95850kB time=00:00:30.90 bitrate=25411.1kbits/s speed=1.05x    
2024-03-27 02:52:20.683003089  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : frame=  291 fps=9.7 q=-0.0 size=   97875kB time=00:00:31.50 bitrate=25453.7kbits/s speed=1.05x    
2024-03-27 02:52:20.683192959  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : frame=  297 fps=9.7 q=-0.0 size=   99900kB time=00:00:32.10 bitrate=25494.7kbits/s speed=1.04x    
2024-03-27 02:52:20.683377286  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : frame=  303 fps=9.7 q=-0.0 size=  101925kB time=00:00:32.70 bitrate=25534.2kbits/s speed=1.04x    
2024-03-27 02:52:20.683634529  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : frame=  308 fps=9.6 q=-0.0 size=  103612kB time=00:00:33.30 bitrate=25489.3kbits/s speed=1.04x    
2024-03-27 02:52:20.683783857  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : frame=  314 fps=9.7 q=-0.0 size=  105638kB time=00:00:33.90 bitrate=25527.5kbits/s speed=1.04x    
2024-03-27 02:52:20.683985685  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : [Parsed_scale_rkrga_0 @ 0x5595640630] RGA blit failed: -12
2024-03-27 02:52:20.684170888  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : [vf#0:0 @ 0x559562ccf0] Error while filtering: Generic error in an external library
2024-03-27 02:52:20.684359299  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : Failed to inject frame into filter network: Generic error in an external library
2024-03-27 02:52:20.684550335  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : Error while filtering: Generic error in an external library
2024-03-27 02:52:20.684741370  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : [out#0/rawvideo @ 0x5595600090] video:108000kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
2024-03-27 02:52:20.684930948  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : frame=  320 fps=9.6 q=-0.0 Lsize=  108000kB time=00:00:34.40 bitrate=25719.1kbits/s speed=1.03x    
2024-03-27 02:52:20.685118776  [2024-03-27 02:52:20] ffmpeg.camera_two.detect ERROR   : Conversion failed!

camera four

2024-03-27 02:52:21.257455741  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : Input #0, rtsp, from 'rtsp://*:*@445.445.445.1:8787/cam/realmonitor?channel=1&subtype=2':
2024-03-27 02:52:21.257773065  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :   Metadata:
2024-03-27 02:52:21.258018349  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :     title           : Media Server
2024-03-27 02:52:21.258260134  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :   Duration: N/A, start: 0.000000, bitrate: N/A
2024-03-27 02:52:21.258504251  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :   Stream #0:0: Video: hevc (Main), yuvj420p(pc, bt470bg/bt470bg/smpte170m), 1920x1080, 25 fps, 25 tbr, 90k tbn
2024-03-27 02:52:21.258745452  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :   Stream #0:1: Audio: aac (LC), 64000 Hz, mono, fltp
2024-03-27 02:52:21.258984029  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : Stream mapping:
2024-03-27 02:52:21.259315352  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :   Stream #0:0 -> #0:0 (hevc (hevc_rkmpp) -> rawvideo (native))
2024-03-27 02:52:21.259559178  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : Press [q] to stop, [?] for help
2024-03-27 02:52:21.259798921  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : Output #0, rawvideo, to 'pipe:':
2024-03-27 02:52:21.260036622  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :   Metadata:
2024-03-27 02:52:21.260273448  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :     title           : Media Server
2024-03-27 02:52:21.260532149  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :     encoder         : Lavf60.16.100
2024-03-27 02:52:21.260773933  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :   Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(pc, bt470bg/bt470bg/smpte170m, progressive), 1920x1080, q=2-31, 622080 kb/s, 25 fps, 25 tbn
2024-03-27 02:52:21.261009009  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :     Metadata:
2024-03-27 02:52:21.261250794  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   :       encoder         : Lavc60.31.102 rawvideo
2024-03-27 02:52:21.261490537  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : frame=    0 fps=0.0 q=0.0 size=       0kB time=N/A bitrate=N/A dup=1 drop=0 speed=N/A    
2024-03-27 02:52:21.261729113  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : frame=   24 fps=0.0 q=-0.0 size=   69862kB time=00:00:00.96 bitrate=596160.0kbits/s dup=1 drop=0 speed=1.79x    
2024-03-27 02:52:21.261968856  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : frame=   38 fps= 36 q=-0.0 size=  112388kB time=00:00:01.52 bitrate=605709.5kbits/s dup=1 drop=0 speed=1.46x    
2024-03-27 02:52:21.262208599  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : frame=   51 fps= 32 q=-0.0 size=  151875kB time=00:00:02.04 bitrate=609882.4kbits/s dup=1 drop=0 speed=1.28x    
2024-03-27 02:52:21.262447758  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : frame=   66 fps= 30 q=-0.0 size=  197438kB time=00:00:02.60 bitrate=622080.0kbits/s dup=1 drop=0 speed=1.17x    
2024-03-27 02:52:21.262700625  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : frame=   81 fps= 28 q=-0.0 size=  243000kB time=00:00:03.24 bitrate=614400.0kbits/s dup=1 drop=0 speed=1.11x    
2024-03-27 02:52:21.262944160  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : frame=   98 fps= 29 q=-0.0 size=  294638kB time=00:00:03.92 bitrate=615732.2kbits/s dup=1 drop=0 speed=1.14x    
2024-03-27 02:52:21.263184194  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : [Parsed_scale_rkrga_0 @ 0x559db31ff0] RGA blit failed: -12
2024-03-27 02:52:21.263423354  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : [vf#0:0 @ 0x559db03d00] Error while filtering: Generic error in an external library
2024-03-27 02:52:21.263663388  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : Failed to inject frame into filter network: Generic error in an external library
2024-03-27 02:52:21.263917714  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : Error while filtering: Generic error in an external library
2024-03-27 02:52:21.264160373  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : [out#0/rawvideo @ 0x559dadbf70] video:328050kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
2024-03-27 02:52:21.264400699  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : frame=  108 fps= 27 q=-0.0 Lsize=  328050kB time=00:00:04.28 bitrate=627893.8kbits/s dup=1 drop=0 speed=1.07x    
2024-03-27 02:52:21.264641901  [2024-03-27 02:52:21] ffmpeg.camera_four.detect ERROR   : Conversion failed!

Encoder endofstream problem (lost few final frames)

Hello.
I found the problem in encoder logic.

if packet (encoded frame) is not available (still encoding), EAGAIN is returned

ffmpeg-rockchip/libavcodec/rkmppenc.c

Lines 687 to 691 in 27136b0

    
           if ((ret = r->mapi->encode_get_packet(r->mctx, &mpp_pkt)) != MPP_OK) { 
        
               int log_level = (ret == MPP_NOK) ? AV_LOG_DEBUG : AV_LOG_ERROR; 
        
               ret = (ret == MPP_NOK) ? AVERROR(EAGAIN) : AVERROR_EXTERNAL; 
        
               av_log(avctx, log_level, "Failed to get packet from encoder output queue: %d\n", ret); 
        
               return ret;

error return is handled in encode function, *got_packet=0

ffmpeg-rockchip/libavcodec/rkmppenc.c

Lines 776 to 779 in 27136b0

    
           ret = rkmpp_get_packet(avctx, packet); 
        
           if (ret == AVERROR_EOF || 
        
               ret == AVERROR(EAGAIN)) 
        
               *got_packet = 0;

encoder loop check got_packet and if in draining state, ends draining and lost all remaining encoded packet

ffmpeg-rockchip/libavcodec/encode.c

Lines 353 to 354 in 27136b0

if (avci->draining && !got_packet)

avci->draining_done = 1;

I did not discover why "ffmpeg" loop does not suffer this problem but my program have this problem. So I try to add busy-loop over rkmpp_get_packet() (if loop is only to "get:" label it sometimes stops for few seconds, please review the logic):

diff --git a/libavcodec/rkmppenc.c b/libavcodec/rkmppenc.c
index 1688acc..afbb59f 100644
--- a/libavcodec/rkmppenc.c
+++ b/libavcodec/rkmppenc.c
@@ -774,6 +774,8 @@ send:
 
 get:
     ret = rkmpp_get_packet(avctx, packet);
+    if (!frame && ret == AVERROR(EAGAIN))
+        goto send;
     if (ret == AVERROR_EOF ||
         ret == AVERROR(EAGAIN))
         *got_packet = 0;

Typical problem (penultimate column shows PTS inserted to encoder, last column shows PTS outputted from encoder, so lost 2 frames (1456376400, 1456380000)):

main.c:712: flt->enc                     1456369200
main.c:709: >ENC
main.c:720: .ENC
main.c:744: enc->out                               1456358400
main.c:720: .ENC
main.c:744: enc->out                               1456362000
main.c:720: .ENC
main.c:709: >ENC
main.c:712: flt->enc                     1456372800
main.c:709: >ENC
main.c:712: flt->enc                     1456376400
main.c:709: >ENC
main.c:720: .ENC
main.c:744: enc->out                               1456365600
main.c:720: .ENC
main.c:709: >ENC
main.c:712: flt->enc                     1456380000
main.c:698: =send EOF to ENC (signaling EOF to encoder)
main.c:720: .ENC
main.c:744: enc->out                               1456369200
main.c:720: .ENC
[h264_rkmpp @ 0x557c23e3b0] End of stream
[h264_rkmpp @ 0x557c23e3b0] Wrote 0 bytes to encoder
[h264_rkmpp @ 0x557c23e3b0] Received a packet
main.c:744: enc->out                               1456372800
main.c:720: .ENC
[h264_rkmpp @ 0x557c23e3b0] End of stream
[h264_rkmpp @ 0x557c23e3b0] Wrote 0 bytes to encoder
[h264_rkmpp @ 0x557c23e3b0] Failed to get packet from encoder output queue: -11
main.c:733: =send EOF to OUT (received EOF from encoder)
======================= aframe 100 98

编译opencv with ffmpeg 时出错

描述：由于ffmpeg 5.0开始弃用了avcodec_get_context_defaults3，编译opencv时出错，导致编译失败
问题：能否集成一个低版本的ffmpeg version<5.0

SIG11 when running ffmpeg

Stone:
Linux orangepi5plus 5.10.160-rockchip-rk3588
(rkmppenc works fine, compiled from scratch)

Input:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'uhd_vid.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.45.100
Duration: 00:02:27.17, start: 0.000000, bitrate: 45285 kb/s
Stream #0:00x1: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 3840x2160 [SAR 1:1 DAR 16:9], 45061 kb/s, 29.97 fps, 29.97 tbr, 16k tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream #0:10x2: Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, stereo, fltp, 224 kb/s (default)
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
Side data:
audio service type: main

Trtanscoding:
./ffmpeg_g -i ./uhd_vid.mp4 -c:v h264_rkmpp -qp_init 26 -profile:v main -level 4.1 -y tmp.mp4

SIG11:
warning: Loadable section "" outside of ELF segments in /lib/aarch64-linux-gnu/libmali.so.1 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1". Core was generated by ./ffmpeg_g -i ./uhd_vid.mp4 -c:v h264_rkmpp -qp_init 26 -profile:v main -level'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000007f990076ac in MppBufferService::get_group(char const*, char const*, MppBufferMode, MppBufferType, unsigned int) () from /lib/aarch64-linux-gnu/librockchip_mpp.so.1
[Current thread is 1 (Thread 0x7f99cc7020 (LWP 24452))]
(gdb) where
#0 0x0000007f990076ac in MppBufferService::get_group(char const*, char const*, MppBufferMode, MppBufferType, unsigned int) () from /lib/aarch64-linux-gnu/librockchip_mpp.so.1
#1 0x0000007f99007ae0 in mpp_buffer_group_init () from /lib/aarch64-linux-gnu/librockchip_mpp.so.1
#2 0x0000005583a98020 in rkmpp_frames_init (hwfc=0x558795d720) at libavutil/hwcontext_rkmpp.c:302
#3 0x0000005583a93f68 in av_hwframe_ctx_init (ref=0x558795cfd0) at libavutil/hwcontext.c:368
#4 0x00000055835eb1c4 in init_hwframes_ctx (avctx=0x55878ef4a0) at libavcodec/rkmppenc.c:916
#5 rkmpp_encode_init (avctx=0x55878ef4a0) at libavcodec/rkmppenc.c:1080
#6 0x00000055832dc734 in avcodec_open2 (avctx=avctx@entry=0x55878ef4a0, codec=codec@entry=0x5583fc7b08 <ff_h264_rkmpp_encoder>, options=options@entry=0x558793ee90) at libavcodec/avcodec.c:337
#7 0x0000005582ede9bc in enc_open (ost=ost@entry=0x558793ed80, frame=frame@entry=0x55878ef0c0) at fftools/ffmpeg_enc.c:311
#8 0x0000005582ee02a4 in enc_frame (ost=ost@entry=0x558793ed80, frame=frame@entry=0x55878ef0c0) at fftools/ffmpeg_enc.c:855
#9 0x0000005582ee199c in fg_output_frame (ofp=ofp@entry=0x55878f2100, frame=frame@entry=0x55878ef2b0) at fftools/ffmpeg_filter.c:2125
#10 0x0000005582ee449c in fg_output_step (flush=0, ofp=0x55878f2100) at fftools/ffmpeg_filter.c:2220
#11 reap_filters (flush=, fg=) at fftools/ffmpeg_filter.c:2239
#12 reap_filters (fg=0x558790e570, flush=flush@entry=0) at fftools/ffmpeg_filter.c:2228
#13 0x0000005582ed6554 in transcode_step (ost=) at fftools/ffmpeg.c:1155
#14 transcode (err_rate_exceeded=) at fftools/ffmpeg.c:1204
#15 main (argc=, argv=) at fftools/ffmpeg.c:1330
(gdb)
`

Push and pull streaming delay problem

I try to use the following commands to pull and forward the video stream. No errors were reported during the encoding and decoding process, but the speed was very slow, speed = 1x, and the frame was also very slow.当我尝试用vlc或者ffplay进行取流的时候，视频需要很长的时候才能显示，而且在播放时有大约2s的延迟，并且存在卡顿现象，请问该如何解决

ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -i rtsp://xxxx/Channels/101 -c:a copy -strict -2 -vf scale_rkrga=w=1920:h=1080:format=nv12:afbc=1 -c:v h264_rkmpp -r 15 -rtsp_transport udp -rc_mode VBR -b:v 1M -maxrate 2M -bufsize 12M -profile:v high -g:v 120 -f rtsp rtsp://xxxx/amera1

Tone-mapping slow after upgrading kernel.

After upgrading the kernel from 5.1 to 6.1 (orangepi-linux latest kernel),tone-mapping got super slow even upgraded opencl driver.
Both latest and v1.9-1-3381696 is not functioning correctly.

Before upgrading the kernel, everything is fine with legacy opencl driver.

Orange Pi 1.1.8 Bookworm with Linux 6.1.43-rockchip-rk3588

clinfo output:
https://pastebin.com/eH0YEEfB

Jellyfin-ffmpeg log:
https://pastebin.com/fpUN4JMG

docker-compose.yml:
https://pastebin.com/jrbsDvrK

leaking file descriptors

Hello.
I am using code in alloc/free loop.

When I used "scale_rkrga" filter in cycle (avfilter_graph_alloc() ... avfilter_graph_free()) usually leaks 1 FD "anon_inode:sync_file" in /proc/PID/fd (maybe some sort of sync primitive leak).
When I used "h264_rkmpp" encoder in cycle (avcodec_alloc_context3() ... avcodec_free_context()) usually leaks 1 FD "/dmabuf:276910-main" in /proc/PID/fd.
("hevc_rkmpp" decoder does not need to restart, because it supports avcodec_flush_buffers()).

I cannot determine if there is problem with rk libraries or ffmpeg integration code.
Does anyone have hint/solution to this ?

Thanks, Martin

h264_rkmpp 接收 rawvideo 灰度图数据失败。Failed to get key input frame from packet meta: -1

ffmpeg 通过管道线接收灰度图 bytes 数据，在使用 libx264 编码时可以推流，但换用 h264_rkmpp 接收数据时报错。

ffmpeg 启动命令为：
ffmpeg -y -f rawvideo -pix_fmt gray -s 1280x100 -i - -c:v h264_rkmpp -f rtsp rtsp://127.0.0.1:8554/live/test

请问我需要如何处理？

调试信息如下：
ffmpeg version ba84e56 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.3)
configuration: --prefix=/usr/local/ffmpeg --toolchain=hardened --arch=arm64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-frei0r --enable-libx264 --enable-shared --enable-version3 --enable-libdrm --enable-rkmpp --enable-rkrga
libavutil 58. 29.100 / 58. 29.100
libavcodec 60. 31.102 / 60. 31.102
libavformat 60. 16.100 / 60. 16.100
libavdevice 60. 3.100 / 60. 3.100
libavfilter 9. 12.100 / 9. 12.100
libswscale 7. 5.100 / 7. 5.100
libswresample 4. 12.100 / 4. 12.100
libpostproc 57. 3.100 / 57. 3.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'gray'.
Reading option '-s' ... matched as option 's' (set frame size (WxH or abbreviation)) with argument '1280x100'.
Reading option '-i' ... matched as output url with argument '-'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'h264_rkmpp'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rtsp'.
Reading option 'rtsp://192.168.2.100:8554/live/24033403' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url -.
Applying option f (force format) with argument rawvideo.
Applying option pix_fmt (set pixel format) with argument gray.
Applying option s (set frame size (WxH or abbreviation)) with argument 1280x100.
Successfully parsed a group of options.
Opening an input file: -.
[rawvideo @ 0x558a20c6d0] Opening 'fd:' for reading
[fd @ 0x558a20cc30] Setting default whitelist 'crypto,data'
[rawvideo @ 0x558a20c6d0] Before avformat_find_stream_info() pos: 0 bytes read:65536 seeks:0 nb_streams:1
[rawvideo @ 0x558a20c6d0] All info found
[rawvideo @ 0x558a20c6d0] After avformat_find_stream_info() pos: 128000 bytes read:128000 seeks:0 frames:1
Input #0, rawvideo, from 'fd:':
Duration: N/A, start: 0.000000, bitrate: 25600 kb/s
Stream #0:0, 1, 1/25: Video: rawvideo, 1 reference frame (Y800 / 0x30303859), gray, 1280x100, 0/1, 25600 kb/s, 25 tbr, 25 tbn
Successfully opened the file.
Parsing a group of options: output url rtsp://192.168.2.100:8554/live/24033403.
Applying option c:v (codec name) with argument h264_rkmpp.
Applying option f (force format) with argument rtsp.
Successfully parsed a group of options.
Opening an output file: rtsp://192.168.2.100:8554/live/24033403.
[out#0/rtsp @ 0x558a23d200] No explicit maps, mapping streams automatically...
[vost#0:0/h264_rkmpp @ 0x558a240570] Created video stream from input stream 0:0
Successfully opened the file.
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_rkmpp))
[rawvideo @ 0x558a23d410] PACKET SIZE: 128000, STRIDE: 1280
detected 4 logical cores
[graph 0 input from stream 0:0 @ 0x558a2431e0] Setting 'video_size' to value '1280x100'
[graph 0 input from stream 0:0 @ 0x558a2431e0] Setting 'pix_fmt' to value '8'
[graph 0 input from stream 0:0 @ 0x558a2431e0] Setting 'time_base' to value '1/25'
[graph 0 input from stream 0:0 @ 0x558a2431e0] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 0x558a2431e0] Setting 'frame_rate' to value '25/1'
[graph 0 input from stream 0:0 @ 0x558a2431e0] w:1280 h:100 pixfmt:gray tb:1/25 fr:25/1 sar:0/1
[format @ 0x558a243730] Setting 'pix_fmts' to value 'gray|yuv420p|yuv422p|yuv444p|nv12|nv21|nv16|nv24|yuyv422|yvyu422|uyvy422|rgb24|bgr24|rgba|rgb0|bgra|bgr0|argb|0rgb|abgr|0bgr|drm_prime'
[AVFilterGraph @ 0x558a23d0c0] query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
[h264_rkmpp @ 0x558a2408b0] Rate Control mode is set to CBR
[h264_rkmpp @ 0x558a2408b0] Bitrate Target/Min/Max is set to 2000000/1875000/2125000
[h264_rkmpp @ 0x558a2408b0] QP Init/Max/Min/Max_I/Min_I is set to 26/48/0/48/0
[h264_rkmpp @ 0x558a2408b0] Profile is set to HIGH
[h264_rkmpp @ 0x558a2408b0] 8x8 Transform is enabled
[h264_rkmpp @ 0x558a2408b0] Level is set to 0
[h264_rkmpp @ 0x558a2408b0] Coder is set to CABAC
[h264_rkmpp @ 0x558a2408b0] Created a RKMPP hardware device
[tcp @ 0x558a2438f0] No default whitelist set
[tcp @ 0x558a2438f0] Original list of addresses:
[tcp @ 0x558a2438f0] Address 192.168.2.100 port 8554
[tcp @ 0x558a2438f0] Interleaved list of addresses:
[tcp @ 0x558a2438f0] Address 192.168.2.100 port 8554
[tcp @ 0x558a2438f0] Starting connection attempt to 192.168.2.100 port 8554
[tcp @ 0x558a2438f0] Successfully connected to 192.168.2.100 port 8554
[rtsp @ 0x558a23db90] SDP:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 192.168.2.100
t=0 0
a=tool:libavformat 60.16.100
m=video 0 RTP/AVP 96
b=AS:2000
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z2QQFawbGqBQP+eEAAADAAQAAAMAyjwiEag=,aO4xshsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA; profile-level-id=641015
a=control:streamid=0

[rtp @ 0x558a251840] No default whitelist set
[udp @ 0x558a251a70] No default whitelist set
[udp @ 0x558a251a70] end receive buffer size reported is 425984
[udp @ 0x558a251b20] No default whitelist set
[udp @ 0x558a251b20] end receive buffer size reported is 425984
Output #0, rtsp, to 'rtsp://192.168.2.100:8554/live/24033403':
Metadata:
encoder : Lavf60.16.100
Stream #0:0, 0, 1/90000: Video: h264 (High), 1 reference frame, gray(progressive), 1280x100 (0x0), 0/1, q=2-31, 2000 kb/s, 25 fps, 90k tbn
Metadata:
encoder : Lavc60.31.102 h264_rkmpp
[h264_rkmpp @ 0x558a2408b0] Configured with size: 1280x100 | pix_fmt: gray | sw_pix_fmt: gray
[h264_rkmpp @ 0x558a2408b0] Wrote 204800 bytes to encoder
[h264_rkmpp @ 0x558a2408b0] Received a packet
[rawvideo @ 0x558a23d410] PACKET SIZE: 128000, STRIDE: 1280 bitrate=N/A speed=N/A
[rtp @ 0x558a251bd0] Sending NAL 7 of len 26 M=0
[rtp @ 0x558a251bd0] Sending NAL 8 of len 5 M=1
[h264_rkmpp @ 0x558a2408b0] Wrote 204800 bytes to encoder
[h264_rkmpp @ 0x558a2408b0] Received a packet
[h264_rkmpp @ 0x558a2408b0] Failed to get key input frame from packet meta: -1
[vost#0:0/h264_rkmpp @ 0x558a240570] Error submitting video frame to the encoder
Error while filtering: Generic error in an external library
[vist#0:0/rawvideo @ 0x558a20cae0] Decoder thread received EOF packet
[vist#0:0/rawvideo @ 0x558a20cae0] Decoder returned EOF, finishing
[vist#0:0/rawvideo @ 0x558a20cae0] Terminating decoder thread
[h264_rkmpp @ 0x558a2408b0] End of stream
[h264_rkmpp @ 0x558a2408b0] Wrote 0 bytes to encoder
[h264_rkmpp @ 0x558a2408b0] Received an EOS packet
[out#0/rtsp @ 0x558a23d200] All streams finished
[out#0/rtsp @ 0x558a23d200] Terminating muxer thread
[AVIOContext @ 0x558a252d90] Statistics: 76 bytes written, 0 seeks, 2 writeouts
[out#0/rtsp @ 0x558a23d200] Output file #0 (rtsp://192.168.2.100:8554/live/24033403):
[out#0/rtsp @ 0x558a23d200] Output stream #0:0 (video): 2 frames encoded; 1 packets muxed (39 bytes);
[out#0/rtsp @ 0x558a23d200] Total: 1 packets (39 bytes) muxed
[out#0/rtsp @ 0x558a23d200] video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
frame= 1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed= 0x
[in#0/rawvideo @ 0x558a20c5d0] Terminating demuxer thread
[in#0/rawvideo @ 0x558a20c5d0] Input file #0 (fd:):
[in#0/rawvideo @ 0x558a20c5d0] Input stream #0:0 (video): 4 packets read (512000 bytes); 2 frames decoded; 0 decode errors;
[in#0/rawvideo @ 0x558a20c5d0] Total: 4 packets (512000 bytes) demuxed
[AVIOContext @ 0x558a21cdb0] Statistics: 512000 bytes read, 0 seeks
Conversion failed!

use scale_rkrga filter error in rk3588(ubuntu22.02 desktop)

cmd: ./ffmpeg -timeout 10000000 -rtsp_transport tcp -loglevel debug -c:v h264_rkmpp -i rtsp://xxxx:[email protected]/h264/ch2/main/av_stream -vf fps=4,scale_rkrga=-1:720 -c:v h264_rkmpp -global_quality 19 -g 16 -rw_timeout 5000000 -f flv -an rtmp://127.0.0.1/cams/live

log:
h264_rkmpp @ 0x55ba0f6360] Noticed an info change
[h264_rkmpp @ 0x55ba0f6360] Format nv12 chosen by get_format().
[h264_rkmpp @ 0x55ba0f6360] Configured with size: 1920x1080 | pix_fmt: nv12 | sw_pix_fmt: nv12
[h264_rkmpp @ 0x55ba0f6360] Received a frame
[AVFilterGraph @ 0x55b9f214c0] Setting 'fps' to value '4'
[AVFilterGraph @ 0x55b9f214c0] Setting 'w' to value '-1'
[AVFilterGraph @ 0x55b9f214c0] Setting 'h' to value '720'
detected 8 logical cores
[graph 0 input from stream 0:0 @ 0x55b9f52bb0] Setting 'video_size' to value '1920x1080'
[graph 0 input from stream 0:0 @ 0x55b9f52bb0] Setting 'pix_fmt' to value '23'
[graph 0 input from stream 0:0 @ 0x55b9f52bb0] Setting 'time_base' to value '1/90000'
[graph 0 input from stream 0:0 @ 0x55b9f52bb0] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 0x55b9f52bb0] Setting 'frame_rate' to value '25/1'
[graph 0 input from stream 0:0 @ 0x55b9f52bb0] w:1920 h:1080 pixfmt:nv12 tb:1/90000 fr:25/1 sar:0/1
[format @ 0x55b9f52f70] Setting 'pix_fmts' to value 'gray|yuv420p|yuv422p|yuv444p|nv12|nv21|nv16|nv24|yuyv422|yvyu422|uyvy422|rgb24|bgr24|rgba|rgb0|bgra|bgr0|argb|0rgb|abgr|0bgr|drm_prime'
[auto_scale_0 @ 0x55b9f53b90] w:iw h:ih flags:'' interl:0
[Parsed_scale_rkrga_1 @ 0x55b9f52b00] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_fps_0' and the filter 'Parsed_scale_rkrga_1'
Impossible to convert between the formats supported by the filter 'Parsed_fps_0' and the filter 'auto_scale_0'
[Parsed_fps_0 @ 0x55b9f733b0] 0 frames in, 0 frames out; 0 frames dropped, 0 frames duplicated.
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
[AVIOContext @ 0x55ba0d2aa0] Statistics: 0 bytes written, 0 seeks, 0 writeouts
[rtmp @ 0x55b9f7c4f0] UnPublishing stream...
[rtmp @ 0x55b9f7c4f0] Deleting stream...
Terminating demuxer thread 0
Conversion failed!

The problem of Chinese garbled code in drawtext

chip:rk3588
board: orange pi5

root@orangepi5:~# cat /proc/version
Linux version 5.10.160-rockchip-rk3588 (root@orangepi5) (aarch64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #1.1.8 SMP Mon Nov 13 11:22:15 CST 2023

Sometimes some Chinese can be partly displayed, and sometimes Chinese can not be displayed.

root@orangepi5:~# ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -i 1.mp4 -vf scale_rkrga=w=1920:h=1080,hwdownload,format=nv12,drawtext=text='中文':fontfile=msyh.ttc:fontsize=200 -c:v h264_rkmpp -y -t 10  2.mp4
ffmpeg version 6.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-libdrm --enable-rkmpp --enable-rkrga --enable-filter=drawtext --enable-libharfbuzz --enable-libfreetype --enable-libfontconfig --enable-libfreetype --enable-libfribidi
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, mpeg, from '1.mp4':
  Duration: 00:02:59.85, start: 6095.275000, bitrate: 2593 kb/s
  Stream #0:0[0x1e0]: Video: hevc (Main), yuvj420p(pc, bt709), 2560x1440, 15 fps, 25 tbr, 90k tbn
  Stream #0:1[0x1c0]: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (hevc_rkmpp) -> h264 (h264_rkmpp))
  Stream #0:1 -> #0:1 (pcm_alaw (native) -> aac (native))
Press [q] to stop, [?] for help
[aac @ 0x5577682660] Too many bits 8832.000000 > 6144 per frame requested, clamping to max
rga_api version 1.10.0_[8]
Output #0, mp4, to '2.mp4':
  Metadata:
    encoder         : Lavf60.16.100
  Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), nv12(pc, bt709, progressive), 1920x1080, q=2-31, 2000 kb/s, 25 fps, 12800 tbn
    Metadata:
      encoder         : Lavc60.31.102 h264_rkmpp
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 8000 Hz, mono, fltp, 48 kb/s
    Metadata:
      encoder         : Lavc60.31.102 aac
[out#0/mp4 @ 0x55775c7590] video:2433kB audio:52kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.149914%
frame=  250 fps=0.0 q=-0.0 Lsize=    2490kB time=00:00:09.99 bitrate=2041.1kbits/s dup=100 drop=0 speed=12.6x    
[aac @ 0x5577682660] Qavg: 40287.332

root@orangepi5:~# ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -i 1.mp4 -vf "scale_rkrga=w=1920:h=1080,hwdownload,format=nv12,drawtext=text='中文字幕测试':fontfile=msyh.ttc:fontsize=200" -c:v h264_rkmpp -y -t 10  3.mp4
ffmpeg version 6.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-libdrm --enable-rkmpp --enable-rkrga --enable-filter=drawtext --enable-libharfbuzz --enable-libfreetype --enable-libfontconfig --enable-libfreetype --enable-libfribidi
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, mpeg, from '1.mp4':
  Duration: 00:02:59.85, start: 6095.275000, bitrate: 2593 kb/s
  Stream #0:0[0x1e0]: Video: hevc (Main), yuvj420p(pc, bt709), 2560x1440, 15 fps, 25 tbr, 90k tbn
  Stream #0:1[0x1c0]: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (hevc_rkmpp) -> h264 (h264_rkmpp))
  Stream #0:1 -> #0:1 (pcm_alaw (native) -> aac (native))
Press [q] to stop, [?] for help
[aac @ 0x55d26df660] Too many bits 8832.000000 > 6144 per frame requested, clamping to max
rga_api version 1.10.0_[8]
Output #0, mp4, to '3.mp4':
  Metadata:
    encoder         : Lavf60.16.100
  Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), nv12(pc, bt709, progressive), 1920x1080, q=2-31, 2000 kb/s, 25 fps, 12800 tbn
    Metadata:
      encoder         : Lavc60.31.102 h264_rkmpp
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 8000 Hz, mono, fltp, 48 kb/s
    Metadata:
      encoder         : Lavc60.31.102 aac
[out#0/mp4 @ 0x55d2624590] video:2427kB audio:52kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.150300%
frame=  250 fps=0.0 q=-0.0 Lsize=    2483kB time=00:00:09.99 bitrate=2035.8kbits/s dup=100 drop=0 speed=11.5x    
[aac @ 0x55d26df660] Qavg: 40287.332

Reducing Latency in ffplay with ffmpeg-rockchip Compared to gstreamer for RTSP Streams

Hello ffmpeg-rockchip community,

I am experiencing a noticeable delay issue when using ffplay compared to gstreamer for RTSP stream handling. With ffplay, using the following command:

ffplay -fflags nobuffer -flags low_delay -framedrop -strict experimental -rtsp_transport udp rtsp://[ANONYMIZED]

I encounter a delay of about 1-2 seconds compared to gstreamer:

gst-launch-1.0 rtspsrc location=rtsp://[ANONYMIZED] latency=100 ! rtph265depay ! h265parse ! decodebin ! autovideosink

Could you please explain why there is such a delay difference between these two approaches? Is there a way to optimize ffplay performance in this context, especially with ffmpeg-rockchip, to reduce latency and achieve similar performance as gstreamer?

Any insights or guidance would be greatly appreciated.

Thank you!

Device: Orange pi 5 plus
OS: Kernel .5.10.160-28

Green screen with the attached video sample

Originally reported at 7Ji-PKGBUILDs/.meta#29

I'm using kodi-mpp-git and ffmpeg-rockchip-git.

With the attached video sample, mpv --profile=fast --hwdec=rkmpp path-to-file displays green screen.

kodi under GBM (https://github.com/hbiyik/ffmpeg-rockchip/wiki/Rendering#kodi-under-gbm) seems to play it, although displays black screen after stopping the video.

After stopping the video, kodi shows black screen (it doesn't freeze, it really is just a blank or black screen).

I see these messages in journalctl:

kernel: [drm:rockchip_vop2_mod_supported] *ERROR* Unsupported format modifier 0x800000000000051

And these messages in kodi log:

2024-03-14 21:28:00.101 T:328297   error <general>: CDRMAtomic::DrmAtomicCommit - test commit failed: (Invalid argument) - falling back to last successful atomic request
2024-03-14 21:28:00.102 T:328297   error <general>:
                                                   Object: plane        ID: 71
                                                     Property: SRC_X    ID: 9   Value: 0
                                                     Property: SRC_Y    ID: 10  Value: 0
                                                     Property: SRC_W    ID: 11  Value: 251658240
                                                     Property: SRC_H    ID: 12  Value: 141557760
                                                     Property: CRTC_X   ID: 13  Value: 0
                                                     Property: CRTC_Y   ID: 14  Value: 0
                                                     Property: CRTC_W   ID: 15  Value: 3840
                                                     Property: CRTC_H   ID: 16  Value: 2160
                                                     Property: FB_ID    ID: 17  Value: 322
                                                     Property: CRTC_ID  ID: 20  Value: 85
2024-03-14 21:28:00.102 T:328297   error <general>: CDRMAtomic::DrmAtomicCommit - atomic commit failed: Invalid argument

Sorry for gzip'ed mkv, couldn't upload to github otherwise :(

out.mkv.gz

avcodec_receive_packet blocking

大佬您好：

我在同时使用ffmpeg库进行推拉流的时候发现，在编码过程调用avcodec_receive_packet会出现阻塞卡死的现象

代码调用的方式像这样。
另给您发了一份邮件，望回复。

请教使用scale_rkrga进行YUV420P/nv12转BGR24

下面代码是我编写的一个YUV420P转BGR24的方法，请问下使用scale_rkrga的硬件方式是这样的编写的吗？
int DRtspDecoder::YUV420p2BGR24(AVFrame *src_frame, AVFrame **dst_frame)
{
if ((src_frame == NULL) || (dst_frame == NULL)) {
std::cerr << "invalid input parameter" << std::endl;
return -1;
}

*dst_frame = NULL;
AVFilterContext *buffersink_ctx = NULL;
AVFilterContext *buffersrc_ctx = NULL;
AVFilterGraph *filter_graph = NULL;
const char *filter_descr = "scale_rkrga=format=bgr24";

// 初始化filter graph
filter_graph = avfilter_graph_alloc();
if (!filter_graph) {
    std::cerr << "failed to allocate filter graph" << std::endl;
    return -2;
}

// 创建buffersrc上下文
AVStream *input_stream = src_frame->opaque;  // 从源帧的opaque字段获取输入流信息
AVCodecParameters *input_codecpar = input_stream->codecpar;
AVPixelFormat input_pix_fmt = static_cast<AVPixelFormat>(input_codecpar->format);

AVFilter *buffersrc = avfilter_get_by_name("buffer");
if (!buffersrc) {
    std::cerr << "failed to get buffer filter" << std::endl;
    return -3;
}

AVCodecContext *codec_ctx = input_stream->codec;
char args[512];
snprintf(args, sizeof(args),
         "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
         codec_ctx->width, codec_ctx->height, input_pix_fmt,
         input_stream->time_base.num, input_stream->time_base.den,
         codec_ctx->sample_aspect_ratio.num, codec_ctx->sample_aspect_ratio.den);
         
int ret = avfilter_graph_create_filter(&buffersrc_ctx, buffersrc, "in",
                                       args, NULL, filter_graph);
if (ret < 0) {
    std::cerr << "failed to create buffer source filter" << std::endl;
    avfilter_graph_free(&filter_graph);
    return -4;
}

// 创建buffersink上下文
AVFilter *buffersink = avfilter_get_by_name("buffersink");
if (!buffersink) {
    std::cerr << "failed to get buffer sink filter" << std::endl;
    return -5;
}

AVPixelFormat output_pix_fmt = AV_PIX_FMT_BGR24;
AVCodecContext *output_codec_ctx = (*dst_frame)->opaque;  // 从目标帧的opaque字段获取输出流信息

ret = avfilter_graph_create_filter(&buffersink_ctx, buffersink, "out",
                                   NULL, NULL, filter_graph);
if (ret < 0) {
    std::cerr << "failed to create buffer sink filter" << std::endl;
    avfilter_graph_free(&filter_graph);
    return -6;
}

// 链接filters
ret = avfilter_link(buffersrc_ctx, 0, buffersink_ctx, 0);
if (ret < 0) {
    std::cerr << "failed to link filters" << std::endl;
    avfilter_graph_free(&filter_graph);
    return -7;
}

// 配置filter graph
ret = avfilter_graph_config(filter_graph, NULL);
if (ret < 0) {
    std::cerr << "failed to configure filter graph" << std::endl;
    avfilter_graph_free(&filter_graph);
    return -8;
}

AVFrame *temp_frame = av_frame_alloc();
if (!temp_frame) {
    std::cerr << "failed to allocate temporary frame" << std::endl;
    avfilter_graph_free(&filter_graph);
    return -9;
}

// 将输入帧数据填充到buffersrc_ctx中
ret = av_buffersrc_add_frame_flags(buffersrc_ctx, src_frame,
                                   AV_BUFFERSRC_FLAG_KEEP_REF);
if (ret < 0) {
    std::cerr << "failed to send source frame to buffer source filter" << std::endl;
    av_frame_free(&temp_frame);
    avfilter_graph_free(&filter_graph);
    return -10;
}

// 从buffersink_ctx中读取输出帧数据
ret = av_buffersink_get_frame(buffersink_ctx, temp_frame);
if (ret < 0) {
    std::cerr << "failed to get destination frame from buffer sink filter" << std::endl;
    av_frame_free(&temp_frame);
    avfilter_graph_free(&filter_graph);
    return -11;
}

// 分配目标帧并复制颜色转换后的数据
*dst_frame = av_frame_alloc();
if (!*dst_frame) {
    std::cerr << "failed to allocate destination frame" << std::endl;
    av_frame_free(&temp_frame);
    avfilter_graph_free(&filter_graph);
    return -12;
}

(*dst_frame)->width = output_codec_ctx->width;
(*dst_frame)->height = output_codec_ctx->height;
(*dst_frame)->format = output_pix_fmt;

ret = av_frame_get_buffer(*dst_frame, 0);
if (ret < 0) {
    std::cerr << "failed to allocate destination frame buffer" << std::endl;
    av_frame_free(&temp_frame);
    av_frame_free(&*dst_frame);
    avfilter_graph_free(&filter_graph);
    return -13;
}

ret = av_frame_copy_props(*dst_frame, temp_frame);
if (ret < 0) {
    std::cerr << "failed to copy frame properties" << std::endl;
    av_frame_free(&temp_frame);
    av_frame_free(&*dst_frame);
    avfilter_graph_free(&filter_graph);
    return -14;
}

// 将颜色转换后的数据复制到目标帧
ret = av_frame_copy(*dst_frame, temp_frame);
if (ret < 0) {
    std::cerr << "failed to copy destination frame" << std::endl;
    av_frame_free(&temp_frame);
    av_frame_free(&*dst_frame);
    avfilter_graph_free(&filter_graph);
    return -15;
}

av_frame_free(&temp_frame);
avfilter_graph_free(&filter_graph);
return 0;

}

Questions about cross-compilation

Cross-compilation Error for ffmpeg-rockchip

libdrm and rockchip_mpp pkg-config Not Found

Build files for mpp, rga, and drm have been successfully generated through cross-compilation. Using the following commands, pkg-config files have been imported, and pkg-config information has been successfully queried, as shown below:

rk@rk-VirtualBox:~/linux/ffmpeg-rockchip-master$ pkg-config --modversion libdrm
2.4.107
rk@rk-VirtualBox:~/linux/ffmpeg-rockchip-master$ pkg-config --modversion rockchip_mpp
1.3.8

Then, attempting to compile ffmpeg with the following command:

./configure \
--enable-shared \
--enable-static \
--prefix=$PWD/_install \
--enable-cross-compile \
--cross-prefix=/home/rk/linux/rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu- \
--arch=aarch64 \
--target-os=linux \
--extra-cflags="-I/home/rk/linux/airockchip/librga/installed/include;-I/home/rk/linux/airockchip/mpp/installed/include;-I/home/rk/linux/libdrm-2.4.107/install/include" \
--extra-ldflags="-L/home/k/linux/airockchip/librga/installed/lib;-L/home/rk/linux/airockchip/mpp/installed/lib;-L/home/rk/linux/libdrm-2.4.107/install/lib" \
--enable-gpl \
--enable-version3 \
--enable-libdrm \
--enable-rkmpp \
--enable-rkrga \
--disable-stripping

Encountering the following errors:

ERROR: libdrm not found using pkg-config
ERROR: rockchip_mpp not found using pkg-config

Using with Photoprism

Hello
Is it possible to use compiled ffmpeg with Photoprism to encode videos?

RTSP使用h264_rkmpp无法使用

使用的fixup! lavf/rkrga: add force_{yuv,chroma} options for vpp filter这版FFmepg
rkmpp使用的是HermanChen/mpp@fdeb8c3

现在出现问题如下，请问这个是什么原因呢？
[tcp @ 0x2cbc850] No default whitelist set
[tcp @ 0x2cbc850] Original list of addresses:
[tcp @ 0x2cbc850] Address 192.168.11.65 port 554
[tcp @ 0x2cbc850] Interleaved list of addresses:
[tcp @ 0x2cbc850] Address 192.168.11.65 port 554
[tcp @ 0x2cbc850] Starting connection attempt to 192.168.11.65 port 554
[tcp @ 0x2cbc850] Successfully connected to 192.168.11.65 port 554
[rtsp @ 0x2ce0500] SDP:
v=0
o=- 1709553511739851 1709553511739851 IN IP4 192.168.11.65
s=Media Presentation
e=NONE
b=AS:5050
t=0 0
a=control:rtsp://192.168.11.65:554/Streaming/Channels/101/
m=video 0 RTP/AVP 96
b=AS:5000
a=control:rtsp://192.168.11.65:554/Streaming/Channels/101/trackID=1
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=Z0IAKpY1QPAET8s3AQEBQAABwgAAV+Qh,aM4xsg==
a=Media_header:MEDIAINFO=494D4B48010100000400000100000000000000000000000000000000000000000000000000000000;
a=appversion:1.0

[rtsp @ 0x2ce0500] video codec set to: h264
[rtsp @ 0x2ce0500] RTP Profile IDC: 42 Profile IOP: 0 Level: 29
[rtsp @ 0x2ce0500] RTP Packetization Mode: 1
[rtsp @ 0x2ce0500] Extradata set to 0x2ce13d0 (size: 36)
[rtsp @ 0x2ce0500] setting jitter buffer size to 0
[rtsp @ 0x2ce0500] hello state=0
mVideoIndex = 0, width = 0, height = 0, codecId = 27
pixfmt: 0, width: 0, height: 0
[h264_mp4toannexb @ 0x2cda170] The input looks like it is Annex B already
[h264_rkmpp @ 0x2cd67c0] Format nv12 chosen by get_format().
[h264_rkmpp @ 0x2cd67c0] Failed to init MPP context: -1
open codec failed, return: -542398533, errstr: Generic error in an external library

Compile into static form

Hi,
I want to compile FFmpeg into a static form. However, when compiling the MPP library, I specified -DBUILD_STATIC_LIBS=ON, and for rga, I used --default-library=static. When compiling FFmpeg, I set --disable-shared --enable-static. Despite these settings, the resulting MPP library is a dynamic library, as is the rga library, and FFmpeg still has dynamic dependencies. How can I resolve this issue?
Thank you.

lost the last frame in complex filter

Hello.
I found that complex filter (double RGA filter) lost the last frame. in.ts has 100 frames.

single RGA filter (overlay or scale)
- ./ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -i in.ts -i ovr.png -filter_complex '[0:v]hwupload[main];[1:v]hwupload[overlay];[main][overlay]overlay_rkrga=eof_action=repeat:repeatlast=true:format=nv12' -c:v h264_rkmpp out.ts -y --> 100 frames
- ./ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -i in.ts -filter_complex 'hwupload,scale_rkrga=w=1280:h=720' -c:v h264_rkmpp out.ts -y --> 100 frames
double RGA filter (overlay and scale)
- ./ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -i in.ts -i ovr.png -filter_complex '[0:v]hwupload[main];[1:v]hwupload[overlay];[main][overlay]overlay_rkrga=eof_action=repeat:repeatlast=true:format=nv12,scale_rkrga=w=1280:h=720' -c:v h264_rkmpp out.ts -y --> 99 frames

Rockchip BSP/vendor kernel is necessary, 5.10 and 6.1 are two tested versions...?

Hi, thank you for your work on this. What is this v6.1 BSP kernel you mention being tested? Does it support rk3588? I have some OPi5pluses that I've been playing with and im only aware of a v5.10 BSP kernel. Would be nice to have a 6.1 bsp kernel option. I've tinkered with building various 6.X mainline/collabora based kernels and random patches, but I don't believe theres been any working hardware video decoder support yet. Thanks!

当摄像头设备分辨率为1920x1080时，tbr帧率识别错误？

Hi,nyanmisaka

我在推流摄像头设备时发现当/dev/video0的分辨率为1920x1080时，ffprobe出帧率为1000k tbr，提示increasing probesize，导致没有输出，设置为1280x720时ffprobe帧率为 59.94 tbr，正常输出
v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat='NV12'

./dev/ffmpeg/ffmpeg  -i /dev/video0  -c:v h264_rkmpp -payload_type 126 -f rtp udp://127.0.0.1:8004
ffmpeg version ba84e56 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-libdrm --enable-rkmpp --enable-rkrga
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[video4linux2,v4l2 @ 0x55a6fcac70] ioctl(VIDIOC_G_INPUT): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0x55a6fcac70] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0x55a6fcac70] Time per frame unknown
[video4linux2,v4l2 @ 0x55a6fcac70] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 1371.892581, bitrate: N/A
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080, 1000k tbr, 1000k tbn

How to get buffer from drm_prime?

I use av_hwframe_transfer_data to convert drm_prime to nv12. nv12 has two planes, plane 0 size should be 2073600, plane 1 size should be 1036800, for case 1920x1080. But in fact nv12 plane 0 size is 3133568 and plane 1 will cause Segmentation fault.

I use it in my Qt project.

void onFrame(AVFrame *frame) {
    if (videoSink == nullptr) {
        return;
    }

    auto        fmt = getFormat(static_cast<AVPixelFormat>(frame->format));
    QVideoFrame videoFrame({QSize(frame->width, frame->height), fmt});
    videoFrame.map(QVideoFrame::WriteOnly);
    for (int i = 0; i < videoFrame.planeCount(); i++) {
        auto buf = av_frame_get_plane_buffer(frame, i);
        memcpy(videoFrame.bits(i), buf->data, buf->size);
    }
    videoFrame.unmap(); // TODO use drm prime

    emit videoSink->videoFrameChanged(videoFrame);
}

RTSP使用h264_rkmpp无法使用

使用的fixup! lavf/rkrga: add force_{yuv,chroma} options for vpp filter这版FFmepg
rkmpp使用的是HermanChen/mpp@fdeb8c3

`Internal bug, should not have happened` 在同时编码与解码的时候

System.ApplicationException: Internal bug, should not have happened
   at IPHost.Desktop.FFmpegHelper.ThrowExceptionIfError(Int32 error) in E:\code\IPHost\IPHost.Desktop\FFmpegHelper.cs:line 76
   at IPHost.Desktop.VideoDecoder.<.ctor>b__28_0() in E:\code\IPHost\IPHost.Desktop\VideoDecoder.cs:line 192
Aborted

if (packs.TryDequeue(out var packet))
{
    var data = (byte*)ffmpeg.av_malloc((ulong)packet.Size);
    Marshal.Copy(packet.Data, 0, (IntPtr)data, packet.Size);

    ffmpeg.av_packet_from_data(_pPacket, data, packet.Size)
        .ThrowExceptionIfError();

    ffmpeg.avcodec_send_packet(_videoCodecContext, _pPacket)
        .ThrowExceptionIfError();

    //ffmpeg.av_packet_unref(_pPacket);
    ffmpeg.av_free(data);

    error = ffmpeg.avcodec_receive_frame(_videoCodecContext, _videoFrame);
    if (error == 0)
    {
        AVFrame frame = *_videoFrame;

        if (_videoCodecContext->hw_device_ctx != null)
        {
            ffmpeg.av_hwframe_transfer_data(_receivedFrame, _videoFrame, 0)
                .ThrowExceptionIfError();
            frame = *_receivedFrame;
            if (first_frame)
            {
                first_frame = false;
            }
        }

        if (vfc == null)
        {
            var sourceSize = FrameSize;
            var sourcePixelFormat = HWDeviceType == AVHWDeviceType.AV_HWDEVICE_TYPE_NONE
            ? _videoCodecContext->pix_fmt
                     : DecodeHelper.GetHWPixelFormat(HWDeviceType);
            var destinationSize = sourceSize;
            var destinationPixelFormat = AVPixelFormat.AV_PIX_FMT_BGRA;
            vfc = new VideoFrameConverter(sourceSize, sourcePixelFormat, destinationSize, destinationPixelFormat);
        }

        var convertedFrame = vfc.Convert(frame);
        IPHostApi.SetVideoPtr(_uuid, convertedFrame.width, convertedFrame.height, (IntPtr)convertedFrame.data[0]);

        start_time = ffmpeg.av_gettime() - ffmpeg.av_rescale_q(_videoFrame->pts, TimeBase, ffmpeg.av_make_q(1, ffmpeg.AV_TIME_BASE));
    }
    else if (error != -11)
    {
        error.ThrowExceptionIfError();
    }
}
else
{
    Thread.Sleep(1);
}

SEI information support

I set var of side_data for AVFrame with a value of “side_data”. I debugged the code and found that MPP didn't use this value when encoding. Is it MPP doesn't support setting side_data?

当前支持rk3528处理器吗？

在rk3528的armbian上（内核使用rk6.1）安装ffmpeg后可以使用mpv硬解码视频
sudo mpv --hwdec=rkmpp 1.mp4
[ffmpeg/demuxer] mov,mp4,m4a,3gp,3g2,mj2: stream 0, timescale not set
(+) Video --vid=1 () (h264 1280x720 24.000fps)
Video --vid=2 [P] (mjpeg 1.000fps)
(+) Audio --aid=1 --alang=eng () (aac 2ch 48000Hz)
Audio --aid=2 --alang=eng (eac3 6ch 48000Hz)
File tags:
Date: 2024
Description: Toranaga is forced to carve out a new deal with a long lost family member.
Title: A Stick of Time
error: XDG_RUNTIME_DIR not set in the environment.
Using hardware decoding (rkmpp).
AO: [alsa] 48000Hz stereo 2ch float
VO: [gpu] 1280x720 drm_prime[nv12]
AV: 00:00:59 / 00:54:30 (2%) A-V: 0.000 Dropped: 8

Audio/Video desynchronisation detected! Possible reasons include too slow
hardware, temporary CPU spikes, broken drivers, and broken files. Audio
position will not match to the video (see A-V status field).

但是视频颜色不对，显示偏红色，RGA2没有中断产生
内核提示：
mpp_info: mpp version: unknown mpp version for missing VCS info

Use `h264_rkmpp` play rtsp stream, `avcodec_receive_frame` allways return -11.

When debugging with breakpoints, it was found that avcodec_receive_frame allways return -11.
But using h264 is normal, it can display.

try
{
    ffmpeg.av_frame_unref(_videoFrame);
    ffmpeg.av_frame_unref(_audioFrame);
    ffmpeg.av_frame_unref(_receivedFrame);
    int old = 0;
    byte* data = null;
    AVRational time_base = _pFormatContext->streams[_streamIndex]->time_base;
    long start_time = 0;
    bool first_frame = true;
    do
    {
        ffmpeg.av_packet_unref(_pPacket);
        error = ffmpeg.av_read_frame(_pFormatContext, _pPacket);

        if (error == ffmpeg.AVERROR_EOF || cancellationToken.IsCancellationRequested)
        {
            return;
        }

        error.ThrowExceptionIfError();

        if (_pPacket->stream_index == _audioStreamIndex)
        {
            ffmpeg.avcodec_send_packet(_audioCodecContext, _pPacket)
                .ThrowExceptionIfError();

            error = ffmpeg.avcodec_receive_frame(_audioCodecContext, _audioFrame);
            if (error == 0 && IPHostApi.EnableAudio)
            {
                BufferFormat format;
                var frame = *_audioFrame;
                if (frame.ch_layout.nb_channels == 1)
                {
                    format = frame.format == (int)AVSampleFormat.AV_SAMPLE_FMT_U8 ? BufferFormat.Mono8 : BufferFormat.Mono16;
                }
                else if (frame.ch_layout.nb_channels == 2)
                {
                    format = frame.format == (int)AVSampleFormat.AV_SAMPLE_FMT_U8 ? BufferFormat.Stereo8 : BufferFormat.Stereo16;
                }
                else
                {
                    continue;
                }

                int outputSampleCount = (int)ffmpeg.av_rescale_rnd(ffmpeg.swr_get_delay(swr_ctx, frame.sample_rate) + frame.nb_samples, 48000, frame.sample_rate, AVRounding.AV_ROUND_UP)
                    * 2 * 2;

                if (old != outputSampleCount)
                {
                    if (data != null)
                    {
                        Marshal.FreeHGlobal((nint)data);
                    }

                    data = (byte*)Marshal.AllocHGlobal(outputSampleCount);
                    old = outputSampleCount;
                }

                int size = ffmpeg.swr_convert(swr_ctx, &data, outputSampleCount, frame.extended_data, frame.nb_samples) * 4;

                if (size > 0 && IPHostApi.EnableAudio)
                {
                    OpenALPlayer.Write(format, data, size, 48000);
                }
            }
            else if (error != -11)
            {
                error.ThrowExceptionIfError();
            }
        }
        else if (_pPacket->stream_index == _streamIndex)
        {
            ffmpeg.avcodec_send_packet(_videoCodecContext, _pPacket)
                .ThrowExceptionIfError();
            error = ffmpeg.avcodec_receive_frame(_videoCodecContext, _videoFrame);
            if (error == 0)
            {
                AVFrame frame = *_videoFrame;

                if (_videoCodecContext->hw_device_ctx != null)
                {
                    ffmpeg.av_hwframe_transfer_data(_receivedFrame, _videoFrame, 0)
                        .ThrowExceptionIfError();
                    frame = *_receivedFrame;
                }

                var convertedFrame = vfc.Convert(frame);
                IPHostApi.SetVideoPtr(uuid, convertedFrame.width, convertedFrame.height, (IntPtr)convertedFrame.data[0]);

                if (first_frame)
                {
                    first_frame = false;
                    start_time =  ffmpeg.av_gettime() - ffmpeg.av_rescale_q(_videoFrame->pts, time_base, ffmpeg.av_make_q(1, ffmpeg.AV_TIME_BASE));
                }
            }
            else if (error != -11)
            {
                error.ThrowExceptionIfError();
            }
        }

        if (!first_frame)
        {
            long pts_time = ffmpeg.av_rescale_q(_videoFrame->pts, time_base, ffmpeg.av_make_q(1, ffmpeg.AV_TIME_BASE));
            long now = ffmpeg.av_gettime() - start_time;
            if (pts_time > now)
            {
                ffmpeg.av_usleep((uint)(pts_time - now));
            }
        }
    } while (!cancellationToken.IsCancellationRequested);
}
catch (Exception e)
{
    Console.WriteLine(e);
}

The decoding time of a single frame is too long.

When I calculated the time to decode, I found out the average time is almost 84ms.

Open the log and find such a problem “Decoder buffer is full”. This happens regardless of whether I export to NV12 format or drm_prime.

This is the test code:

更换了Armbian_23.11.1_Orangepi5-plus_bookworm_legacy_5.10.160.img后好像hevc 10解码失效了😭

n大好!更换了底层系统后，我做了以下操作:

更新系统的操作
下载了libmali-valhall-g610-g13p0-x11-wayland-gbm_1.9-1_arm64.deb
sudo apt install ./libmali-valhall-g610-g13p0-x11-wayland-gbm_1.9-1_arm64.deb
chmod -R 777 /dev
之前也是同样的系统，同样的步骤，是可以解码的，但是这次不知道什么原因不行了。求指导😭

出现的问题:

4k hevc sdr 的可以解码播放，4k hevc 10bit 解码转圈，3分钟播放1秒这样子(开启了色调映射)，不开色调映射可以点开就能解码

docker日志和配置截图:

_jellyfin_logs.txt

Use `h264_rkmpp` decoder video to fast and Use `h264` is no issue.

this is my code.

ffmpeg.avformat_open_input(&pFormatContext, url, null, null).ThrowExceptionIfError();
ffmpeg.avformat_find_stream_info(_pFormatContext, null).ThrowExceptionIfError();

_streamIndex = ffmpeg.av_find_best_stream(_pFormatContext,
    AVMediaType.AVMEDIA_TYPE_VIDEO, -1, -1, null, 0);
AVCodec* codec;

if (_streamIndex >= 0)
{
    codec = ffmpeg.avcodec_find_decoder_by_name("h264_rkmpp");
    if (codec != null)
    {
        Console.WriteLine("Using rkmpp for decoding\n");
        _videoCodecContext = ffmpeg.avcodec_alloc_context3(codec);
        ffmpeg.avcodec_parameters_to_context(_videoCodecContext,
            _pFormatContext->streams[_streamIndex]->codecpar)
            .ThrowExceptionIfError();
        ffmpeg.avcodec_open2(_videoCodecContext, codec, null).ThrowExceptionIfError();
    }
    else
    {
        codec = ffmpeg.avcodec_find_decoder_by_name("h264");
        Console.WriteLine("rkmpp decoder not found\n");
        _videoCodecContext = ffmpeg.avcodec_alloc_context3(codec);
        ffmpeg.avcodec_parameters_to_context(_videoCodecContext,
            _pFormatContext->streams[_streamIndex]->codecpar)
            .ThrowExceptionIfError();
        ffmpeg.avcodec_open2(_videoCodecContext, codec, null).ThrowExceptionIfError();
    }

do
{
    ffmpeg.av_packet_unref(_pPacket);
    error = ffmpeg.av_read_frame(_pFormatContext, _pPacket);

    if (error == ffmpeg.AVERROR_EOF || cancellationToken.IsCancellationRequested)
    {
        return;
    }

    error.ThrowExceptionIfError();
if (_pPacket->stream_index == _streamIndex)
                {
                    ffmpeg.avcodec_send_packet(_videoCodecContext, _pPacket)
                        .ThrowExceptionIfError();
                    error = ffmpeg.avcodec_receive_frame(_videoCodecContext, _videoFrame);
                    if (error == 0)
                    {
                        AVFrame frame;
                        //if (_videoCodecContext->hw_device_ctx != null)
                        //{
                        //    if (HWDevice == (AVHWDeviceType)12)
                        //    {
                        //        _receivedFrame->format = (int)AVPixelFormat.AV_PIX_FMT_NV12;

                        //        _receivedFrame->width = _videoFrame->width;
                        //        _receivedFrame->height = _videoFrame->height;
                        //    }
                        //    ffmpeg.av_hwframe_transfer_data(_receivedFrame, _videoFrame, 0)
                        //        .ThrowExceptionIfError();
                        //    frame = *_receivedFrame;
                        //}
                        //else
                            frame = *_videoFrame;

                        var convertedFrame = vfc.Convert(frame);
                        IPHostApi.SetVideoPtr(uuid, convertedFrame.width, convertedFrame.height, (IntPtr)convertedFrame.data[0]);
                    }
                    else if (error != -11)
                    {
                        error.ThrowExceptionIfError();
                    }

The playback speed is like 1.5 times

[hevc_rkmpp] Zerolatency encoding

I am using hevc_rkmpp encoder in C++ and trying to figure out how to enable zero-latency mode.
I.e. send frame = receive packet without delays, buffers and etc.

What I am trying to do - send real-time captures of game frames over the LAN, display and save them for later usage.
I was using other implementation of ffmpeg for rkmpp (Orange PI 5 Plus) and summary delay wasn't bigger than 1 frame, but it had issues with bit-rate control.
Now, with your project, I can easily control bit-rate and quality, but summary delay is bigger than 3-4 frames, which is not real-time for me. (its hard to react in action games with such delay)

Current problem with encoder is that I always get AVPacket for AVFrame with 1 frame delay.
I.e. sending and receiving data like this:

send frame     : 0 - 1 - 2 - 3 - 4 - ...
receive packet : X - 0 - 1 - 2 - 3 - ...

Code flow and set options are pretty basic: (I'll skip really basic ones, those are to test zero-latency mode)

 - gop_size = 0
 - max_b_frames = 0
 - refs = 0
 - flags |= AV_CODEC_FLAG_LOW_DELAY
 - flags2 |= AV_CODEC_FLAG2_FAST
 - preset = ultrafast
 - tune = zerolatency

 1. receive frame from internal queue
 2. use `sws_scale` to convert to correct format
 3. `av_rescale_q` to fix pts
 4. `avcodec_send_frame` to send frame to encoder
 5. right after that `avcodec_receive_packet` but there is no data for the first frame
 6. doing steps from 1 to 4 and calling `avcodec_receive_packet` for the second time gives packet for the first frame

Is there is any option to force encoder give exactly one packet per frame without buffering it on its side?

P.S. It may be another issue, but will ask in any way - hevc_rkmpp decoder always crashes after working for a few seconds if using it from C++ code. Is it intended behavior or I need to create another issue?

use filter scale_rkrga failed in rk3588（ubuntu22.02 desktop）

cmd:
ffmpeg-rockchip-master$ ./ffmpeg -timeout 10000000 -rtsp_transport tcp -loglevel debug -c:v h264_rkmpp -i rtsp://xxx:[email protected]/h264/ch2/main/av_stream -vf scale_rkrga=1280:720 -c:v h264_rkmpp -global_quality 19 -g 16 -rw_timeout 5000000 -f flv -an rtmp://127.0.0.1/cams/live
log:

[AVFilterGraph @ 0x556afe0190] Setting 'w' to value '1280'
[AVFilterGraph @ 0x556afe0190] Setting 'h' to value '720'
detected 8 logical cores
[graph 0 input from stream 0:0 @ 0x556af59940] Setting 'video_size' to value '1920x1080'
[graph 0 input from stream 0:0 @ 0x556af59940] Setting 'pix_fmt' to value '23'
[graph 0 input from stream 0:0 @ 0x556af59940] Setting 'time_base' to value '1/90000'
[graph 0 input from stream 0:0 @ 0x556af59940] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 0x556af59940] Setting 'frame_rate' to value '25/1'
[graph 0 input from stream 0:0 @ 0x556af59940] w:1920 h:1080 pixfmt:nv12 tb:1/90000 fr:25/1 sar:0/1
[format @ 0x556af571d0] Setting 'pix_fmts' to value 'gray|yuv420p|yuv422p|yuv444p|nv12|nv21|nv16|nv24|yuyv422|yvyu422|uyvy422|rgb24|bgr24|rgba|rgb0|bgra|bgr0|argb|0rgb|abgr|0bgr|drm_prime'
[auto_scale_0 @ 0x556af57ec0] w:iw h:ih flags:'' interl:0
[Parsed_scale_rkrga_0 @ 0x556b0815d0] auto-inserting filter 'auto_scale_0' between the filter 'graph 0 input from stream 0:0' and the filter 'Parsed_scale_rkrga_0'
Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:0' and the filter 'auto_scale_0'
[vf#0:0 @ 0x556af7a8e0] Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
[vist#0:0/h264 @ 0x556af4c8a0] Decoder returned EOF, finishing
[vist#0:0/h264 @ 0x556af4c8a0] Terminating decoder thread
Error while filtering: Function not implemented
[out#0/flv @ 0x556af50cf0] Nothing was written into output file, because at least one of its streams received no packets.
frame= 0 fps=0.0 q=0.0 Lsize= 0kB time=N/A bitrate=N/A speed=N/A
[AVIOContext @ 0x556af58b90] Statistics: 0 bytes written, 0 seeks, 0 writeouts
[rtmp @ 0x556af56660] UnPublishing stream...
[rtmp @ 0x556af56660] Deleting stream...
[in#0/rtsp @ 0x556af45fb0] Terminating demuxer thread
[in#0/rtsp @ 0x556af45fb0] Input file #0 (rtsp://admin:[email protected]/h264/ch2/main/av_stream):
[in#0/rtsp @ 0x556af45fb0] Input stream #0:0 (video): 7 packets read (295283 bytes); 2 frames decoded; 0 decode errors;
[in#0/rtsp @ 0x556af45fb0] Total: 7 packets (295283 bytes) demuxed
Conversion failed!
请问是什么原因啊
谢谢！

addr = mpp_buffer_get_ptr(desc->buffers[i]); core dump

大佬您好，又来请教一个问题。

我在进行网络流解码或者文件解码操作时hwcontext_rkmpp.c:397出现了core dump。
命令行如下：
gdb --args ffmpeg -re -c:v h264_rkmpp -hwaccel_output_format drm_prime -i /xxx/8.mkv -an -sn -vframes 5000 -f null -

堆栈：

采用调用ffmpeg接口的方式出现了同样的问题，mpp已经更新到官网最新，使用的是最新的6.1分支的代码。

附件是我测试用的视频
8.zip

Error native compilation in Armbian 5.10.160-legacy-rk35xx

I'm using this wiki to compile in an orange pi 5 ARM: Armbian 24.2.1 Bookworm with Linux 5.10.160-legacy-rk35xx

https://github.com/nyanmisaka/ffmpeg-rockchip/wiki/Compilation

MPP and RGA compiles OK.

When I'm trying to compile the ffmpeg:

CC	libavcodec/rkmppenc.o
libavcodec/rkmppdec.c: In function ‘rkmpp_get_coding_type’:
libavcodec/rkmppdec.c:43:44: error: ‘MPP_VIDEO_CodingAV1’ undeclared (first use in this function); did you mean ‘MPP_VIDEO_CodingAVS’?
   43 |     case AV_CODEC_ID_AV1:           return MPP_VIDEO_CodingAV1;
      |                                            ^~~~~~~~~~~~~~~~~~~
      |                                            MPP_VIDEO_CodingAVS
libavcodec/rkmppdec.c:43:44: note: each undeclared identifier is reported only once for each function it appears in
libavcodec/rkmppdec.c: In function ‘rkmpp_decode_init’:
libavcodec/rkmppdec.c:243:35: error: ‘MPP_FRAME_FBC_AFBC_V2’ undeclared (first use in this function); did you mean ‘MPP_FRAME_FBC_AFBC_V1’?
  243 |         MppFrameFormat afbc_fmt = MPP_FRAME_FBC_AFBC_V2;
      |                                   ^~~~~~~~~~~~~~~~~~~~~
      |                                   MPP_FRAME_FBC_AFBC_V1
libavcodec/rkmppdec.c: In function ‘rkmpp_export_frame’:
libavcodec/rkmppdec.c:550:27: error: implicit declaration of function ‘mpp_frame_get_offset_y’; did you mean ‘mpp_frame_get_fmt’? [-Werror=implicit-function-declaration]
  550 |         frame->crop_top = mpp_frame_get_offset_y(mpp_frame);
      |                           ^~~~~~~~~~~~~~~~~~~~~~
      |                           mpp_frame_get_fmt
cc1: some warnings being treated as errors
make: *** [ffbuild/common.mak:81: libavcodec/rkmppdec.o] Error 1
make: *** Waiting for unfinished jobs....
libavcodec/rkmppenc.c: In function ‘rkmpp_get_mpp_fmt_h26x’:
libavcodec/rkmppenc.c:46:37: error: ‘MPP_FMT_YUV444P’ undeclared (first use in this function); did you mean ‘MPP_FMT_YUV444SP’?
   46 |     case AV_PIX_FMT_YUV444P: return MPP_FMT_YUV444P;
      |                                     ^~~~~~~~~~~~~~~
      |                                     MPP_FMT_YUV444SP
libavcodec/rkmppenc.c:46:37: note: each undeclared identifier is reported only once for each function it appears in
libavcodec/rkmppenc.c: In function ‘rkmpp_set_enc_cfg_prep’:
libavcodec/rkmppenc.c:301:20: error: ‘MPP_FRAME_FBC_AFBC_V2’ undeclared (first use in this function); did you mean ‘MPP_FRAME_FBC_AFBC_V1’?
  301 |         mpp_fmt |= MPP_FRAME_FBC_AFBC_V2;
      |                    ^~~~~~~~~~~~~~~~~~~~~
      |                    MPP_FRAME_FBC_AFBC_V1
libavcodec/rkmppenc.c: In function ‘rkmpp_submit_frame’:
libavcodec/rkmppenc.c:654:20: error: ‘MPP_FRAME_FBC_AFBC_V2’ undeclared (first use in this function); did you mean ‘MPP_FRAME_FBC_AFBC_V1’?
  654 |         mpp_fmt |= MPP_FRAME_FBC_AFBC_V2;
      |                    ^~~~~~~~~~~~~~~~~~~~~
      |                    MPP_FRAME_FBC_AFBC_V1
libavcodec/rkmppenc.c:658:13: error: implicit declaration of function ‘mpp_frame_set_offset_y’; did you mean ‘mpp_frame_set_fmt’? [-Werror=implicit-function-declaration]
  658 |             mpp_frame_set_offset_y(mpp_frame, afbc_offset_y);
      |             ^~~~~~~~~~~~~~~~~~~~~~
      |             mpp_frame_set_fmt
libavcodec/rkmppenc.c:671:9: error: implicit declaration of function ‘mpp_frame_set_fbc_hdr_stride’; did you mean ‘mpp_frame_set_hor_stride’? [-Werror=implicit-function-declaration]
  671 |         mpp_frame_set_fbc_hdr_stride(mpp_frame, hor_stride);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |         mpp_frame_set_hor_stride
cc1: some warnings being treated as errors
make: *** [ffbuild/common.mak:81: libavcodec/rkmppenc.o] Error 1

Test filter error

chip:rk3566
board: lubancat zero

Encoder and Decoder test normal

root@lubancat:~/dev/ffmpeg# cat /proc/version
Linux version 4.19.232 ([email protected]) (gcc version 6.3.1 20170404 (Linaro GCC 6.3-2017.05), GNU ld (Linaro_Binutils-2017.05) 2.27.0.20161019) #17 SMP Fri Mar 15 15:24:13 CST 2024

root@lubancat:~/dev/ffmpeg# ls /dev/rga -lh
crw-rw---- 1 root video 10, 58 Mar 21 17:00 /dev/rga

root@lubancat:~/dev/ffmpeg# ./ffmpeg -init_hw_device rkmpp=hw -filter_hw_device hw -f lavfi -i testsrc2=s=1920x1080,format=nv12 \
-vf hwupload,scale_rkrga=w=1280:h=720:format=nv12 -c:v h264_rkmpp -b:v 4M -maxrate 4M -vframes 1000 -y /tmp/tmp.mp4
ffmpeg version 1fa586a Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-libdrm --enable-rkmpp --enable-rkrga --enable-libass --enable-libfreetype --enable-filter=drawtext --enable-filter=subtitles --enable-filter=subtitles --enable-libharfbuzz
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, lavfi, from 'testsrc2=s=1920x1080,format=nv12':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: wrapped_avframe, nv12, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> h264 (h264_rkmpp))
Press [q] to stop, [?] for help
librga fail to get driver version! Compatibility mode will be enabled.

47 im2d_rga_impl rga_version_below_minimun_range_user_driver(304): The driver may be compatible, but it is best to update the driver to version 1.2.4. You can try to update the SDK or update the <SDK>/kernel/drivers/video/rockchip/rga3 directory individually. current version: librga 1.9.2, driver .
rga_api version 1.9.2_[1]
[Parsed_scale_rkrga_1 @ 0x55669235f0] RGA async blit returned invalid fence_fd: 0
[vf#0:0 @ 0x5566920eb0] Error while filtering: Generic error in an external library
Failed to inject frame into filter network: Generic error in an external library
Error while filtering: Generic error in an external library
[out#0/mp4 @ 0x556691f1f0] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=N/A bitrate=N/A speed=N/A    
Conversion failed!

undefined reference to ff_h264_*

Hello. I'm using Raxda's Debian and I'm getting these errors:

root@radxa-zero3:/opt/ffmpeg-rockchip# make -j 4
HTML	doc/ffmpeg-scaler.html
HTML	doc/ffmpeg-utils.html
HTML	doc/ffmpeg-resampler.html
HTML	doc/ffprobe-all.html
HTML	doc/ffmpeg-codecs.html
HTML	doc/ffmpeg-bitstream-filters.html
HTML	doc/ffmpeg-formats.html
HTML	doc/ffmpeg-protocols.html
HTML	doc/ffmpeg-devices.html
HTML	doc/ffmpeg-filters.html
HTML	doc/libavutil.html
HTML	doc/libswscale.html
HTML	doc/libswresample.html
HTML	doc/libavcodec.html
HTML	doc/libavformat.html
HTML	doc/libavdevice.html
HTML	doc/libavfilter.html
HTML	doc/community.html
HTML	doc/developer.html
HTML	doc/faq.html
HTML	doc/fate.html
HTML	doc/general.html
HTML	doc/git-howto.html
HTML	doc/mailing-list-faq.html
HTML	doc/nut.html
HTML	doc/platform.html
POD	doc/ffmpeg.pod
POD	doc/ffplay.pod
POD	doc/ffprobe.pod
POD	doc/ffmpeg-all.pod
POD	doc/ffplay-all.pod
POD	doc/ffprobe-all.pod
POD	doc/ffmpeg-utils.pod
POD	doc/ffmpeg-scaler.pod
POD	doc/ffmpeg-resampler.pod
POD	doc/ffmpeg-codecs.pod
POD	doc/ffmpeg-bitstream-filters.pod
POD	doc/ffmpeg-formats.pod
POD	doc/ffmpeg-protocols.pod
POD	doc/ffmpeg-devices.pod
POD	doc/ffmpeg-filters.pod
POD	doc/libavutil.pod
POD	doc/libswscale.pod
POD	doc/libswresample.pod
POD	doc/libavcodec.pod
POD	doc/libavformat.pod
POD	doc/libavdevice.pod
POD	doc/libavfilter.pod
MAN	doc/ffmpeg.1
MAN	doc/ffplay.1
MAN	doc/ffprobe.1
MAN	doc/ffmpeg-all.1
MAN	doc/ffplay-all.1
MAN	doc/ffprobe-all.1
MAN	doc/ffmpeg-utils.1
MAN	doc/ffmpeg-scaler.1
MAN	doc/ffmpeg-resampler.1
MAN	doc/ffmpeg-codecs.1
MAN	doc/ffmpeg-bitstream-filters.1
MAN	doc/ffmpeg-formats.1
MAN	doc/ffmpeg-protocols.1
MAN	doc/ffmpeg-devices.1
MAN	doc/ffmpeg-filters.1
MAN	doc/libavutil.3
MAN	doc/libswscale.3
MAN	doc/libswresample.3
MAN	doc/libavcodec.3
MAN	doc/libavformat.3
MAN	doc/libavdevice.3
MAN	doc/libavfilter.3
LD	ffmpeg_g
LD	ffplay_g
/usr/bin/ld: /lib/aarch64-linux-gnu/libtirpc.so.3: warning: common of `rpc_createerr@@GLIBC_2.17' overridden by definition from /lib/aarch64-linux-gnu/libc.so.6
/usr/bin/ld: /lib/aarch64-linux-gnu/libtirpc.so.3: warning: common of `rpc_createerr@@GLIBC_2.17' overridden by definition from /lib/aarch64-linux-gnu/libc.so.6
LD	ffprobe_g
/usr/bin/ld: /lib/aarch64-linux-gnu/libtirpc.so.3: warning: common of `rpc_createerr@@GLIBC_2.17' overridden by definition from /lib/aarch64-linux-gnu/libc.so.6
/usr/bin/ld: /usr/bin/ld: libavcodec/libavcodec.a(h264_cavlc.o): in function `decode_mb_skip':
/opt/ffmpeg-rockchip/libavcodec/h264_mvpred.h:951libavcodec/libavcodec.a(h264_cavlc.o): in function `decode_mb_skip':
: undefined reference to `ff_h264_pred_direct_motion'
/opt/ffmpeg-rockchip/libavcodec/h264_mvpred.h:951: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: /usr/bin/ld: libavcodec/libavcodec.a(h264_cavlc.o): in function `ff_h264_decode_mb_cavlc':
/opt/ffmpeg-rockchip/libavcodec/h264_cavlc.c:855: undefined reference to `ff_h264_pred_direct_motionlibavcodec/libavcodec.a(h264_cavlc.o): in function `ff_h264_decode_mb_cavlc':
/opt/ffmpeg-rockchip/libavcodec/h264_cavlc.c:855'
: undefined reference to `ff_h264_pred_direct_motion/usr/bin/ld: '
/opt/ffmpeg-rockchip/libavcodec/h264_cavlc.c:946: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: /opt/ffmpeg-rockchip/libavcodec/h264_cavlc.c:946: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld/usr/bin/ld: : libavcodec/libavcodec.a(allcodecs.o):(libavcodec/libavcodec.a(allcodecs.o).data.rel.ro:+0x(.data.rel.ro+0xa0a0)): undefined reference to `: undefined reference to `ff_ffv1_encoderff_ffv1_encoder'
'
/usr/bin/ld: /usr/bin/ld: libavcodec/libavcodec.a(h264_slice.o): in function `h264_slice_init':
/opt/ffmpeg-rockchip/libavcodec/h264_slice.c:1935: undefined reference to `ff_h264_direct_dist_scale_factor'
/usr/bin/ld: /opt/ffmpeg-rockchip/libavcodec/h264_slice.c:1937: undefined reference to `ff_h264_direct_ref_list_init'
libavcodec/libavcodec.a(h264_slice.o): in function `h264_slice_init':
/opt/ffmpeg-rockchip/libavcodec/h264_slice.c:1935: undefined reference to `ff_h264_direct_dist_scale_factor'
/usr/bin/ld: /opt/ffmpeg-rockchip/libavcodec/h264_slice.c:1937: undefined reference to `ff_h264_direct_ref_list_init'
/usr/bin/ld: /usr/bin/ld: libavcodec/libavcodec.a(h264_cabac.o): in function `decode_mb_skip':
/opt/ffmpeg-rockchip/libavcodec/h264_mvpred.h:951: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: libavcodec/libavcodec.a(h264_cabac.o): in function `ff_h264_decode_mb_cabac':
/opt/ffmpeg-rockchip/libavcodec/h264_cabac.c:2221: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: /opt/ffmpeg-rockchip/libavcodec/h264_cabac.c:2124: undefined reference to `ff_h264_pred_direct_motion'
libavcodec/libavcodec.a(h264_cabac.o): in function `decode_mb_skip':
/opt/ffmpeg-rockchip/libavcodec/h264_mvpred.h:951: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: libavcodec/libavcodec.a(h264_cabac.o): in function `ff_h264_decode_mb_cabac':
/opt/ffmpeg-rockchip/libavcodec/h264_cabac.c:2221: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: /opt/ffmpeg-rockchip/libavcodec/h264_cabac.c:2124: undefined reference to `ff_h264_pred_direct_motion'
collect2: error: ld returned 1 exit status
make: *** [Makefile:133: ffmpeg_g] Error 1
make: *** Waiting for unfinished jobs....
collect2: error: ld returned 1 exit status
make: *** [Makefile:133: ffplay_g] Error 1
/usr/bin/ld: libavcodec/libavcodec.a(h264_cavlc.o): in function `decode_mb_skip':
/opt/ffmpeg-rockchip/libavcodec/h264_mvpred.h:951: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: libavcodec/libavcodec.a(h264_cavlc.o): in function `ff_h264_decode_mb_cavlc':
/opt/ffmpeg-rockchip/libavcodec/h264_cavlc.c:855: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: /opt/ffmpeg-rockchip/libavcodec/h264_cavlc.c:946: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: libavcodec/libavcodec.a(allcodecs.o):(.data.rel.ro+0xa0): undefined reference to `ff_ffv1_encoder'
/usr/bin/ld: libavcodec/libavcodec.a(h264_slice.o): in function `h264_slice_init':
/opt/ffmpeg-rockchip/libavcodec/h264_slice.c:1935: undefined reference to `ff_h264_direct_dist_scale_factor'
/usr/bin/ld: /opt/ffmpeg-rockchip/libavcodec/h264_slice.c:1937: undefined reference to `ff_h264_direct_ref_list_init'
/usr/bin/ld: libavcodec/libavcodec.a(h264_cabac.o): in function `decode_mb_skip':
/opt/ffmpeg-rockchip/libavcodec/h264_mvpred.h:951: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: libavcodec/libavcodec.a(h264_cabac.o): in function `ff_h264_decode_mb_cabac':
/opt/ffmpeg-rockchip/libavcodec/h264_cabac.c:2221: undefined reference to `ff_h264_pred_direct_motion'
/usr/bin/ld: /opt/ffmpeg-rockchip/libavcodec/h264_cabac.c:2124: undefined reference to `ff_h264_pred_direct_motion'
collect2: error: ld returned 1 exit status
make: *** [Makefile:133: ffprobe_g] Error 1

The configure command was copied from https://github.com/nyanmisaka/ffmpeg-rockchip/wiki/Compilation:

./configure --prefix=/usr --enable-gpl --enable-version3 --enable-libdrm --enable-rkmpp --enable-rkrga

Could somebody help to fix this?

root@radxa-zero3:/opt/ffmpeg-rockchip# git rev-parse HEAD
ba84e56c51d9cde1f3b1fead2a21e4d271028709

RK3566 performance issues on debian

Hello, I built and ran the performance test on Debian's RK3566. I compared it with gst. The performance gap is a bit big. What is the source of this problem?

OS:debian
HW:RK3566
Test file duration: HH:MMSS: 00:03:12
File encoding format: 3840x2160 28.72fps VP90

ffmpeg result:

root@debian:~# time ffmpeg -hwaccel rkmpp -i TCL4K.mkv -an -benchmark -f null -
ffmpeg version 4c1997e5a8 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --enable-gpl --enable-version3 --disable-doc --disable-shared --enable-static --enable-libdrm --enable-rkmpp --enable-rkrga
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, matroska,webm, from 'TCL4K.mkv':
  Metadata:
    COMPATIBLE_BRANDS: isomiso2avc1mp41
    MAJOR_BRAND     : isom
    MINOR_VERSION   : 512
    ENCODER         : IDMmkvlib0.1
    LANGUAGE        : und
    HANDLER_NAME    : AudioHandler
  Duration: 00:03:13.28, start: 0.000000, bitrate: 13694 kb/s
  Stream #0:0: Video: vp9 (Profile 0), yuv420p(tv, bt709/unknown/unknown), 3840x2160, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 1k tbn (default)
  Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp (default)
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (vp9_rkmpp) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    COMPATIBLE_BRANDS: isomiso2avc1mp41
    MAJOR_BRAND     : isom
    MINOR_VERSION   : 512
    HANDLER_NAME    : AudioHandler
    LANGUAGE        : und
    encoder         : Lavf60.16.100
  Stream #0:0: Video: wrapped_avframe, nv12(tv, bt709/unknown/unknown, progressive), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn (default)
    Metadata:
      encoder         : Lavc60.31.102 wrapped_avframe
[out#0/null @ 0x55bd821bb0] video:2718kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
frame= 5798 fps= 36 q=-0.0 Lsize=N/A time=00:03:13.23 bitrate=N/A speed= 1.2x
bench: utime=98.383s stime=37.910s rtime=160.763s
bench: maxrss=79848kB

real    2m41.066s
user    1m38.532s
sys     0m38.053s

gstreamer result:

root@debian:~# time gst-launch-1.0 filesrc location=TCL4K.mkv ! decodebin ! videoconvert ! videoscale ! fakesink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:01:47.828021557
Setting pipeline to NULL ...
Freeing pipeline ...

real    1m48.100s
user    0m35.405s
sys     0m10.839s

almost four times the difference。

external library error? rkmpp decoder issue

2024-02-08 15:33:35.670125850  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : Input #0, rtsp, from 'rtsp://*:*@0.0.0.1:554/cam/realmonitor?channel=1&subtype=2':
2024-02-08 15:33:35.670394175  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :   Metadata:
2024-02-08 15:33:35.670695166  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :     title           : Media Server
2024-02-08 15:33:35.670903993  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :   Duration: N/A, start: 0.080000, bitrate: N/A
2024-02-08 15:33:35.671047780  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :   Stream #0:0: Video: hevc (Main), yuvj420p(pc, bt470bg/bt470bg/smpte170m), 1920x1080, 25 fps, 25 tbr, 90k tbn
2024-02-08 15:33:35.671235607  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : Stream mapping:
2024-02-08 15:33:35.671427518  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :   Stream #0:0 -> #0:0 (hevc (hevc_rkmpp) -> rawvideo (native))
2024-02-08 15:33:35.671616512  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : Press [q] to stop, [?] for help
2024-02-08 15:33:35.671804048  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : Output #0, rawvideo, to 'pipe:':
2024-02-08 15:33:35.671993626  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :   Metadata:
2024-02-08 15:33:35.672180870  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :     title           : Media Server
2024-02-08 15:33:35.672371323  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :     encoder         : Lavf60.16.100
2024-02-08 15:33:35.672562650  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :   Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(pc, bt470bg/bt470bg/smpte170m, progressive), 1920x1080, q=2-31, 622080 kb/s, 25 fps, 25 tbn
2024-02-08 15:33:35.672759519  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :     Metadata:
2024-02-08 15:33:35.672947347  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   :       encoder         : Lavc60.31.102 rawvideo
2024-02-08 15:33:35.673150341  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=    0 fps=0.0 q=0.0 size=       0kB time=N/A bitrate=N/A dup=0 drop=1 speed=N/A    
2024-02-08 15:33:35.673332627  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=    0 fps=0.0 q=0.0 size=       0kB time=N/A bitrate=N/A dup=0 drop=1 speed=N/A    
2024-02-08 15:33:35.673523079  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=   38 fps=9.4 q=-0.0 size=  112388kB time=00:00:01.80 bitrate=511488.0kbits/s dup=0 drop=1 speed=0.445x    
2024-02-08 15:33:35.673711490  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  103 fps= 23 q=-0.0 size=  309825kB time=00:00:04.40 bitrate=576837.8kbits/s dup=0 drop=1 speed=0.967x    
2024-02-08 15:33:35.673923234  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  141 fps= 28 q=-0.0 size=  425250kB time=00:00:05.64 bitrate=617668.1kbits/s dup=0 drop=1 speed=1.11x    
2024-02-08 15:33:35.674118353  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  155 fps= 28 q=-0.0 size=  467775kB time=00:00:06.16 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.09x    
2024-02-08 15:33:35.674309680  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  161 fps= 26 q=-0.0 size=  486000kB time=00:00:06.44 bitrate=618216.1kbits/s dup=0 drop=1 speed=1.04x    
2024-02-08 15:33:35.674503049  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  167 fps= 23 q=-0.0 size=  504225kB time=00:00:06.64 bitrate=622080.0kbits/s dup=0 drop=1 speed=0.926x    
2024-02-08 15:33:35.674690293  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  192 fps= 23 q=-0.0 size=  580162kB time=00:00:07.64 bitrate=622080.0kbits/s dup=0 drop=1 speed=0.925x    
2024-02-08 15:33:35.674876371  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  217 fps= 24 q=-0.0 size=  656100kB time=00:00:08.64 bitrate=622080.0kbits/s dup=0 drop=1 speed=0.959x    
2024-02-08 15:33:35.675067115  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  253 fps= 26 q=-0.0 size=  765450kB time=00:00:10.08 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.05x    
2024-02-08 15:33:35.675254360  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  266 fps= 26 q=-0.0 size=  804938kB time=00:00:10.64 bitrate=619741.4kbits/s dup=0 drop=1 speed=1.05x    
2024-02-08 15:33:35.675450354  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  280 fps= 26 q=-0.0 size=  847462kB time=00:00:11.16 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.05x    
2024-02-08 15:33:35.675640514  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  289 fps= 26 q=-0.0 size=  874800kB time=00:00:11.56 bitrate=619927.5kbits/s dup=0 drop=1 speed=1.03x    
2024-02-08 15:33:35.675829800  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  292 fps= 25 q=-0.0 size=  883912kB time=00:00:11.64 bitrate=622080.0kbits/s dup=0 drop=1 speed=0.986x    
2024-02-08 15:33:35.676017920  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  295 fps= 23 q=-0.0 size=  893025kB time=00:00:12.08 bitrate=605601.1kbits/s dup=0 drop=1 speed=0.96x    
2024-02-08 15:33:35.676206914  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  297 fps= 22 q=-0.0 size=  899100kB time=00:00:12.16 bitrate=605709.5kbits/s dup=0 drop=1 speed=0.902x    
2024-02-08 15:33:35.676396200  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  305 fps= 22 q=-0.0 size=  923400kB time=00:00:12.48 bitrate=606129.2kbits/s dup=0 drop=1 speed=0.891x    
2024-02-08 15:33:35.676588110  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  309 fps= 21 q=-0.0 size=  935550kB time=00:00:12.64 bitrate=606331.1kbits/s dup=0 drop=1 speed=0.87x    
2024-02-08 15:33:35.676783229  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  312 fps= 21 q=-0.0 size=  944662kB time=00:00:12.76 bitrate=606479.2kbits/s dup=0 drop=1 speed=0.845x    
2024-02-08 15:33:35.676975140  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  366 fps= 23 q=-0.0 size= 1108688kB time=00:00:14.92 bitrate=608737.8kbits/s dup=0 drop=1 speed=0.956x    
2024-02-08 15:33:35.677174926  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  411 fps= 26 q=-0.0 size= 1245375kB time=00:00:16.52 bitrate=617561.3kbits/s dup=0 drop=1 speed=1.03x    
2024-02-08 15:33:35.677384919  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  432 fps= 26 q=-0.0 size= 1309162kB time=00:00:17.24 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.04x    
2024-02-08 15:33:35.677586747  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  441 fps= 25 q=-0.0 size= 1336500kB time=00:00:17.64 bitrate=620669.4kbits/s dup=0 drop=1 speed=1.02x    
2024-02-08 15:33:35.677778949  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  461 fps= 26 q=-0.0 size= 1397250kB time=00:00:18.40 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.03x    
2024-02-08 15:33:35.677971151  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  466 fps= 25 q=-0.0 size= 1412438kB time=00:00:18.64 bitrate=620745.1kbits/s dup=0 drop=1 speed=1.02x    
2024-02-08 15:33:35.678168604  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  488 fps= 26 q=-0.0 size= 1479262kB time=00:00:19.48 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.03x    
2024-02-08 15:33:35.678365764  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  499 fps= 26 q=-0.0 size= 1512675kB time=00:00:19.96 bitrate=620833.3kbits/s dup=0 drop=1 speed=1.03x    
2024-02-08 15:33:35.678553592  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  514 fps= 26 q=-0.0 size= 1558238kB time=00:00:20.56 bitrate=620869.7kbits/s dup=0 drop=1 speed=1.03x    
2024-02-08 15:33:35.678749003  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  525 fps= 26 q=-0.0 size= 1591650kB time=00:00:21.00 bitrate=620895.1kbits/s dup=0 drop=1 speed=1.03x    
2024-02-08 15:33:35.678938580  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  536 fps= 25 q=-0.0 size= 1625062kB time=00:00:21.40 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.01x    
2024-02-08 15:33:35.679126408  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  540 fps= 25 q=-0.0 size= 1637212kB time=00:00:21.64 bitrate=619780.3kbits/s dup=0 drop=1 speed=0.986x    
2024-02-08 15:33:35.679323860  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  544 fps= 24 q=-0.0 size= 1649362kB time=00:00:22.04 bitrate=613048.0kbits/s dup=0 drop=1 speed=0.978x    
2024-02-08 15:33:35.679540853  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  545 fps= 24 q=-0.0 size= 1652400kB time=00:00:22.08 bitrate=613064.3kbits/s dup=0 drop=1 speed=0.955x    
2024-02-08 15:33:35.679714098  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  548 fps= 23 q=-0.0 size= 1658475kB time=00:00:22.16 bitrate=613096.9kbits/s dup=0 drop=1 speed=0.926x    
2024-02-08 15:33:35.679903676  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  610 fps= 25 q=-0.0 size= 1849838kB time=00:00:24.68 bitrate=614014.1kbits/s dup=0 drop=1 speed=1.01x    
2024-02-08 15:33:35.680097920  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  641 fps= 26 q=-0.0 size= 1944000kB time=00:00:25.60 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.03x    
2024-02-08 15:33:35.680290414  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  651 fps= 26 q=-0.0 size= 1974375kB time=00:00:26.00 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.02x    
2024-02-08 15:33:35.680481158  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  659 fps= 25 q=-0.0 size= 1998675kB time=00:00:26.36 bitrate=621136.0kbits/s dup=0 drop=1 speed=1.01x    
2024-02-08 15:33:35.680699318  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  667 fps= 24 q=-0.0 size= 2022975kB time=00:00:26.64 bitrate=622080.0kbits/s dup=0 drop=1 speed=0.975x    
2024-02-08 15:33:35.680896770  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  695 fps= 25 q=-0.0 size= 2108025kB time=00:00:28.08 bitrate=614990.8kbits/s dup=0 drop=1 speed=1.01x    
2024-02-08 15:33:35.681086640  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  716 fps= 25 q=-0.0 size= 2171812kB time=00:00:28.64 bitrate=621211.2kbits/s dup=0 drop=1 speed=1.01x    
2024-02-08 15:33:35.681279425  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  741 fps= 26 q=-0.0 size= 2247750kB time=00:00:29.60 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.02x    
2024-02-08 15:33:35.681462295  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  751 fps= 25 q=-0.0 size= 2278125kB time=00:00:30.04 bitrate=621251.7kbits/s dup=0 drop=1 speed=1.02x    
2024-02-08 15:33:35.681648956  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  768 fps= 26 q=-0.0 size= 2329762kB time=00:00:30.68 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.02x    
2024-02-08 15:33:35.681852241  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  780 fps= 26 q=-0.0 size= 2366212kB time=00:00:31.16 bitrate=622080.0kbits/s dup=0 drop=1 speed=1.02x    
2024-02-08 15:33:35.682038902  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  787 fps= 25 q=-0.0 size= 2387475kB time=00:00:31.52 bitrate=620501.1kbits/s dup=0 drop=1 speed=1.02x    
2024-02-08 15:33:35.682225271  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  791 fps= 25 q=-0.0 size= 2399625kB time=00:00:31.64 bitrate=621293.6kbits/s dup=0 drop=1 speed=0.996x    
2024-02-08 15:33:35.682425349  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  796 fps= 24 q=-0.0 size= 2414812kB time=00:00:32.12 bitrate=615882.4kbits/s dup=0 drop=1 speed=0.984x    
2024-02-08 15:33:35.682614634  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  797 fps= 24 q=-0.0 size= 2417850kB time=00:00:32.16 bitrate=615890.1kbits/s dup=0 drop=1 speed=0.97x    
2024-02-08 15:33:35.682804212  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : [Parsed_scale_rkrga_0 @ 0x55a1a43f10] RGA blit failed: -12
2024-02-08 15:33:35.682989123  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : [vf#0:0 @ 0x55a1a16150] Error while filtering: Generic error in an external library
2024-02-08 15:33:35.683178992  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : Failed to inject frame into filter network: Generic error in an external library
2024-02-08 15:33:35.683375570  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : Error while filtering: Generic error in an external library
2024-02-08 15:33:35.683587313  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : [out#0/rawvideo @ 0x55a1a47bd0] video:2454300kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
2024-02-08 15:33:35.683766391  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : frame=  808 fps= 24 q=-0.0 Lsize= 2454300kB time=00:00:32.28 bitrate=622850.9kbits/s dup=0 drop=1 speed=0.949x    
2024-02-08 15:33:35.683989801  [2024-02-08 15:33:35] ffmpeg.camera2.detect ERROR   : Conversion failed!

MJPEG/JPEG

什么时候会更新MJPEG/JPEG的MPP解码呢？

Generic error in external library

[hevc_rkmpp @ 0xaaaaf62be8a0] Failed to init MPP context: -1
[vist#0:0/hevc @ 0xaaaaf62bcbd0] Error while opening decoder: Generic error in an external library
[vost#0:0/h264_rkmpp @ 0xaaaaf62dfa90] Error initializing a simple filtergraph
Error opening output file /ssd/Aliens.Special.Edition.1080p_h264_6M.mkv.
Error opening output files: Generic error in an external library

I get this on my rock 3c which has Rockchip RK3566 SoC and im using 5.10 rk35xx kernel

avcodec_send_frame(codec_ctx_, frame_) failed.

This work is fantastic. But I'm having a problem when using this library.

Do you know how to fix it?

Segmentation fault: mpp_buffer_group_init

I'm running command:

ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -c:v h264_rkmpp /tmp/1.flv

And it segfaults:

root@radxa-zero3:/dev/shm# ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -c:v h264_rkmpp /tmp/1.flv
ffmpeg version ba84e56c51 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr --enable-gpl --enable-version3 --enable-libdrm --enable-rkmpp --enable-rkrga
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 9571.528469, bitrate: N/A
  Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1000k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (h264_rkmpp))
Press [q] to stop, [?] for help
[swscaler @ 0xaaaaf7a9fd70] deprecated pixel format used, make sure you did set range correctly
Segmentation fault (core dumped)s

gdb says:

root@radxa-zero3:/dev/shm# gdb ffmpeg core
GNU gdb (Debian 10.1-1.7) 10.1.90.20210103-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ffmpeg...
(No debugging symbols found in ffmpeg)
[New LWP 32677]
[New LWP 32678]
[New LWP 32679]
[New LWP 32683]
[New LWP 32681]
[New LWP 32680]
[New LWP 32696]
[New LWP 32698]
[New LWP 32682]
[New LWP 32700]
[New LWP 32697]
[New LWP 32699]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".

warning: Loadable section ".note.gnu.build-id" outside of ELF segments
Core was generated by `ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -c:v h264_rkmpp /tmp/1.flv'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000ffff842b4c0c in MppBufferService::get_group(char const*, char const*, MppBufferMode, MppBufferType, unsigned int) ()
   from /lib/aarch64-linux-gnu/librockchip_mpp.so.1
[Current thread is 1 (Thread 0xffff8072e010 (LWP 32677))]
(gdb) bt
#0  0x0000ffff842b4c0c in MppBufferService::get_group(char const*, char const*, MppBufferMode, MppBufferType, unsigned int) ()
   from /lib/aarch64-linux-gnu/librockchip_mpp.so.1
#1  0x0000ffff842b5040 in mpp_buffer_group_init () from /lib/aarch64-linux-gnu/librockchip_mpp.so.1
#2  0x0000aaaacdaeb318 in ?? ()
#3  0x0000aaaacdae9db8 in ?? ()
#4  0x0000aaaacd652d18 in ?? ()
#5  0x0000aaaacd34d690 in ?? ()
#6  0x0000aaaaccf6b438 in ?? ()
#7  0x0000aaaaccf6cc50 in ?? ()
#8  0x0000aaaaccf6e294 in ?? ()
#9  0x0000aaaaccf70bfc in ?? ()
#10 0x0000aaaaccf63350 in main ()

解码后的图像颜色不对

consider supporting RV1126？

Integrating Rockchip's encoding and decoding SDK into ffmpeg is very convenient to use. not sure if author has considered supporting RV1126 Thanks

	if ((ret = r->mapi->encode_get_packet(r->mctx, &mpp_pkt)) != MPP_OK) {
	int log_level = (ret == MPP_NOK) ? AV_LOG_DEBUG : AV_LOG_ERROR;
	ret = (ret == MPP_NOK) ? AVERROR(EAGAIN) : AVERROR_EXTERNAL;
	av_log(avctx, log_level, "Failed to get packet from encoder output queue: %d\n", ret);
	return ret;

	ret = rkmpp_get_packet(avctx, packet);
	if (ret == AVERROR_EOF \|\|
	ret == AVERROR(EAGAIN))
	*got_packet = 0;

	if (avci->draining && !got_packet)
	avci->draining_done = 1;