jlennox / nvencsharp Goto Github PK

Hi,

I was very excited when I found your project, especially due to lack of any solution which would help us read frames into GPU for further processing on Windows.
For example, there is no way to get OpenCV+FFMPEG, or any other way into NVIDIA GPU. Only CPU works resulting in reading rate being just to slow with any other solution (Gstreamer doesn't even have any NVIDIA plugins for Windows yet).
However, I'm not able to get your VideoDecode project sample to work.

If I try with 4K video, it raises an exception on line 159, where the following code can be found: "backbufferStream.Write(packet.Packet)".

The exception states:
"System.NotSupportedException: 'Memory stream is not expandable.'"

However, if I try with HD video (1920x1080), then the loop seems run (if I put a breakpoint, it will be hit), but the display window is frozen (gray), and no video is being displayed, so I'm not sure if it works.

4K video
https://drive.google.com/file/d/1MWniv_hCi-EXiQFflnt3PGJ7cdwJRGRd/view?usp=sharing

Command-line used:
--input c:\t\x.mp4

HD video
https://drive.google.com/file/d/1uFQIyKy-_vKFSyF4Fy0TdL7SE_hnXF4T/view?usp=sharing

Command-line used:
--input c:\t\x2.mp4

I would appreciate any help.

If Let's say I have 2 Displays, and I start the execution of encoder it shows black screen in the presence of 2 display which are in extended mode.

Does anyone here have any idea about the legal status of redistributing this work? Especially, the native bits, that get linked against the proprietary NVidia SDKs.

Following your earlier comment about this test code, I converted my video to raw h264 like so:
ffmpeg -i input.mp4 -c copy output.h264
The file output.h264 displays the converted video but I only see the blue channel. Is there any special setup involved?

Hi @jlennox,

Thank you for clarifying this issue.
It is awesome, how much effort you have put into this project.
There is so much code inside.

My use case is simple:

• grab frame by frame from the input video
• convert it to RGBA
• get the pointer of the frame which should still reside in device memory.
• forward the frame to the Darknet (YOLOV wrapper) Detect method (the overload which accepts IntPtr as function argument).
• resize image
• write frame using NVIDIA encoder

All steps need to be performed in GPU, so no device-host copying should take place.

Is that possible with your project?
If yes, can you please provide some hints, or an example?

P.S.
It would be also also brilliant if GPU frame from your project can be converted to OpenCV GpuMat. The C# wrapper which can be used for that is EmguCV, so all other GPU operations can be done using EmguCV (i.e. OpenCV). The number of possible operations, which can be done on GpuMat by OpenCV is enormous.

Any help is appreciated.

Thanks

I believe this happens when the DirectX device backing the encoder gets lost.

The error needs to be caught and propagated to .NET

This is a known issue. The file NvEncSharp.Cuda.Library.dll is not being properly copied from its sub directory to the output directory. This can be remediated by manually copying as part of the post-build script.

This is only an issue if LibCudaLibrary is being used.

There was a note about linux testing so I wanted to post some limited results.
Wanted to try and figure out what nvidia options were avail on a machine to get it working I did:

export DOTNET_ROOT="/tmp/dotnet"
wget https://dot.net/v1/dotnet-install.sh && bash dotnet-install.sh --channel STS  --install-dir "${DOTNET_ROOT}"  && ln -s "${DOTNET_ROOT}/dotnet" /usr/bin/
dotnet new console
dotnet add package Lennox.NvEncSharp

Then for an app

var ret = LibNvEnc.TryInitialize(out var err);
if (ret != LibNcEncInitializeStatus.Success)
	throw new Exception($"Unable to initialize nvidia {ret} {err}");
LibCuda.Initialize();
var descriptions = CuDevice.GetDescriptions().ToArray();
if (descriptions.Length == 0)
	throw new Exception("Cannot find any nvidia cuda devices");
var first = descriptions.First();
var dev = CuDevice.GetDevice(first.Device.Handle);
using var context = dev.CreateContext();
var ps = new NvEncOpenEncodeSessionExParams {
	Version = LibNvEnc.NV_ENC_OPEN_ENCODE_SESSION_EX_PARAMS_VER,
	ApiVersion = LibNvEnc.NVENCAPI_VERSION,
	Device = context.Handle,
	DeviceType = NvEncDeviceType.Cuda
};
using var encoder = LibNvEnc.OpenEncoder(ref ps);
var cap = new NvEncCapsParam { CapsToQuery = NvEncCaps.SupportMultipleRefFrames };
var ret = 0;
encoder.GetEncodeCaps(NvEncCodecGuids.Hevc, ref cap, ref ret);

finally after running dotnet build I needed to link in the system libraries to match the names .net looked for:

ln -s /usr/lib/x86_64-linux-gnu/libnvidia-encode.so bin/Debug/net7.0/libnvEncodeAPI64.dll.so
ln -s /usr/lib/x86_64-linux-gnu/libcuda.so bin/Debug/net7.0/nvcuda.dll.so

I am not sure if there is value in switching NvEncoder to be a class without a parameterless constructor to make it a bit more obvious you can't just init it that way.

Hello,
I'm looking for a solution to extract all frames from a video (1080p 29.97fps h264) convert them as Bitmap and store them in a List.
I'm currently decoding videos using Emgu.CV, but it only decodes videos using CPU (i7-10700k) at 96 frames each second, I recently bought an RTX 4000 and I would like to decode videos using hardware acceleration but that's not possible with emgu.cv and OpenCV.
I tried your VideoDecode Sample converting the video to raw h264 but it only decodes at 70 fps. I can see on the task manager that it's running a video decode task on the GPU and I can't display the result on the window because I'm getting this excpetion: 'Unable to load DLL 'NvEncSharp.Cuda.Library.dll' so I commented _window.FrameArrived(frame); row.

Am I missing something?
Would I get a better result trying the plain c++ NVIDIA Video Codec SDK?

Any help is appreciated.

Thanks

Hello,

Maybe I did something wrong I don't know. I try something based on your exemple, but it's not possible to define a NvEncInitializeParams struct with NvEncConfig.

The one in the struct end with * like this NvEncConfig* because of this I can't compile the code. If I remove the * in your source code in Structs.cs it give me a Memory Violation Exception.

Here is my code:

        private NvEncoder CreateEncoder(Texture2D texture)
        {
            if (Initialized) return _encoder;

            _stream = File.OpenWrite(Settings.Output);

            //var desc = texture.Description;
            var encoder = OpenEncoderForDirectX(texture.Device.NativePointer);

            NvEncPresetConfig presetConfig = encoder.GetEncodePresetConfig(NvEncCodecGuids.H264, NvEncPresetGuids.LowLatencyHq);
            presetConfig.Version = NV_ENC_PRESET_CONFIG_VER;
            presetConfig.PresetCfg.RcParams.AverageBitRate = (uint)Settings.Bitrate;

            var initparams = new NvEncInitializeParams
            {
                Version = NV_ENC_INITIALIZE_PARAMS_VER,
                EncodeGuid = NvEncCodecGuids.H264,
                EncodeHeight = (uint)Settings.Height,
                EncodeWidth = (uint)Settings.Width,
                DarHeight = (uint)Settings.Height,
                DarWidth = (uint)Settings.Width,
                FrameRateNum = (uint)Settings.Framerate,
                FrameRateDen = 1,
                ReportSliceOffsets = false,
                EnableSubFrameWrite = false,
                PresetGuid = NvEncPresetGuids.LowLatencyHq,
                EncodeConfig = presetConfig.PresetCfg,
                EnableEncodeAsync = 0
            };

            encoder.InitializeEncoder(ref initparams);

            _bitstreamBuffer = encoder.CreateBitstreamBuffer();

            _encoder = encoder;
            Initialized = true;

            return encoder;
        }

Have you any idea how to achieve this ? Or maye a sample / demo that can help me ?

Thank's in advance