How to get more then one layer onEncode? Is it possible? about openh264lib.net

[timikrutik]

@secile
Can you help me to solve this problem as soon as it possible. Your project is my last chance to do the things right.
I can get you some ☕☕☕ if we will be able to handle it. ☺

[secile]

Hello. I understand you are in very trouble. but I do not understand your question. Could you explain a little more clearly.
As I stated in README.md, this is only wrapper library, It may not be possible to solve it depending on the problem.

[timikrutik]

@secile, thank you for your response. I'll try to explain
A single SVC codec actually generates several bit streams, called layers. The lower or base layer base layers (level 0) is a stream that is decoded by a standard single - level decoder, such as an H. 264 decoder, and contains a video sequence with the lowest available quality (resolution) parameters. One or more of the more advanced layers, levels 1 and 2 for example example, are encoded as SVC streams. To get a better quality sequence, the SVC decoder decodes the base level and one or more improved levels.
And as I understand it, OpenH264 implements such a mechanism. This is also indicated by the following lines of code in the file Encoder.cpp in project OpenH264Lib.Net (especially this : info.iLayerNum):

void Encoder::OnEncode(const SFrameBSInfo% info)
	{
		for (int i = 0; i < info.iLayerNum; ++i) {
			const SLayerBSInfo& layerInfo = info.sLayerInfo[i];
			int layerSize = 0;
			for (int j = 0; j < layerInfo.iNalCount; ++j) {
				layerSize += layerInfo.pNalLengthInByte[j];
			}


			//bool keyFrame = (info.eFrameType == videoFrameTypeIDR) || (info.eFrameType == videoFrameTypeI);
			//OnEncodeFunc(layerInfo.pBsBuf, layerSize, keyFrame);

			array<Byte>^ data = gcnew array<Byte>(layerSize);
			//for(int j = 0; j < layerSize; j++) ary[j] = layerInfo.pBsBuf[j];
			/*pin_ptr<Byte> dataPtr = &data[0];
			memcpy(dataPtr, layerInfo.pBsBuf, layerSize);
			dataPtr = nullptr;*/
			System::Runtime::InteropServices::Marshal::Copy((IntPtr)layerInfo.pBsBuf, data, 0, layerSize);
			OnEncodeFunc(data, layerSize, (FrameType)info.eFrameType);
		}
	}

So that's the question. Can I get multiple layers?

[secile]

I tested my OpenH264Sample. I set brake point at OnEncode function and selected some jpeg images.
In case info.iLayerNum == 2, info.eFrameType is always FrameType.IDR.
In case info.iLayerNum == 1, info.eFrameType is always FrameType.P.

You said that if info.iLayerNum is greater than 1,
The lower layer contains lowest quality, and higher layer contains higher quality video data.

So, you want to use only high level layer data to encode?
So, what you want is following code?

void Encoder::OnEncode(const SFrameBSInfo% info)
{
	// only use high layer.
	int i = info.iLayerNum - 1;
	const SLayerBSInfo& layerInfo = info.sLayerInfo[i];
	int layerSize = 0;
	for (int j = 0; j < layerInfo.iNalCount; ++j) {
		layerSize += layerInfo.pNalLengthInByte[j];
	}

	array<Byte>^ data = gcnew array<Byte>(layerSize);

	System::Runtime::InteropServices::Marshal::Copy((IntPtr)layerInfo.pBsBuf, data, 0, layerSize);
	OnEncodeFunc(data, layerSize, (FrameType)info.eFrameType);
}

[timikrutik]

I use the Encode function very often - for a large number of images. And info.iLayerNum == 2 only when keyframes are generated (FrameType. IDR). This happens at intervals of, for example, 2 seconds (this depends on the float keyFrameInterval = 2 * 1.0f parameter that we specify in encoder. Setup(widthWebCam, heightWebCam, bpsVideo, 30, keyFrameInterval, onEncode)). And all the other frames (FrameType.P) between the keys (IDR) have info.iLayerNum == 1 (that is, within two seconds of all frames have info.iLayerNum == 1). I just thought it was possible to get multiple layers of each frame type (both IDR and P, not just in IDR, always).

I started thinking about it when I tried to deal with OpenH264 on my own. They (developers) often talk about it. These layers may not be two, but even more.

[secile]

You want to get multiple layers both IDR and P, not just in IDR! I understand! But...
It could be beyond what wrapper library can do.

[timikrutik]

Sorry of course, but thank you!