Shaka-packager's subtitle output and timing is broken for real-time flows when the input has "gaps/periods" without subtitle streams. about shaka-packager

[tobbee]

@cosmin Thanks for your quick response. I just checked a live TS stream that @Brainiarc7's team suggested as test source and it has PES packets with PTS timestamp every 40ms on the teletext PID.

With those triggers I'll try to enhance the teletext parser to generate a zero-duration TextSample every 500ms starting when a TextSamples has ended, but no new one has started. By setting the duration to zero, I think it should be fairly easy to update the TextChunker::OnTextSample method to only write chunks whose end time has passed but NOT add the TextSample itself as a non-rendered sample.

One could implement alternative 2 as a fallback, but I leave that for later.

[tobbee]

@cosmin I'm trying to look solve this problem of text segment generation when there is no output. It seems that you are working quite a lot on improving Shaka-packager so maybe you have some thoughts about the best approach?

I think that the core problem is that all text segment generation is triggered by cue or text samples.

For example, looking at the OutputsEmptySegments unit test there should be three output segments:

• segment 0, 0-100ms with text 0-50ms
• segment 1, 100-200ms with no text
• segment 2, 200-300ms with text 200-250ms

There are then events simulating text samples being read.
The first one happens at 50ms and stores a text sample.
The second one happens at 250ms, and it is only at this time that segment 0 and the empty segment 1 is generated. The segments are generated by a loop in TextChunker::OnTextSample that loops over all old segments
and generate them with appropriate segment timing.

If text sample number 2 would have been even later, segment 0 and all intermediate empty segment would be generated even later.

Please correct me if I have misunderstood something.

I see three type of possible remedies (for the case of an incoming MPEG TS stream with teletext or DVB subtitles)

1. Rely on some incoming data on the text PID with PTS timestamps, so that it is possible to detect time progress in the teletext parser and generate empty cues/samples
2. Let the video track trigger text segment generation if there is no data (possibly with some delay)
3. Introduce a timer in the text_chunker that triggers empty chunks if there are no data for some finite time like 500ms

Do you have a take on the preferred method, or maybe some other way of tackling this problem?

[cosmin]

I think some heartbeat approach to generate and pass through empty cues from the input to the output to indicate absence of text seems right.

As to how generate these heartbeat cues, to the extent it's possible when parsing the input to know that there is no text data then that would be ideal. When text and audio/video would arrive separately then having some configurable buffering delay for how long to wait for text after video or audio is received would be good.

So at a high level #1 where possible otherwise #2, but probably not #3.