Infinite download loop when playing m3u8 about hls.js

[mangui]

@arikgilad , indeed the video loading is looping. I also got video decode error reported on the video element.
chrome://media-internals told me the following when appending the second fragment:
Media segment did not begin with key frame. Support for such segments will be available in a future version. Please see https://crbug.com/229412

I then checked with http://www.flashls.org/latest/examples/chromeless/?src=http%3A%2F%2Fmakostore-vh.akamaihd.net%2Fi%2FSHORT%2FCH22_NEWS%2F2015%2F09%2Fpress_vtr2_n20150903_v1%2Fpress_vtr2_n20150903_v1_%2C500%2C850%2C.mp4.csmil%2Fmaster.m3u8%3Fhdcore%3D3.0.3
it is just playing fine. I then turned on the debug/debug2 logs.
keyframes are detected properly at the beginning of the media segment.
while investigating further, it happens that some NDR NAL unit in your stream are in fact keyframes.
slice_type should be parsed to know whether a NAL is a keyframe or not.
the following logic needs to be imported in hls.js to be able to support your stream: https://github.com/mangui/flashls/blob/dev/src/org/mangui/hls/demux/TSDemuxer.as#L631-L649

we will check this soon.

[mangui]

@arikgilad attached commits will fix the infinite loop loading, and detect keyframes embedded in NDR NAL unit.
but still there is an issue with your stream: only the first fragment of each quality level contains SPS and PPS NAL unit.
Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) contain information that an h.264 decoder needs to decode the video data, for example the resolution and frame rate of the video.

this means that the only way to play your stream is to start from position 0 and stay on the same quality level.

[arikgilad]

Thanks for the quick response.
Flashls is playing video without an issue. Is the problem because of chrome MediaSource implementation? Is it a bug in chrome?

[mangui]

flashls is indeed playing it without any issue, although when switching between level 0 and 1, the video width and height are different...
As there is no SPS/PPS in the second fragment of level 1, there won't be any FLV AVC header pushed into NetStream. I think flash is just more tolerant and able to cope with it.

so, IMHO it is not a MediaSource bug : when changing video dimensions/attributes, a new initialization segment with video info (dimension, color space, ...) should be appended into the MediaSource sourcebuffer so that it could know how to decode the video.
same behaviour (freezed frame, but audio playing) could be observed with both Chrome and FF.

[matb33]

Another stream that exhibits looping: https://s3.amazonaws.com/tablo-theoplayer-testing/playlist.m3u8
When I start playing it from the beginning, it appears to grab segments in order properly, but if I seek ahead, then it downloads the appropriate segment, then the next, then back to the segment before, then the next, and so on. This occurs on all of our streams, so if you'd like a shorter one, let me know.

@mangui Let me know if you'd like the video in a form that you can download locally.

[matb33]

I have been doing some digging, and I'm not sure what to do with these findings. I'm consulting the video engineers on my end to see if they have any insights, but just in case this is useful:

For the example stream and any other stream generated by the product in question, the start position of each fragment (as inferred by the playlist via EXTINF) starts to drift away from the start position in the PTS data. This appears to get progressively worse over time.

For example:

Segment	EXTINF range	PTS range	Difference
#53	426 to 436	426.459377778 to 436.369277778	0.459
#368	3576 to 3586	3579.609366667 to 3589.519277778	3.609
#586	5756 to 5766	5761.789366667 to 5771.699266667	5.789

This gradual drift means that whenever one seeks, the player can't find an appropriate segment, as the data doesn't match up.

Perhaps the PTS data is diverging during the transmuxing process, or the original stream has bad PTS data to start with. That being said, we've been using flashls for a few years with no problems with these same streams.

[matb33]

The video engineers got back to me on this:

HLS has always had drift between actual PTS and HLS time tracking. The HLS time values were only meant as an estimation, in fact it wasn't until a later version of HLS even allowed floating point values in the EXTINF. (A few years ago, we couldn't even enable that because players didn't support it). No HLS players have had issue with this until these guys. I know for a fact that we will never get perfect timing values for the playlist. It really shouldn't matter. When seeking, it was just meant to put you in the ball park. It shouldn't cause the player to fail if there is skew. The authoritative time value in the stream is the PTS, not the playlist.

So that being said, it looks like the logic in BufferController (probably https://github.com/dailymotion/hls.js/blob/master/src/controller/buffer-controller.js#L230-L243) can't rely on frag.start completely, as those values are pulled from EXTINF, and won't necessarily match up with the PTS values.

Some form of tolerance needs to be added, perhaps seeking to the nearest buffered segment.

What I've done as a hack workaround in the meantime is I handled the FRAG_LOOP_LOADING_ERROR event, analyzed video.buffered for the nearest buffered section and seeked to that buffered start position. This works, but of course I have to wait for the loop to be detected before my logic can kick in.

@mangui any comments/thoughts on all this?

[matb33]

Any reason why this was disabled: https://github.com/dailymotion/hls.js/blob/master/src/controller/buffer-controller.js#L915

[mangui]

i started implementing it but it was introducing other issues with live playlist.
thus it is disabled for now
dailymotion@99a1f23

[mangui]

The drift things need to be completely rethought, as first implementation you might find around was a bit simplistic.
in flashls, fragment duration are recomputed after fragment parsing, and the whole level (fragment start offset and duration) is re-adjusted to take into account any potential drift: https://github.com/mangui/flashls/blob/dev/src/org/mangui/hls/model/Level.as#L274-L368

also, when switching to a level on which PTS is unknown, a PTS analysis is first performed to find the right fragment : https://github.com/mangui/flashls/blob/dev/src/org/mangui/hls/loader/FragmentLoader.as#L682-L775

similar logic should be imported in hls.js to cope with fragment drift.
this deserve a separate ticket.

[Zharv]

Another problem stream http://meta.vcdn.biz/3433feecbbd36498280ec51006375f82_megogo/vod/fs/o/84771/p/CUPERTINO/b/450_900_1350_1500_2000_5000/u_sid/500unh97pmftvgk9lbsrck0dl1/u_uid/22080/u_vod/4/u_device/flash/g/ua/a/0/type.amlst/playlist.m3u8
If the quality is above 480 and then everything is OK, but if you switch on the bottom are the problem.

switching to level 0
level-controller.js:115 (re)loading playlist for level 0
buffer-controller.js:607 immediateLevelSwitch
buffer-controller.js:569 flush audio [0,113.744266], of [0,113.744266], pos:12.321
buffer-controller.js:569 flush video [0,113.743599], of [0,113.743599], pos:12.321
buffer-controller.js:785 level 0 loaded [1,54],duration:600.4999999999999
buffer-controller.js:595 buffer flushed
buffer-controller.js:260 Loading 2 of [1 ,54],level 0, currentTime:12.3209,bufferEnd:12.321
buffer-controller.js:851 Demuxing 2 of [1 ,54],level 0
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:video/10.167/21.750/10.167/21.750/278
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:audio/10.170/21.757/10.170/21.757/499
buffer-controller.js:969 video buffered : [10.170311,21.743821]
buffer-controller.js:260 Loading 3 of [1 ,54],level 0, currentTime:12.3209,bufferEnd:22.044
buffer-controller.js:851 Demuxing 3 of [1 ,54],level 0
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:video/21.750/33.042/21.750/33.042/271
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:audio/21.757/33.065/21.757/33.065/487
buffer-controller.js:969 video buffered : [10.170311,21.743821]
buffer-controller.js:256 SN just loaded, load next one: 4
buffer-controller.js:260 Loading 4 of [1 ,54],level 0, currentTime:12.3209,bufferEnd:22.044
buffer-controller.js:851 Demuxing 4 of [1 ,54],level 0
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:video/33.042/45.042/33.042/45.042/288
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:audio/33.065/45.047/33.065/45.047/516
buffer-controller.js:969 video buffered : [10.170311,21.743821]
buffer-controller.js:260 Loading 3 of [1 ,54],level 0, currentTime:12.3209,bufferEnd:22.044
buffer-controller.js:851 Demuxing 3 of [1 ,54],level 0
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:video/21.750/33.042/21.750/33.042/271
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:audio/21.757/33.065/21.757/33.065/487
buffer-controller.js:969 video buffered : [10.170311,21.743821]
buffer-controller.js:256 SN just loaded, load next one: 4
buffer-controller.js:260 Loading 4 of [1 ,54],level 0, currentTime:12.3209,bufferEnd:22.044
buffer-controller.js:851 Demuxing 4 of [1 ,54],level 0
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:video/33.042/45.042/33.042/45.042/288
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:audio/33.065/45.047/33.065/45.047/516
buffer-controller.js:969 video buffered : [10.170311,21.743821]
buffer-controller.js:260 Loading 3 of [1 ,54],level 0, currentTime:12.3209,bufferEnd:22.044
buffer-controller.js:851 Demuxing 3 of [1 ,54],level 0
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:video/21.750/33.042/21.750/33.042/271
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:audio/21.757/33.065/21.757/33.065/487
buffer-controller.js:969 video buffered : [10.170311,21.743821]
buffer-controller.js:256 SN just loaded, load next one: 4
buffer-controller.js:260 Loading 4 of [1 ,54],level 0, currentTime:12.3209,bufferEnd:22.044
buffer-controller.js:851 Demuxing 4 of [1 ,54],level 0
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:video/33.042/45.042/33.042/45.042/288
buffer-controller.js:915 parsed data, type/startPTS/endPTS/startDTS/endDTS/nb:audio/33.065/45.047/33.065/45.047/516
buffer-controller.js:969 video buffered : [10.170311,21.743821]
buffer-controller.js:260 Loading 3 of [1 ,54],level 0, currentTime:12.3209,bufferEnd:22.044
level-controller.js:189 cannot recover fragLoopLoadingError error

[mangui]

@Zharv regarding http://meta.vcdn.biz/3433feecbbd36498280ec51006375f82_megogo/vod/fs/o/84771/p/CUPERTINO/b/450_900_1350_1500_2000_5000/u_sid/500unh97pmftvgk9lbsrck0dl1/u_uid/22080/u_vod/4/u_device/flash/g/ua/a/0/type.amlst/playlist.m3u8

there is an audio decode error raised by Chrome on lowest bitrate.
However it works fine on FF, check here:
http://dailymotion.github.io/hls.js/demo/?src=http%3A%2F%2Fmeta.vcdn.biz%2F3433feecbbd36498280ec51006375f82_megogo%2Fvod%2Ffs%2Fo%2F84771%2Fp%2FCUPERTINO%2Fb%2F450_900_1350_1500_2000_5000%2Fu_sid%2F500unh97pmftvgk9lbsrck0dl1%2Fu_uid%2F22080%2Fu_vod%2F4%2Fu_device%2Fflash%2Fg%2Fua%2Fa%2F0%2Ftype.amlst%2Fchunklist-sid562976-b450000.m3u8

not clear why ...
it seems to be a AAC mono channel, 44.1 kHz right

however adaptive switching is also broken on FF...
this deserves a separate ticket

[pavelbinar]

I am experiencing the fragLoopLoadingError on Samsung Tizen TV (J6400).
What can I provide for debugging?

[pavelbinar]

@matb33 What was the hack you did to bypass that error - just to be able to run the player?

[matb33]

@pavelbinar I detail the hack here: https://github.com/dailymotion/hls.js/issues/24#issuecomment-147397520

I'm also working on porting to the new Samsung Tizen TVs -- I implemented HLS support via their AVPlay library (as videojs tech). So far it works fine... Any reason you're using hls.js in that case?

[pavelbinar]

@matb33 Thanks for super fast reply! Check your email, but in general:
I want be able to run purely HTML5 + JS HLS video player on any TV platform.

[mangui]

outdated, closing