streamroot / fmse Goto Github PK
View Code? Open in Web Editor NEWMedia Source Extensions polyfill in Flash
Home Page: http://www.streamroot.io
Media Source Extensions polyfill in Flash
Home Page: http://www.streamroot.io
During the decoupling of this polyfill from our player, we removed a feedback loop that would get the key frame time from the JS dash player, and then:
A better way to work around these limitations is to give every frame located before the seek target the timestamp of the seek target itself.
This can be done in the transmuxer: this loop goes over every frame to write them in a ByteArray
I don't think we broke anything else during the decoupling phase, but a good way to start might be to seek programmatically exactly to the beginning of a DASH segment (expecting that it starts with a key frame), using a video only stream.
In appendBuffer call, as of now, we still need to pass the segment's start and end timestamp, and wether it's an initialization segment. See here
This is the main thing that still require some modifications in the JS player to work with this polyfill.
We need to parse these infos from the MP4 segments themselves in order to get rid of this issue
It's important to define from the start what our target browsers will be, to have a more precise idea of the scope of our polyfill and the workarounds we are going to implement.
If we want to support IE 9- for example, then we want to have a satisfying degraded version without web workers and that would solve #1 . If we don't care about Safari at all since MSE is in the latest versions, then we can close #1 already
In Safari, btoa
is not supported in web workers, but we make use of it here:
https://github.com/streamroot/fmse/blob/master/src/js/lib/B64Worker.js#L22
Dash.js, and probably others, test if the video is an instance of HTMLMediaElement : https://github.com/Dash-Industry-Forum/dash.js/blob/development/src/streaming/utils/Capabilities.js#L66
We used to inherit HTMLMediaElement prototype to work around this, but it broke when updating the version (Error: illegal invocation when setting video.src): https://github.com/Dash-Industry-Forum/dash.js/blob/ae5330eb3a6112fda9f4b831b2620f4a58990b44/src/streaming/models/VideoModel.js#L88
We don't have anything like the controls given by <video controls>...</video>
.
I don't think it is useful for anybody in production, but on the other hand it would be great to have a reusable standalone js tool that would add basic control to any sample page we would build for other MSE based player
in my player just if mediasource.sourceopen is called the player is visible.
but here if player is hidden sourceopen is not executed.
I m investigating if it simple to modify the code
dash.js scheduler sometimes stops downloading segments, sometimes resumes after a while.
This doesn't happen on regular MSE, so I suspect some part of the API (getter or event) is not implemented correctly.
Very easily reproductible on http://wowza.streamroot.io/vodorigin/tos.mp4/manifest.mpd .
If you reconfigure dash.js buffer to download 15s of content, it stops after downloading 3 segments:
player.setStableBufferTime(15);
player.setBufferTimeAtTopQuality(15);
player.setBufferTimeAtTopQualityLongForm(15);
I can t call video.volume
I think this is a great idea to provide a fallback for Media Source Extensions. Thanks for putting this concept out there!
I am interested in why we need the Service Worker dependency?
If you look at browser support for MSE and the browser support for Service Workers I'm not sure what browsers this is beneficial for. Are there any browsers that don't have MSE that support Service Workers?
Since the service worker requirement makes this apply to such a small subset of browsers, what is the goal of this library?
hi ,
your project it is very interesting i m working for integrating it in my player :)
I emproved a bit the init call:
function init(onReady, options) {
if (!options) options={}
var polyfillSwfUrl=options.polyfillSwfUrl || 'fMSE.swf';
var flashByDefault=options.flashByDefault || false;
var videoElement=options.videoElement||null;
var parent=options.videoContainer||(videoElement&&videoElement.parentElement)||document.body;
var isMSESupported = !!window.MediaSource;
if (isMSESupported && !flashByDefault) {
return onReady(videoElement);
}
window.fMSE.debug=options.debug||false;
window.MediaSource = MediaSourceFlash;
window.fMSE.callbacks = window.fMSE.callbacks || {};
window.fMSE.callbacks.onFlashReady = function () {
onReady(new VideoExtension(swfObj));
};
var readyFunctionString = "window.fMSE.callbacks.onFlashReady";
var height = options.height||(videoElement&&videoElement.height) || 150;
var width = options.width || (videoElement&&videoElement.width) || 300;
var oldId , oldIdClasses ;
if (videoElement){
oldId = videoElement.id;
oldIdClasses = videoElement.className;
}
var swfObj = document.createElement("object");
oldId='e'
if (oldId) {
swfObj.setAttribute('id',oldId);
swfObj.setAttribute('name',oldId)
}
swfObj.setAttribute('type','application/x-shockwave-flash')
swfObj.setAttribute('data',polyfillSwfUrl)
swfObj.setAttribute('width',width)
swfObj.setAttribute('height',height)
if (oldIdClasses)swfObj.setAttribute('class',oldIdClasses)
swfObj.setAttribute('style','display:'+(videoElement &&videoElement.style.display?videoElement.style.display:'block'))
var param = document.createElement("param");
param.setAttribute('name','movie')
param.setAttribute('value',polyfillSwfUrl)
swfObj.appendChild(param);
param = document.createElement("param");
param.setAttribute('name','flashvars')
param.setAttribute('value',"readyFunction="+readyFunctionString)
swfObj.appendChild(param);
param = document.createElement("param");
param.setAttribute('name','allowScriptAccess')
param.setAttribute('value','always')
swfObj.appendChild(param);
param = document.createElement("param");
param.setAttribute('name','wmode')
param.setAttribute('value','opaque')
swfObj.appendChild(param);
param = document.createElement("param");
param.setAttribute('name','allowNetworking')
param.setAttribute('value','all')
swfObj.appendChild(param);
param = document.createElement("param");
param.setAttribute('name','bgcolor')
param.setAttribute('value','#000000')
swfObj.appendChild(param);
parent.appendChild(swfObj);
}
Considering the browsers for the future wants support just mp4 and webm it would be usefull fmse support also trascoding for webm format to flash.
Seek adds a lot of complexity in the appending/transmuxing pipeline: NetStream has limitations and will crash quite spectacularly if you append segments out of order.
That's why we need to make sure every segment is properly flushed, at every step of the pipeline, and we added a lot of state variables along the way to handle situations like: "do I need to discard this segment when it comes out of the worker?"
It would be great to improve the SourceBuffer.as class to handle segments out of order, from disjoint time intervals.
We would need to account for #4 : if we don't be careful about this some segments could have their first frames squashed if they were appended during a seek
This is probably a wontfix
We're far from covering 100% of the specification of Video, MediaSource, and SourceBuffer APIs.
This is meant as a tracking bug to centralize this info although we could most probably find a way better tool to discuss, list and prioritize the API details we're missing.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.