Explainer for using MediaRecorder to mux WebCodecs generated content.
dalecurtis / mediarecorder-muxer Goto Github PK
View Code? Open in Web Editor NEWExplainer for using MediaRecorder to mux WebCodecs generated content.
Explainer for using MediaRecorder to mux WebCodecs generated content.
I was curious if this could be achieved by using a MediaStreamTrackGenerator
. I build a little demo for it.
https://stackblitz.com/edit/js-1jfqfv?file=index.js,index.html
It uses a canvas to render red and green rectangles which get then converted into VideoFrame
s and fed into the writabe
of a MediaStreamTrackGenerator
. It almost works.
The problem is that the MediaStreamTrackGenerator
doesn't seem to respect the duration
and the timestamp
of a VideoFrame
. I'm not sure if this is a bug or if that's intentional. If it would respect that one could generate and "record" a MediaStream
using the MediaRecorder
at least in real time.
I know that it's already possible to use captureStream()
on a canvas to achieve a similar thing. The problem with that solution however is that the timing is unreliable. Being able to craft VideoFrame
s with a precise duration
and timestamp
would solve this problem.
Maybe this idea could even be expanded to support faster than realtime rendering.
Instead of plugging the muxer API as an extension to MediaRecorder API, I think one option is to have a new standard API for W3C Muxers. I take as inspiration the WebM-Muxer API, as described here:
https://github.com/Vanilagy/webm-muxer
I think it would be a cleaner solution, which would support all the features for an audio / video / subtitles muxer, which is what we want when designing an audio or video editor, for example. I suggest starting with just webm as mandatory muxer, and than adding more as stack-holders demand it.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.