bbc / videocontext Goto Github PK

When using our application on a non-localhost domain, we immediately see
No available video element in the cache, creating a new one. This may break mobile, make your initial cache larger.
in the logs, which is intermittently followed immediately by the video (canvas) playback stopping.

On localhost we never experience the message or the video playback problem. We're using hash-based routing which might be the culprit, but we're using that on localhost too.

There's a line of code that inspects the url of the window so I suspect that might be the problem ¯_(ツ)_/¯

if ((element.src === "" || element.src === undefined || element.src === stripHash(window.location)) && element.srcObject == null )return element;

The docs at http://bbc.github.io/VideoContext/doc/ are older than the JSDocs in the actual code.

Can these be updated?

Hi,

First this is a great project!

I've started integrating this with Clappr at https://github.com/tjenkinson/clappr-videocontext-playback

Demo: http://tjenkinson.me/clappr-videocontext-playback

At the moment to make it work I am having to access a few private variables so I have submitted a few PR's here. Let me know what you think.

ImageNodes fail to correctly render gifs. Only rendering the first frame. As discovered by @jontutcher .

Negative playbackRate should play the video in reverse mode or it should not be allowed to have negative values.

To change the src for a node, I used node.element.src = newSrc; which of course works, however doesn't align with the given ctx.video(src); API. In video() src can be different types, while node.element.src is explicitely a string with a url. Should be more dev friendly to offer:

class VideoNode {
   set src() {}
}

and then to process the input accordingly.

I give the library a try and it made all my code a lot easier (since I was working around the same issues you had (= no fixed currentTime, etc.)). I followed the basic example, create video context, connect a video node and play, nothing fancy.

Before that, I had a <div> container in which I placed many <video> elements, making one of them active (= hide the others). The <video> elements took up all the space from the surrounding <div>. Videos played on the space are automatically scaled to their own ratio (some had 16:9, others 4:3 but the <video> element was clever enough to arrange it quite fine. Now, instead of the video elements, I have my canvas, which also fits into the surrounding div, however, the videos take the full canvas and because of that become skewed. So I asked for the dimensions of the videos at first and set the width and height of the canvas to fit their ratio.

I feel like I'm doing it wrong as I was also possibly expecting to auto do that for me.

Second (which I thought is a reason of bad scaling) is the quality of the video is lost. I do have a 1080p video but when rendered on the canvas it looks horrible (see screenshot below).

How to get the original quality at auto scaling videos?

Thanks a lot.

hi,
I am checking your example, and it can't work in mobile chrome . Here is the error in console:

Uncaught TypeError: Array.from is not a function
85 videocontext.js:641 Uncaught TypeError: Array.from is not a function

Am I missing something ? Or it's no support in mobile yet?

thank you !

Hey friends, has anyone had luck moving VideoContext to a web worker? Would it be something worth while doing?

I made the switch from desktop to mobile development, using cordova.

Cordova and mobile platforms are kinda weird when it comes to path and URLs for files. Cordova mainly offers two urls: file:// and cdvfile:// to access videos on the mobile device. The apps itself are served via file://android_asset/www/index.html (as prod build) or via http:// (dev build). Also there are many restrictions that need to be adjusted to load locally available videos (CORS and mobile device access whitelisting). Using a standard <video> element with one of the urls above as element.src works quite nice. Setting the crossorigin attribute to anonymous can make the video unloadable (which is a must to display it onto canvas). I was able to load a video in a prod build (served as file://) from file:// with anonymous crossorigin. At least I could instruct the VideoContext to play, however the screen was black, yet the audio gives sound.

From MDN video element reference:

crossorigin This enumerated attribute indicates whether to use CORS to fetch the related image. CORS-enabled resources can be reused in the element without being tainted

Since I set CORS to whitelist my video origin, which finally made the VideoContext play but only sound, I'm wondering whether you experienced this field before and if you have a good advise to make this library usable on mobile devices (using cordova).

Thanks

Probably related to #14 and prevention of autoplay.

Safari 11, both desktop and mobile.

DOMError {name: "AbortError", message: "The operation was aborted."}

once VideoContext run .play() on a cached video element.

const canvas = document.getElementById('canvas');
this.ctx = new VideoContext(canvas);
const node = this.ctx.video('video.mp4');
node.startAt(0);
node.stopAt(15);
node.connect(this.ctx.destination);

function play () {
this.ctx.play();
}

function pause () {
this.ctx.pause();
}

sometimes first play(); the VideoContext is not change the frame but ctx.currentTime is adding; then after click pause to start, the frame is change but currentTime is not right.

1. click play(): ctx.currentTime is adding; frame not change
2. click pause(): ctx.currentTime is 10s,the ctx is pause
3. click play(): ctx.currentTime is adding from 10s, but video is play from 0s, after 5s the ctx is stop(because i set node.stopAt(15))

It just happens sometimes, not every time

const node = this.ctx.video('mp4');
node.startAt(0);
node.stopAt(5);

const node2 = this.ctx.video('mp4');
node2.startAt(2);
node2.stopAt(10);

const node3 = this.ctx.video('mp4');
node3.startAt(7);
node3.stopAt(15);

const node4 = this.ctx.video('mp4');
node4.startAt(15);
node4.stopAt(20);

const node5 = this.ctx.video('mp4');
node5.startAt(20);
node5.stopAt(27);

const crossFade = this.ctx.transition(VideoContext.DEFINITIONS.HORIZONTAL_WIPE);
const crossFade2 = this.ctx.transition(VideoContext.DEFINITIONS.CROSSFADE); // 渐显
const crossFade3 = this.ctx.transition(VideoContext.DEFINITIONS.CROSSFADE); // 渐显
const crossFade4 = this.ctx.transition(VideoContext.DEFINITIONS.CROSSFADE); // 渐显

// 1 2 平移
node.connect(crossFade);
node2.connect(crossFade);
crossFade.transition(2, 5, 0.0, 1.0);

// 2 3 渐显
crossFade.connect(crossFade2); // connect to input of crossfade2 rather than destination.
node3.connect(crossFade2);
crossFade2.connect(this.ctx.destination);
crossFade2.transition(7, 10, 0.0, 1.0);

// 4 无特效
node4.connect(this.ctx.destination);

// 5 开始
node5.connect(crossFade3);
crossFade3.connect(this.ctx.destination);
crossFade3.transition(20, 23, 1.0, 0.0);

The code is like this; first play is all right; but when i replay it after first end; some node show nocontant in this.ctx.registerCallback('nocontent', function () { console.log('nocontent'); }); has something wrong with it or did I make a mistake?

I have a videocontext graph that sometimes produces this error on loading

Uncaught TypeError: Cannot set property 'playbackRate' of undefined

This error occurs in videonode.js if, on loading, this._startTime === this._currentTime, This prevents this._load() being called before this._state is set to playing in sourcenode.js

this._load is only called if this._startTime - this._currentTime < this._preloadTime (https://github.com/bbc/VideoContext/blob/master/src/SourceNodes/videonode.js#L172), so this can be fixed by changing this condition to "less than or equal to" this._startTime - this._currentTime <= this._preloadTime

Or, if the bug is that playing is set prematurely, you could update the condition that sets the playing state https://github.com/bbc/VideoContext/blob/master/src/SourceNodes/sourcenode.js#L354 to be less than (but not equal to)

Hi,

We have been testing WebM videos with alpha in VideoContext, overlaying on top of another video. It works well except for when the video ends, the final frame is still visible on the canvas until VideoContext is restarted.

We have checked the source node with the WebM video and it definitely looks like it ends correctly and is unloaded by VideoContext.

Is there any other reason we would be seeing this behaviour?

Thanks,

John

When I use this to play video, it look like this:

When I go to https://www.shadertoy.com/view/Mdf3zr, play the same video(comment the 39 line of the glsl script), it look like this:

Can VideoContext do some anti-aliasing to let the video look more smoothly?

[shadertoy.com use piLibJs(http://www.iquilezles.org/code/piLibsJS/piLibsJS.htm) for rendering, but it's too complex for me...😭]

Is it possible to dynamically create the timeline and have the last video in the timeline loopable?

So I'm working on a live coding language to wrap around VideoContext; the language is going well but playback is off. Videos and images are extremely pixellated. It may be a problem with my implementation but Ian mentioned having the problem internally at the BBC before, any ideas? I've attached an example below.

I plan to use the library not only for desktop, but also for mobile phones, thus performance is important to me.

I've read that especially low performance devices would benefit from a "perserveDrawingBuffer": false situation, where the graphics driver can swap instead of copy the frame buffer. At the moment, the VideoContext constructor always uses "preserveDrawingBuffer: true.

I want to open this issue to make sure we can check (later on) whether this is possible for the implementation and how big the performance speedup is on low end devices.

Some links for reference:
http://stackoverflow.com/questions/27746091/preservedrawingbuffer-false-is-it-worth-the-effort
https://www.khronos.org/webgl/public-mailing-list/archives/1110/msg00003.php

Georg

var canvasElement = document.getElemenyById("canvas");
this.ctx = new VideoContext(canvasElement);
var imageNode = this.ctx.image("image.png");
this.ctx.startAt(0);
this.ctx.stopAt(15);
imageNode.connect(this.ctx.destination);

btn.click(function(){
ctx.currentTime = 10;
ctx.play();
})

After i click btn the ctx is not play; and i saw the ctx.currentTime is still 10s。But change image to video is play ok.

i'm not sure if it was a bug.
html5 video will change frame after currenttime is change;
but after i change currentTime the frame is change black.

the frame is not change when VideoContext is paused and set currentTime。。。
like this, the VideoContext is pause in 5s， the frame is stop in 5s，then change the currentTime to 10s but the frame is still in 5s,
Is that right? Or what do I need to do to change the picture?

const canvas = document.getElementById('canvas');
this.ctx = new VideoContext(canvas);
this.ctx.registerCallback("ended", function(){console.log("Playback ended");});

but when ctx.currentTime == ctx.duration the callback is not happen,there is no console

Any plans to add support for simple audio mixing/audio follow video or similar?

I do have a very strange situation. I developed a video player around VideoContext, which is great. It runs in electron, which sounds like a pretty stable context for running a webapp. However, video behaves different across systems. I have 4 buttons to change playback rate: 0.5x, 1x, 2x and 5x. It works well for me, yet for some of my colleagues, when they select 5x, the time updates accordingly, yet the frames are still being rendered at 1x speed. It's difficult to narrow this error down to what's responsible for that. What might be cause for VideoContext to still render the frames at 1x speed while actually it is 5x ?

Thanks a lot

Hi,

What is the best way to clean a video context timeline? I mean removing all the videos and effects (or destroy and create again)?

Thnks

I do wrote myself my own timeline, which has a cursor to move the playhead. When the cursor is changed, I call: ctx.currentTime = newTime. I have enabled the visualiseVideoContextTimeline. I can see the cursor in the visualization is the same as my own one, so the sync here is good. What lacks behind is the video. There is always some lag on the web, yet the video basically only changes, when I stop moving the cursor around or slow down my cursor movement drastically. I'm only on a local machine, so there is bandwith limit for me at the moment. I was wondering how speed up rendering the video in "time-critic" situations.

There was a talk at jsconf eu this year as well adressing this issue: https://youtu.be/NpKLt_YO3o8?list=PL37ZVnwpeshFmAPr65sU2O5WMs7_CGjs_&t=722 I think this is a solution to this. Although there is a drawback of using tons of RAM. Maybe as an opt-in option? Ref #58

When
ctx.playbackRate++;
The currentPosition is increased properly but the audio/video do not speed up.

Work around: The playbackRate only changes the VIDEO speed after
ctx.currentTime = ctx.currentTime;

Ps. Tried with the video Big_Buck_Bunny_4K.webm.160p.webm

Missing documentation for:

• VideoContext constructor
• transition return values

Hi,

I need to add 2 clips in a row on a timeline. But I don't know in advance the durations.

videoNode1 = videoCtx.video(url1);
videoNode1.startAt(0);
videoNode1.stopAt(???); // video1FullDuration
videoNode1.connect(videoCtx.destination);
videoNode2 = videoCtx.video(url2);
videoNode2.startAt(???); // video1FullDuration
videoNode1.stopAt(???); // video1FullDuration + video2FullDuration
videoNode2.connect(videoCtx.destination);

How to query for the VideoNode full duration (maybe even if it is not connected to any context)?

this.ctx.registerCallback('ended', this.hanldeEnded);
hanldeEnded = () => {
console.log('====>Playback ended');
this.ctx.pause();
this.ctx.currentTime = 0;
}

the console.log is show but this.ctx is still playing form 0s last for a few seconds, and frame is not show, i saw videoNode registerCallback is still loading

http://bbc.github.io/VideoContext/api/VideoContext.html
<VideoContext.STATE.PLAYING: all sources are active>; I think the ‘all sources’ maybe change to ‘current sources’ is more precise;
because i set a right url from 0s to 10s;and then set a error url from 15s to 20s; the VideoContext.state is STALLED after play 15s; so i think it is the current source state.
other i want to know is there some mark or status let me know from 10s to 15s VideoContext not has sources it‘s empty content

This is more of a question and less of an issue, but I'd been using the node deprecated BBC HTML5 Video Compositor, and I really liked the possibility of being able to pass in a playlist that consisted of an array of clips, as an EDL of some sort.

Is there a way to do the same in VideoContext?

The transition function is:
transition(startTime, endTime, currentValue, targetValue, propertyName)

currentValue and targetValue define opacity or transparency?

1.0 is 100% opaque or 100% transparent?

It is working like 100% transparency but for a CrossFade transition I think it should indicate the oposite.

Does this make sense to you?

I do get this warning fairly often, populating my console: WebGL: INVALID_VALUE: texImage2D: no video

I tracked it down to the function:

Background: I do have parts of my videocontext, which are black (nothing to show at that time). I also see this, when the video is loading.

this.ctx = new VideoContext();
var videoNode = this.ctx.createVideoSourceNode('errorvideo.mp4');
videoNode.start(0);
videoNode.stop(10);
var videoNode2 = ctx.createVideoSourceNode('errorvideo.mp4');
videoNode2.start(10);
videoNode2.stop(20);

the first videoNode will still loading, so how can i check the src is right; or can you give me a prompt when it is wrong like 'registerCallback('nocontent', ... )'

this.ctx = new VideoContext(canvas)
const node = this.ctx.video('mp4');
node.startAt(0);
node.stopAt(10);

const node2 = this.ctx.video('mp4');
node2.startAt(5);
node2.stopAt(20);

const node3 = this.ctx.video('mp4');
node3.startAt(15);
node3.stopAt(25);

const crossFade = this.ctx.transition(VideoContext.DEFINITIONS.HORIZONTAL_WIPE);
const crossFade2 = this.ctx.transition(VideoContext.DEFINITIONS.CROSSFADE);

node.connect(crossFade);
node2.connect(crossFade);
crossFade.connect(this.ctx.destination);
crossFade.transition(5, 10, 0.0, 1.0);

node3.connect(crossFade2);
crossFade2.connect(this.ctx.destination);
crossFade2.transition(15, 20, 0.0, 1.0);

The z-layers of my tracks change and I would like to be able to change them programmatically, yet haven't found a way.

As far as I can see, the z-layer is determined at the time a sourcenode is connected to the videocontext. Changing the z-layer later on is impossible? Do I have to kill all sourcenodes first and then add them in the new order again? What about a sourceNode.layer = value? (layer was just the best name I could come up with at the moment though I'm pretty sure, you already have some internal value here)

Hi,

i've noticed that the video transition example does not work on Chrome for Android. It blends the video as expected, but the charachter is not moving. It looks like two images - instead of videos - get transitioned.

I've used the Chrome Developer tools to inspect the error log and came to this message:
videocontext.js:1623 Uncaught (in promise) DOMException: play() can only be initiated by a user gesture

The problem is that the play() cal is not within the stack trace of the click() handler for the play button. Thus, Chrome does not allow it to play.

I've read some comments in the chrome issue tracker and there is a workaround that seems to work for tags. It may work also for video (haven't had time to test it yet): Create an empty

By the way: On Firefox for Android it works as expected. But i've read that mobile safari has the same restrictions as Chrome for Android. Is here somebody with an iPhone device to test this?

Georg

About 60% of the time when we instantiate multiple videocontext instances at the same time where some have multiple source nodes, we see the following error:

Uncaught TypeError: Cannot set property 'playbackRate' of undefined
    at VideoNode._update (videocontext.js:1561)
    at VideoContext._update (videocontext.js:945)
    at UpdateablesManager._update (videocontext.js:3074)
    at UpdateablesManager._updateRAFTime (videocontext.js:3066)

which is followed by the videocontext instances failing. Based on the traceback above, there seems to be some situation where a SourceNode can have no this._element but where the this._state is SOURCENODESTATE.playing.

We've not seen this issue occur when loading videocontext instances where each has just a single source node.

I've looked to see if I can find any glaring error conditions at our end and can't. We're not sending any options to the videocontext constructor, just the canvas, so it could perhaps have something to do with the videoelementcache.

this.ctx = new VideoContext();
var videoNode = this.ctx.createVideoSourceNode('video.mp4');
videoNode.start(0);
videoNode.stop(10);
var videoNode2 = ctx.createVideoSourceNode('');
videoNode2.start(10);
videoNode2.stop(20);

In this code the this.ctx.duration is 20s,
i want to show 20s video and the last 10s has no content(during 10s~20s i want listen in nocontent callback) but it looks like something wrong leads to videocontext is not continue after 10s.

createElementTexutre Should be changed to createElementTexture

Hey guys,

I am really impressed by your library and might consider using it. I was wondering, if - mostly technically spoken - the rendered video (as it is put on canvas) can be transmitted to chromecast?

Answer: Yes, it is possible. See comment below.

thanks

Sometimes when I switch away from and then come back to the tab with the VideoContext, the canvas goes black.

In the snippet below, the video is paused after 2 seconds by setting stretchPaused=true. Once it has paused, if you move to a different tab for more than 8 seconds (the video's stop time (10 seconds) minus the video's current time (2 seconds)) the canvas goes black when you return to the tab. The issue doesn't seem to be there if the stop time is not set.

var canvas = document.getElementById("canvas");

var videoCtx = new VideoContext(canvas);
var node = videoCtx.createVideoSourceNode("./video.mp4");

node.start(0);
node.stop(10);
node.connect(videoCtx.destination);

videoCtx.play();

setTimeout(function() {
    node.stretchPaused = true;
}, 2000);

I could not find the volume for the VideoContext. Instead, each source has its own volume. I - and I guess most others as well - treat the whole composition as one video with one volume control. However if I do have build an UI around video context, setting the volume, is cycle through all source nodes and set it on each. I was wondering, whether there is a default volume, that is set on all source nodes, when they are added and when set is applied onto all present source nodes as well.

In pseudo code:

VideoContext {
    get volume(); // returns the default volume
    set volume(value); // sets the default volume and applies it to all present source nodes
}

Would be wonderful to have that.