Playback hooks for audio syncing about asciinema-player

[ku1ik]

I implemented the following:

Properties:

• currentTime (writable)
• duration

Methods:

• play()
• pause()

Events:

• loadedmetadata
• loadeddata
• canplay
• canplaythrough
• play
• pause

It's in master now, and you can see the docs for these in the README: https://github.com/asciinema/asciinema-player#controlling-the-player-programmatically

Please take a look at this and let me know what you think. If it looks good I'll make a new release (2.4.0).

[ku1ik]

@remisharrock I'm actually working now on native audio support in the player. If it goes well it should land in next few weeks 😎

[remisharrock]

@sickill Hi Marcin, here is a small demo for you ;-)
Juste push the play button and put the sound up !
https://recorder.epixode.fr/player?id=1462872844655
And here is the open source tool to do this:
https://github.com/France-ioi/fioi-recorder

[remisharrock]

@sickill Hey Marcin, I will be in KRK starting the 15 of august, let's meet !
Also, where are you now with audio recording / playing ?
I wanted to record some Linux command line classes with audio for a massive open online course. Let me know if you think I will be able to do that this summer otherwise I have to do a screencast ... bad héhé :-)
Anyway, let me know if you are by KRK this summer so that we can meet !
See ya

[remisharrock]

@egrosclaude
I want to have audio synchronized with asciinema even if you click pause and play or even if you shuffle in time ; like a video actually. Or like our stuff we implemented here for example:
https://codecast.france-ioi.org/player?id=1467232086946

[ku1ik]

Hey @metasoarous, that would be awesome! The code doesn't include any docstrings at the moment so some parts may need some explanation from my side. I guess that's a good moment for me to document it.

[ku1ik]

I started adding proper JS interface here: https://github.com/asciinema/asciinema-player/blob/next/src/cljs/asciinema_player/core.cljs#L222-L226

I was reading through YouTube player's API doc and I think it's not a bad idea to be inspired by it.

The idea is to:

• accept events object in options object which would be used to register your event handlers
• return JS object from this function which will have some control methods like play(), pause() etc

I haven't started on these yet but they're the next things on my list.

[remisharrock]

@sickill Hey Marcin (Hope to see you soon in KRK by the way...)
I would love to have audio support for education purposes. Indeed, I'm an assistant professor at university and would love to do audio asciicasts !
We started an "audio code recorder / player" (here : https://github.com/France-ioi/fioi-editor2 ): based on the ACE editor (https://ace.c9.io/), it records/plays audio and keystrokes in the ACE editor for education purposes .
What's the status on audio support, particularly theses hooks ? How can we help ?

[ku1ik]

I started investigating the possibility of making the player a proper HTML5 element (<asciinema-player>), with API similar/compatible with <video> element (HTMLMediaElement). See asciinema/asciinema-server#18. I believe it would be great to have this as a main API. There would still be the current JS API under the hood though (if anyone would prefer that). This is related to hooks (they would need to be supported on <asciinema-player> element).

What would be the minimal set of hooks needed to integrate with a separate <audio> element?

• player starts -> trigger audio start
• player pauses -> trigger audio pause
• player resumes -> trigger audio resume
• player jumps to position -> trigger audio jump

Is there a need to differentiate between initial start and resuming from pause? Both could be seen as "play", right?

The player could support audio "natively" (eventually), but it would be nice to be able to integrate with external <audio> element manually via hooks (initially).

There is one thing with starting the playback initially that I'm not sure about: syncing the start of both terminal playback and audio playback. Both have a need to make a HTTP request to fetch data first and we can't assume they'll both start at the exact same time. This could be solved by pre-fetching both asciicast and audio, but this feature should work fine too without pre-fetching.

The solution could be to wait for both to be loaded and only then start the playback. Does <audio> tag trigger any event when it's actually ready play (you called .play() on it, it fetched enough data to be able to start playing immediately)?

[remisharrock]

@sickill Great idea !
What about using the hooks here: https://github.com/bnoguchi/hooks-js ?

Does audio tag trigger any event when it's actually ready play (you called .play() on it, it fetched enough data to be able to start playing immediately)?

yes:

"canplay" https://html.spec.whatwg.org/multipage/embedded-content.html#event-media-canplay

"canplaythrought" https://html.spec.whatwg.org/multipage/embedded-content.html#event-media-canplaythrough

"loadeddata" https://html.spec.whatwg.org/multipage/embedded-content.html#event-media-loadeddata

<audio id="audio" hidden loop>
<source src="music.mp3" type="audio/mpeg">
No support for HTML5 audio!
</audio>

<div id="playButton" onclick="play()">PLAY</div>
<script type="text/javascript">
    var audio = document.getElementById("audio");

    // Binding event  oncanplay for example
    audio.oncanplay = function() {
        alert("Can start playing video");
    };

    // Playing the audio
    function play() {
        audio.play();
    }
</script>

[remisharrock]

Hi all, any news on this ? I would really love sync with audio (or other things) some day !
Let me know how we can help !

[ku1ik]

@remisharrock ha! Thanks for the demo, your project looks great. Hopefully soon you'll be able to use asciinema with sound for teaching purposes. Let me know when you'll be in Krakow, we should definitely 🍻

[spork87]

Looks great under Chrome on Mac OS X, but the demo didn't launch with Safari.

[remisharrock]

@spork87 absolutely, it's only compatible with chrome firefox (maybe IE) for the moment ;-)

[egrosclaude]

@remisharrock May I suggest you take a look at reveal.js, you may insert audioclips into each slide and I have very easily integrated asciinema-player.

[Dowwie]

@sickill doesn't seem that the native Audio support feature was rolled out, or is that not the case?

[johngalt2600]

I was thinking about adding something similar to close caption but with audio. Multiple audioclips would eat up storage space. I was thinking about a single mp3 and close caption style start and stop points if you're doing clips. If you have sound throughout the video the playback times could just be synced. Good thing I checked to see if it was a planned feature so I didn't have to reinvent the wheel.
Would love to see this feature rolled out. Any updated ETA? Is there some alpha code we could play with in the mean time?

[remisharrock]

@johngalt2600 I don't know if this can help but we a demo, a writing code synced with audio and seekable, here https://codecast.france-ioi.org/player?id=1467232086946
the project is open source: https://github.com/France-ioi/fioi-recorder and https://github.com/France-ioi/fioi-editor2

[ku1ik]

Ok, <asciinema-player> HTML5 element just landed in v2.3.0 of the player 🎉. See the README for the details.

By having a simple way of embedding as a tag (same as <video>), we can now use Javascript's exisiting addEventListener for various hooks we need, without reinventing the wheel. Let's look at events triggered by <video> element, it would probably be good idea to use the same names for them.

[Nateowami]

I agree. The important ones would be play, pause, timeupdate and currentTime (writable property). MDN has a list of events, but I doubt most of them are necessary. "canplay" and "canplaythrough" (mentioned previously in this issue) would be useful for knowing when the ASCII cast is ready.

[remisharrock]

COOL ! let's try to build a demo for asciinema audio recorder and player !

[ku1ik]

I have just released v2.4.0 with the mentioned properties, methods and events.