Define the interaction with Media Session about remote-playback

[avayvod]

@foolip @mounirlamouri do you think this is a good candidate for the F2F in May given Philip will be attending remotely in the best case and I doubt most group participants are closely familiar with Media Session API and user agent's default behaviors. Brainstorming use cases and requirements might still be useful though?

[avayvod]

Let's discuss the behavior w/r/t other tabs making sound before and after remote playback of the current tab and also how multiple remotely played media elements interact. Also reuse of metadata.
We can define the behavior that makes sense and then express it via MediaSession API later.

[tidoust]

Discussed at the F2F:
http://www.w3.org/2016/05/24-webscreens-minutes.html#item10

ACTION: Mounir to update issue with comments about remote and local don't fight over output, and suggestion that remote playback can access keys. No spec changes at this juncture.

[mounirlamouri]

François said it all, not much to add here. The conclusion from our discussion was that remote playback should not compete for audio focus with local playback. A remote playback should fight for audio focus on the device it is playing.

However, remote playback should compete for media keys on the local playback. That means that if there is a remote and local playback, one or the other might get media keys access. I don't think the Media Session spec is mature enough to allow us to hook into this so the conclusion was to do no mention of this in Remote Playback API at the moment. Later, we can either mention this in Media Session API or Remote Playback API.

[foolip]

This behavior makes sense, I agree. It will require changes to Media Session to decouple media key access from "active local session". If it's not a problem, I suggest leaving this and w3c/mediasession#123 until we figure that out, and implementation could inform the design.

[tidoust]

See related discussion at TPAC

The group will wait until the Media Session spec matures before it revisits this issue.

[avayvod]

Ok, revisiting the issue as both APIs are now shipping in Chrome.

I'd like the metadata at least to be applied to the remote playback if set on navigator.

The actions and event handlers are harder to define though - it doesn't seem like a good experience to pause both local and remote playback, for instance. It would be okay if only remote playback is happening (e.g. on iOS as I remember, remote and local playback are mutually exclusive).

Otherwise we could add a separate, per-element MediaSession object under remote that would override the one set on navigator.mediaSession if not null.

The corresponding issue on the MediaSession spec (closed as it was mostly about audio focus) is here.

[mounirlamouri]

One principle of the Media Session specification is that the instance on navigator.mediaSession will hook itself to the default audio focus used by the user agent. With remote playback, the user agent uses a different audio focus than the one on the device which means that handling remote playback in the default media session would break this principle.

Adding a MediaSession instance on the media.remote instance can help but it sounds slightly limited as other things could benefit from a "remote" media session. I don't have any good idea though. The only thing I can think of is to allow a remote audio focus (when the Audio Focus API happens) that could receive a media session. Though, I am not sure how useful the remote audio focus would be apart from linking to the media session.

[mfoltzgoogle]

I agree with keeping the global, user-agent media session distinct from a media session used to control remote playback.

One question is, do we expect at most one remote media session per user agent, or could there be multiple active sessions (one per remote device)?

[avayvod]

@mounirlamouri Do you agree we can say that if the only media playing is playing remotely, the UA is only involved in one audio focus - the remote one? Using navigator.mediaSession.metadata alone for remote playback sounds useful on its own to me agnostic to audio focus and controls.

it sounds slightly limited as other things could benefit from a "remote" media session.

What other things did you have in mind? Some things not attached to the specific media element?

[chrisn]

One question is, do we expect at most one remote media session per user agent, or could there be multiple active sessions (one per remote device)?

I'd prefer a design that allows multiple active sessions.

[avayvod]

@mfoltzgoogle In Chrome we only support one remote media session per system (limitation of the Cast SDK). However the Remote Playback API specification doesn't prevent remote playback of multiple elements on multiple remote playback devices in the same spirit as the Presentation API that IIRC allows multiple PresentationConnections to be established with different presentation screens.