aomediacodec / id3-emsg Goto Github PK

The specification may want to give some recommendations with regard to using the id field in the emsg.

The specification uses 'emsg' to carry id3 but not all fields are defined, especially those related to timing. I outline my questions related to the timing with suggested answers.

what should the duration of the emsg version 1 be ?
=> if emsg is send with each fragment it may be the fragment duration in each case
=> in case the duration is just the time for which the metadata applies this should be mentioned
=> if the duration is indefenete/unknown this should be mentioned and het duration must be set to 0xFFFFFFFF

The specification contains (for now) one (duplicate) 'must' statement, and maybe more if #17 is approved. In order to validate these statements in a validation tool, the tool would need to know either from the bitstream or from the manifest that the bitstream claims to be compliant. This could also be useful to a client to know before downloading the stream that it contains an ID3 stream. This is typically done via a brand, e.g. aid3 (meaning AOM id3), or via a urn (as used in CMAF to signal compliance to 'single initialization constraints' using urn:mpeg:cmaf:siss) which could be urn:aomedia:cmaf:id3.

The current title of the spec is:

Timed Metadata in the Common Media Application Format (CMAF)

This is too broad. AOM cannot claim to define all timed metadata in CMAF. This is only applicable to ID3. I suggest renaming it to:

Carriage of ID3 Timed Metadata in the Common Media Application Format (CMAF)

Here:
https://aomediacodec.github.io/id3-emsg/
The link to ID3 is not working:
[ID3]
ID3.org, “The ID3 audio file data tagging format”, http://www.id3.org/Developer_Information.

It says: "No wiki configuration matching the URL found!"

The introduction cites company names. This is not really appropriate in a specification. I suggest removing the sentence:

Companies in this ecosystem include Disney, Sony, and Nielsen.

AV1 should not be in the name.

what should the constraints on presentation time of the emsg version 1 be ?

=> is it allowed to send metadata that applies to a later or earlier fragment ?

=> I would recommend the presentation time to be the earliest presentation time of the fragment, or at least to overlap with the fragment it is send with.
=> if this is not the case it should be explicitly mentioned what the presentation time could be in the specification

I hope this is the right forum for this conversation.

There is a section in the specification describing the carriage of ID3 tags in DASHEventMessageBoxes. I want to suggest adding a section to the specification describing the carriage of ID3 tags in EventMessageInstanceBoxes. This would standardize the carriage of ID3 tags in ISO/IEC 23001-18 event message tracks.

This new section (proposed naming: “ID3 Metadata in an Event Message Instance Box”) would be very similar to the existing “ID3 Metadata in an Event Message Box” section. The one difference I want to propose is that the event_duration for a durationless ID3 tag should be 0 in an event message instance box (‘emib’). The 0xFFFFFFFF value works well in ‘emsg’ boxes, but doesn’t generalize well to ISO/IEC 23001-18 event message tracks.

Specifically, the ISO/IEC 23001-18 specification introduces the notion of an active interval for each event. Section 7.4 of the specification states:

Each EventMessageInstanceBox documents an event message which has or will have an active interval. If the media presentation time of the containing sample is T, the active interval is defined to run: from (T + presentation_time_delta) to, but not including (T + presentation_time_delta + event_duration).

Each event has an event duration, represented by the event_duration field. Each sample in the track also has a duration D.

Each sample shall contain all events that have an active interval that overlaps the sample's time interval [T, T+D).

The problem is that an event_duration of 0xFFFFFFFF will result in the active interval for the event being 0xFFFFFFFF ticks long, which is not what we want for a durationless ID3 tag.

If there is support for this proposal, I’m happy to put up a PR. I’m also interested in alternative suggestions. I’m also interested in any concerns about client-side handling of a 0 tick event.

The emsg box is normatively defined in section 5.10.3.3 of ISO/IEC 23009-1:2017 (DASH), but section 7.4.5 of ISO/IEC 23000-19:2018 (CMAF) places additional constraints on the use of emsg boxes with CMAF. So both CMAF and DASH should be normative references in this metadata specification.

A document referred to does not exist
ID3.org, “The ID3 audio file data tagging format”, http://www.id3.org/Developer_Information.

Signaling added in #26 could be confusing. DASH §5.10.3.3.1 mentions that "Event message boxes with scheme identifier and value pairs that are not defined in the MPD should not be present. If a DASH Client detects an event message box with a scheme that is not defined in MPD, the client is expected to ignore it." Some implementation might therefore prefer using "https://aomedia.org/emsg/ID3" rather than "urn:aomedia:cmaf:id3" for schemeIdUri to signal ID3 in a DASH manifest.

If the spec gets updated, the URL of the DASH spec could also be updated to https://www.iso.org/standard/83314.html.

I would also remove the mention of the section 5.10.3.3.3. The copy of the definition of the DASHEventMessageBox is enough.

The abstract should be reworded to be a sentence with a verb, e.g. "This specification defines how ..."

An update is needed to the "Event Message Box" section, to reflect an aomedia.org namespace URL instead of developer.apple.com

The spec only contains one (duplicate) 'must' statement, about the use of v1 emsg boxes. In order to make testing clearer, 'must' statements should be used to mandate the use of the defined scheme id uri and the use of the ID3 v2.4 in the payload.

A question from our team, why is only version 1 supported, some devices may only support version 0 as this was introduced earlier.

Is there any chance of also supporting version 0 for broader compatibility ?

The first sentence of the overview is misleading. Using event message boxes is not the only way to have timed metadata in CMAF. I suggest deleting the sentence.

The second sentence is a duplicate of the sentence in the next section. I suggest deleting it too.

The current specification, although it cites the DASH specification normatively, does not indicate that it is intended (I believe) to be applicable to DASH streaming as well. I would suggest adding the following sentence (or similar) at the end of the introduction:

CMAF-compatible fragmented MP4 can also be used in DASH. The elements defined in this specification may also be used with DASH.

https://www.id3.org/Developer_Information returns Internal Server Error