Start and end trimming specification about iamf

[shawnisingh]

Here are the three options that were discussed, with some additional details that I could think of:

(1) Use media_time in ISOBMFF edit list, plus all the subsequent timestamp and duration changes scattered across the mp4 boxes. In particular, duration of last packet in TRUN and corresponding SubSegmentDuration would need to be shorter as well, so that playback could seek sample-accurately and know how to trim off the end as well.

(2) Start and end trimming described within ia_static_meta(), colocated with the description of scalable multi-channel and ambisonic. In my proposal that would be within AudioElement.

(3) for Opus only, there is preSkip in the typical opus config structure - I assume preSkip is the number of audio samples to be trimmed from the first audio frame? If I understand correctly, Opus audio frames are capable of expressing their shorter duration, so then end trimming does not need to be explicit.

From these three options, perhaps we should use all three, except that we don't keep preSkip?
The reason I think all three are meaningful: (1) ISOBMFF approach allows sample-accurate seeking, and helps to represent sample-accurate presentation durations that may be useful for some users. (2) will allow us to express correct trimming if any parser reads OBU syntax without the MP4 container. It seems OK that the redundancy exists when the OBU is contained in MP4. (3) -- preSkip would be redundant and unnecessary, but for shortened duration of last audio frame, I would expect it will be easier to keep the semantics of the Opus encoder and not force the frame to be full size unnecessarily.

Would be interested to hear what others think, thanks for considering!

[tdaede]

For (3), Opus audio packets themselves cannot express their shorter duration, however as a whole the stream can because of the inclusion of preSkip. This is technically redundant with the edit list in normal Opus-in-ISOBMFF packaging, and is mainly there to keep the same config box format when Opus is put into other containers without an equivalent feature to edit list start trimming.

[cconcolato]

In my view, it should be possible to store the information in OBU streams, when not stored into MP4. We should not reproduce the errors of AAC where the round tripping from MP4 to AAC looses the information.

Also, ideally, when storing in MP4, it should be possible to remove the information from the bitstream, to avoid redundancy and possibility of divergence. If that is not possible, then the spec should be clear that the 2 places (bitstream and container) shall match.

[sunghee-hwang]

In my view, it should be possible to store the information in OBU streams, when not stored into MP4. We should not reproduce the errors of AAC where the round tripping from MP4 to AAC looses the information.

Also, ideally, when storing in MP4, it should be possible to remove the information from the bitstream, to avoid redundancy and possibility of divergence. If that is not possible, then the spec should be clear that the 2 places (bitstream and container) shall match.

It has been resolved as implementing obu_trimming_status information in Audio Frame OBU.

[tdaede]

OK. One thing that seems to be still missing is how this interacts with edit lists.