aomediacodec / av1-hdr10plus

Official AOM repository for the development of the specification for the use of AV1 and HDR10+.

AV1 and HDR10+

This is the official AOM repository for the development of the specification for the use of AV1 and HDR10+.

Conformance

A set of files used to verify the conformance to the specification can be found in conformance directory. This directory includes both, valid and invalid files. The validation tool ComplianceWarden AOM AV1 HDR10+ can be used to validate conformance of your files to this specification.

Specification development

This specification is written using Bikeshed.

The main file is called index.bs and is used by the Bikeshed tool to generate index.html, which is then published via GitHub Pages.

If you have modifications to make (e.g. in pull requests), change the index.bs file. Then, run Bikeshed to produce the HTML file and verify your changes.

Bikeshed can be run locally on your machine after installation:

bikeshed spec

Or executed remotely e.g.:

curl https://api.csswg.org/bikeshed/ -F [email protected] -F force=1 > index.html

See the Bikeshed Documentation for more details.

av1-hdr10plus's Issues

Contribute the conformance file provided by Samsung

We currently have a file in our wiki which is not yet included into conformance suite.

@quantizationbit could you try to create a new branch either directly in the main repo, or in your fork and follow the guidelines from https://github.com/AOMediaCodec/av1-hdr10plus/tree/main/conformance and see if you can add that file to the conformance suite?

HDR10 Static Metadata definition placement

Currently HDR10 Static Metadata is defined in section 2.2.2 which specifies the placement of HDR10+ Metadata OBUs. This probably have to move to section 2.1

Link to the coverage page from the spec

Can we link the coverage page from the specification?

Reducing overhead in ISOBMFF

The current specification requires HDR10+ Metadata OBU to be present at every frame. Given the size of the HDR10+ data, this leads to an overhead in the order of 16kbps @ 30fps. This may be a problem when the video is a low bitrate.
A possible approach would be to alter the storage in MP4. An ISOBMFF packager would remove duplicate HDR10+ metadata when constant and add them back upon un-packaging when passing OBUs to the decoder. Obviously, this would require a new brand to signal it to players.

References terms and definitions

Some referenced terms seem not to be defined in the referenced specifications in the same way. For example, look at this sentence:

A TU contains a series of OBUs starting from a Temporal Delimiter, optional sequence headers, optional metadata OBUs, a sequence of one or more frame headers, each followed by zero or more tile group OBUs as well as optional padding OBUs.

• TU is not defined in the AV1 specification. Section 2 of AV1 specifications defines Temporal unit and I could not find the term TU in the spec.
• Temporal Delimiter should be Temporal delimiter OBU
• sequence headers -> sequence header OBUs
• frame headers -> frame header OBUs

which would lead to something like this:

A temporal unit contains a series of OBUs starting from a Temporal delimiter OBU, optional sequence header OBUs, optional metadata OBUs, a sequence of one or more frame header OBUs, each followed by zero or more tile group OBUs as well as optional padding OBUs.

Other parts of the specification should also use properly defined terms as defined in referenced specifications.

Clarify the color config needed in HDR10+ streams

AV1 video streams can have various color configuration parameters: bit depth, color primaries, transfer function, matrix coefficients, range, subsampling. The HDR10+ specification should clarify if there are any requirements or recommendations on those parameters.

Use of configOBUs

https://aomediacodec.github.io/av1-isobmff/#av1codecconfigurationbox-semantics indicates:

NOTE: The configOBUs field is expected to contain only one Sequence Header OBU and zero or more Metadata OBUs when applicable to all the associated samples.

A question is: should we allow HDR10+ metadata in the configOBUs array?
Given that HDR10+ metadata is supposed to change at every sample, I don't think it makes sense to have it in the configOBUs array.

So the question is rather: should we forbid the use of HDR+ metadata in the configOBUs array?
I would suggest:

HDR10+ Metadata OBUs should not be present in the configOBUs field of the AV1CodecConfigurationRecord.

Update section 2.2 to indicate that MDCV is mandatory

Need to update this sentance in section 2.2:

"HDR10 Static Metadata (defined as MDCV, MaxCLL and MaxFALL) may be present.”

While HDR10+ does not need these parameters in the outside standards: ATSC, SCTE, DVB they were written that MDCV is mandatory and MaxCLL/FALL are optional. We will sync this.

Multi-layer coding, Metadata OBUs and extension header

When an AV1 stream is encoded in multiple layers, metadata may apply to a specific layer in which case the OBU header for that Metadata OBU should use the extension header and have the same value as the OBU header of the frame with which it is associated.

Consistency: AV1 streams vs AV1 bitstreams vs AV1 coded bitstreams

The text seems not to be consistent

Reference CICP

color_primaries = 9 ([BT-2020])
transfer_characteristics = 16 ([SMPTE-ST-2084] / [BT-2100])
matrix_coefficients = 9 ([BT-2020])

The values 9, 16 etc. are defined in the CICP specification and not in BT-2020 etc. It would be good to be more specific here and make sure that even if value 9 represents BT-2020 the value itself is defined in CICP.

Further improvements

Better name for Section 2.1 would be "HDR10+ Metadata"

We should add text on how HDR10+ Metadata OBU is defined in section 2.1. Currently we have the definition in the Figure 1 but it would be better to have this written as text. Then we can embed the sentences in the assert statements for conformance purposes.

Figure 1 caption should be: "HDR10+ Metadata OBU"

Section 2.2 could be renamed to "HDR10+ bitstream constraints"

Figure 2 issues:

• TEMPORAL_DELIMITER_OBU is not correct and should be OBU_TEMPORAL_DELIMITER
• Show_existing_frame -> show_existing_frame = 1
• Legend as a table looks very strange and could be improved.

Both figures should use the same styling.

AV1 Film grain handling and HDR10+

One of the advanced features of the AV1 specification is film grain synthesis. Noise, such as film grain, could be removed at the source and prior to encoding, while information to resynthesize the noise at the decoder can be included in the AV1 bitstream. The AV1 specification requires that such noise is always added during reconstruction, but allows some flexibility on the specifics of the noise synthesis and the addition, to better cater for different architectures. For example, a decoder may decide to strictly follow the specification and in this case the output of the decoder will include the film grain, while another decoder may decide to only reconstruct the video signal up to and prior to the film grain synthesis. In this case, the film grain parameters can be passed to a secondary process, e.g. the display, where that process will determine in which domain/format the noise can be added. Different domains/formats include changes in chroma sampling, resolution, color representation (e.g. RGB), among others.

This raises the question of how HDR10+ should be handled when film grain is signalled in the AV1 bitstream. If the HDR10+ metadata are based on the original content (with the grain) then the process should account of where the grain is to be added in the decoding/display chain. Some clarification may be needed in this document of how this should be handled.

Create a section for Example Stream & Tools

Add a section 8, to a wiki page listing tools and link to example stream file.

Deploy pipeline fails

I noticed that the latest version was not deployed to GitHub pages. Seems that something is failing in the publishing pipeline.

File Storage for Example Files

TBD
Currently Wiki would have links to example files. Need to discuss the storage. @cyril will look at their storage.

How to get rid of file in repository and history: https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github

Contribute encrypted AV1 file

Figure 2 should be updated to include HDR10+ Metadata OBU

Figure 2 should be updated as it only documents the structure of a stream without HDR10+ metadata OBUs. Maybe a Figure 3 could be added to show where the HDR10+ metadata OBUs shall be inserted.

List conformance files in the coverage report

It would good to also have a table simply listing the conformance files, with their descriptions, maybe at the end of https://aomediacodec.github.io/av1-hdr10plus/conformance/

Bikeshed formatting issues

It seems that the Bikeshed tool has a few issues with rendering the Terms defined by reference section as shown below.

Problem 1: all references are transformed to lowercase e.g.: tu should be TU.

Problem 2: When using the dark Theme the text is rendered black and is not visible.

Automate Compliance Warden status checks

we can automate the process to show if CW already implements the latest rules or not. If we detect that CW does not implement the latest rule of the spec, report.

cdm4 brand should vs shall

Currently there is only a should for cdm4 brand in the spec.

And even the Sample AV1 Encoded HDR10+ File provided by Samsung has no cdm4 brand in it.

It looks like the presence of this brand is the only way to figure out if .mp4 file has HDR10+ in it (without parsing samples and T.35 metadata OBUs).

Why is there a should in av1-hdr10plus spec, when in the SCTE spec (referenced in CTA-5001, referenced in av1-hdr10plus) cdm4 shall be in the ftyp?

Conformance section reference to RFC 2119

Our bikeshed template currently adds the reference to the RFC2119 which points to https://datatracker.ietf.org/doc/html/rfc2119 should it be pointing to https://www.ietf.org/rfc/rfc2119.txt instead?

Also the name of the specification should be changed. Currently it is:

[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://datatracker.ietf.org/doc/html/rfc2119

But it should be:

[RFC2119]
IETF RFC 2119, Key words for use in RFCs to Indicate Requirement Levels, S. Bradner, March 1997. Best Current Practice. URL: https://www.ietf.org/rfc/rfc2119.txt

Restrict use of av1M in AV1 HDR10+ ISOBMFF files

Wrap normative statements into Bikeshed asserts

Fix ITU references

It would be good to change the ITU references from

BT.2020. Standard. URL: https://www.itu.int/rec/R-REC-BT.2020

to something like:

Recommendation ITU-R BT.2020-2 (2015), Parameter values for ultra-high definition television systems for production and international programme exchange.

Sentences about placement for show_frame = 0

The sentence:

HDR10+ Metadata OBUs are not provided when show_frame = 0.

is a consequence of the sentences in the previous paragraph, because:

• a show_frame = 0 cannot be on its own in a Temporal unit
• if the show_frame = 0 had a HDR10+ Metadata OBU, either the TU would have 2 HDR10+ Metadata OBU and that is invalid, or the show_frame = 1 in the same TU would not have an HDR10+ Metadata OBU, and this is also invalid.

We should:

• remove the assert so that it does not appear in the conformance report
• consider making the sentence a note to make it clear that it is not an additional statement

The second sentence:

For non-layered streams, there is only one HDR10+ Metadata OBU per temporal unit.

should also not be wrapped into an assert and maybe put into a NOTE. It's a rephrase/consequence of the previous paragraph.

Assert statement for encryption is incomplete

The assert in the sentence:

This specification requires that HDR10 Static Metadata and HDR10+ Metadata OBUs are unprotected.

should cover the whole sentence otherwise the report simply uses "are" instead of showing the word "requires"

Normative vs Informative references

When should we use normative and when informative references to the specs? Should we just merge everything to normative?

Analyze specification to determine how many conformance bitstreams are needed

Automate conformance report generation

This should be based on the asserts generated by Bikeshed #26 and based on contributed files #15.

Invalid files show up in Covered section

Currently the coverage page shows a table with successful checks and lists all the files which execute that particular rule successfully.

Do we want to keep it like that or should we only list files which execute the rule successfully AND do not have any errors in other rules? Do we still show the ones with warnings? @cconcolato any thoughts?