Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cocallaghan/writing colr atom #9

Draft
wants to merge 15 commits into
base: cocallaghan/av1-codec-string
Choose a base branch
from

Conversation

CaitlinOCallaghan
Copy link
Owner

@CaitlinOCallaghan CaitlinOCallaghan commented Jul 24, 2023

This PR is an extension of the full AV1 codec string feature.

As per the AV1 spec, the codec string may contain optional color values. These color values are critical for detecting HDR video streams - see Issue 1007. Color information is extracted from the input mp4's colr atom and used to generate the full AV1 codec string. This PR preserves the color information by writing the colr atom to the muxed mp4.

References:

Testing

Manual: AV1 video WITH colr atom

Note: Presence of the colr atom in bbbhdr_av1.mp4 was verified with a hex editor.
image

  1. Generate Packager MPD with: ./out/Release/packager in=~/Downloads/bbbhdr_av1.mp4,stream=video,output=av1_with_colr.mp4 --mpd_output av1_with_colr.mpd
  2. Check manifest generated by Shaka Packager for Representation codecs string:
<?xml version="1.0" encoding="UTF-8"?>
<!--Generated with https://github.com/shaka-project/shaka-packager version 31129ee-release-->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 DASH-MPD.xsd" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011" minBufferTime="PT2S" type="static" mediaPresentationDuration="PT1.100000023841858S">
  <Period id="0">
    <AdaptationSet id="0" contentType="video" width="640" height="360" frameRate="15360/512" subsegmentAlignment="true" par="16:9">
      <Representation id="0" bandwidth="778146" codecs="av01.1.01M.10.0.000.09.16.09.0" mimeType="video/mp4" sar="1:1">
        <BaseURL>av1_with_colr.mp4</BaseURL>
        <SegmentBase indexRange="808-851" timescale="15360">
          <Initialization range="0-807"/>
        </SegmentBase>
      </Representation>
    </AdaptationSet>
  </Period>
</MPD>
  1. Check av1_with_colr.mp4 bitstream for the colr atom and cross reference the values of the atom's properties with the codec representation string found in the manifest:
    image

Format

git clang-format --style Chromium --binary /usr/bin/clang-format 80e024013df87a4bfeb265c8ea83cfa2a0c5db0f

unsigned int transfer_characteristics;
unsigned int matrix_coefficients;
unsigned int video_full_range_flag;
video.colr.color_parameter_type = FOURCC_nclx;
Copy link
Owner Author

@CaitlinOCallaghan CaitlinOCallaghan Jul 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cosmin, Is there an issue if we always write the colr atom as nclx? Even if the original atom was type nclc, we can make the output nclx by including the video_full_range_flag in the bitstream (it's value would be 0). Other than the inclusion of the video_full_range_flag data, is there any other differences between nclc and nclx?

Writing as nclx saves us from having to propagate the color parameter type through VideoStreamInfo.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I can tell for MP4 we always want to write nclx. The use of nclc should be just for reading from QuickTime MOV.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants