AOMedia AV1 Codec
|
The AV1 library source code is stored in the Alliance for Open Media Git repository:
CMake replaces the configure step typical of many projects. Running CMake will produce configuration and build files for the currently selected CMake generator. For most systems the default generator is Unix Makefiles. The basic form of a makefile build is the following:
The above will generate a makefile build that produces the AV1 library and applications for the current host system after the make step completes successfully. The compiler chosen varies by host platform, but a general rule applies: On systems where cc and c++ are present in $PATH at the time CMake is run the generated build will use cc and c++ by default.
The AV1 codec library has a great many configuration options. These come in two varieties:
ENABLE_FEATURE
.CONFIG_FEATURE
.Both types of options are set at the time CMake is run. The following example enables ccache and disables the AV1 encoder:
The available configuration options are too numerous to list here. Build system configuration options can be found at the top of the CMakeLists.txt file found in the root of the AV1 repository, and AV1 codec configuration options can currently be found in the file build/cmake/aom_config_defaults.cmake
.
A dylib (shared object) build of the AV1 codec library can be enabled via the CMake built in variable BUILD_SHARED_LIBS
:
This is currently only supported on non-Windows targets.
Depending on the generator used there are multiple ways of going about debugging AV1 components. For single configuration generators like the Unix Makefiles generator, setting CMAKE_BUILD_TYPE
to Debug is sufficient:
For Xcode, mainly because configuration controls for Xcode builds are buried two configuration windows deep and must be set for each subproject within the Xcode IDE individually, CMAKE_CONFIGURATION_TYPES
should be set to Debug:
For Visual Studio the in-IDE configuration controls should be used. Simply set the IDE project configuration to Debug to allow for stepping through the code.
In addition to the above it can sometimes be useful to debug only C and C++ code. To disable all assembly code and intrinsics set AOM_TARGET_CPU
to generic at generation time:
For the purposes of building the AV1 codec and applications and relative to the scope of this guide, all builds for architectures differing from the native host architecture will be considered cross compiles. The AV1 CMake build handles cross compiling via the use of toolchain files included in the AV1 repository. The toolchain files available at the time of this writing are:
The following example demonstrates use of the x86-macos.cmake toolchain file on a x86_64 MacOS host:
To build for an unlisted target creation of a new toolchain file is the best solution. The existing toolchain files can be used a starting point for a new toolchain file since each one exposes the basic requirements for toolchain files as used in the AV1 codec build.
As a temporary work around an unoptimized AV1 configuration that builds only C and C++ sources can be produced using the following commands:
In addition to the above it's important to note that the toolchain files suffixed with gcc behave differently than the others. These toolchain files attempt to obey the $CROSS environment variable.
Sanitizer integration is built-in to the CMake build system. To enable a sanitizer, add -DSANITIZE=<type>
to the CMake command line. For example, to enable address sanitizer:
Sanitizers available vary by platform, target, and compiler. Consult your compiler documentation to determine which, if any, are available.
Building the AV1 codec library in Microsoft Visual Studio is supported. Visual Studio 2019 (16.0) or later is required. The following example demonstrates generating projects and a solution for the Microsoft IDE:
NOTE: The build system targets Windows 7 or later by compiling files with -D_WIN32_WINNT=0x0601
.
Building the AV1 codec library in Xcode is supported. The following example demonstrates generating an Xcode project:
Building the AV1 codec library with Emscripten is supported. Typically this is used to hook into the AOMAnalyzer GUI application. These instructions focus on using the inspector with AOMAnalyzer, but all tools can be built with Emscripten.
It is assumed here that you have already downloaded and installed the EMSDK, installed and activated at least one toolchain, and setup your environment appropriately using the emsdk_env script.
Three variables allow for passing of additional flags to the build system.
The build system attempts to ensure the flags passed through the above variables are passed to tools last in order to allow for override of default behavior. These flags can be used, for example, to enable asserts in a release build:
After installing libvmaf.a, you can use it with the encoder:
Please note that the default VMAF model ("/usr/local/share/model/vmaf_v0.6.1.json") will be used unless you set the following flag when running the encoder:
There are several methods of testing the AV1 codec. All of these methods require the presence of the AV1 source code and a working build of the AV1 library and applications.
The unit tests can be run at build time:
The example tests require a bash shell and can be run in the following manner:
When making a change to the encoder run encoder tests to confirm that your change has a positive or negligible impact on encode quality. When running these tests the build configuration should be changed to enable internal encoder statistics:
The repository contains scripts intended to make running these tests as simple as possible. The following example demonstrates creating a set of baseline clips for comparison to results produced after making your change to libaom:
After making your change and creating the baseline clips, you'll need to run encodes that include your change(s) to confirm that things are working as intended:
After creating both data sets you can use test/visual_metrics.py
to generate a report that can be viewed in a web browser:
You can view the report by opening mytweak.html in a web browser.
By default the generated projects files created by CMake will not include the runtests and testdata rules when generating for IDEs like Microsoft Visual Studio and Xcode. This is done to avoid intolerably long build cycles in the IDEs– IDE behavior is to build all targets when selecting the build project options in MSVS and Xcode. To enable the test rules in IDEs the ENABLE_IDE_TEST_HOSTING
variable must be enabled at CMake generation time:
The fastest and easiest way to obtain the test data is to use CMake to generate a build using the Unix Makefiles generator, and then to build only the testdata rule. By default the test files will be downloaded to the current directory. The LIBAOM_TEST_DATA_PATH
environment variable can be used to set a custom one.
The above make command will only download and verify the test data.
First, add the new test data file to the aom-test-data
bucket of the aomedia-testing
project on Google Cloud Platform. You may need to ask someone with the necessary access permissions to do this for you.
NOTE: When a new test data file is added to the aom-test-data
bucket, its "Public access" is initially "Not public". We need to change its "Public access" to "Public" by using the following gsutil
command:
This command grants the AllUsers
group READ access to the file named "test-data-file-name" in the aom-test-data
bucket.
Once the new test data file has been added to aom-test-data
, create a CL to add the name of the new test data file to test/test_data_util.cmake
and add the SHA1 checksum of the new test data file to test/test-data.sha1
. (The SHA1 checksum of a file can be calculated by running the sha1sum
command on the file.)
The test data mentioned above is strictly intended for unit testing.
Additional input data for testing the encoder can be obtained from: https://media.xiph.org/video/derf/
The AV1 codec library unit tests are built upon gtest which supports sharding of test jobs. Sharded test runs can be achieved in a couple of ways.
To create a test shard for each CPU core available on the current system set GTEST_TOTAL_SHARDS
to the number of CPU cores on your system minus one.
The maximum number of test targets that can run concurrently is determined by the number of CPUs on the system where the build is configured as detected by CMake. A system with 24 cores can run 24 test shards using a value of 24 with the -j
parameter. When CMake is unable to detect the number of cores 10 shards is the default maximum value.
We are using the Google C Coding Style defined by the Google C++ Style Guide.
The coding style used by this project is enforced with clang-format using the configuration contained in the .clang-format file in the root of the repository.
You can download clang-format using your system's package manager, or directly from llvm.org. You can also view the documentation on llvm.org. Output from clang-format varies by clang-format version, for best results your version should match the one used on Jenkins. You can find the clang-format version by reading the comment in the .clang-format
file linked above.
Before pushing changes for review you can format your code with:
Check the .clang-format file for the version used to generate it if there is any difference between your local formatting and the review system.
Some Git installations have clang-format integration. Here are some examples:
Use the following comment block in new C/C++ source files, replacing "${year}" with the current year. The same comment should be added to other file types, adjusting the comment syntax as necessary.
We manage the submission of patches using the Gerrit code review tool. This tool implements a workflow on top of the Git version control system to ensure that all changes get peer reviewed and tested prior to their distribution.
Browse to AOMedia Git index and login with your account (Gmail credentials, for example). Next, follow the Generate Password
Password link at the top of the page. You’ll be given instructions for creating a cookie to use with our Git repos.
You must also have a Gerrit account associated with your Google account. To do this visit the Gerrit review server and click "Sign in" (top right).
You will be required to execute a contributor agreement to ensure that the AOMedia Project has the right to distribute your changes.
Note: If you are pushing changes on behalf of an Alliance for Open Media member organization this step is not necessary.
The testing basics are covered in the testing section above.
In addition to the local tests, many more (e.g. asan, tsan, valgrind) will run through Jenkins instances upon upload to gerrit.
Gerrit requires that each submission include a unique Change-Id. You can assign one manually using git commit –amend, but it’s easier to automate it with the commit-msg hook provided by Gerrit.
Copy commit-msg to the .git/hooks
directory of your local repo. Here's an example:
See the Gerrit documentation for more information.
The command line to upload your patch looks like this:
If you previously uploaded a change to Gerrit and the Approver has asked for changes, follow these steps:
In general, you should not rebase your changes when doing updates in response to review. Doing so can make it harder to follow the evolution of your change in the diff view.
Once your change has been Approved and Verified, you can “submit” it through the Gerrit UI. This will usually automatically rebase your change onto the branch specified.
Sometimes this can’t be done automatically. If you run into this problem, you must rebase your changes manually:
If there are any conflicts, resolve them as you normally would with Git. When you’re done, reupload your change.
To check the status of a change that you uploaded, open Gerrit, sign in, and click My > Changes.
This library is an open source project supported by its community. Please please email aomed.nosp@m.iaco.nosp@m.dec@j.nosp@m.oint.nosp@m.devel.nosp@m.opme.nosp@m.nt.ka.nosp@m.vi.c.nosp@m.om for help.
Bug reports can be filed in the Alliance for Open Media issue tracker.