Reproducible builds

From ArchWiki

Arch Linux is currently working on making all packages reproducible. This lets users and researchers verify the distributed packages from Arch Linux. For the exact definition of reproducible builds and its benefits take a look at the project website.

Verification builds

Repository vs. rebuild

An experimental rebuilderd instance has been setup on our own infrastructure with a status page. Rebuilderd rebuilds our repository packages and checks if they are bit for bit identical. If they are not reproducible there is either a bug in the tooling, the package is not reproducible or the package has not been built cleanly.

A list of known issues is located on /Status.

Rebuild vs. another rebuild with variations

The Reproducible Builds project rebuilds Arch Linux packages and compares them with another rebuild in a different environment. The package status and environment variations are listed in the dedicated Arch Linux page.

Helping out

Tooling

Help out on fixing bugs and adding features for repro.

Running a Rebuilder instance

Setting up rebuilderd to build Arch Linux packages helps to independently verify repository packages.

Verifying packages with repro and finding issues

A great way to help out is to find an unreproducible package and figuring out how it can be made reproducible.

  • Download an Arch Linux package or get one from the Arch Linux Archive
  • Run repro on the downloaded package or a package from your pacman cache. Ideally with repro -d to get diffoscope output. For example, repro -d /var/cache/pacman/pkg/curl-7.73.0-1-x86_64.pkg.tar.zst
Note: https://reproducible.archlinux.org skips the check() step because running all tests for all packages is too flakey. This creates false unreproducible packages status if check() has side-effects, such as Python packages.

Work on issues in the tests.reproducible-builds.org infrastructure

Arch users can help contribute to Reproducible Build issues by looking at the continuous reproducing environment. There are various issues which can be sorted out:

  • FTBS (failed to build from source): reproduce the build failure locally and create a bug report if the package cannot be built from a clean chroot (extra-x86_64-build or multilib-build).
  • Failed to download sources, reproduce the issue (makepkg -o -d) and create a bug report on the Arch bugtracker.
  • Failed to reproduce. Locally you can reproduce packages using reprotest. Note that not all variations can be used. For simple time related testing: 
    $ reprotest --variations '+time' 'sudo extra-x86_64-build' '*.pkg.tar.zst'

There might be various reasons for a package to not be reproducible, but before digging in take a look at the upstream repository or the reproducible status in Debian

This article or section is out of date.

Reason: glibc 2.35-6 ships with C.UTF-8 included. (Discuss in Talk:Reproducible builds)
  • Failed to run tests, these failures are heavily on the testing environment. Most likely due to to LANG=C being set and Arch not supporting LANG=C.UTF-8.

If you are interested in the code which runs the continuous reproducing environment, the first build code starts here on salsa

Note: The continuous reproducing environment on tests.reproducible-builds.org does NOT reproduce official arch packages. Its purpose is only to fuzz the packages and attempt to trigger as many bugs as possible, so they can be reported upstream.

Known issues

GPG verification

There is a possible rebuild scenario where GPG keys will not verify as the packager was removed from the keyring or revoked as we use the latest keyring and a package in the archive which we need might need be signed by a revoked key we are unable to verify it and the build will fail.

Contact