Arch build system

From ArchWiki

Related articles

The Arch build system (ABS) is a system for building and packaging software from source code. While pacman is the specialized Arch tool for binary package management, the Arch build system is a collection of tools for compiling source into installable .pkg.tar.zst packages.

The Arch build system can be compared to ports for *BSD, which automates the process of building software from source code. The system uses a port to download, unpack, patch, compile, and install the given software. A port is merely a small directory on the user's computer, named after the corresponding software to be installed, that contains a few files with the instructions for building and installing the software from source. This makes installing software as simple as typing make or make install clean within the port's directory.

The Arch build system is based on a similar concept. It comprises a collection of git repositories for every package available in Arch Linux. Each repository contains a PKGBUILD file (and sometimes other files), and does not contain the software source nor binary. By running makepkg inside a directory, the software sources are downloaded, the software is compiled, and then packaged within the build directory. Then you can use pacman to install the package.

Toolchain

This article or section needs expansion.

Reason: Add pkgctl/devtools to the list. (Discuss in Talk:Arch build system)

The Arch build system includes and relies on several components and tools that are used in the process of building packages from source:

Repository
The directory structure containing files needed to build all official packages but not the packages themselves nor the source files of the software. It is available in the form of Git repositories hosted on gitlab.archlinux.org. See the section #Repository structure for more information.
PKGBUILD
A Bash script that contains the URL of the source code along with the compilation and packaging instructions.
makepkg
A shell command tool which reads the PKGBUILDs, automatically downloads and compiles the sources and creates a .pkg.tar* according to the PKGEXT array in makepkg.conf. You may also use makepkg to make your own custom packages from the AUR or third-party sources. See Creating packages for more information.
pacman
pacman is completely separate, but is necessarily invoked, either by makepkg or manually, to install and remove the built packages and for fetching dependencies.
AUR
The Arch User Repository is separate from the official repository of PKGBUILDs, but packages from the AUR can be built using the same tools. It contains many thousands of user-contributed PKGBUILDs for software which is unavailable as an official Arch package. If you need to build a package outside the official Arch tree, chances are it is in the AUR.
Warning: Official PKGBUILDs assume that packages are built in a clean chroot. Building software on a dirty build system may fail or cause unexpected behaviour at runtime, because if the build system detects dependencies dynamically, the result depends on what packages are available on the build system.

Repository structure

The core, extra, core-testing, and extra-testing official repositories are hosted on the Arch Linux GitLab instance.

Each package has its own repository in the archlinux/packaging/packages namespace. Each repository contains the PKGBUILD and files used in official builds. Also some files which are used by the developers for the build process can be found there.

For example, the tree for acl looks like this: 

acl
├── keys
│   └── pgp
│       ├── 259B3792B3D6D319212CC4DCD5BF9FEB0313653A.asc
│       ├── 600CD204FBCEA418BD2CA74F154343260542DF34.asc
│       └── B902B5271325F892AC251AD441633B9FE837F581.asc
└── PKGBUILD

The source code for the package is not present in the directory. Instead, the PKGBUILD contains a URL that will download the source code when the package is built.

Use cases

The Arch build system automates certain tasks related to compilation from source. Its use cases are:

This article or section needs expansion.

Reason: The main use case is obviously building official binary packages for Arch Linux by developers and packagers. User customizations and unofficial builds are secondary. (Discuss in Talk:Arch build system)
  • Any use case that requires you to compile or recompile a package.
  • Make and install new packages from source of software for which no packages are yet available (see Creating packages).
  • Customize existing packages to fit your needs (e.g. enabling or disabling options, patching).
  • Rebuild your entire system using your compiler flags, "à la FreeBSD".
  • Cleanly build and install your own custom kernel (see Kernel compilation).
  • Get kernel modules working with a custom kernel.
  • Easily compile and install a newer, older, beta, or development version of an Arch package by editing the version number in the PKGBUILD.

Usage

Retrieve PKGBUILD source

To retrieve the PKGBUILD file required to build a certain package from source, you can either use the pkgctl tool or directly use Git.

Using the pkgctl tool

As a precondition, install the devtools package. pkgctl is a tool to help work with building source files for Arch Linux packages using git.

To clone the git repository that contains the latest build files for the package pkgname using pkgctl, the following command is used: 

$ pkgctl repo clone pkgname
Tip: This clones over SSH by default so if you have not set an SSH key in your Arch GitLab account, you need to clone over HTTPS: pkgctl repo clone --protocol=https pkgname.

Note that here, build source files refers to PKGBUILD, possibly with some few other required files, such as keys. That is, the essential files that are required for Arch Linux build system. It does not refer to the source files of the package that were written by the team that authored the package, such as C or Python files.

This will give you not only the current source build files, but also their previous versions. Furthermore, you can use all other git commands to checkout an older version of the package or to track custom changes.

If you want to get a specific version of a package you can use something like the following: 

$ pkgctl repo clone --switch="2:1.19.5-1" go

Do read pkgctl-repo-clone(1) for more insight, and for the other commands available.

Using git directly

Use the following git command to clone a package: 

$ git clone https://gitlab.archlinux.org/archlinux/packaging/packages/pkgname.git

For example, to copy the Apache build files: 

$ git clone https://gitlab.archlinux.org/archlinux/packaging/packages/apache.git

Build package

Configure makepkg for building packages from the PKGBUILDs you have checked out, as explained in makepkg#Configuration.

Then, copy the directory containing the PKGBUILD you wish to modify to a new location. Make the desired modifications there and use makepkg there as described in makepkg#Usage to create and install the new package.

Tips and tricks

Preserve modified packages

Updating the system with pacman will replace a modified package with the package of the same name from the official repositories. See the following instructions for how to avoid this.

Insert a group array into the PKGBUILD, and add the package to a group called modified. 

PKGBUILD
groups=('modified')

Add this group to the section IgnoreGroup in /etc/pacman.conf. 

/etc/pacman.conf
IgnoreGroup = modified

If new versions are available in the official repositories during a system update, pacman prints a note that it is skipping this update because it is in the IgnoreGroup section. At this point, the modified package should be rebuilt to avoid partial upgrades.