Preboot Execution Environment
From Wikipedia:Preboot Execution Environment:
- The Preboot eXecution Environment (PXE, most often pronounced as pixie) specification describes a standardized client-server environment that boots a software assembly, retrieved from a network, on PXE-enabled clients. On the client side it requires only a PXE-capable network interface controller (NIC), and uses a small set of industry-standard network protocols such as DHCP and TFTP.
In this guide, PXE is used to boot the installation medium with an appropriate option ROM that supports PXE on the target. This works well when you already have a server set up.
Preparation
Overview
It is useful to give an overview of the PXE boot process in order to understand the #Server setup, the #Installation on the client side and the Arch Linux files needed.
The client starts by broadcasting packets asking for a DHCP server and containing specific PXE options. The DHCP server responds with networking information (the IP address assigned to the client) and also provides, by using specific bootstrap protocol (BOOTP) parameters of the DHCP, additional information like the TFTP server address, the path of the initial network bootstrap program (NBP) to download or the boot configuration file name.
The NBP is transferred from the PXE server to the client using TFTP, to be loaded into memory and executed. The kernel and the initramfs are also transferred this way.
Then the root file system is transferred using one of the following protocols: HTTP, NFS or NBD.
Boot from install medium
In order to gather the files that will be transferred from the server to the client for booting, get the latest official install image from the download page.
Next mount the image:
# mount --mkdir -o loop,ro archlinux-release_date-x86_64.iso /mnt/archiso
where release_date is the release date in the ISO filename like, e.g., 2022.10.01.
Boot from netboot
Arch Linux netboot images can be used to download the latest Arch Linux release on the fly upon system boot. Download the image compatible with your configuration.
Pixiecore
An all-in-one solution is provided by pixiecore
- Install pixiecore-gitAUR
- Run
pixiecore quick arch --dhcp-no-bindas root - Boot via PXE
Server setup
You will need to setup a DHCP server, a TFTP server for transferring the NBP and one of the following services for transferring the root file system: HTTP server, NFS or NBD.
Network
Bring up your wired network adapter, and assign it an address appropriately.
# ip link set eth0 up # ip addr add 192.168.0.1/24 dev eth0
You can also use dhcpcd to configure the static IP.
DHCP + TFTP
You will need both a DHCP server and a TFTP server to configure networking on the install target and to facilitate the transfer of files between the server and the client. dnsmasq does both, and is extremely easy to set up.
dnsmasq needs to be configured. See instructions on how to setup a dnsmasq#TFTP server and a dnsmasq#PXE server.
Are provided below some common configuration instructions. tftp_root is the directory where the Arch ISO is mounted (e.g. /mnt/archiso) or where the network boot program is located.
# /etc/dnsmasq.conf
# Listen only to the specified interface interface=eth0 # Do not function as DNS server port=0 # TFTP server setup enable-tftp tftp-root=tftp_root # Log extra information about DHCP transactions (for debug purposes) log-dhcp
To enable the DHCP server and give IPv4 addresses within a range, add to the configuration file a line similar to:
dhcp-range=192.168.0.50,192.168.0.150
Or in case there is already a DHCP-server running on the network and you want to interoperate with it, see dnsmasq#Proxy DHCP.
Two examples covering different boot style and installation media are provided below.
Once configured according to your needs, start dnsmasq.service.
BIOS boot from install medium
The path of the initial bootstrap program to be transferred is defined with the dhcp-boot option in the configuration file.
dhcp-boot=/boot/syslinux/lpxelinux.0
In order to send specific bootstrap protocol (BOOTP) parameters, like the configuration file path, the dhcp-option-force=flag,value line is used.
dhcp-option-force=209,archiso_pxe.cfg # this file might be under /mnt/archiso/boot/syslinux dhcp-option-force=210,
UEFI boot from netboot
To send a file depending on the architecture, here the netboot image for UEFI-style boot, use:
pxe-service=BC_EFI, "Boot from network BC EFI", ipxe.efi pxe-service=X86-64_EFI, "Boot from network X86-64 EFI", ipxe.efi
If using netboot, the rest of the server setup section which focuses on the Arch ISO does not apply.
Transferring archiso root file system
Thanks to archiso_pxe_http, archiso_pxe_nfs and archiso_pxe_nbd initcpio hooks in archiso, it is possible to boot using HTTP, NFS or NBD. Boot time is approximately the same in all three methods, but HTTP method allows you to watch a state of downloading airootfs.sfs in percents.
HTTP
Among all alternatives, darkhttpd is by far the most trivial to setup (and the lightest-weight).
First, install the darkhttpd package.
Then start darkhttpd using our /mnt/archiso as the document root:
# darkhttpd /mnt/archiso
darkhttpd/1.8, copyright (c) 2003-2011 Emil Mikulic. listening on: http://0.0.0.0:80/
80. If you start darkhttpd without root access it will default to 8080, the client will still try to access port 80 and the boot will fail.NFS
You will need to set up an NFS server with an export at the root of your mounted installation medium, which would be /mnt/archiso if you followed #Preparation. After setting up the server, add the following line to your /etc/exports file:
/etc/exports
/mnt/archiso 192.168.0.0/24(ro,no_subtree_check)
If the server was already running, re-export the file systems with exportfs -r -a -v.
The default settings in the installer expect to find the NFS at /run/archiso/bootmnt/, so you will need to edit the boot options. To do this, press Tab on the appropriate boot menu choice and edit the archiso_nfs_srv option accordingly:
archiso_nfs_srv=${pxeserver}:/mnt/archiso
Alternatively, you can use /run/archiso/bootmnt for the entire process.
After the kernel loads, the Arch bootstrap image will copy the root file system via NFS to the booting host. This can take a little while. Once this completes, you should have a running system.
NBD
Install the nbd package and configure it:
/etc/nbd-server/config
[generic]
[archiso]
readonly = true
exportname = /srv/archlinux-release_date-x86_64.iso
where release_date is the release date in the ISO filename like, e.g., 2022.10.01.
Start nbd.service.
Existing PXE server
If you have an existing PXE server with a PXELINUX system setup (e.g. a combination of DHCP and TFTP), you can add the following menu items to your /tftpboot/pxelinux.cfg/default file in order to boot Arch via your preferred method.
Since PXELINUX supports HTTP, only the boot loader needs to be transferred over TFTP, everything else can use HTTP. E.g.:
LABEL archlinux
MENU LABEL Arch Linux x86_64
LINUX http://httpserver/path/to/extracted/Arch/ISO/arch/boot/x86_64/vmlinuz-linux
INITRD http://httpserver/path/to/extracted/Arch/ISO/arch/boot/x86_64/initramfs-linux.img
APPEND archisobasedir=arch archiso_http_srv=http://httpserver/path/to/extracted/Arch/ISO/ cms_verify=y
SYSAPPEND 3
TEXT HELP
Arch Linux 2022.10.01 x86_64
ENDTEXT
For NFS and NBD, the kernel and initramfs must be downloaded from TFTP. E.g., for NFS:
LABEL archlinux
MENU LABEL Arch Linux x86_64
LINUX /path/to/extracted/Arch/ISO/arch/boot/x86_64/vmlinuz-linux
INITRD /path/to/extracted/Arch/ISO/arch/boot/x86_64/initramfs-linux.img
APPEND archisobasedir=arch archiso_nfs_srv=pxeserver:/run/archiso/bootmnt cms_verify=y
SYSAPPEND 3
TEXT HELP
Arch Linux 2022.10.01 x86_64
ENDTEXT
The LINUX and INITRD paths are relative to TFTP root. For NBD, replace archiso_nfs_srv with archiso_nbd_srv in the above example. See usage examples in boot/syslinux/archiso_pxe.cfg file resided on Arch Linux ISO.
Whichever method you choose, you must pass ip= parameter to instruct the kernel to bring up the network interface before it attempts to mount the installation medium over the network. Passing BOOTIF= is required when there are several wired interfaces on the client side and/or you want resolv.conf to be already configured inside booted archiso. You can use sysappend mask 3 (which is 1+2) to pass these parameters automatically. For available boot parameters see README.bootparams.
Installation
For this portion you will need to figure out how to tell the client to attempt a PXE boot; in the corner of the screen along with the normal post messages, usually there will be some hint on which key to press to try PXE booting first. On an IBM x3650 F12 brings up a boot menu, the first option of which is Network; on a Dell PE 1950/2950 pressing F12 initiates PXE booting directly.
Boot
Looking at journald on the PXE server will provide some additional insight to what exactly is going on during the early stages of the PXE boot process:
# journalctl -u dnsmasq.service -f
dnsmasq-dhcp[2544]: DHCPDISCOVER(eth1) 00:1a:64:6a:a2:4d dnsmasq-dhcp[2544]: DHCPOFFER(eth1) 192.168.0.110 00:1a:64:6a:a2:4d dnsmasq-dhcp[2544]: DHCPREQUEST(eth1) 192.168.0.110 00:1a:64:6a:a2:4d dnsmasq-dhcp[2544]: DHCPACK(eth1) 192.168.0.110 00:1a:64:6a:a2:4d dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/pxelinux.0 to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/archiso.cfg to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/whichsys.c32 to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/archiso_pxe_choose.cfg to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/ifcpu64.c32 to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/archiso_pxe_both_inc.cfg to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/archiso_head.cfg to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/archiso_pxe32.cfg to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/archiso_pxe64.cfg to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/archiso_tail.cfg to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/vesamenu.c32 to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/boot/syslinux/splash.png to 192.168.0.110
After you load pxelinux.0 and archiso.cfg via TFTP, you will (hopefully) be presented with a syslinux boot menu with several options, where you can select Boot Arch Linux (x86_64) (HTTP).
Next the kernel and initramfs (appropriate for the architecture you selected) will be transferred, again via TFTP:
dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/x86_64/vmlinuz to 192.168.0.110 dnsmasq-tftp[2544]: sent /mnt/archiso/arch/boot/x86_64/initramfs-linux.img to 192.168.0.110
If all goes well, you should then see activity on darkhttpd coming from the PXE-target; at this point the kernel would be loaded on the PXE-target, and in init:
1348347586 192.168.0.110 "GET /arch/aitab" 200 678 "" "curl/7.27.0" 1348347587 192.168.0.110 "GET /arch/x86_64/root-image.fs.sfs" 200 107860206 "" "curl/7.27.0" 1348347588 192.168.0.110 "GET /arch/x86_64/usr-lib-modules.fs.sfs" 200 36819181 "" "curl/7.27.0" 1348347588 192.168.0.110 "GET /arch/any/usr-share.fs.sfs" 200 63693037 "" "curl/7.27.0"
After the root file system is downloaded via HTTP, you will eventually end up at the normal live system root zsh prompt.
Post-boot
Unless you want all traffic to be routed through your PXE server (which will not work anyway unless you set it up properly), you will want to stop dnsmasq.service and get a new lease on the install target, as appropriate for your network layout.
You can also kill darkhttpd; the target has already downloaded the root file system, so it is no longer needed. While you are at it, you can also unmount the installation image:
# umount /mnt/archiso
At this point you can follow the Installation guide.
Low memory systems
The copytoram initramfs option can be used to control whether the root file system should be copied to ram in its entirety in early-boot.
It highly recommended to leave this option alone, and should only be disabled if entirely necessary (systems with less than ~256MB physical memory). Append copytoram=n to your kernel line if you wish to do so.
copytoram=n and archiso_pxe_http are mutually exclusive.Sharing internet with PXE clients
If your network for PXE clients is private (for example, 192.168.1.0/24), and you want them to be able to access internet (for example, for packages installation), you should configure masquerade/source nat properly. Your PXE server must have a separate NIC connected to the internet. You can use such command to pass through the internet to clients:
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -j MASQUERADE
To make this rule persistent after reboot, run the following command:
iptables-save -f /etc/iptables/iptables.rules
and enable iptables.service.
See Simple stateful firewall#Setting up a NAT gateway and Internet sharing#Enable NAT for more information.
Troubleshooting
DHCP interface rename bug
FS#36749 causes default predictable network interface renaming to fail and then DHCP client to fail because of it. A workaround is to add the kernel boot parameter net.ifnames=0 to disable predictable interface names.
VirtualBox cannot boot while real machines can
When using VirtualBox to test your configuration, the virtual machine may get stuck at:
Probing EDD (edd=off to disable)... ok
While PXE booting with a real machine works fine. The problem may be because you have set several CPU cores to your client machine, and you set its type as Other and version as Other/Unknown (64 bit). So VirtualBox does not know which paravirtualization interface to use by default.
Adding loglevel=7 to the kernel parameters lets you see where it actually got stuck:
[ 0.063697] smp: Bringing up secondary CPUs... [ 0.103768] x86: Booting SMP configuration:
To resolve this, either use one CPU core, or go to Machine > Settings > System > Acceleration and set one of the following paravirtualization interface: Minimal, Hyper-V or KVM.