UEFI HTTP Boot with Libvirt

Background

I’ve been a big proponent of network based provisioning pretty much my entire career. My second job out of college involved imaging ~800 computers multiple times a week. When I was hired, my predecessors used floppy disks to load a small OS, matching NIC driver, and imaging client (remember Ghost?!). The bottom line was it was very time/labor intensive and a horrible process. Imaging a group of systems took about 30-60 min. Long story short we reduced that time to about 5 min after we leveraged a combination of PXE, wake-on-lan, UNDI drivers, vlans, and IGMP snooping. My second iteration of the solution took the total attended time to less than 30 seconds. Anyway, it’s amazing technology for provisioning, and I even got hired at Red Hat by giving a presentation on PXE. Needless to say, I’m a huge fan!

Anyway, the problem here is PXE dates back to the 90s and is quite limited by its reliance on technologies such as TFTP. Intel has been threatening to deprecate PXE for years now and they are finally doing it. Even though PXE is pervasive today, it’s likely that over the next 2-10 years UEFI HTTP boot will become the default for most environments. Other benefits and technical details are outlined here. The TL;DR is PXE relies on DHCP & TFTP and UEFI HTTP boot needs DHCP & HTTP. Sound easy? That’s because it is. You might even say it’s as Trivial as File Transfer Protocols get. ….sorry network booting humor is pretty difficult to come by.

Switching from PXE to HTTP Boot

The best documentation I can find on this topic currently is from the good folks at Suse here. Essentially, we drop the need for TFTP and need to tweak the DHCP options to fetch the NBP from a HTTP/HTTPS endpoint. I’ll be using GRUB as the NBP in this post. Environments using dhcpd can simply drop-in the configuration they provide and everything should just work assuming you tweak the NBP for your environment:

   class "pxeclients" {
     match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
     next-server 192.168.111.1;
     filename "/bootx64.efi";
   }
   class "httpclients" {
     match if substring (option vendor-class-identifier, 0, 10) = "HTTPClient";
     option vendor-class-identifier "HTTPClient";
     filename "http://www.httpboot.local/sle/EFI/BOOT/bootx64.efi";
   }

This example is nice as it allows UEFI systems loading a legacy PXE rom to also boot.

Let’s Apply this to Libvirt

I had originally planned to set this up on my home network, but pfSense doesn’t yet support this nor does it allow the dhcpd config to be modified directly. I’ve opened an issue and hopefully this will be added in the future. The next simplest option I have for testing is using Libvirt and VMs w/ the OVMF (Tianocore) UEFI firmware. Libvirt works great for creating a self-contained environment for experimenting with technology like this and it’s super easy to replicate. All we really need to do is provide the correct DHCP options.

The default libvirt network will look something similar to this:

<network>
  <name>default</name>
  <uuid>75f4d9cd-9af2-4df5-afcc-f8f9145f7e34</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' zone='trusted' stp='on' delay='0'/>
  <mac address='52:54:00:95:95:84'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.100.128' end='192.168.100.254' />
    </dhcp>
  </ip>
</network>

Enabling legacy PXE is well documented, but I’m including the changes here in case it’s helpful for anyone reading. We basically define the next-server and filename DHCP options.

<network>
  <name>default</name>
  <uuid>75f4d9cd-9af2-4df5-afcc-f8f9145f7e34</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' zone='trusted' stp='on' delay='0'/>
  <mac address='52:54:00:95:95:84'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
  <tftp root='/var/lib/tftpboot'/>
    <dhcp>
      <range start='192.168.100.128' end='192.168.100.254' />
      <bootp file='pxelinux.0'/>
    </dhcp>
  </ip>
</network>

Unfortunately the libvirt schema doesn’t support a lot of available config options for dnsmasq. Luckily, recent versions of libvirt support an XML namespace that will append options directly to the bottom of the generated config. This email shows the only working dnsmasq config that I was able to find that works as http boot is not documented well for the project. For now I’m leaving the tftp info in place so VMs using bios can continue to do network installs. Note: don’t ignore the first line of this example!

<network xmlns:dnsmasq='http://libvirt.org/schemas/network/dnsmasq/1.0'>
  <name>default</name>
  <uuid>75f4d9cd-9af2-4df5-afcc-f8f9145f7e34</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' zone='trusted' stp='on' delay='0'/>
  <mac address='52:54:00:95:95:84'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
  <tftp root='/var/lib/tftpboot'/>
    <dhcp>
      <range start='192.168.100.128' end='192.168.100.254' />
      <bootp file='pxelinux.0'/>
    </dhcp>
  </ip>
  <dnsmasq:options>
    <dnsmasq:option value='dhcp-vendorclass=set:efi-http,HTTPClient:Arch:00016'/>
    <dnsmasq:option value='dhcp-option-force=tag:efi-http,60,HTTPClient'/>
    <dnsmasq:option value='dhcp-boot=tag:efi-http,&quot;http://192.168.122.1/rhel8/EFI/BOOT/BOOTX64.EFI&quot;'/>
  </dnsmasq:options>
</network>

With the network config in place, a simple sudo virsh net-destroy default && sudo virsh net-start default will load the new config. Next we need a web server. I’m simply running httpd on my system as you can see it’s defined in the above example as 192.168.122.1 . One of the major benefits of this setup is the web end point can be anywhere so please use whatever makes sense for your setup. For reasons I don’t understand Silverblue includes httpd; all I needed to do on my system was run systemctl start httpd.

Setting up the boot menu.

Since I’m working with the RHEL 8.4 Beta, all that’s necessary is to download the minimal boot.iso, mount it, and copy the contents to /var/www/html/rhel8/. We won’t mount the ISO in this location as we need to tweak the GRUB config and will need to write the file. Next we need to tweak the kernel and initrd paths in the default grub config found under /var/www/html/rhel8/EFI/BOOT/grub.cfg as this won’t support relative paths. For my setup it’s only necessary to add /rhel8 on each line. Pass any other option needed and you’re good to go.

menuentry 'Install Red Hat Enterprise Linux 8.4' --class fedora --class gnu-linux --class gnu --class os {
      linuxefi /rhel8/images/pxeboot/vmlinuz inst.stage2=http://192.168.122.1/rhel8 inst.ks=http://192.168.122.1/ks/84.ks quiet
      initrdefi /rhel8/images/pxeboot/initrd.img
}

Enough already, let’s boot!

Now we’re ready to create and boot a VM which leads to the annoying part. By default a network boot with OVMF will attempt an IPv4 PXE -> IPv6 PXE -> IPv4 HTTP -> IPv6 HTTP in that order. It takes a long to let them fail so we’ll want to disrupt the standard boot process by quickly pressing the escape key repeatedly once the VM console comes up to manually select the IPv4 HTTP. In the screencast below I’m pressing the escape key quickly to select the correct option. This is less than ideal and I’m sure there’s a better way to either configure the firmware to disable the legacy option(s). Please let me know if you know how to do this.

sudo virt-install \
  --name=8.4-uefi-httpboot \
  --ram=2048 \
  --vcpus=1 \
  --os-type=linux \
  --os-variant=rhel8.4 \
  --graphics=vnc \
  --pxe \
  --disk=path=/var/home/bbreard/data/distros/uefi.qcow2 \
  --check path_in_use=off \
  --network=network=default,model=virtio \
  --boot=uefi

That’s all there is to it! I hope you’ll join me in a world that’s free from the limitations of TFTP.

References:

https://community.theforeman.org/t/enabling-httpboot-plugin-for-foreman-1-20/13058

https://en.opensuse.org/UEFI_HTTPBoot_Server_Setup

https://bitbin.de/blog/2019/04/uefi_http_boot/

https://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2018q3/012451.html

https://thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html