Infrastructure - FreeBSD Buildbox

This may not be necessary for everyone, but my goal is to run my jails as minimally as possible: no userland, no root, minimal dependencies. This means that a lot of things will need to be manually built - and in so far as possible, I am aiming for reproducible builds. This means a controlled build environment, ideally down to the compiler flags and package options. This build environment will be a standard base FreeBSD install - there's a ton of tutorials out there for how to install such a system.

There is a chicken-and-egg problem here in that maintaining a source and ports tree practically requires git, but git is not part of the base. Circumventing that is easiest by installing the ports and src components of the base system. We'll get there.

The nature of a build box dictates that you will need a fair amount of RAM and hard disk space - on my box, I currently have 6.3 GByte in assorted built packages, 1 GByte for the compiled FreeBSD source code, and quite a bit of free space to hold the files while building things. Finally, I recommend using compatible sub-architectures for build box and your final host. In my case, and in all the following instructions, that is assumed to be "x86-64-v3".

With all that said, I am going to assume there is a VM dedicated as build system, and you have installed and configured the FreeBSD base system - at time of this writing, 14.3p6. I know, 15.0 is out, but there's something going on with that release I haven't run down yet. So, stick with the 14.x releases for now. Without further configuration, the Ports tree builds as root - one more reason for the dedicated (and disposable!) build box. So, since so far, there is nothing set up, log in as root to your shiny new builder. As said earlier, you'll want git. So, after logging as root, run

make -C /usr/ports/devel/git FLAVOR=tiny all install clean

If everything goes right (on a naked system, it should), that will get you a git client. Probably an old-ish version, but that's fine for now. Next, you'll want to grab poudriere. It's a build tool for FreeBSD ports, and takes care of all the annoying details like jail management for building.

make -C /usr/ports/ports-mgmt/poudriere all install clean

Poudriere needs some setup to function. You'll need - and quite possibly want - a number of options set in /usr/local/etc/poudriere.conf:

#Storage for jails, log files and build packages
ZPOOL=<your storage pool>
ZROOTFS=<root dataset for Poudriere>
BASEFS=<mount point for Poudriere's root dataset>

#Building in a tmpfs (RAM disk) can speed things up, but watch large packages
USE_TMPFS=yes
#My suggestion. You might find you need to add others.
TMPFS_BLACKLISE="llvm* rust gcc*"

#Source code needs to be downloaded somewhere
DISTFILES_CACHE=${BASEFS}/data/cache/distfiles

#This shouldn't be required, but I feel it's a good idea.
#Terminates a compile if a dependency does not exist
BAD_PKGNAME_DEPS_ARE_FATAL=yes

#Build in parallel
PARALLEL_JOBS=<number of jobs. I have mine at 1/2 the number of CPU cores>
ALLOW_MAKE_JOBS=yes

#Hostname of the build box
BUILDER_HOSTNAME=<set to whatever>

#Do not build these in parallel. This is currently only on the devel branch
MUTUALLY_EXCLUSIVE_BUILD_PACKAGES="llvm* rust* gcc* node*"

In the next chapter, we'll start building actual packages.