FreeBSD Tools for Running in the Cloud

Intro

A lot goes on behind the scenes when a FreeBSD instance boots up on a public cloud. The following is my attempt to document the various services that are responsible for provisioning a new cloud instance. The are several that are specific to EC2 which will be covered in a future article.

Tools

First Boot Sentinel File

When FreeBSD boots, it looks for the firstboot sentinel file. The contents of the file are never read, so the system is only concerned with whether the file exists. The name/path of this file is set by the rc.conf variable firstboot_sentinel (The default is /firstboot). If a file exists with this path, rc.d scripts with the firstboot keyword will be run on startup and the sentinel file will be deleted after the boot process completes. The sentinel file must be located on a writable file system which is mounted no later than early_late_divider to function properly.

To get a list of all the services which are controlled by the firstboot keyword, run the following command:

# rcorder -k firstboot /etc/rc.d/* /usr/local/etc/rc.d/*

/etc/rc.d/zpoolreguid
/etc/rc.d/growfs
/etc/rc.d/growfs_fstab
/usr/local/etc/rc.d/ec2_configinit
/usr/local/etc/rc.d/ec2_fetchkey
/usr/local/etc/rc.d/firstboot_pkgs
/usr/local/etc/rc.d/firstboot_freebsd_update
/usr/local/etc/rc.d/ec2_bootmail

More info: https://man.freebsd.org/cgi/man.cgi?rc.conf

firstboot-pkgs Service

When the system first boots, the firstboot-pkgs service will install the pkg(8) tools (if not already installed) and packages listed in the firstboot_pkgs_list rc.conf variable. If the installed packages added new rc.d scripts, the service will request a reboot.

Add the following lines to /etc/rc.conf.local or /etc/rc.conf to enable this and place a list of packages in firstboot_pkgs_list.

firstboot_pkgs_enable="YES"
firstboot_pkgs_list="apache24 php83 mysql80-server

More info: https://www.freshports.org/sysutils/firstboot-pkgs/

firstboot-freebsd-update Service

This service will perform a freebsd-update fetch when the system first boots. If updates are downloaded, install them and request a reboot.

More info: https://www.freshports.org/sysutils/firstboot-freebsd-update/

configinit Service

Anyone who has been around the world of cloud computing for long is likely to have heard of CloudInit. It is a system originally written for Ubuntu which performs configuration of a system at boot-time based on “user-data” provided via EC2 or from a similar environment (e.g., OpenStack). CloudInit works well for its original purpose, but is less than ideal for FreeBSD systems, for two reasons: First, it relies on Python, which is not part of the FreeBSD base system; and second, it is designed around a concept of configuring a system by running commands rather than editing configuration files.

Now, there are merits to both approaches — most notably, configuring a system by running commands is easier to script, while configuring a system by editing text files has the advantage that given a working configuration there’s no doubt about how to reproduce it — but the fact that BSD systems are far more edit-configuration-files oriented (to the point that /etc/rc.conf might be the only configuration file which needs to be edited on some systems), and thus CloudInit is less than optimal for configuring FreeBSD systems.

Rather than providing instructions such as “tell apt to use this mirror” or “run this python code”, configinit handles four types of input:

  1. If the configuration data starts >/path/to/file then the data, minus the first line, will be written to the specified location.
  2. If the configuration data starts >>/path/to/file. then the data, minus the first line, will be appended to the specified location.
  3. If the configuration data starts #!, it will be executed (in most cases this would be a shell script).
  4. For any other inputs, configinit attempts to extract the file as an archive, and (if extraction was successful) runs on each part in turn. The extraction is performed using bsdtar, so archives in tar, pax, cpio, zip, jar, ar, xar, rpm and ISO 9660 formats, optionally compressed using gzip, bzip2, xz, or compress can all be used.

This is much simpler than CloudInit, but in combination with other tools which are already available on FreeBSD, such as firstboot-pkgs, it provides very powerful yet easy-to-use functionality. For example, launching a FreeBSD EC2 instance with the following user-data:

> >>/etc/rc.conf
> firstboot_pkgs_list="apache22"
> apache22_enable="YES"

will provide a system with Apache 2.2 installed and running (in my test, within 150 seconds of when I clicked “launch” in the EC2 Management Console) — in addition to performing the other default system initialization behaviors of my EC2 images: checking for updates to the FreeBSD base system, downloading an SSH public key to allow SSH logins as ec2-user, logging SSH host keys to the EC2 console, and auto-configuring swap space using EC2 ephemeral disks.

To enable configinit, set ec2_configinit_enable=YES in rc.conf.

More info: http://www.daemonology.net/blog/2013-12-09-FreeBSD-EC2-configinit.html

cloud-init Service

cloud-init is an Ubuntu product handles early initialization of a cloud instance, and has become the de facto standard for Linux instances.

Some of the things it configures are:

  • Setting a default locale
  • Setting hostname
  • Generate ssh private keys
  • Adding ssh keys to user’s .ssh/authorized_keys so they can log in
  • Setting up ephemeral mount points

Unfortunately, cloud-init support for operating systems other than Linux has been rather poor, and the lack of cloud-init support on FreeBSD has been a hindrance to cloud providers who want to offer FreeBSD as a Tier 1 platform. Read more about the effort to bring cloud-init to FreeBSD here.

More info: https://cloud-init.io/

firstboot-cloudsetup Service

This service is an alternative to CloudInit and configinit, but for the OpenNebula cloud platform.

More info: https://git.sr.ht/~jornane/cloudsetup

growfs Service

The growfs script normally runs at the first boot after system installation. If the boot disk is larger than the root file system and boot partitions, and the root file system is in the last partition, growfs can expand the root file system. It can also add a swap partition, with a default size of 10% of the boot disk. Swap is limited to twice the memory size up to 4 GB, 8 GB up to 8 GB of memory, and memory size over 8 GB. It is also limited to the sysctl(8) value of vm.swap_maxpages divided by 2. By default, no swap partition is created if an existing swap partition is found or is listed in /etc/fstab, or the disk is under 15 GB. The growfs_fstab script adds any new swap partition to /etc/fstab after the root file system is made writable, and enables its use as a dump partition if the dumpdev variable from rc.conf(5) is set to AUTO.

This service can also be run when a cloud volume is resized to expand the filesystem to fill the partition.

More info: https://man.freebsd.org/cgi/man.cgi?query=growfs&sektion=7