Bootstrap Containers

Setting up the host with containers that start during boot.

Bootstrap containers are a form of host container that you can use to run critical software before the node connects to an orchestrator. They give you substantial power to configure and modify the system in ways that would otherwise be difficult or impossible.

Lifecycle

When you define a bootstrap container using user-data or the Bottlerocket API, the following sequence occurs on next boot:

  • systemd starts all of the bootstrap containers concurrently,
  • systemd will not move to the next target until all of the bootstrap containers start, run, and exit,
  • any bootstrap container configured with mode=once will change to mode=off once complete,
  • any bootstrap container set to essential=true that exits with an non-zero exit code halts the boot process,
  • unless stopped in the previous step, the boot process continues by loading the orchestrator agent ( kubelet on *-k8s-* variants or the ECS agent on *-ecs-* variants).

Examples

Considerations

As a consequence of this lifecycle, you should keep a few things in mind when using bootstrap containers:

  1. Bootstrap containers should be short-running and cleanly exit when complete otherwise the node will hang.
  2. Since systemd starts all the bootstrap containers concurrently, you cannot rely on a deterministic starting order.
  3. Bootstrap containers do not have access to the container orchestrator nor the rest of the cluster.
  4. Bootstrap containers should not modify critical configuration files (e.g. /etc/).

Capabilities and permissions

Bootstrap containers sit somewhat between default host containers and superpowered host container permissions. They have access to the underlying host filesystem at /.bottlerocket/ which contains the root filesystem (/.bottlerocket/rootfs). Additionally, bootstrap containers run with the CAP_SYS_ADMIN capability, allowing for the creation of files, directories, and mounts accessible to the host (however the root filesystem remains immutable). You cannot elevate the permissions of bootstrap containers.

Use cases

Bootstrap containers have a variety of uses.

  1. Configure large setting values. On many platforms there are limits to the size of user-data but you can use a bootstrap container to pass large values into apiclient.
  2. Conditionally halt boot. You can perform checks or evaluation of the system in a bootstrap container with `essential` set to `true`.
  3. Apply settings you do not want to store in plaintext. In cases where you need to pass values that you would not want to store in plain-text on external systems (such as credentials), you can instead use a bootstrap container that sets the values via apiclient.
  4. Configure ephemeral disks. Since bootstrap containers have access to /.bottlerocket/rootfs/mnt and all bootstrap or superpowered host containers share this bind mount, you can configure ephemeral disks attached to your host in a bootstrap container and use it elsewhere.

Also see


See a problem with this page? File an issue. All feedback is appreciated.
You can also directly contribute a change to the source file of this page on GitHub.