Bootstrap Containers
Bootstrap containers are a form of host container that you can use to run critical software before the node connects to an orchestrator. They give you substantial power to configure and modify the system in ways that would otherwise be difficult or impossible.
Lifecycle
When you define a bootstrap container using user-data or the Bottlerocket API, the following sequence occurs on next boot:
systemd
starts all of the bootstrap containers concurrently,systemd
will not move to the next target until all of the bootstrap containers start, run, and exit,- any bootstrap container configured with
mode=once
will change tomode=off
once complete, - any bootstrap container set to
essential=true
that exits with an non-zero exit code halts the boot process, - unless stopped in the previous step, the boot process continues by loading the orchestrator agent (
kubelet
on*-k8s-*
variants or the ECS agent on*-ecs-*
variants).
Examples
This example shows four bootstrap containers (B1
,B2
,B3
, and B4
) starting.
The start order of these containers is not sequential and could be in any order.
Container B2
runs the longest and the next stage of the boot does not proceed until it completes.
This example is the same as the previous one except container B2
is configured with essential=true
.
Container B2
exits with a non-zero exit code and consequently the boot halts.
This example is the same as the first except container B4
is configured with mode=once
.
After all the bootstrap containers finish, Bottlerocket (via a systemd
unit) updates container B4
’s mode
to off
.
Considerations
As a consequence of this lifecycle, you should keep a few things in mind when using bootstrap containers:
- Bootstrap containers should be short-running and cleanly exit when complete otherwise the node will hang.
- Since
systemd
starts all the bootstrap containers concurrently, you cannot rely on a deterministic starting order. - Bootstrap containers do not have access to the container orchestrator nor the rest of the cluster.
- Bootstrap containers should not modify critical configuration files (e.g.
/etc/
).
Capabilities and permissions
Bootstrap containers sit somewhat between default host containers and superpowered
host container permissions.
They have access to the underlying host filesystem at /.bottlerocket/
which contains the root filesystem (/.bottlerocket/rootfs
).
Additionally, bootstrap containers run with the CAP_SYS_ADMIN capability, allowing for the creation of files, directories, and mounts accessible to the host (however the root filesystem remains immutable).
You cannot elevate the permissions of bootstrap containers.
Use cases
Bootstrap containers have a variety of uses.
- Configure large setting values.
On many platforms there are limits to the size of user-data but you can use a bootstrap container to pass large values into
apiclient
. - Conditionally halt boot. You can perform checks or evaluation of the system in a bootstrap container with `essential` set to `true`.
- Apply settings you do not want to store in plaintext.
In cases where you need to pass values that you would not want to store in plain-text on external systems (such as credentials), you can instead use a bootstrap container that sets the values via
apiclient
. - Configure ephemeral disks. Since bootstrap containers have access to
/.bottlerocket/rootfs/mnt
and all bootstrap orsuperpowered
host containers share this bind mount, you can configure ephemeral disks attached to your host in a bootstrap container and use it elsewhere.
Also see
See a problem with this page? File an issue. All feedback is appreciated.
You can also directly contribute a change to the source file of this page on GitHub.