Configure Brupop
When you install Brupop, the operator comes pre-configured with reasonable defaults. Labeling your nodes is the only required configuration step.
Aside from labeling nodes, you configure Brupop with helm or with a manifest. Helm reduces the configuration burden for Brupop substantially with few down sides, so this documentation focuses on configuration with helm. If you choose to not use helm, refer to the pre-baked manifest for an example .
Required Configuration
Label nodes
Warning
You can fully install Brupop but if you do not apply the proper node labels the operator will not update your nodes.
Kubernetes node labels control which nodes Brupop updates; specifically, the label bottlerocket.aws/updater-interface-version=2.0.0 dictates which nodes in the cluster get automatic updates.
You can label nodes using
settings.kubernetes.node-labels
with TOML
(including instance user data)
, using apiclient in a host container, or kubectl.
Label a node with apiclient
From the control or admin container, run the following:
apiclient set settings.kubernetes.node-labels.bottlerocket.aws/updater-interface-version=2.0.0
Label all nodes when starting an EKS cluster with eksctl
...
nodeGroups:
- name: name-of-your-nodegroup
labels: { bottlerocket.aws/updater-interface-version: 2.0.0 }
...
Labeling nodes with kubectl
Label a single node
# replace MY_NODE_NAME with the name of your node
kubectl label node MY_NODE_NAME bottlerocket.aws/updater-interface-version=2.0.0
Label all nodes
If you are running Bottlerocket on all nodes in your cluster, you can use kubectl to label all nodes at once:
kubectl label node $(kubectl get nodes -o jsonpath='{.items[*].metadata.name}') bottlerocket.aws/updater-interface-version=2.0.0
Labeling a node with the Bottlerocket API
Add the following TOML to your instance user data:
...
[settings.kubernetes.node-labels]
"bottlerocket.aws/updater-interface-version" = "2.0.0"
...
Optional Configuration
API Server Ports
Helm Configuration: apiserver_internal_port for internal traffic, apiserver_service_port for node agent traffic.
Brupop uses two ports for communication between components: apiserver_internal_port for the controller and the BottlerocketShadow custom resource and the apiserver_service_port for the conversion webhook.
Refer to the the
manifest
for more information on the usage of each port.
By default, the operator’s API server uses port 8443 for internal traffic and port 443 for node agents, but you can change these ports via this configuration.
Both ports must be set or the operator will fail to start.
Example:
apiserver_internal_port: "8443"
Concurrent Updates
Helm Configuration: max_concurrent_updates
You can set the maximum concurrency of updates that Brupop will perform.
You either set a specific number of concurrent updates or, alternately, "unlimited" to update as many nodes as possible concurrently.
In either case, Brupop always respects PodDisruptionBudget.
Conflicts between load balancing and concurrency
Take caution when setting concurrency and excluding load balancers together, as misconfiguration can result in a condition where all nodes exclude load balancing and can never drain fully to complete the update.
Setting up PodDisruptionBudget guards against this condition.
Example:
max_concurrent_updates: "1"
Namespace
Helm Configuration: brupop-bottlerocket-aws
You can change the namespace where Kubernetes deploys Brupop (default: brupop-bottlerocket-aws).
Example:
namespace: "brupop-bottlerocket-aws"
Load balancer exclusion
Helm Configuration: exclude_from_lb_wait_time_in_sec
With this option, you control the exclusion of the node from load balancing and delays draining the node for the number of seconds specified.
Internally, Brupop uses node.kubernetes.io/exclude-from-external-load-balancers to exclude the node from load balancing.
See Concurrent Updates for an important warning about concurrency and load balancer exclusion.
Example:
exclude_from_lb_wait_time_in_sec: "0"
Logging
Brupop emits logs from the controller, agent, and API server through standard Kubernetes logging mechanisms but you configure the log format and filter.
Format
Helm Configuration: logging.formatter
Log formatting has four options:
full: Human-readable, single-line logs,compact: A shorter version offull,pretty: “Excessively pretty”, terminal-optimized human-readable logs (default),json: New line-delimited JSON-formatted (machine-readable) logs.
Example:
logging:
formatter: "pretty"
Colours
Helm Configuration: logging.ansi_enabled
You can optionally set the logs to add ANSI colour information (true/false), which is helpful if viewing in a terminal, but adds garbage characters for non-terminal logging utilities.
Example:
logging:
ansi_enabled: "true"
Filter
Helm Configuration: The controller, agent, and API server are configured vialogging.controller.tracing_filter, logging.agent.tracing_filter, and logging.apiserver.tracing_filter (respectively).
Log filtering accepts on both typical log levels (info (default), debug, error) or through filter directives.
Example:
controller:
tracing_filter: "info"
agent:
tracing_filter: "debug"
apiserver:
tracing_filter: "error"
Placement
Helm Configuration: placement.agent, placement.controller, placement.apiserver
With these configurations, you can control the tolerations for the agent, controller and API server. For the controller and API server you can also control the node selector, and pod affinitiy and anti-affinity.
Example:
# Placement controls
# See the Kubernetes documentation about placement controls for more details:
# * https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
# * https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector
# * https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
placement:
agent:
# The agent is a daemonset, so the only controls that apply to it are tolerations.
tolerations: []
controller:
tolerations: []
nodeSelector: {}
podAffinity: {}
podAntiAffinity: {}
apiserver:
tolerations: []
nodeSelector: {}
podAffinity: {}
# By default, apiserver pods prefer not to be scheduled to the same node.
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
podAffinityTerm:
labelSelector:
matchExpressions:
- key: brupop.bottlerocket.aws/component
operator: In
values:
- apiserver
topologyKey: kubernetes.io/hostname
Private Image Registry
Helm Configuration: image_pull_secrets
If you are testing Brupop with a private image registry, you can configure pull secrets to fetch images.
Example:
image_pull_secrets:
- name: "brupop"
Scheduling
Helm Configuration: scheduler_cron_expression
Brupop schedules node updates based on a cron expression in the following format:
┌───────────── seconds (0 - 59)
│ ┌───────────── minute (0 - 59)
│ │ ┌───────────── hour (0 - 23)
│ │ │ ┌───────────── day of the month (1 - 31)
│ │ │ │ ┌───────────── month (Jan, Feb, Mar, Apr, Jun, Jul, Aug, Sep, Oct, Nov, Dec)
│ │ │ │ │ ┌───────────── day of the week (Mon, Tue, Wed, Thu, Fri, Sat, Sun)
│ │ │ │ │ │ ┌───────────── year (formatted as YYYY)
│ │ │ │ │ │ │
│ │ │ │ │ │ │
* * * * * * *
Example:
# Every day at 3 AM
scheduler_cron_expression: "* * 3 * * * *"