Parameters

The parent key for all of the following parameters is openshift_upgrade_controller.

namespace

type

string
default

syn-openshift-upgrade-controller

The namespace in which to deploy this component.

namespaceMetadata

type

dict
default 
annotations:
  openshift.io/node-selector: "node-role.kubernetes.io/infra="
labels:
  openshift.io/cluster-monitoring: "true"

Metadata to be added to the namespace.

images

type

dictionary

The images to use for this component.

manifests_version

type

string
default

${openshift_upgrade_controller:images:openshift_upgrade_controller:tag}

The Git reference to the controller deployment manifests. The default is the tag of the controller image.

upgrade_configs

type

dict
default

{}
example 
upgrade_configs:
  monday-afternoon:
    spec:
      maxSchedulingDelay: 1h
      maxUpgradeStartDelay: 1h
      schedule:
        cron: "0 10 * * 2"
        location: Europe/Zurich
      jobTemplate:
        spec:
          config:
            upgradeTimeout: 12h
            preUpgradeHealthChecks:
              timeout: 1h
            postUpgradeHealthChecks:
              timeout: 1h

This parameter is used to configure the UpgradeConfig objects. The dictionary key is used as the name of the UpgradeConfig object. The value is the configuration of the UpgradeConfig object.

See github.com/appuio/openshift-upgrade-controller for the manifest definition.

upgrade_job_hooks

type

dict
default

{}
example 
upgrade_job_hooks:
  notify:
    spec:
      events:
        - Create
        - Start
        - UpgradeComplete
        - Finish
        - Success
        - Failure
      run: Next
      failurePolicy: Ignore
      selector:
        matchLabels: ${openshift_upgrade_controller:upgrade_configs:monday-afternoon:spec:jobTemplate:metadata:labels}
      template:
        spec:
          template:
            spec:
              containers:
              - name: notify
                image: curlimages/curl:8.1.2 # sponsored OSS image
                args:
                - -XPOST
                - -H
                - 'Content-Type: application/json'
                - -d
                - '{"event": $(EVENT_NAME), "version": $(JOB_spec_desiredVersion_image)}'
                - https://example.com/webhook
              restartPolicy: Never
          backoffLimit: 3
          ttlSecondsAfterFinished: 43200 # 12h
          activeDeadlineSeconds: 300 # 5m

This parameter is used to configure the UpgradeJobHooks objects. The dictionary key is used as the name of the UpgradeJobHooks object. The value is the configuration of the UpgradeJobHooks object.

UpgradeJobHooks default to priorityClassName: system-cluster-critical.

See github.com/appuio/openshift-upgrade-controller for the manifest definition.

spec.selector can be referenced from the UpgradeConfig object using Commodore variable expansion.

upgrade_suspension_windows

type

dict
default

{}
example 
upgrade_suspension_windows:
  end-of-year-holidays-2023:
    spec:
      start: "2023-12-25T00:00:00Z"
      end: "2024-01-08T00:00:00Z"
      reason: "End of year holidays"
      configSelector: {}
      jobSelector:
        matchLabels: ${openshift_upgrade_controller:upgrade_configs:monday-afternoon:spec:jobTemplate:metadata:labels}

This parameter is used to configure the UpgradeSuspensionWindow objects. The dictionary key is used as the name of the UpgradeSuspensionWindow object. The value is the configuration of the UpgradeSuspensionWindow object.

See github.com/appuio/openshift-upgrade-controller for the manifest definition.

spec.jobSelector and spec.configSelector can be referenced from the UpgradeConfig and UpgradeJob object using Commodore variable expansion.
An empty selector (jobSelector: {}) matches all objects. A null selector (jobSelector: null, jobSelector:, omitted) matches no objects.

node_force_drains

type

dict
default

{}
example 
node_force_drains:
  node-role-app:
    spec:
      nodeDrainGracePeriod: 1m
      nodeSelector:
        matchLabels:
          node-role.kubernetes.io/app: ""
      podForceDeleteGracePeriod: 2m

This parameter is used to configure the NodeForceDrain objects. The dictionary key is used as the name of the NodeForceDrain object. The value is the configuration of the NodeForceDrain object.

See github.com/appuio/openshift-upgrade-controller for the manifest definition.

upgrade_silence

type

object
default:: default

See class/defaults.yml

Creates a pre-configured UpgradeJobHook object to silence alerts given by alert_matchers during the upgrade.

upgrade_silence.alertmanager_host, upgrade_silence.alertmanager_operated_service, upgrade_silence.alertmanager_namespace

type

string
default 
alertmanager_host: "alertmanager-main"
alertmanager_operated_service: "alertmanager-operated"
alertmanager_namespace: "openshift-monitoring"

The host name, service name and namespace of the Alertmanager to use.

upgrade_silence.upgrade_job_selector

type

dict
default

{}
example 
upgrade_job_selector:
  matchLabels: ${openshift_upgrade_controller:upgrade_configs:appuio-monday-afternoon:spec:jobTemplate:metadata:labels}

Selects which UpgradeJob objects this hook applies to.

upgrade_silence.alert_matchers

type

dict
default

{}
example 
alert_matchers:
  "only maintenance without SLOs":
    matchers:
      - name: alertname
        value: "Watchdog"
        isRegex: false
        isEqual: false
      - name: Maintenance
        value: "true"
        isRegex: false
        isEqual: false

Selects the alerts to silence. The dictionary key is added as a comment to the silence. The value is the configuration of the silence. List values are ANDed together, and multiple silence configurations are ORed together.

upgrade_silence.silence_timeout_hours

type

integer
default

12

The duration of the silence in hours.

upgrade_silence.silence_after_finish_minutes

type

integer
default

30

The duration to wait after the upgrade job has finished before expiring the silence in minutes.

upgrade_silence.handle_delayed_worker_pools

type

bool
default

false

Whether to create separate silences for the initial maintenance and the delayed maintenance of one or more MachineConfigPools.

If set to true, the upgrade silence UpgradeJobHook is executed for events UpgradeComplete (when the upgrade is complete except for delayed MachineConfigPools) and MachineConfigPoolUnpause (when the delayed MachineConfigPool maintenance starts) in addition to the Start and Finish events.

When the hook script runs for the UpgradeComplete event it expires the silence after upgrade_silence.silence_after_finish_minutes. When the hook script runs for the MachineConfigPoolUnpause event it creates a silence which ends after upgrade_silence.silence_timeout_hours.

The Start and Finish logic remains unchanged, the Finish run will expire the silence created by the run for MachineConfigPoolUnpause when the handle_delayed_worker_pools parameter is true.

Currently, the upgrade-controller doesn’t have an event for MachineConfigPoolComplete. Enabling this parameter may produce undesired silences for configurations which have different delays for multiple delayed MachineConfigPools.

upgrade_silence.additional_job_configuration

type

dict
default 
additional_job_configuration:
  metadata: {}
  spec:
    template:
      spec:
        template:
          spec:
            nodeSelector:
              node-role.kubernetes.io/infra: ''

Additional configuration for the UpgradeJob object. This is merged with the configuration generated by the component.

admin_ack

type

object
default:: default

See class/defaults.yml

Creates a pre-configured UpgradeJobHook object to automatically acknowledge OpenShift minor upgrades. See OpenShift documentation for an example of this.

admin_ack.upgrade_job_selector

type

dict
default

{}
example 
upgrade_job_selector:
  matchLabels: ${openshift_upgrade_controller:upgrade_configs:appuio-monday-afternoon:spec:jobTemplate:metadata:labels}

Selects which UpgradeJob objects this hook applies to.

admin_ack.overrides

type

dict
default

{}
example 
overrides:
  "4.17": "special-ack-key"

Allows to manually set the key used to acknowledge the upgrade for a given minor version. Check the OpenShift documentation for the correct key to use.

If not set a default key is generated based on the upgrade jobs desiredVersion field. The default key is ack-4.${lastMinor}-kube-1.${kubernetesMinor}-api-removals-in-4.${minor}.

admin_ack.config_map_ref

type

dict
default 
config_map_ref:
  name: admin-acks
  namespace: openshift-config

The reference to the ConfigMap that holds the ack keys.

The ConfigMap was observed to have the following format:

apiVersion: v1
kind: ConfigMap
metadata:
  name: admin-acks
  namespace: openshift-config
data:
  ack-4.12-kube-1.25-api-removals-in-4.12: "true"
  ack-4.13-kube-1.26-api-removals-in-4.13: "true"
  ack-4.14-kube-1.27-api-removals-in-4.14: "true"

admin_ack.additional_job_configuration

type

dict
default 
additional_job_configuration:
  metadata: {}
  spec:
    template:
      spec:
        template:
          spec:
            nodeSelector:
              node-role.kubernetes.io/infra: ''

Additional configuration for the UpgradeJob object. This is merged with the configuration generated by the component.

cluster_version.openshiftVersion

type

object
default 
Major: '4'
Minor: '8'

This parameter is used to conditionally add configurations in the ClusterVersion object.

The component currently uses this parameter to set default values for * field channel. The component sets this field to stable-<Major>.<Minor>, where <Major> and <Minor> are replaced with the values of fields Major and Minor of this parameter.

cluster_version.spec

type

object
default 
upstream: https://api.openshift.com/api/upgrades_info/v1/graph

See the OpenShift docs for available parameters and values.

Field clusterID must be set in the ClusterVersion object. The value for this field is supposed to be extracted from the cluster as a fact. However, the corresponding dynamic fact isn’t implemented yet. We recommend that users set spec.clusterID to a non-component parameter such as openshift.clusterID in the config hierarchy.

Values specified in this parameter take precedence over default values derived from parameter openshiftVersion.

The desiredUpdate field is removed from the ClusterVersion object. The desired update is determined by the controller based on the UpgradeConfig and upstream ClusterVersion objects.

Example

We recommend configuring a reference for component parameter openshift_upgrade_controller.cluster_version.spec.clusterID for all OpenShift 4 clusters:

openshift4.yml
parameters:
  openshift:
    clusterID: 'OVERRIDE_THIS_IN_THE_CLUSTER_CONFIG'
  openshift_upgrade_controller:
    cluster_version:
      spec:
        clusterID: ${openshift.clusterID}

With this approach, each individual cluster config only needs to set generic parameter openshift.clusterID.

cluster.yml
parameters:
  openshift:
    clusterID: '6d8329e3-7098-4bab-b7d8-11f1dc353481'

  openshift_upgrade_controller:
    cluster_version:
      spec: ...

This example assumes that openshift_upgrade_controller.cluster_version.spec.clusterID is set to ${openshift.clusterID} somewhere in the inventory. Due to github.com/projectsyn/commodore/issues/138, this can not yet be done in the defaults.

cluster_version.overlays

type

object
default

{}
example 
cluster_version:
  spec:
    template:
      spec:
        channel: stable-4.13
  overlays:
    "2024-06-13T17:00:00+02:00":
      spec:
        channel: stable-4.14

This parameter can be used to schedule changes to the ClusterVersion object. The overlay with the latest from timestamp after the current time is merged onto the base configuration in spec.template. Overlays do not accumulate. See the upgrade controller ClusterVersion CRD for details.

The dictionary key is used as the from field of the overlay. The value is used as the overlay field of the overlay.

Entries can be removed by setting the value to null in the hierarchy.

alerts

type

dict
example 
BadThingsHappening:
  enabled: true
  rule:
    annotations:
      description: Bad things have been happening on {{$labels.node}} for more than 10 minutes.
      message: Bad things have been happening on {{$labels.node}} for more than 10 minutes.
      runbook_url: https://hub.syn.tools/openshift-upgrade-controller/runbooks/BadThingsHappening.html
    expr: |
      bad_thing_happening == 1
    for: 10m
    labels:
      severity: warning

alerts defines the alerts to be installed. The dictionary key is used as the name of the alert.

alerts.<name>.enabled

type

bool

Defines whether to install the alert.

alerts.<name>.rule

type

dict

Holds the configuration of the alert rule.

See Prometheus Alerting Rules for details.