UpgradeJobHook examples

This is a collection of upgradejobhooks for frequently used one-off tasks.

Scale down a machineset

Scale down a machineset to remove a node after upgradejob is finished. Disables ArgoCD auto-sync for the root and openshift4-nodes apps. Only runs once on the next UpgradeJob due to the .spec.run: Next setting. A clusterrolebinding for cluster-admin for the appuio-openshift-upgrade-controller default serviceaccount is included below.

---
apiVersion: managedupgrade.appuio.io/v1beta1
kind: UpgradeJobHook
metadata:
  name: scale-down-machineset
  namespace: appuio-openshift-upgrade-controller
spec:
  events:
    - Finish
  selector:
    matchLabels:
      appuio-managed-upgrade: "true"
  run: Next
  template:
    spec:
      template:
        spec:
          containers:
            - args:
                - -c
                - |
                  #!/bin/sh
                  set -e
                  oc -n syn patch apps root --type=json \
                    -p '[{"op":"replace", "path":"/spec/syncPolicy", "value": {}}]'
                  oc -n syn patch apps openshift4-nodes --type=json \
                    -p '[{"op":"replace", "path":"/spec/syncPolicy", "value": {}}]'
                  oc -n openshift-machine-api scale --replicas=3 machineset app
              command:
                - sh
              image: quay.io/appuio/oc:v4.13
              name: remove-nodes
              env:
                - name: HOME
                  value: /export
              volumeMounts:
                - mountPath: /export
                  name: export
              workingDir: /export
          restartPolicy: Never
          volumes:
            - emptyDir: {}
              name: export
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: drain-nodes-upgrade-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: default
    namespace: appuio-openshift-upgrade-controller

Replace nodes during maintenance

Replaces all nodes labelled with appuio.io/remove at the start of the maintenance sequentially. Only works on clusters that use machinesets. This is useful for example when cloudscale requests to power off certain VMs completely for cold migration. With this upgradejobhook we instead simply replace all the affected nodes. Only runs once on the next UpgradeJob due to the .spec.run: Next setting. A clusterrolebinding for cluster-admin for the appuio-openshift-upgrade-controller default serviceaccount is included below.

---
apiVersion: managedupgrade.appuio.io/v1beta1
kind: UpgradeJobHook
metadata:
  name: replace-nodes
  namespace: appuio-openshift-upgrade-controller
spec:
  events:
    - Start
  selector:
    matchLabels:
      appuio-managed-upgrade: "true"
  run: Next
  template:
    spec:
      template:
        spec:
          containers:
            - args:
                - -c
                - |
                  for node in $(kubectl get no -l appuio.io/remove -ojsonpath='{.items[*].metadata.name}')
                  do
                    kubectl drain $node --ignore-daemonsets --delete-emptydir-data --force
                    kubectl -n openshift-machine-api delete machine $node
                    max_replicas=$(kubectl -n openshift-machine-api get machineset worker -ojsonpath='{.status.replicas}')
                    kubectl -n openshift-machine-api wait --for=jsonpath='{.status.readyReplicas}'=$max_replicas machineset worker --timeout=900s
                  done
              command:
                - sh
              image: quay.io/appuio/oc:v4.20
              name: replace-nodes
          restartPolicy: Never
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: remove-nodes-upgrade-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: default
    namespace: appuio-openshift-upgrade-controller

Manually rotate the service CA certificate

Force a rotation of the service CA certificate during the maintenance. See the OpenShift documentation for details. The rotation will be skipped for noop upgradejobs. Only runs once on the next UpgradeJob due to the .spec.run: Next setting. A clusterrolebinding for cluster-admin for the appuio-openshift-upgrade-controller default serviceaccount is included below.

---
apiVersion: managedupgrade.appuio.io/v1beta1
kind: UpgradeJobHook
metadata:
  name: rotate-service-ca-cert
  namespace: appuio-openshift-upgrade-controller
spec:
  events:
    - Start
  selector:
    matchLabels:
      appuio-managed-upgrade: "true"
  run: Next
  template:
    spec:
      template:
        spec:
          containers:
            - args:
                - -c
                - |
                  #!/bin/bash
                  set -xeuo pipefail

                  version="${JOB_spec_desiredVersion_version:-}"

                  if [ -z "${version}" ]; then
                    echo "Noop upgradejob, skipping ca rotation"
                  else
                    oc delete secret/signing-key -n openshift-service-ca
                  fi
              command:
                - bash
              image: quay.io/appuio/oc:v4.15
              name: delete-service-ca-cert
              env:
                - name: HOME
                  value: /export
              volumeMounts:
                - mountPath: /export
                  name: export
              workingDir: /export
          restartPolicy: Never
          volumes:
            - emptyDir: {}
              name: export
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: rotate-service-ca-cert
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: default
    namespace: appuio-openshift-upgrade-controller