Restore Keycloak from a Backup

The following steps will guide you through restoring a backup of Keycloak.

This guide only covers how to restore the built-in database. If you use an external database, please consult the documentation of your database provider on how to backup and restore it.

You can only restore a database that has been backed up. Please refer to the built-in database setup guide on how to enable backups.

Requirements

  • kubectl With access to the cluster running Keycloak

  • base64

  • restic

  • (optional) jq

  • pwgen

  • vault

  1. Configure access to backups

    # The namspace containing the Keycloak instance. Change if necessary.
export NAMESPACE=syn-keycloak

export reposecret=$( \
    kubectl -n $NAMESPACE get schedule backup \
    -o go-template="{{.spec.backend.repoPasswordSecretRef.name}}" \
  )
export s3secret=$( \
    kubectl -n $NAMESPACE get schedule backup \
    -o jsonpath="{.spec.backend.s3.accessKeyIDSecretRef.name}" \
  )

export RESTIC_REPOSITORY=$( \
    kubectl -n $NAMESPACE get schedule backup \
    -o go-template="s3:{{.spec.backend.s3.endpoint}}/{{.spec.backend.s3.bucket}}/" \
  )
export RESTIC_PASSWORD=$( \
    kubectl -n $NAMESPACE get secrets $reposecret \
    -o jsonpath={.data.password} \
    | base64 -d \
  )
export AWS_ACCESS_KEY_ID=$( \
    kubectl -n $NAMESPACE get secrets $s3secret \
    -o jsonpath="{.data.username}" \
    | base64 -d \
  )
export AWS_SECRET_ACCESS_KEY=$( \
    kubectl -n $NAMESPACE get secrets $s3secret \
    -o jsonpath="{.data.password}" \
    | base64 -d \
  )

  2. List backups and choose the one to restore

    restic snapshots

export SNAPSHOT_ID=XXXXXX # Choose a snapshot id from the list

    To choose the last available backup you can simply run

    export SNAPSHOT_ID=$(restic snapshots --json --latest 1 --path /$NAMESPACE-keycloak-postgresql.sql | jq -r '.[0].id')

  3. Disable ArgoCD auto sync

    # The ArgoCD app of the Keycloak instance. Change if necessary.
export ARGO_APP=keycloak

kubectl -n syn patch apps root --type=json \
  -p '[{"op":"replace", "path":"/spec/syncPolicy", "value": {}}]'
kubectl -n syn patch apps ${ARGO_APP} --type=json \
  -p '[{"op":"replace", "path":"/spec/syncPolicy", "value": {}}]'

  4. Scale down Keycloak

    kubectl -n $NAMESPACE patch statefulset keycloakx --type=json \
  -p '[{"op":"replace", "path":"/spec/replicas", "value": 0}]'

# Wait until statefulset has been scaled down
kubectl -n $NAMESPACE get statefulset keycloakx -w

  5. Load the backup and restore it

    export POD=keycloak-postgresql-0

restic dump "${SNAPSHOT_ID}" /$NAMESPACE-keycloak-postgresql.sql \
  | kubectl -n $NAMESPACE exec -i $POD \
  -- sh -c 'PGPASSWORD="${POSTGRES_PASSWORD}" psql --set ON_ERROR_STOP=on -U "${POSTGRES_USER}" ${POSTGRES_DATABASE}'

  6. Re-enable ArgoCD auto sync and scale up Keycloak

    kubectl -n syn patch apps root --type=json \
  -p '[{
    "op":"replace",
    "path":"/spec/syncPolicy",
    "value": {"automated": {"prune": true, "selfHeal": true}}
  }]'

# Wait until Keycloak has started sucessfully
kubectl -n $NAMESPACE get statefulset keycloakx -w

This guide assumes that you have direct access to the S3 bucket holding the backup. If the access is restricted to the Kubernetes cluster, you will need to adapt these steps. You could:

  1. Perform the commands in a container running on the cluster

  2. Restore the database backup to a PVC and copy it over

Consult the official K8up documentation on options for restoring backups.