Migrating from component-managed per-namespace egress IP to self service for namespace egress IPs

Starting situation

  • You have a cluster which uses the component’s support for per-namespace egress IPs (parameter egress_gateway.egress_ip_ranges.<range>.namespace_egress_ips).

  • You want to switch the cluster to enable the customer to configure namespace egress IPs via namespace annotation.

  • Espejote is installed on the cluster via component-espejote.

  • The cluster is already using component-cilium v1.13.0 or newer

    Make sure to update the cluster to component-cilium v1.13.0 or newer before applying the steps in this how-to to ensure that the egress policies aren’t deleted during the migration.

Prerequisites

  • Cluster-admin access to the cluster

  • Local Commodore setup

  • yq

Steps

  1. Switch to a temporary working directory

    WDIR=$(mktemp -d)
    pushd $WDIR
  2. Compile the cluster which you want to update

    CLUSTER_ID=c-cluster-id-1234 (1)
    commodore catalog compile ${CLUSTER_ID}
    1 The Project Syn ID of the cluster
  3. Make sure you have access to the cluster

    export KUBECONFIG=/path/to/cluster.kubeconfig
    kubectl cluster-info
  4. Update the cluster config to use the self service mechanism for namespace egress IPs

    TENANT_ID=$(yq .parameters.cluster.tenant inventory/classes/params/cluster.yml)
    pushd inventory/classes/${TENANT_ID}
    
    yq -i ".parameters.cilium.egress_gateway.self_service_namespace_ips = true" ${CLUSTER_ID}.yml
    yq -i ".parameters.cilium.egress_gateway.egress_ip_ranges[] |= del(.namespace_egress_ips)" ${CLUSTER_ID}.yml
    
    git commit -am "Enable per-namespace self-service egress IPs on ${CLUSTER_ID}"
    git push
    
    popd
  5. Compile and push the cluster catalog

    commodore catalog compile ${CLUSTER_ID} --push -i
  6. Add the cilium.syn.tools/egress-ip annotation to all namespaces which have a policy

    This step assumes that the only IsovalentEgressGatewayPolicy resources on the cluster are per-namespace egress IP policies managed by the Commodore component.
    kubectl get isovalentegressgatewaypolicy -l argocd.argoproj.io/instance=cilium -oyaml | \
      yq '.items[] |
        "kubectl --as=cluster-admin annotate namespace \(.metadata.name) cilium.syn.tools/egress-ip="
        + .metadata.annotations["cilium.syn.tools/egress-ip"]
      ' | \
      bash (1)
    1 If you want to verify the commands that are generated, remove the | bash for the first run.
  7. Verify that the Espejote ManagedResource has been deployed

    kubectl -n cilium get managedresource
  8. Remove the argocd.argoproj.io/instance label from the egress policies to bring the cilium ArgoCD app back in sync

    This step assumes that the only IsovalentEgressGatewayPolicy resources on the cluster are per-namespace egress IP policies managed by the Commodore component.
    kubectl --as=cluster-admin label isovalentegressgatewaypolicy -l argocd.argoproj.io/instance=cilium argocd.argoproj.io/instance-