Migrating from component-managed per-namespace egress IP to self service for namespace egress IPs
Starting situation
-
You have a cluster which uses the component’s support for per-namespace egress IPs (parameter
egress_gateway.egress_ip_ranges.<range>.namespace_egress_ips). -
You want to switch the cluster to enable the customer to configure namespace egress IPs via namespace annotation.
-
Espejote is installed on the cluster via component-espejote.
-
The cluster is already using component-cilium v1.13.0 or newer
Make sure to update the cluster to component-cilium v1.13.0 or newer before applying the steps in this how-to to ensure that the egress policies aren’t deleted during the migration.
Steps
-
Switch to a temporary working directory
WDIR=$(mktemp -d) pushd $WDIR -
Compile the cluster which you want to update
CLUSTER_ID=c-cluster-id-1234 (1) commodore catalog compile ${CLUSTER_ID}1 The Project Syn ID of the cluster -
Make sure you have access to the cluster
export KUBECONFIG=/path/to/cluster.kubeconfig kubectl cluster-info -
Update the cluster config to use the self service mechanism for namespace egress IPs
TENANT_ID=$(yq .parameters.cluster.tenant inventory/classes/params/cluster.yml) pushd inventory/classes/${TENANT_ID} yq -i ".parameters.cilium.egress_gateway.self_service_namespace_ips = true" ${CLUSTER_ID}.yml yq -i ".parameters.cilium.egress_gateway.egress_ip_ranges[] |= del(.namespace_egress_ips)" ${CLUSTER_ID}.yml git commit -am "Enable per-namespace self-service egress IPs on ${CLUSTER_ID}" git push popd -
Compile and push the cluster catalog
commodore catalog compile ${CLUSTER_ID} --push -i -
Add the
cilium.syn.tools/egress-ipannotation to all namespaces which have a policyThis step assumes that the only IsovalentEgressGatewayPolicyresources on the cluster are per-namespace egress IP policies managed by the Commodore component.kubectl get isovalentegressgatewaypolicy -l argocd.argoproj.io/instance=cilium -oyaml | \ yq '.items[] | "kubectl --as=cluster-admin annotate namespace \(.metadata.name) cilium.syn.tools/egress-ip=" + .metadata.annotations["cilium.syn.tools/egress-ip"] ' | \ bash (1)1 If you want to verify the commands that are generated, remove the | bashfor the first run. -
Verify that the Espejote
ManagedResourcehas been deployedkubectl -n cilium get managedresource -
Remove the
argocd.argoproj.io/instancelabel from the egress policies to bring theciliumArgoCD app back in syncThis step assumes that the only IsovalentEgressGatewayPolicyresources on the cluster are per-namespace egress IP policies managed by the Commodore component.kubectl --as=cluster-admin label isovalentegressgatewaypolicy -l argocd.argoproj.io/instance=cilium argocd.argoproj.io/instance-