Parameters

The parent key for all of the following parameters is cilium.

The namespace is cilium and currently non configurable.

`install_method`

type	string
default	`helm`
possible values	`helm`, `olm`

The installation method for Cilium. olm uses the Openshift Operator Lifecycle Manager. OLM installation is required for OpenShift clusters. The Cilium OLM is a thin wrapper around Helm, because of this the Helm values are used for OLM configuration too.

Table 1. Supported Combinations
	Helm	OLM
Opensource	✅	✅
Enterprise	✅	✅

`release`

type	string
default	`opensource`
possible values	`opensource`, `enterprise`

Two version of Cilium exist. The open-source version and the enterprise version.

See Upgrade Cilium OSS to Cilium Enterprise (OpenShift 4) for upgrading from the OSS version to the enterprise version.

`charts.cilium.source`

type	string
default	`helm.cilium.io`

The Helm repository from which to download the cilium Helm chart.

`charts.cilium.version`

type	string
default	See `class/defaults.yml`

The version to use for the cilium Helm chart.

`charts.cilium-enterprise.source`

type	string
default	`<CILIUM-ENTERPRISE-CHART-REPO-URL>`

The chart repository URL of the cilium-enterprise Helm chart. Users must provide the chart repository URL themselves in their Project Syn global or tenant configuration.

The component default is an invalid string (<CILIUM-ENTERPRISE-CHART-REPO-URL>) instead of ~ to make the Kapitan error message somewhat useful when the user hasn’t reconfigured the chart repository.

Example

parameters:
  cilium:
    charts:
      cilium-enterprise:
        source: https://cilium-ee-charts.example.org

`charts.cilium-enterprise.version`

type	string
default	See `class/defaults.yml`

The version to use for the cilium-enterprise Helm chart.

`olm.source`

type	object
default	`opensource: https://github.com/cilium/cilium-olm/archive/master.tar.gz enterprise: <CILIUM-ENTERPRISE-OLM-MANIFESTS-TARGZ-URL>`

The source for the OLM manifests. The component selects the opensource or enterprise field based on the value of component parameter release. The component doesn’t provide the URL of the Cilium Enterprise OLM manifests .tar.gz archive. Users must provide the URL themselves in their Project Syn configuration hierarchy.

The component default is an invalid string (<CILIUM-ENTERPRISE-OLM-MANIFESTS-TARGZ-URL>) instead of ~ to make the Kapitan error message somewhat useful when the user hasn’t reconfigured the chart repository.

Example

parameters:
  cilium:
    olm:
      source:
        enterprise: https://cilium-ee.example.com/downloads/v${cilium:olm:version}/cilium-ee-${cilium:olm:full_version}.tar.gz (1)

1	The example configuration uses Reclass references to construct URL parts containing the desired version. The component explicitly provides separate parameters for the OLM minor version and patchlevel.

`olm.version`

type	string
example	`1.10`
default	See `class/defaults.yml`

The minor version of the OLM release to download.

`olm.patchlevel`

type	string
example	`4`
default	See `class/defaults.yml`

The patch level of the OLM release to download.

`olm.full_version`

type	string
default	`${cilium:olm:version}.${cilium:olm:patchlevel}`

The complete version of the OLM release to download. By default, the component constructs the value for this parameter from parameters version and patchlevel.

`olm.resources`

type	object
default	See `class/defaults.yml`

The resource requests and limits for the Cilium OLM Deployment.

`olm.log_level`

type	string
default	`info`

Zap log level for the OLM operator.

`cilium_helm_values`

type	object
default	See `class/defaults.yml`

The configuration values of the underlying Cilium helm chart. See Reference.

`egress_gateway`

This section allows users to configure the [Cilium EgressGatewayPolicy] feature.

`egress_gateway.enabled`

type	boolean
default	`false`

This parameter allows users to set all the configurations necessary to enable the egress gateway policy feature through a single parameter. The parameter sets the following Helm values:

egressGateway:
  enabled: true
bpf:
  masquerade: true
l7Proxy: false

Notably, the L7 proxy feature is disabled by default when egress gateway policies are enabled. This is recommended by the Cilium documentation, see also the upstream documentation.

`egress_gateway.policies`

type	object
default	`{}`

This parameter allows users to deploy CiliumEgressGatewayPolicy resources. Each key-value pair in the parameter is converted to a CiliumEgressGatewayPolicy resource. Entries can be removed by setting the value to null.

Example

egress_gateway:
  policies:
    all-example:
      metadata:
        annotations:
          syn.tools/description: |
            Route all egress traffic from example-namespace through
            203.0.113.100.
      spec:
        destinationCIDRs:
          - 0.0.0.0/0
        egressGroups:
          - nodeSelector:
              matchLabels:
                node-role.kubernetes.io/infra: ""
            egressIP: 203.0.113.100
        selectors:
          - podSelector:
              matchLabels:
                io.kubernetes.pod.namespace: example-namespace
    removed: null

The component configuration shown above is rendered as follows by the component:

apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  annotations:
    syn.tools/description: |
      Route all egress traffic from example-namespace through
      203.0.113.100.
  labels:
    name: all-example-namespace
  name: all-example-namespace
spec:
  destinationCIDRs:
    - 0.0.0.0/0
  egressGroups:
    - egressIP: 203.0.113.100
      nodeSelector:
        matchLabels:
          node-role.kubernetes.io/infra: ''
  selectors:
    - podSelector:
        matchLabels:
          io.kubernetes.pod.namespace: example-namespace

`egress_gateway.generate_shadow_ranges_configmap`

type	boolean
default	`false`

When this parameter is set to true, the component will deploy a ConfigMap which is suitable for the systemd unit which creates dummy interfaces managed by component openshift4-nodes. Additionally, the component will deploy one or more DaemonSets (depending on the contents of egress_gateway.egress_ip_ranges) to ensure the Kubelets on all nodes where egress interfaces need to be created can access the ConfigMap.

See also the documentation for parameter egressInterfaces in openshift4-nodes.

`egress_gateway.egress_ip_ranges`

type	object
default	`{}`

This parameter allows users to configure CiliumEgressGatewayPolicy resources which assign a single egress IP to a namespace according to the design selected in Floating egress IPs with Cilium on cloudscale.

Each entry in the parameter is intended to describe a group of dummy interfaces that can be used in CiliumEgressGatewayPolicy resources. The component expects that each value is an object with fields egress_range, node_selector, namespace_egress_ips, and shadow_ranges.

Field shadow_ranges is optional, see the section on shadow ranges for more details.

Prerequisites

The component expects that the key for each entry matches the prefix of the dummy interface names that are assigned the shadow IPs which map to the egress IP range defined in egress_range. To expand, the component expects that each node matching the selector in node_selector has a dummy interfaces which is named <prefix>_<i> for \$i \in [0, n)\$ where \$n\$ is the number of IPs contained in the specified egress range.

Additionally, the component expects that the network environment of the cluster ensures that all traffic which originates from the IPs assigned to the dummy interfaces on each node is mapped to the IPs in the range given in egress_range. The details of the mapping are left to the operator of the cluster’s network environment, but the component expects that traffic that originates from the IPs assigned to the same dummy interface on different nodes is mapped to a single egress IP.

We recommend that cluster operators allocate a shadow egress IP range of the same size as the egress IP range specified in field egress_range for each node that matches the node selector defined in node_selector.

For example, a cluster operator could select shadow IP CIDRs 192.0.2.32/27, 192.0.2.64/27 and 192.0.2.96/27 for an egress IP CIDR of 192.0.2.0/27 if 3 nodes in the cluster match the node selector.

In this case, the operator would need to ensure that traffic originating from each shadow IP CIDR is mapped to the egress CIDR. One option to realize this mapping are iptables NETMAP rules on the cluster’s default gateway:

iptables -t nat -A POSTROUTING -j NETMAP -s 192.0.2.32/27 --to 192.0.2.0/27 -p all
iptables -t nat -A POSTROUTING -j NETMAP -s 192.0.2.64/27 --to 192.0.2.0/27 -p all
iptables -t nat -A POSTROUTING -j NETMAP -s 192.0.2.96/27 --to 192.0.2.0/27 -p all

This approach assumes that the default gateway has suitable routes to ensure that traffic to 192.0.2.0/25 is routed to the nodes matching the node selector. The easiest option to do so is to define a link-local route for 192.0.2.0/25 on the network interface of the default gateway which is attached to the L2 network of the cluster nodes.

Policy generation

The component will generate one CiliumEgressGatewayPolicy for each key-value pair in field namespace_egress_ips for each egress range.

The compilation will abort with an error if the same namespace appears in multiple egress range definitions.

The component doesn’t enforce that different egress ranges are non-overlapping.

The component expects that keys in field namespace_egress_ips are namespace names. Additionally, the component expects that values in that field are IPs in the defined egress IP range.

The component allows users to assign the same egress IP to multiple namespaces.

The component expects that the value of egress_range has format 192.0.2.32-192.0.2.63. If the range isn’t given in the expected format or if the component detects that the given range is empty (for example if the first IP is larger than the last IP) compilation is aborted with an error. Additionally, the component also aborts compilation with an error if an egress IP that’s assigned to a namespace is outside the specified egress range.

Finally, entries in egress_ip_ranges and namespace_egress_ips can be removed by setting the value to null.

Shadow ranges

The component optionally deploys a ConfigMap which contains a map from node names to egress shadow IP ranges. This ConfigMap is only deployed when parameter egress_gateway.generate_shadow_ranges_configmap is set to true. If the shadow ranges ConfigMap is enabled, the component will check that each shadow range is the same size as the egress IP range it’s associated with. If there are any mismatches compilation aborts with an error.

If that parameter is set to true, the component will extract the node names and associated shadow IP ranges from field shadow_ranges in each entry of egress_ip_ranges. Ranges which don’t define this field (or where it’s set to null) are skipped.

This ConfigMap is intended to be consumed by a component running on the node (such as the systemd unit deployed by openshift4-nodes, see the documentation for parameter egressInterfaces in openshift4-nodes.

To ensure the Kubelet Kubeconfig on the nodes can be used to access this ConfigMap, the component also deploys a DaemonSet which mounts the ConfigMap for each unique node selector in all egress IP ranges. The DaemonSet pods establish a link between the node and the ConfigMap which is required in order for the Kubernetes Node authorization mode to allow the Kubelet to access the ConfigMap.

Example

egress_ip_ranges:
  egress_a:
    egress_range: '192.0.2.32 - 192.0.2.63'
    node_selector:
      node-role.kubernetes.io/infra: ''
    namespace_egress_ips:
      foo: 192.0.2.32
      bar: 192.0.2.61
    shadow_ranges:
      infra-foo: 198.51.100.32 - 198.51.100.63
      infra-bar: 198.51.100.64 - 198.51.100.95

The configuration shown above results in the two CiliumEgressGatewayPolicy resources shown below.

apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  annotations: (1)
    cilium.syn.tools/description: Generated policy to assign egress IP 192.0.2.61
      in egress range "egress_a" (192.0.2.32 - 192.0.2.63) to namespace bar.
    cilium.syn.tools/egress-ip: 192.0.2.61
    cilium.syn.tools/egress-range: 192.0.2.32 - 192.0.2.63
    cilium.syn.tools/interface-prefix: egress_a
    cilium.syn.tools/source-namespace: bar
  labels:
    name: bar
  name: bar (2)
spec:
  destinationCIDRs:
    - 0.0.0.0/0 (3)
  egressGroups:
    - interface: egress_a_29 (4)
      nodeSelector:
        matchLabels:
          node-role.kubernetes.io/infra: '' (5)
  selectors:
    - podSelector:
        matchLabels:
          io.kubernetes.pod.namespace: bar (6)
---
apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  annotations: (1)
    cilium.syn.tools/description: Generated policy to assign egress IP 192.0.2.32
      in egress range "egress_a" (192.0.2.32 - 192.0.2.63) to namespace foo.
    cilium.syn.tools/egress-ip: 192.0.2.32
    cilium.syn.tools/egress-range: 192.0.2.32 - 192.0.2.63
    cilium.syn.tools/interface-prefix: egress_a
    cilium.syn.tools/source-namespace: foo
  labels:
    name: foo
  name: foo (2)
spec:
  destinationCIDRs:
    - 0.0.0.0/0 (3)
  egressGroups:
    - interface: egress_a_0 (4)
      nodeSelector:
        matchLabels:
          node-role.kubernetes.io/infra: '' (5)
  selectors:
    - podSelector:
        matchLabels:
          io.kubernetes.pod.namespace: foo (6)

1	The component adds a number of annotations that contain the input data that was used to generate the policy. Additionally, the component adds an annotation that gives a human-readable description of the policy.
2	The namespace name is used as the name for the `CiliumEgressGatewayPolicy` resource.
3	The policy always masquerades all traffic from the namespace with the defined egress IP.
4	The policy uses the key in `egress_ip_ranges` and the offset of the selected egress IP into the range to generate the name of the dummy interface that’s expected to be assigned the shadow IPs that map to the egress IP.
5	The policy uses the node selector that’s defined in the parameter.
6	The policy always matches all traffic originating in the specified namespace.

Additionally, if parameter egress_gateway.generate_shadow_ranges_configmap is set to true, the ConfigMap and DaemonSet shown below are created.

apiVersion: v1
kind: ConfigMap
metadata:
  name: eip-shadow-ranges
data: (1)
  infra-foo: '{"egress_a":{"base":"198.51.100","from":"32","to":"63"}}'
  infra-bar: '{"egress_a":{"base":"198.51.100","from":"64","to":"95"}}'
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  annotations:
    cilium.syn.tools/description: Daemonset which ensures that the Kubelet on the
      nodes where the pods are scheduled can access configmap eip-shadow-ranges in
      namespace cilium.
  name: eip-shadow-ranges-e70e8
spec:
  selector:
    matchLabels:
      name: eip-shadow-ranges-e70e8
  template:
    metadata:
      labels:
        name: eip-shadow-ranges-e70e8
    spec:
      containers:
        - command:
            - /bin/sh
            - -c
            - 'trap : TERM INT; sleep infinity & wait'
          image: docker.io/bitnami/kubectl:1.29.4@sha256:f3cee231ead7d61434b7f418b6d10e1b43ff0d33dca43b341bcf3088fcaaa769
          imagePullPolicy: IfNotPresent
          name: sleep
          volumeMounts: (2)
            - mountPath: /data/eip-shadow-ranges
              name: shadow-ranges
      nodeSelector:
        node-role.kubernetes.io/infra: '' (3)
      volumes: (2)
        - configMap:
            name: eip-shadow-ranges
          name: shadow-ranges

1	The contents of the ConfigMap are generated in the format that the systemd unit managed by component `openshift4-nodes` expects.
2	The DaemonSet mounts the `eip-shadow-ranges` ConfigMap as a volume.
3	The DaemonSet is scheduled using the same node selector that’s used for the `CiliumEgressGatewayPolicy` resources

`helm_values`

type

object

The final merged Helm values provided to Helm when rendering the chart. This parameter is configured dynamically to match the expected structure for Helm chart parameters for the opensource and enterprise charts.

When the enterprise chart is used, the actual Cilium configuration is nested under key cilium in the Helm values and additional top-level keys exist to configure the Cilium Enterprise components. Users should prefer to configure Cilium itself through parameter cilium_helm_values.

Example

release_name: cilium

install_method: helm
release: opensource

cilium_helm_values:
  prometheus:
    serviceMonitor:
      enabled: false