Parameters

The parent key for all of the following parameters is prometheus.

A shared instance of the following kube-prometheus component can be configured:

  • prometheus_operator

Multiple instances of the following components can be configured:

  • prometheus

  • alertmanager

  • grafana

  • nodeExporter

  • blackboxExporter

  • kubernetesControlPlane

  • prometheusAdapter

  • kubeStateMetrics

kubernetes_version

type

string
default

1.21
example 
kubernetes_version: '${dynamic_facts:kubernetesVersion:major}.${dynamic_facts:kubernetesVersion:minor}'

The Kubernetes version of the cluster is used to consume the corresponding kube-prometheus version.

registries

type

dictionary
default

See class/defaults.yml
example 
registries:
  docker.io: 'docker-io.proxy.com'
  quay.io: 'quay-io.proxy.com'
  k8s.gcr.io: 'k8s-gcr-io.proxy.com'

A dictionary of registries to use for the Prometheus components. Allows to map upstream images to a custom registry. Useful for air gapped clusters.

images

type

dictionary
default

See class/defaults.yml

A dictionary of images used by this component. Doesn’t include the kube-prometheus upstream images.

namespaces

type

dict
default

{}
example 
namespaces:
  my-stack:
    labels:
      stack: my-stack
    annotations:
      description: Used to monitor my stack.

The namespaces parameter can be used to create a list of namespaces for the Prometheus stacks to be deployed. The key is used as the name of the namespace and the value is used as the namespace metadata. Namespaces can be removed from the hierarchy by setting the value to null.

The prometheus-operator deployed by this component will only reconcile resources in these namespaces.

secrets

type

dict
default

{}
example 
secrets:
  my-secret:
    stringData:
      foo: bar
      long: |
        A long
        multiline secret
  object-storage:
    metadata:
      namespace: foo-ns
    stringData:
      thanos:
        type: S3
        config:
          bucket: thanos
          endpoint: s3.example.com
          access_key: '?{vaultkv:${cluster:tenant}/${cluster:name}/s3_access_key}'
          secret_key: '?{vaultkv:${cluster:tenant}/${cluster:name}/s3_secret_key}'

The secrets parameter can be used to create a list of arbitrary secrets. The key is used as the name of the secret and the value is transformed into Secret resources.

When specifying an object as value for a key in stringData, the component will render the object as JSON and add it as a string. This can be very helpful if you need so specify JSON or YAML configuration through a secret, for example for Thanos object-storage configuration.

addons

type

array
default

[]
example 
addons:
  - podsecuritypolicies

A list of addons to configure the Prometheus stack. Upstream addons can be found in the addons folder of the kube-prometheus project. Addons provided by this component can be found in the component/addons folder.

addon_configs

type

dict

A list of configuration options for various addons. For supported configuration options, see the documentation of the respective addon. The key is set to the addon name (replace hyphens with underscore - eg. for addon disable-alerts the key is disable_alerts)

addon_configs.alert_routing_discovery

type

dict
default 
debug_config_map: false
team_receiver_format: team_default_%s
additional_alert_matchers: []
prepend_routes: []
append_routes: []
fallback_team: null

This parameter configures the alert-routing-discovery addon which enables users to easily implement dynamic alert routing based on each Commodore component’s Project Syn team.

When the addon is enabled (referenced in component parameter addons), it injects a partial Alertmanager config into the Alertmanager of each monitoring stack managed by this component.

The addon expects that the user defines a receiver for each team that could potentially receive alerts with a name matching the format string defined in field team_receiver_format. The addon expects that this format string contains exactly one %s which will be replaced with the team identifier.

If field debug_config_map is set to true, the addon emits a configmap which contains debug information that can be helpful to investigate incorrect alert routing rules.

Field additional_alert_matchers enables users to provide additional matchers which are added to each of the dynamically defined routing rules. This can be used to further restrict which alerts are routed to teams.

Fields prepend_routes and append_routes enable users to add additional alert routing rules before and after the dynamic rules generated by the addon.

If field fallback_team is not null, it’s used as the default/fallback Project Syn team name for clusters which don’t set syn.owner and all alerts not assigned to a specific team are sent to that team’s receiver.

ingressNetworkPolicySource

type

dict
default 
namespaceSelector:
  matchLabels: {}
  podSelector: {}

A dict containing the namespaceSelector and podSelector to match the ingress controller pods. Required when configuring ingress for Grafana.

base

type

dict
default 
common:
  namespace: syn-prometheus
prometheus:
  enabled: false
  config: {}
  overrides: {}
alertmanager:
  enabled: false
  config: {}
  overrides: {}
grafana:
  enabled: false
  config: {}
  overrides: {}
  ingress:
    enabled: false
    host: ""
    annotations: {}
    tls: {}
    ingressClassName: ""
  persistence:
    enabled: false
    storageClass: ""
    size: ""
nodeExporter:
  enabled: false
  config: {}
  overrides: {}
  containers:
    nodeExporter:
      additionalArgs: []
blackboxExporter:
  enabled: false
  config: {}
  overrides: {}
kubernetesControlPlane:
  enabled: false
  config: {}
  overrides: {}
prometheusAdapter:
  enabled: false
  config: {}
  overrides: {}
kubeStateMetrics:
  enabled: false
  config: {}
  overrides: {}
  containers:
    kubeStateMetrics:
      additionalArgs: []
kubePrometheus:
  enabled: false
  config: {}
  overrides: {}

The base configuration shared by all instances. Can be overridden by the instance-specific configuration.

General rules and alerts are grouped in the kubePrometheus component.

alerts

type

object

This parameter holds configurations that are used by component library prometheus-alert-patching.libsonnet. The configurations are also used by component addons additional-rules and disable-alerts.

ignoreNames

type

list
default

See class/defaults.yml

A list of alert names to remove when rendering Prometheus rule groups.

customAnnotations

type

object
default

{}

This parameter allows users to apply custom annotations on individual alerts.

The object keys are expected to be strings and used as alert names. The object values are expected to be objects with string keys and values and are added to the target alert’s annotations.

Example

The following configuration will add (or overwrite) annotation runbook_url for alert PrometheusBadConfig

parameters:
  prometheus:
    alerts:
      customAnnotations:
        PrometheusBadConfig:
          runbook_url: https://hub.syn.tools/prometheus/runbooks/PrometheusBadConfig.html

defaultInstance

type

string
default

null
example

infra

The key of the default instance in the instances dict.

The specified instance is used for library functions when no instance is provided.

instances

type

dict
default

{}
example 
infra:
  common:
    namespace: monitoring-infra
  prometheus:
    enabled: true
    config:
      scrape_interval: 15s
      scrape_timeout: 10s
      evaluation_interval: 15s
  nodeExporter:
    enabled: true

Instances contains the configuration for each instance of the stack that should be deployed. base is used as the default values for the instance.

Every deployable component can be configured in its corresponding key.

Users can set fields in instances to null to disable stacks defined higher up in the hierarchy.

base.COMPONENT, instances.*.COMPONENT

type

dict
default 
prometheus:
  enabled: false
  config: {}
  overrides: {}
example 
prometheus:
  enabled: true
  config:
    scrape_interval: 15s
  overrides: {}

Every component can be configured with the following keys:

  • enabled: Whether the component should be deployed.

  • config: The configuration for how the component should be rendered. Warning: Configuring one component can have side effects on other components.

  • overrides: The configuration overrides for the component. Warning: The overrides are applied after the manifests are rendered. This means configuration side effects don’t apply and the configuration can contain invalid values.

config parameters can be found in the corresponding kube-prometheus library here. The easiest way to find the allowed parameters is to look at the local defaults variable. See the kube state metrics defaults as an example: kube-prometheus/components/kube-state-metrics.libsonnet

instances.*.(prometheus|alertmanager|grafana).networkPolicy.additionalIngressRules

type

list
default

null
example 
prometheus:
  networkPolicy:
    additionalIngressRules:
      - from:
          - namespaceSelector:
              matchLabels:
                kubernetes.io/metadata.name: kube-system
            podSelector:
              matchLabels:
                app: konnectivity-agent

Can be used to supply additional ingress rules to the NetworkPolicies for Prometheus, Grafana or Alertmanager. Configured rules will be appended to the existing NetworkPolicy and must adhere to the Kubernetes NetworkPolicy spec.

base.common, instances.common

type

dict
default

{}
example 
common:
  images:
    alertmanager: 'my.quaymirror.tld/prometheus/alertmanager:v${kube_prometheus.common.versions.alertmanager}'

Common parameters to be passed down to all components. The supported fields can be found here under the values.common key. At the time of writing this dict supported the following fields:

common.namespace

type

string
default

syn-prometheus
example 
common:
  namespace: my-monitoring-stack

The default namespace for all components.

common.platform

type

string
default

null
example 
common:
  platform: aws

Using a predefined mixin for a given platform. A list of supported platforms can be found here: github.com/prometheus-operator/kube-prometheus/blob/main/docs/customizations/platform-specific.md

common.ruleLabels

type

dict
default

{ role: 'alert-rules', prometheus: $.values.prometheus.name }

Default alert rule labels for all components.

common.versions

type

dict
default

See github.com/prometheus-operator/kube-prometheus/blob/main/jsonnet/kube-prometheus/versions.json
example 
common:
  versions:
    grafana: 8.0.2
    prometheus: 2.20.3

The default version to be used for the various components.

common.images

type

dict
default

See github.com/prometheus-operator/kube-prometheus/blob/main/jsonnet/kube-prometheus/main.libsonnet
example 
common:
  images:
    grafana: 'my.quaymirror.tld/grafana/grafana:${kube_prometheus.common.versions.grafana}'
    prometheus: 'my.quaymirror.tld/prometheus/prometheus:${kube_prometheus.common.versions.prometheus}'

The default image to be used for the various components.