Operator Managed PrometheusRules
This how-to describes how to manage PrometheusRules that are deployed by an operator. Using Espejote’s ManagedResources, we can copy, filter and patch the PrometheusRules on-the-fly inside the cluster.
What does component-openshift4-monitoring provide?
This component provides a JsonnetLibrary (from Espejote) which: * contains a global configuration for managing the PrometheusRules * contains a jsonnet function for filtering and patching the PrometheusRules
This way you only need a minimal template in your ManagedResource to parse the rules.
Global configuration
The global configuration contains the following fields:
openshift4_monitoring:
alerts:
ignoreGroups: [] (1)
ignoreNames: [] (2)
ignoreWarnings: [] (3)
customAnnotations: {} (4)
patchRules: [] (5)
includeNamespaces: [] (6)
excludeNamespaces: [] (7)
teamLabel: "" (8)
1 | A list of rule groups to ignore |
2 | A list of rule names to ignore |
3 | A list of rule names to ignore that have severity warning |
4 | A map of custom annotations to add to the rules |
5 | A map of rules with their patches |
6 | A list of namespaces to include in namespace-selectors, modifies the expression namespace=~"(openshift-.|kube-.|default)" |
7 | A list of namespaces to exclude from namespace-selectors, modifies the expression namespace=~"(openshift-.|kube-.|default)" |
8 | The team label to add to the rules |
How to use in your component
To use this in your component, you need to: * provide a component-specific configuration, which will be merged into the global configuration * create a minimal template for your ManagedResource * the necessary RBAC for the ManagedResource
Component-specific configuration
Create a JsonnetLibrary in your component, with the following format:
apiVersion: espejote.io/v1alpha1
kind: JsonnetLibrary
metadata:
labels:
app.kubernetes.io/name: openshift4-monitoring-rules
name: openshift4-monitoring-rules (1)
namespace: <YOUR_NAMESPACE> (2)
spec:
data:
config.json: |-
{
"ignoreGroups": [],
"ignoreNames": [],
"ignoreWarnings": [],
"customAnnotations": {},
"patchRules": {},
"teamLabel": ""
}
1 | The name of the JsonnetLibrary, can be anything but for ease of use with the components helper library use this name. |
2 | The namespace where the ManagedResource will be deployed, usually the same as the component. |
Above configuration will be merged with the global configuration in the rendering function. The component-specific configuration can be empty.
Minimal template for the ManagedResource
This component provides a library for generating the ManagedResource and all RBAC rules needed. see openshift4-monitoring-operator-rules.libsonnet |
Create a minimal template for your ManagedResource, with the following format:
apiVersion: espejote.io/v1alpha1
kind: ManagedResource
metadata:
name: <YOUR_NAME>
namespace: <YOUR_NAMESPACE>
spec:
serviceAccountRef:
name: openshift4-monitoring-rules
context:
- name: op_rules
resource:
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
labelSelector:
matchExpressions:
- key: espejote.io/created-by
operator: DoesNotExist (1)
- key: espejote.io/ignore
operator: DoesNotExist (2)
namespace: <YOUR_NAMESPACE>
triggers:
- name: op_rules
watchContextResource:
name: op_rules
- name: generated_rules
watchResource:
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
labelSelector:
matchExpressions:
- key: espejote.io/created-by
operator: In
values:
- openshift4-monitoring-rules
namespace: <YOUR_NAMESPACE>
template: |
local esp = import 'espejote.libsonnet';
local renderer = import 'lib/openshift4-monitoring-rules/renderer_v1.libsonnet'; (3)
local configGlobal = import 'lib/openshift4-monitoring-rules/config_v1.json'; (4)
local configComponent = import 'openshift4-monitoring-rules/config.json';
local opRules = esp.context().op_rules;
local inDelete(obj) = std.get(obj.metadata, 'deletionTimestamp', '') != '';
if std.member([ 'op_rules', 'generated_rules' ], esp.triggerName()) then (
// if the trigger is 'op_rules' or 'generated_rules', render single op_rule
local trigger = esp.triggerData();
local or = if esp.triggerName() == 'op_rules' then
trigger.resource
else
local cand = std.filter(function(r) r.metadata.name == trigger.resource.metadata.ownerReferences[0].name, opRules);
if std.length(cand) > 0 then cand[0];
if or != null && !inDelete(or) then [
renderer.process(or, configGlobal, configComponent)
]
)
else [
// if the trigger is not 'op_rules' or 'generated_rules', render all op_rules
renderer.process(or, configGlobal, configComponent),
for or in opRules
if !inDelete(or)
]
1 | Ignore PrometheusRules that are created by this ManagedResource |
2 | Ignore PrometheusRules that have the espejote.io/ignore label |
3 | The renderer is versioned, currently v1 is available |
4 | The global configuration is versioned, currently v1 is available |