Skip to content

Commit 28eca5a

Browse files
abrennan89abrennan89
committed
OBSDOCS-213: Loki reliability hardening reference docs
1 parent fcd2e96 commit 28eca5a

File tree

5 files changed

+70
-19
lines changed

5 files changed

+70
-19
lines changed

‎logging/cluster-logging-loki.adoc

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,26 @@ Loki is a horizontally scalable, highly available, multi-tenant log aggregation
1313
include::modules/loki-deployment-sizing.adoc[leveloffset=+1]
1414

1515
include::modules/cluster-logging-loki-deploy.adoc[leveloffset=+1]
16+
1617
////
1718
include::modules/logging-loki-restart-hardening.adoc[leveloffset=+1]
1819

1920
[role="_additional-resources"]
2021
.Additional resources
2122
* link:https://kubernetes.io/docs/concepts/workloads/pods/disruptions/#pod-disruption-budgets[Pod disruption budgets Kubernetes documentation]
23+
24+
include::modules/logging-loki-reliability-hardening.adoc[leveloffset=+1]
25+
26+
[role="_additional-resources"]
27+
.Additional resources
28+
* link:https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.24/#podantiaffinity-v1-core[`PodAntiAffinity` v1 core Kubernetes documentation]
29+
* link:https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity[Assigning Pods to Nodes Kubernetes documentation]
30+
31+
ifdef::openshift-enterprise[]
32+
* xref:../nodes/scheduling/nodes-scheduler-pod-affinity.adoc#nodes-scheduler-pod-affinity[Placing pods relative to other pods using affinity and anti-affinity rules]
33+
endif::[]
2234
////
35+
2336
include::modules/logging-loki-retention.adoc[leveloffset=+1]
2437

2538
include::modules/cluster-logging-forwarding-lokistack.adoc[leveloffset=+1]
@@ -29,8 +42,9 @@ include::modules/loki-rate-limit-errors.adoc[leveloffset=+2]
2942
[role="_additional-resources"]
3043
[id="additional-resources_cluster-logging-loki"]
3144
== Additional Resources
32-
* link:https://grafana.com/docs/loki/latest/logql/[Loki Query Language (LogQL) Documentation]
33-
* link:https://loki-operator.dev/docs/howto_connect_grafana.md/[Grafana Dashboard Documentation]
34-
* link:https://loki-operator.dev/docs/object_storage.md/[Loki Object Storage Documentation]
35-
* link:https://loki-operator.dev/docs/api.md/#loki-grafana-com-v1-IngestionLimitSpec[Loki Operator `IngestionLimitSpec` Documentation]
36-
* link:https://grafana.com/docs/loki/latest/operations/storage/schema/#changing-the-schema[Loki Storage Schema Documentation]
45+
* link:https://grafana.com/docs/loki/latest/get-started/components/[Loki components documentation]
46+
* link:https://grafana.com/docs/loki/latest/logql/[Loki Query Language (LogQL) documentation]
47+
* link:https://loki-operator.dev/docs/howto_connect_grafana.md/[Grafana Dashboard documentation]
48+
* link:https://loki-operator.dev/docs/object_storage.md/[Loki Object Storage documentation]
49+
* link:https://loki-operator.dev/docs/api.md/#loki-grafana-com-v1-IngestionLimitSpec[Loki Operator `IngestionLimitSpec` documentation]
50+
* link:https://grafana.com/docs/loki/latest/operations/storage/schema/#changing-the-schema[Loki Storage Schema documentation]

‎modules/logging-loki-reliability-hardening.adoc

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * logging/cluster-logging-loki.adoc
4+
5+
:_content-type: CONCEPT
6+
[id="logging-loki-reliability-hardening_{context}"]
7+
= Configuring Loki to tolerate node failure
8+
9+
In the {logging} 5.8 and later versions, the Loki Operator supports setting pod anti-affinity rules to request that pods of the same component are scheduled on different available nodes in the cluster.
10+
11+
include::snippets/about-pod-affinity.adoc[]
12+
13+
The Operator sets default, preferred `podAntiAffinity` rules for all Loki components, which includes the `compactor`, `distributor`, `gateway`, `indexGateway`, `ingester`, `querier`, `queryFrontend`, and `ruler` components.
14+
15+
You can override the preferred `podAntiAffinity` settings for Loki components by configuring required settings in the `requiredDuringSchedulingIgnoredDuringExecution` field:
16+
17+
.Example user settings for the ingester component
18+
[source,yaml]
19+
----
20+
apiVersion: loki.grafana.com/v1
21+
kind: LokiStack
22+
metadata:
23+
name: logging-loki
24+
namespace: openshift-logging
25+
spec:
26+
# ...
27+
template:
28+
ingester:
29+
podAntiAffinity:
30+
# ...
31+
requiredDuringSchedulingIgnoredDuringExecution: <1>
32+
- labelSelector:
33+
matchLabels: <2>
34+
app.kubernetes.io/component: ingester
35+
topologyKey: kubernetes.io/hostname
36+
# ...
37+
----
38+
<1> The stanza to define a required rule.
39+
<2> The key-value pair (label) that must be matched to apply the rule.

‎modules/nodes-scheduler-node-affinity-about.adoc

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -99,4 +99,3 @@ If you are using node affinity and node selectors in the same pod configuration,
9999
100100
* If you specify multiple `matchExpressions` associated with `nodeSelectorTerms`, then the pod can be scheduled onto a node only if all `matchExpressions` are satisfied.
101101
====
102-

‎nodes/scheduling/nodes-scheduler-pod-affinity.adoc

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -6,19 +6,7 @@ include::_attributes/common-attributes.adoc[]
66

77
toc::[]
88

9-
10-
11-
Affinity is a property of pods that controls the nodes on which they prefer to be scheduled. Anti-affinity is a property of pods
12-
that prevents a pod from being scheduled on a node.
13-
14-
In {product-title} _pod affinity_ and _pod anti-affinity_ allow you to constrain which nodes your pod is eligible to be scheduled on based on the key/value labels on other pods.
15-
16-
17-
18-
// The following include statements pull in the module files that comprise
19-
// the assembly. Include any combination of concept, procedure, or reference
20-
// modules required to cover the user story. You can also include other
21-
// assemblies.
9+
include::snippets/about-pod-affinity.adoc[]
2210

2311
include::modules/nodes-scheduler-pod-affinity-about.adoc[leveloffset=+1]
2412

‎snippets/about-pod-affinity.adoc

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
// Snippets included in the following assemblies and modules:
2+
//
3+
// * /nodes/scheduling/nodes-scheduler-pod-affinity.adoc
4+
// * /modules/logging-loki-reliability-hardening.adoc
5+
6+
:_content-type: SNIPPET
7+
8+
Affinity is a property of pods that controls the nodes on which they prefer to be scheduled. Anti-affinity is a property of pods
9+
that prevents a pod from being scheduled on a node.
10+
11+
In {product-title}, _pod affinity_ and _pod anti-affinity_ allow you to constrain which nodes your pod is eligible to be scheduled on based on the key-value labels on other pods.

0 commit comments

Comments
 (0)