Skip to content

Commit c5ba1de

Browse files
abrennan89abrennan89
committed
OBSDOCS-460: Clean up collector docs
1 parent 0de7107 commit c5ba1de

8 files changed

+98
-218
lines changed

‎logging/cluster-logging.adoc

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -46,10 +46,6 @@ include::modules/cluster-logging-export-fields.adoc[leveloffset=+2]
4646

4747
For information, see xref:../logging/cluster-logging-exported-fields.adoc#cluster-logging-exported-fields[About exporting fields].
4848

49-
include::modules/cluster-logging-about-collector.adoc[leveloffset=+2]
50-
51-
For information, see xref:../logging/log_collection_forwarding/cluster-logging-collector.adoc#cluster-logging-collector[Configuring the logging collector].
52-
5349
include::modules/cluster-logging-about-logstore.adoc[leveloffset=+2]
5450

5551
For information, see xref:../logging/config/cluster-logging-log-store.adoc#cluster-logging-log-store[Configuring the log store].
@@ -61,5 +57,3 @@ For information, see xref:../logging/config/cluster-logging-visualizer.adoc#clus
6157
include::modules/cluster-logging-eventrouter-about.adoc[leveloffset=+2]
6258

6359
For information, see xref:../logging/log_collection_forwarding/cluster-logging-eventrouter.adoc#cluster-logging-eventrouter[Collecting and storing Kubernetes events].
64-
65-
include::modules/cluster-logging-feature-reference.adoc[leveloffset=+1]

‎logging/log_collection_forwarding/log-forwarding.adoc

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,19 @@ include::_attributes/attributes-openshift-dedicated.adoc[]
77

88
toc::[]
99

10+
The Cluster Logging Operator deploys a collector based on the `ClusterLogForwarder` resource specification. There are two collector options supported by this Operator: the legacy Fluentd collector, and the Vector collector.
11+
12+
include::snippets/logging-fluentd-dep-snip.adoc[]
13+
14+
include::modules/about-log-collection.adoc[leveloffset=+1]
15+
16+
include::modules/logging-vector-fluentd-feature-comparison.adoc[leveloffset=+2]
17+
18+
include::modules/log-forwarding-collector-outputs.adoc[leveloffset=+2]
19+
20+
[id="log-forwarding-about-clf"]
21+
== Log forwarding
22+
1023
Administrators can create `ClusterLogForwarder` resources that specify which logs are collected, how they are transformed, and where they are forwarded to.
1124

1225
`ClusterLogForwarder` resources can be used up to forward container, infrastructure, and audit logs to specific endpoints within or outside of a cluster. Transport Layer Security (TLS) is supported so that log forwarders can be configured to send logs securely.

‎logging/v5_6/logging-5-6-reference.adoc

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,4 @@ include::_attributes/common-attributes.adoc[]
66

77
toc::[]
88

9-
:leveloffset: +1
10-
11-
include::modules/logging-feature-reference-5.6.adoc[]
12-
13-
include::modules/logging-5.6-api-ref.adoc[]
14-
15-
:leveloffset: -1
9+
include::modules/logging-5.6-api-ref.adoc[leveloffset=+1]

‎modules/about-log-collection.adoc

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * logging/log_collection_forwarding/log-forwarding.adoc
4+
5+
:_mod-docs-content-type: CONCEPT
6+
[id="about-log-collection_{context}"]
7+
= Log collection
8+
9+
The log collector is a daemon set that deploys pods to each {product-title} node to collect container and node logs.
10+
11+
By default, the log collector uses the following sources:
12+
13+
* System and infrastructure logs generated by journald log messages from the operating system, the container runtime, and {product-title}.
14+
* `/var/log/containers/*.log` for all container logs.
15+
16+
If you configure the log collector to collect audit logs, it collects them from `/var/log/audit/audit.log`.
17+
18+
The log collector collects the logs from these sources and forwards them internally or externally depending on your {logging} configuration.
19+
20+
[id="about-log-collectors-types_{context}"]
21+
== Log collector types
22+
23+
link:https://vector.dev/docs/about/what-is-vector/[Vector] is a log collector offered as an alternative to Fluentd for the {logging}.
24+
25+
You can configure which logging collector type your cluster uses by modifying the `ClusterLogging` custom resource (CR) `collection` spec:
26+
27+
.Example ClusterLogging CR that configures Vector as the collector
28+
[source,yaml]
29+
----
30+
apiVersion: "logging.openshift.io/v1"
31+
kind: ClusterLogging
32+
metadata:
33+
name: instance
34+
namespace: openshift-logging
35+
spec:
36+
collection:
37+
logs:
38+
type: vector
39+
vector: {}
40+
# ...
41+
----
42+
43+
[id="about-log-collectors-limitations_{context}"]
44+
== Log collection limitations
45+
46+
The container runtimes provide minimal information to identify the source of log messages: project, pod name, and container ID. This information is not sufficient to uniquely identify the source of the logs. If a pod with a given name and project is deleted before the log collector begins processing its logs, information from the API server, such as labels and annotations, might not be available. There might not be a way to distinguish the log messages from a similarly named pod and project or trace the logs to their source. This limitation means that log collection and normalization are considered _best effort_.
47+
48+
[IMPORTANT]
49+
====
50+
The available container runtimes provide minimal information to identify the source of log messages and do not guarantee unique individual log messages or that these messages can be traced to their source.
51+
====

‎modules/cluster-logging-about-collector.adoc

Lines changed: 0 additions & 34 deletions
This file was deleted.

‎modules/cluster-logging-feature-reference.adoc

Lines changed: 0 additions & 159 deletions
This file was deleted.

‎modules/log-forwarding-collector-outputs.adoc

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * logging/log_collection_forwarding/log-forwarding.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="log-forwarding-collector-outputs_{context}"]
7+
= Collector outputs
8+
9+
The following collector outputs are supported:
10+
11+
.Supported outputs
12+
[options="header"]
13+
|==========================================================
14+
| Feature | Fluentd | Vector
15+
| Elasticsearch v6-v8 | ✓ | ✓
16+
| Fluent forward | ✓ |
17+
| Syslog RFC3164 | ✓ | ✓ (Logging 5.7+)
18+
| Syslog RFC5424 | ✓ | ✓ (Logging 5.7+)
19+
| Kafka | ✓ | ✓
20+
| Cloudwatch | ✓ | ✓
21+
| Cloudwatch STS | ✓ | ✓
22+
| Loki | ✓ | ✓
23+
| HTTP | ✓ | ✓ (Logging 5.7+)
24+
| Google Cloud Logging | ✓ | ✓
25+
| Splunk | | ✓ (Logging 5.6+)
26+
|==========================================================

‎modules/logging-feature-reference-5.6.adoc renamed to ‎modules/logging-vector-fluentd-feature-comparison.adoc

Lines changed: 7 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,10 @@
1-
// Module is included in the following assemblies:
1+
// Module included in the following assemblies:
22
//
3-
:_mod-docs-content-type: REFERENCE
4-
[id="logging-5-6-collector-ref_{context}"]
5-
6-
= Collector features
3+
// * logging/log_collection_forwarding/log-forwarding.adoc
74

8-
include::snippets/logging-outputs-5.6-snip.adoc[]
5+
:_mod-docs-content-type: REFERENCE
6+
[id="logging-vector-fluentd-feature-comparison_{context}"]
7+
= Log collector features by type
98

109
.Log Sources
1110
[options="header"]
@@ -49,7 +48,7 @@ include::snippets/logging-outputs-5.6-snip.adoc[]
4948
| Loglevel Normalization | ✓ | ✓
5049
| JSON parsing | ✓ | ✓
5150
| Structured Index | ✓ | ✓
52-
| Multiline error detection | ✓ |
51+
| Multiline error detection | ✓ | ✓
5352
| Multicontainer / split indices | ✓ | ✓
5453
| Flatten labels | ✓ | ✓
5554
| CLF static labels | ✓ | ✓
@@ -81,6 +80,7 @@ include::snippets/logging-outputs-5.6-snip.adoc[]
8180
| Dashboard | ✓ | ✓
8281
| Alerts | ✓ |
8382
|=====================================================
83+
// alerts maybe needs updated for vector in 5.7+?
8484

8585
.Miscellaneous
8686
[options="header"]
@@ -95,8 +95,3 @@ include::snippets/logging-outputs-5.6-snip.adoc[]
9595
| Log event buffering | ✓ |
9696
| Disconnected Cluster | ✓ | ✓
9797
|===========================================================
98-
99-
100-
[role="_additional-resources"]
101-
.Additional resources
102-
* link:https://vector.dev/docs/about/what-is-vector/[Vector Documentation]

0 commit comments

Comments
 (0)