Skip to content

Commit ae52c6b

Browse files
RichiHjuliusv
RichiH
and
juliusv
authored
Add PRW blog post (#1943)
* Add PRW blog post Signed-off-by: Richard Hartmann <[email protected]> Co-authored-by: Julius Volz <[email protected]>
1 parent 0ceb4e4 commit ae52c6b

File tree

1 file changed

+139
-0
lines changed

1 file changed

+139
-0
lines changed

‎content/blog/2021-05-04-prometheus-conformance-remote-write-compliance.md‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
---
2+
title: "Prometheus Conformance Program: Remote Write Compliance Test Results"
3+
created_at: 2021-05-05
4+
kind: article
5+
author_name: Richard "RichiH" Hartmann
6+
---
7+
8+
As [announced by CNCF](https://www.cncf.io/blog/2021/05/03/announcing-the-intent-to-form-the-prometheus-conformance-program/) and by [ourselves](https://prometheus.io/blog/2021/05/03/introducing-prometheus-conformance-program/), we're starting a Prometheus conformance program.
9+
10+
To give everyone an overview of where the ecosystem is before running tests officially, we wanted to show off the newest addition to our happy little bunch of test suites: The Prometheus [Remote Write](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write) compliance test suite tests the sender part of the Remote Write protocol against our [specification](https://docs.google.com/document/d/1LPhVRSFkGNSuU1fBd81ulhsCPR4hkSZyyBj1SZ8fWOM).
11+
12+
During Monday's [PromCon](https://promcon.io/2021-online/), [Tom Wilkie](https://twitter.com/tom_wilkie) presented the test results from the time of recording a few weeks ago. In the live section, he already had an [update](https://docs.google.com/presentation/d/1RcN58LlS3V5tYCUsftqUvNuCpCsgGR2P7-GoH1MVL0Q/edit#slide=id.gd1789c7f7c_0_0). Two days later we have two more updates:
13+
The addition of the [observability pipeline tool Vector](https://github.com/prometheus/compliance/pull/24), as well as [new versions of existing systems](https://github.com/prometheus/compliance/pull/25).
14+
15+
So, without further ado, the current results in alphabetical order are:
16+
17+
| Sender | Version | Score
18+
|--------|---------|------
19+
| Grafana Agent | 0.13.1 | **100%**
20+
| Prometheus | 2.26.0 | **100%**
21+
| OpenTelemetry Collector | 0.26.0 | **41%**
22+
| Telegraf | 1.18.2 | **65%**
23+
| Timber Vector | 0.13.1 | **35%**
24+
| VictoriaMetrics Agent | 1.59.0 | **76%**
25+
26+
The raw results are:
27+
28+
````
29+
--- PASS: TestRemoteWrite/grafana (0.01s)
30+
--- PASS: TestRemoteWrite/grafana/Counter (10.02s)
31+
--- PASS: TestRemoteWrite/grafana/EmptyLabels (10.02s)
32+
--- PASS: TestRemoteWrite/grafana/Gauge (10.02s)
33+
--- PASS: TestRemoteWrite/grafana/Headers (10.02s)
34+
--- PASS: TestRemoteWrite/grafana/Histogram (10.02s)
35+
--- PASS: TestRemoteWrite/grafana/HonorLabels (10.02s)
36+
--- PASS: TestRemoteWrite/grafana/InstanceLabel (10.02s)
37+
--- PASS: TestRemoteWrite/grafana/Invalid (10.02s)
38+
--- PASS: TestRemoteWrite/grafana/JobLabel (10.02s)
39+
--- PASS: TestRemoteWrite/grafana/NameLabel (10.02s)
40+
--- PASS: TestRemoteWrite/grafana/Ordering (26.12s)
41+
--- PASS: TestRemoteWrite/grafana/RepeatedLabels (10.02s)
42+
--- PASS: TestRemoteWrite/grafana/SortedLabels (10.02s)
43+
--- PASS: TestRemoteWrite/grafana/Staleness (10.01s)
44+
--- PASS: TestRemoteWrite/grafana/Summary (10.01s)
45+
--- PASS: TestRemoteWrite/grafana/Timestamp (10.01s)
46+
--- PASS: TestRemoteWrite/grafana/Up (10.02s)
47+
--- PASS: TestRemoteWrite/prometheus (0.01s)
48+
--- PASS: TestRemoteWrite/prometheus/Counter (10.02s)
49+
--- PASS: TestRemoteWrite/prometheus/EmptyLabels (10.02s)
50+
--- PASS: TestRemoteWrite/prometheus/Gauge (10.02s)
51+
--- PASS: TestRemoteWrite/prometheus/Headers (10.02s)
52+
--- PASS: TestRemoteWrite/prometheus/Histogram (10.02s)
53+
--- PASS: TestRemoteWrite/prometheus/HonorLabels (10.02s)
54+
--- PASS: TestRemoteWrite/prometheus/InstanceLabel (10.02s)
55+
--- PASS: TestRemoteWrite/prometheus/Invalid (10.02s)
56+
--- PASS: TestRemoteWrite/prometheus/JobLabel (10.02s)
57+
--- PASS: TestRemoteWrite/prometheus/NameLabel (10.03s)
58+
--- PASS: TestRemoteWrite/prometheus/Ordering (24.99s)
59+
--- PASS: TestRemoteWrite/prometheus/RepeatedLabels (10.02s)
60+
--- PASS: TestRemoteWrite/prometheus/SortedLabels (10.02s)
61+
--- PASS: TestRemoteWrite/prometheus/Staleness (10.02s)
62+
--- PASS: TestRemoteWrite/prometheus/Summary (10.02s)
63+
--- PASS: TestRemoteWrite/prometheus/Timestamp (10.02s)
64+
--- PASS: TestRemoteWrite/prometheus/Up (10.02s)
65+
--- FAIL: TestRemoteWrite/otelcollector (0.00s)
66+
--- FAIL: TestRemoteWrite/otelcollector/Counter (10.01s)
67+
--- FAIL: TestRemoteWrite/otelcollector/Histogram (10.01s)
68+
--- FAIL: TestRemoteWrite/otelcollector/InstanceLabel (10.01s)
69+
--- FAIL: TestRemoteWrite/otelcollector/Invalid (10.01s)
70+
--- FAIL: TestRemoteWrite/otelcollector/JobLabel (10.01s)
71+
--- FAIL: TestRemoteWrite/otelcollector/Ordering (13.54s)
72+
--- FAIL: TestRemoteWrite/otelcollector/RepeatedLabels (10.01s)
73+
--- FAIL: TestRemoteWrite/otelcollector/Staleness (10.01s)
74+
--- FAIL: TestRemoteWrite/otelcollector/Summary (10.01s)
75+
--- FAIL: TestRemoteWrite/otelcollector/Up (10.01s)
76+
--- PASS: TestRemoteWrite/otelcollector/EmptyLabels (10.01s)
77+
--- PASS: TestRemoteWrite/otelcollector/Gauge (10.01s)
78+
--- PASS: TestRemoteWrite/otelcollector/Headers (10.01s)
79+
--- PASS: TestRemoteWrite/otelcollector/HonorLabels (10.01s)
80+
--- PASS: TestRemoteWrite/otelcollector/NameLabel (10.01s)
81+
--- PASS: TestRemoteWrite/otelcollector/SortedLabels (10.01s)
82+
--- PASS: TestRemoteWrite/otelcollector/Timestamp (10.01s)
83+
--- FAIL: TestRemoteWrite/telegraf (0.01s)
84+
--- FAIL: TestRemoteWrite/telegraf/EmptyLabels (14.60s)
85+
--- FAIL: TestRemoteWrite/telegraf/HonorLabels (14.61s)
86+
--- FAIL: TestRemoteWrite/telegraf/Invalid (14.61s)
87+
--- FAIL: TestRemoteWrite/telegraf/RepeatedLabels (14.61s)
88+
--- FAIL: TestRemoteWrite/telegraf/Staleness (14.59s)
89+
--- FAIL: TestRemoteWrite/telegraf/Up (14.60s)
90+
--- PASS: TestRemoteWrite/telegraf/Counter (14.61s)
91+
--- PASS: TestRemoteWrite/telegraf/Gauge (14.60s)
92+
--- PASS: TestRemoteWrite/telegraf/Headers (14.61s)
93+
--- PASS: TestRemoteWrite/telegraf/Histogram (14.61s)
94+
--- PASS: TestRemoteWrite/telegraf/InstanceLabel (14.61s)
95+
--- PASS: TestRemoteWrite/telegraf/JobLabel (14.61s)
96+
--- PASS: TestRemoteWrite/telegraf/NameLabel (14.60s)
97+
--- PASS: TestRemoteWrite/telegraf/Ordering (14.61s)
98+
--- PASS: TestRemoteWrite/telegraf/SortedLabels (14.61s)
99+
--- PASS: TestRemoteWrite/telegraf/Summary (14.60s)
100+
--- PASS: TestRemoteWrite/telegraf/Timestamp (14.61s)
101+
--- FAIL: TestRemoteWrite/vector (0.01s)
102+
--- FAIL: TestRemoteWrite/vector/Counter (10.02s)
103+
--- FAIL: TestRemoteWrite/vector/EmptyLabels (10.01s)
104+
--- FAIL: TestRemoteWrite/vector/Headers (10.02s)
105+
--- FAIL: TestRemoteWrite/vector/HonorLabels (10.02s)
106+
--- FAIL: TestRemoteWrite/vector/InstanceLabel (10.02s)
107+
--- FAIL: TestRemoteWrite/vector/Invalid (10.02s)
108+
--- FAIL: TestRemoteWrite/vector/JobLabel (10.01s)
109+
--- FAIL: TestRemoteWrite/vector/Ordering (13.01s)
110+
--- FAIL: TestRemoteWrite/vector/RepeatedLabels (10.02s)
111+
--- FAIL: TestRemoteWrite/vector/Staleness (10.02s)
112+
--- FAIL: TestRemoteWrite/vector/Up (10.02s)
113+
--- PASS: TestRemoteWrite/vector/Gauge (10.02s)
114+
--- PASS: TestRemoteWrite/vector/Histogram (10.02s)
115+
--- PASS: TestRemoteWrite/vector/NameLabel (10.02s)
116+
--- PASS: TestRemoteWrite/vector/SortedLabels (10.02s)
117+
--- PASS: TestRemoteWrite/vector/Summary (10.02s)
118+
--- PASS: TestRemoteWrite/vector/Timestamp (10.02s)
119+
--- FAIL: TestRemoteWrite/vmagent (0.01s)
120+
--- FAIL: TestRemoteWrite/vmagent/Invalid (20.66s)
121+
--- FAIL: TestRemoteWrite/vmagent/Ordering (22.05s)
122+
--- FAIL: TestRemoteWrite/vmagent/RepeatedLabels (20.67s)
123+
--- FAIL: TestRemoteWrite/vmagent/Staleness (20.67s)
124+
--- PASS: TestRemoteWrite/vmagent/Counter (20.67s)
125+
--- PASS: TestRemoteWrite/vmagent/EmptyLabels (20.64s)
126+
--- PASS: TestRemoteWrite/vmagent/Gauge (20.66s)
127+
--- PASS: TestRemoteWrite/vmagent/Headers (20.64s)
128+
--- PASS: TestRemoteWrite/vmagent/Histogram (20.66s)
129+
--- PASS: TestRemoteWrite/vmagent/HonorLabels (20.66s)
130+
--- PASS: TestRemoteWrite/vmagent/InstanceLabel (20.66s)
131+
--- PASS: TestRemoteWrite/vmagent/JobLabel (20.66s)
132+
--- PASS: TestRemoteWrite/vmagent/NameLabel (20.66s)
133+
--- PASS: TestRemoteWrite/vmagent/SortedLabels (20.66s)
134+
--- PASS: TestRemoteWrite/vmagent/Summary (20.66s)
135+
--- PASS: TestRemoteWrite/vmagent/Timestamp (20.67s)
136+
--- PASS: TestRemoteWrite/vmagent/Up (20.66s)
137+
````
138+
139+
We'll work more on improving our test suites, both by adding more tests & by adding new test targets. If you want to help us, consider adding more of [our list of Remote Write integrations](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage).

0 commit comments

Comments
 (0)