|
9 | 9 | - [Non-Goals](#non-goals)
|
10 | 10 | - [Proposal](#proposal)
|
11 | 11 | - [User Stories](#user-stories)
|
12 |
| - - [Continuous Trace Collection](#continuous-trace-collection) |
13 |
| - - [Example Scenarios](#example-scenarios) |
| 12 | + - [Continuous trace collection](#continuous-trace-collection) |
| 13 | + - [Example scenarios](#example-scenarios) |
14 | 14 | - [Tracing Requests and Exporting Spans](#tracing-requests-and-exporting-spans)
|
| 15 | + - [Connected Traces with Nested Spans](#connected-traces-with-nested-spans) |
15 | 16 | - [Running the OpenTelemetry Collector](#running-the-opentelemetry-collector)
|
16 | 17 | - [Kubelet Configuration](#kubelet-configuration)
|
17 | 18 | - [Design Details](#design-details)
|
|
43 | 44 |
|
44 | 45 | Items marked with (R) are required *prior to targeting to a milestone / release*.
|
45 | 46 |
|
46 |
| -- [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR) |
47 |
| -- [ ] (R) KEP approvers have approved the KEP status as `implementable` |
48 |
| -- [ ] (R) Design details are appropriately documented |
49 |
| -- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input |
50 |
| -- [ ] (R) Graduation criteria is in place |
51 |
| -- [ ] (R) Production readiness review completed |
52 |
| -- [ ] Production readiness review approved |
53 |
| -- [ ] "Implementation History" section is up-to-date for milestone |
54 |
| -- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io] |
55 |
| -- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes |
| 47 | +- [X] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR) |
| 48 | +- [X] (R) KEP approvers have approved the KEP status as `implementable` |
| 49 | +- [X] (R) Design details are appropriately documented |
| 50 | +- [X] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input |
| 51 | +- [X] (R) Graduation criteria is in place |
| 52 | +- [X] (R) Production readiness review completed |
| 53 | +- [X] Production readiness review approved |
| 54 | +- [X] "Implementation History" section is up-to-date for milestone |
| 55 | +- [X] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io] |
| 56 | +- [X] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes |
56 | 57 |
|
57 | 58 | ## Summary
|
58 | 59 |
|
@@ -140,6 +141,14 @@ to generate spans for sampled incoming requests and propagate context with clien
|
140 | 141 |
|
141 | 142 | OpenTelemetry-Go provides the [propagation package](https://github.com/open-telemetry/opentelemetry-go/blob/main/propagation/propagation.go) with which you can add custom key-value pairs known as [baggage](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/baggage/api.md). Baggage data will be propagated across services within contexts.
|
142 | 143 |
|
| 144 | +### Connected Traces with Nested Spans |
| 145 | + |
| 146 | +Nested spans with top-level traces in the kubelet will connect CRI calls together. Nested spans will be created for the following: |
| 147 | +* Sync Loops (e.g. syncPod, eviction manager, various gc routines) where the kubelet initiates new work. |
| 148 | + * [top-level traces for pod sync and GC](https://github.com/kubernetes/kubernetes/pull/114504) |
| 149 | +* Incoming requests (exec, attach, port-forward, metrics endpoints, podresources) |
| 150 | +* Outgoing requests (CNI, CSI, device plugin, k8s API calls) |
| 151 | + |
143 | 152 | ### Running the OpenTelemetry Collector
|
144 | 153 |
|
145 | 154 | Although this proposal focuses on running the [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector), note that any
|
@@ -187,90 +196,47 @@ type TracingConfiguration struct {
|
187 | 196 |
|
188 | 197 | ### Test Plan
|
189 | 198 |
|
190 |
| -<!-- |
191 |
| -**Note:** *Not required until targeted at a release.* |
192 |
| -The goal is to ensure that we don't accept enhancements with inadequate testing. |
193 |
| -All code is expected to have adequate tests (eventually with coverage |
194 |
| -expectations). Please adhere to the [Kubernetes testing guidelines][testing-guidelines] |
195 |
| -when drafting this test plan. |
196 |
| -[testing-guidelines]: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md |
197 |
| ---> |
198 |
| - |
199 | 199 | [x] I/we understand the owners of the involved components may require updates to
|
200 | 200 | existing tests to make this code solid enough prior to committing the changes necessary
|
201 | 201 | to implement this enhancement.
|
202 | 202 |
|
203 | 203 | ##### Prerequisite testing updates
|
204 | 204 |
|
205 |
| -<!-- |
206 |
| -Based on reviewers feedback describe what additional tests need to be added prior |
207 |
| -implementing this enhancement to ensure the enhancements have also solid foundations. |
208 |
| ---> |
209 |
| - |
210 | 205 | An integration test will verify that spans exported by the kubelet match what is
|
211 | 206 | expected from the request. We will also add an integration test that verifies
|
212 | 207 | spans propagated from kubelet to API server match what is expected from the request.
|
213 | 208 |
|
214 | 209 | ##### Unit tests
|
215 | 210 |
|
216 |
| -<!-- |
217 |
| -In principle every added code should have complete unit test coverage, so providing |
218 |
| -the exact set of tests will not bring additional value. |
219 |
| -However, if complete unit test coverage is not possible, explain the reason of it |
220 |
| -together with explanation why this is acceptable. |
221 |
| ---> |
222 |
| - |
223 |
| -<!-- |
224 |
| -Additionally, for Alpha try to enumerate the core package you will be touching |
225 |
| -to implement this enhancement and provide the current unit coverage for those |
226 |
| -in the form of: |
227 |
| -- <package>: <date> - <current test coverage> |
228 |
| -The data can be easily read from: |
229 |
| -https://testgrid.k8s.io/sig-testing-canaries#ci-kubernetes-coverage-unit |
230 |
| -This can inform certain test coverage improvements that we want to do before |
231 |
| -extending the production code to implement this enhancement. |
232 |
| ---> |
233 |
| - |
234 | 211 | - `k8s.io/component-base/traces`: no test grid results - k8s.io/component-base/traces/config_test.go
|
235 | 212 |
|
236 | 213 | ##### Integration tests
|
237 | 214 |
|
238 |
| -<!-- |
239 |
| -This question should be filled when targeting a release. |
240 |
| -For Alpha, describe what tests will be added to ensure proper quality of the enhancement. |
241 |
| -For Beta and GA, add links to added tests together with links to k8s-triage for those tests: |
242 |
| -https://storage.googleapis.com/k8s-triage/index.html |
243 |
| ---> |
244 |
| - |
245 | 215 | An integration test will verify that spans exported by the kubelet match what is
|
246 | 216 | expected from the request. We will also add an integration test that verifies
|
247 | 217 | spans propagated from kubelet to API server match what is expected from the request.
|
248 | 218 |
|
249 | 219 | ##### e2e tests
|
250 | 220 |
|
251 |
| -<!-- |
252 |
| -This question should be filled when targeting a release. |
253 |
| -For Alpha, describe what tests will be added to ensure proper quality of the enhancement. |
254 |
| -For Beta and GA, add links to added tests together with links to k8s-triage for those tests: |
255 |
| -https://storage.googleapis.com/k8s-triage/index.html |
256 |
| -We expect no non-infra related flakes in the last month as a GA graduation criteria. |
257 |
| ---> |
258 |
| - |
259 | 221 | - A test with kubelet-tracing & apiserver-tracing enabled to ensure no issues are introduced, regardless
|
260 | 222 | of whether a tracing backend is configured.
|
261 | 223 |
|
262 | 224 | ### Graduation Requirements
|
263 | 225 |
|
264 | 226 | Alpha
|
265 | 227 |
|
266 |
| -- [] Implement tracing of incoming and outgoing gRPC, HTTP requests in the kubelet |
267 |
| -- [] Integration testing of tracing |
| 228 | +- [X] Implement tracing of incoming and outgoing gRPC, HTTP requests in the kubelet |
| 229 | +- [X] Integration testing of tracing |
268 | 230 |
|
269 | 231 | Beta
|
270 | 232 |
|
271 |
| -- [] Publish examples of how to use the OT Collector with kubernetes |
272 |
| -- [] Allow time for feedback |
| 233 | +- [X] OpenTelemetry reaches GA |
| 234 | +- [X] Publish examples of how to use the OT Collector with kubernetes |
| 235 | +- [X] Allow time for feedback |
| 236 | +- [] Add top level traces to connect spans in sync loops, incoming requests, and outgoing requests. |
| 237 | +- [] Unit/integration test to verify connected traces in kubelet. |
273 | 238 | - [] Revisit the format used to export spans.
|
| 239 | +- [] Parity with the old text-based Traces |
274 | 240 |
|
275 | 241 | GA
|
276 | 242 |
|
@@ -442,6 +408,7 @@ _This section must be completed when targeting beta graduation to a release._
|
442 | 408 | - 2022-07-22: KEP merged, targeted at Alpha in 1.24
|
443 | 409 | - 2022-03-29: KEP deemed not ready for Alpha in 1.24
|
444 | 410 | - 2022-06-09: KEP targeted at Alpha in 1.25
|
| 411 | +- 2023-01-09: KEP targeted at Beta in 1.27 |
445 | 412 |
|
446 | 413 | ## Drawbacks
|
447 | 414 |
|
|
0 commit comments