Merge pull request #44560 from sftim/20231228_merge_main

Merge main branch into dev-1.30

Author: k8s-ci-robot

content/en/blog/_posts/2021-10-05-nsa-cisa-hardening.md

@@ -17,7 +17,7 @@ and are in no way a direct recommendation from the Kubernetes community or autho
 
 USA's National Security Agency (NSA) and the Cybersecurity and Infrastructure
 Security Agency (CISA)
-released, "[Kubernetes Hardening Guidance](https://media.defense.gov/2021/Aug/03/2002820425/-1/-1/1/CTR_KUBERNETES%20HARDENING%20GUIDANCE.PDF)"
+released Kubernetes Hardening Guidance
 on August 3rd, 2021. The guidance details threats to Kubernetes environments
 and provides secure configuration guidance to minimize risk.
 
@@ -29,6 +29,14 @@ _Note_: This blog post is not a substitute for reading the guide. Reading the pu
 guidance is recommended before proceeding as the following content is 
 complementary.
 
+{{% pageinfo color="primary" %}}
+**Update, November 2023:**
+
+The National Security Agency (NSA) and the Cybersecurity and Infrastructure Security Agency (CISA) released the 1.0 version of the Kubernetes hardening guide in August 2021 and updated it based on industry feedback in March 2022 (version 1.1).
+
+The most recent version of the Kubernetes hardening guidance was released in August 2022 with corrections and clarifications. Version 1.2 outlines a number of recommendations for [hardening Kubernetes clusters](https://media.defense.gov/2022/Aug/29/2003066362/-1/-1/0/CTR_KUBERNETES_HARDENING_GUIDANCE_1.2_20220829.PDF).
+{{% /pageinfo %}}
+
 ## Introduction and Threat Model
 
 Note that the threats identified as important by the NSA/CISA, or the intended audience of this guidance, may be different from the threats that other enterprise users of Kubernetes consider important. This section 

content/en/blog/_posts/2023-12-20-contextual-logging-in-kubernetes-1-29.md

@@ -0,0 +1,147 @@
+---
+layout: blog
+title: "Contextual logging in Kubernetes 1.29: Better troubleshooting and enhanced logging"
+slug: contextual-logging-in-kubernetes-1-29
+date: 2023-12-20T09:30:00-08:00
+canonicalUrl: https://www.kubernetes.dev/blog/2023/12/20/contextual-logging/
+---
+
+**Authors**: [Mengjiao Liu](https://github.com/mengjiao-liu/) (DaoCloud), [Patrick Ohly](https://github.com/pohly) (Intel)
+
+On behalf of the [Structured Logging Working Group](https://github.com/kubernetes/community/blob/master/wg-structured-logging/README.md) 
+and [SIG Instrumentation](https://github.com/kubernetes/community/tree/master/sig-instrumentation#readme), 
+we are pleased to announce that the contextual logging feature
+introduced in Kubernetes v1.24 has now been successfully migrated to
+two components (kube-scheduler and kube-controller-manager)
+as well as some directories. This feature aims to provide more useful logs 
+for better troubleshooting of Kubernetes and to empower developers to enhance Kubernetes.
+
+## What is contextual logging?
+
+[Contextual logging](https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/3077-contextual-logging)
+is based on the [go-logr](https://github.com/go-logr/logr#a-minimal-logging-api-for-go) API. 
+The key idea is that libraries are passed a logger instance by their caller
+and use that for logging instead of accessing a global logger.
+The binary decides the logging implementation, not the libraries.
+The go-logr API is designed around structured logging and supports attaching
+additional information to a logger.
+
+This enables additional use cases:
+
+- The caller can attach additional information to a logger:
+  - [WithName](<https://pkg.go.dev/github.com/go-logr/logr#Logger.WithName>) adds a "logger" key with the names concatenated by a dot as value
+  - [WithValues](<https://pkg.go.dev/github.com/go-logr/logr#Logger.WithValues>) adds key/value pairs
+
+  When passing this extended logger into a function, and the function uses it
+  instead of the global logger, the additional information is then included 
+  in all log entries, without having to modify the code that generates the log entries. 
+  This is useful in highly parallel applications where it can become hard to identify 
+  all log entries for a certain operation, because the output from different operations gets interleaved.
+
+- When running unit tests, log output can be associated with the current test.
+  Then, when a test fails, only the log output of the failed test gets shown by go test.
+  That output can also be more verbose by default because it will not get shown for successful tests.
+  Tests can be run in parallel without interleaving their output.
+
+One of the design decisions for contextual logging was to allow attaching a logger as value to a `context.Context`.
+Since the logger encapsulates all aspects of the intended logging for the call,
+it is *part* of the context, and not just *using* it. A practical advantage is that many APIs
+already have a `ctx` parameter or can add one. This provides additional advantages, like being able to
+get rid of `context.TODO()` calls inside the functions.
+
+## How to use it
+
+The contextual logging feature is alpha starting from Kubernetes v1.24,
+so it requires the `ContextualLogging` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to be enabled.
+If you want to test the feature while it is alpha, you need to enable this feature gate
+on the `kube-controller-manager` and the `kube-scheduler`.
+
+For the `kube-scheduler`, there is one thing to note, in addition to enabling 
+the `ContextualLogging` feature gate, instrumentation also depends on log verbosity.
+To avoid slowing down the scheduler with the logging instrumentation for contextual logging added for 1.29,
+it is important to choose carefully when to add additional information:
+- At `-v3` or lower, only `WithValues("pod")` is used once per scheduling cycle.
+  This has the intended effect that all log messages for the cycle include the pod information. 
+  Once contextual logging is GA, "pod" key/value pairs can be removed from all log calls.
+- At `-v4` or higher, richer log entries get produced where `WithValues` is also used for the node (when applicable)
+  and `WithName` is used for the current operation and plugin.
+
+Here is an example that demonstrates the effect:
+> I1113 08:43:37.029524   87144 default_binder.go:53] "Attempting to bind pod to node" **logger="Bind.DefaultBinder"** **pod**="kube-system/coredns-69cbfb9798-ms4pq" **node**="127.0.0.1"
+
+The immediate benefit is that the operation and plugin name are visible in `logger`.
+`pod` and `node` are already logged as parameters in individual log calls in `kube-scheduler` code.
+Once contextual logging is supported by more packages outside of `kube-scheduler`, 
+they will also be visible there (for example, client-go). Once it is GA,
+log calls can be simplified to avoid repeating those values.
+
+In `kube-controller-manager`, `WithName` is used to add the user-visible controller name to log output, 
+for example:
+
+> I1113 08:43:29.284360   87141 graph_builder.go:285] "garbage controller monitor not synced: no monitors" **logger="garbage-collector-controller"**
+
+The `logger=”garbage-collector-controller”` was added by the `kube-controller-manager` core
+when instantiating that controller and appears in all of its log entries - at least as long as the code
+that it calls supports contextual logging. Further work is needed to convert shared packages like client-go.
+
+## Performance impact
+
+Supporting contextual logging in a package, i.e. accepting a logger from a caller, is cheap. 
+No performance impact was observed for the `kube-scheduler`. As noted above, 
+adding `WithName` and `WithValues` needs to be done more carefully.
+
+In Kubernetes 1.29, enabling contextual logging at production verbosity (`-v3` or lower)
+caused no measurable slowdown for the `kube-scheduler` and is not expected for the `kube-controller-manager` either.
+At debug levels, a 28% slowdown for some test cases is still reasonable given that the resulting logs make debugging easier. 
+For details, see the [discussion around promoting the feature to beta](https://github.com/kubernetes/enhancements/pull/4219#issuecomment-1807811995).
+
+## Impact on downstream users
+Log output is not part of the Kubernetes API and changes regularly in each release,
+whether it is because developers work on the code or because of the ongoing conversion
+to structured and contextual logging.
+
+If downstream users have dependencies on specific logs, 
+they need to be aware of how this change affects them.
+
+## Further reading
+
+- Read the [Contextual Logging in Kubernetes 1.24](https://www.kubernetes.dev/blog/2022/05/25/contextual-logging/) article.
+- Read the [KEP-3077: contextual logging](https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/3077-contextual-logging).
+
+## Get involved
+
+If you're interested in getting involved, we always welcome new contributors to join us.
+Contextual logging provides a fantastic opportunity for you to contribute to Kubernetes development and make a meaningful impact.
+By joining [Structured Logging WG](https://github.com/kubernetes/community/tree/master/wg-structured-logging),
+you can actively participate in the development of Kubernetes and make your first contribution.
+It's a great way to learn and engage with the community while gaining valuable experience.
+
+We encourage you to explore the repository and familiarize yourself with the ongoing discussions and projects. 
+It's a collaborative environment where you can exchange ideas, ask questions, and work together with other contributors.
+
+If you have any questions or need guidance, don't hesitate to reach out to us 
+and you can do so on our [public Slack channel](https://kubernetes.slack.com/messages/wg-structured-logging). 
+If you're not already part of that Slack workspace, you can visit [https://slack.k8s.io/](https://slack.k8s.io/)
+for an invitation.
+
+We would like to express our gratitude to all the contributors who provided excellent reviews, 
+shared valuable insights, and assisted in the implementation of this feature (in alphabetical order):
+
+- Aldo Culquicondor ([alculquicondor](https://github.com/alculquicondor))
+- Andy Goldstein ([ncdc](https://github.com/ncdc))
+- Feruzjon Muyassarov ([fmuyassarov](https://github.com/fmuyassarov))
+- Freddie ([freddie400](https://github.com/freddie400))
+- JUN YANG ([yangjunmyfm192085](https://github.com/yangjunmyfm192085))
+- Kante Yin ([kerthcet](https://github.com/kerthcet))
+- Kiki ([carlory](https://github.com/carlory))
+- Lucas Severo Alve ([knelasevero](https://github.com/knelasevero))
+- Maciej Szulik ([soltysh](https://github.com/soltysh))
+- Mengjiao Liu ([mengjiao-liu](https://github.com/mengjiao-liu))
+- Naman Lakhwani ([Namanl2001](https://github.com/Namanl2001))
+- Oksana Baranova ([oxxenix](https://github.com/oxxenix))
+- Patrick Ohly ([pohly](https://github.com/pohly))
+- songxiao-wang87 ([songxiao-wang87](https://github.com/songxiao-wang87))
+- Tim Allclai ([tallclair](https://github.com/tallclair))
+- ZhangYu ([Octopusjust](https://github.com/Octopusjust))
+- Ziqi Zhao ([fatsheep9146](https://github.com/fatsheep9146))
+- Zac ([249043822](https://github.com/249043822))

content/en/case-studies/vsco/index.html

@@ -18,7 +18,7 @@
 
 <h2>Challenge</h2>
 
-<p>After moving from <a href="https://www.rackspace.com/">Rackspace</a> to <a href="https://aws.amazon.com/">AWS</a> in 2015, <a href="https://vsco.co/">VSCO</a> began building <a href="https://nodejs.org/en/">Node.js</a> and <a href="https://golang.org/">Go</a> microservices in addition to running its <a href="http://php.net/">PHP monolith</a>. The team containerized the microservices using <a href="https://www.docker.com/">Docker</a>, but "they were all in separate groups of <a href="https://aws.amazon.com/ec2/">EC2</a> instances that were dedicated per service," says Melinda Lu, Engineering Manager for the Machine Learning Team. Adds Naveen Gattu, Senior Software Engineer on the Community Team: "That yielded a lot of wasted resources. We started looking for a way to consolidate and be more efficient in the AWS EC2 instances."</p>
+<p>After moving from <a href="https://www.rackspace.com/">Rackspace</a> to <a href="https://aws.amazon.com/">AWS</a> in 2015, <a href="https://vsco.co/">VSCO</a> began building <a href="https://nodejs.org/en/">Node.js</a> and <a href="https://go.dev/">Go</a> microservices in addition to running its <a href="http://php.net/">PHP monolith</a>. The team containerized the microservices using <a href="https://www.docker.com/">Docker</a>, but "they were all in separate groups of <a href="https://aws.amazon.com/ec2/">EC2</a> instances that were dedicated per service," says Melinda Lu, Engineering Manager for the Machine Learning Team. Adds Naveen Gattu, Senior Software Engineer on the Community Team: "That yielded a lot of wasted resources. We started looking for a way to consolidate and be more efficient in the AWS EC2 instances."</p>
 
 <h2>Solution</h2>
 

content/en/docs/concepts/cluster-administration/addons.md

@@ -79,6 +79,9 @@ installation instructions. The list does not try to be exhaustive.
   Pods and non-Kubernetes environments with visibility and security monitoring.
 * [Romana](https://github.com/romana) is a Layer 3 networking solution for pod
   networks that also supports the [NetworkPolicy](/docs/concepts/services-networking/network-policies/) API.
+* [Spiderpool](https://github.com/spidernet-io/spiderpool) is an underlay and RDMA
+  networking solution for Kubernetes. Spiderpool is supported on bare metal, virtual machines,
+  and public cloud environments.
 * [Weave Net](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/)
   provides networking and network policy, will carry on working on both sides
   of a network partition, and does not require an external database.

content/en/docs/concepts/configuration/manage-resources-containers.md

@@ -116,8 +116,13 @@ runs on a single-core, dual-core, or 48-core machine.
 
 {{< note >}}
 Kubernetes doesn't allow you to specify CPU resources with a precision finer than
-`1m`. Because of this, it's useful to specify CPU units less than `1.0` or `1000m` using
-the milliCPU form; for example, `5m` rather than `0.005`.
+`1m` or `0.001` CPU. To avoid accidentally using an invalid CPU quantity, it's useful to specify CPU units using the milliCPU form 
+instead of the decimal form when using less than 1 CPU unit. 
+
+For example, you have a Pod that uses `5m` or `0.005` CPU and would like to decrease
+its CPU resources. By using the decimal form, it's harder to spot that `0.0005` CPU
+is an invalid value, while by using the milliCPU form, it's easier to spot that
+`0.5m` is an invalid value.
 {{< /note >}}
 
 ### Memory resource units {#meaning-of-memory}

content/en/docs/concepts/security/pod-security-standards.md

@@ -272,6 +272,10 @@ fail validation.
 					<li><code>net.ipv4.tcp_syncookies</code></li>
 					<li><code>net.ipv4.ping_group_range</code></li>
 					<li><code>net.ipv4.ip_local_reserved_ports</code> (since Kubernetes 1.27)</li>
+					<li><code>net.ipv4.tcp_keepalive_time</code> (since Kubernetes 1.29)</li>
+					<li><code>net.ipv4.tcp_fin_timeout</code> (since Kubernetes 1.29)</li>
+					<li><code>net.ipv4.tcp_keepalive_intvl</code> (since Kubernetes 1.29)</li>
+					<li><code>net.ipv4.tcp_keepalive_probes</code> (since Kubernetes 1.29)</li>
 				</ul>
 			</td>
 		</tr>