Skip to content

Commit 72b0a18

Browse files
ivanceaivancea
authored
Merge branch 'main' into ivancea-esql-text-formats
2 parents 40ad777 + c5f6b13 commit 72b0a18

24 files changed

+748
-38
lines changed

‎deploy-manage/cloud-connect.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
---
2+
applies_to:
3+
deployment:
4+
self:
5+
ece:
6+
eck:
7+
navigation_title: Cloud Connect
8+
---
9+
10+
# Cloud Connect
11+
12+
Cloud Connect enables you to use {{ecloud}} services in your self-managed cluster without having to install and maintain their infrastructure yourself. In this way, you can get faster access to new features while significantly reducing your operational overhead.
13+
14+
Cloud Connect is included with your [self-managed Enterprise license](https://www.elastic.co/subscriptions) and works with self-managed free trials for the duration of the trial period.
15+
16+
AutoOps is the first service available for use with Cloud Connect. More services are coming soon.
17+
18+
### AutoOps
19+
20+
[AutoOps](/deploy-manage/monitor/autoops.md) is a monitoring tool that helps you manage your cluster with real-time issue detection, performance recommendations, and resolution paths. By analyzing hundreds of {{es}} metrics, your configuration, and usage patterns, AutoOps recommends operational and monitoring insights that deliver real savings in administration time and hardware cost.
21+
22+
Learn how to set up and use [](/deploy-manage/monitor/autoops/cc-autoops-as-cloud-connected.md).
23+
24+
## FAQs
25+
26+
Find answers to your questions about Cloud Connect.
27+
28+
:::{dropdown} Does using Cloud Connect require additional payment?
29+
30+
$$$cc-payment$$$
31+
32+
No. Cloud Connect is included with your [self-managed Enterprise license](https://www.elastic.co/subscriptions) and works with self-managed free trials for the duration of the trial period.
33+
34+
However, you may incur additional costs for specific functions within a cloud connected service. For example, when sending metrics data from your cluster in a CSP region to AutoOps in {{ecloud}}, shipping costs will be determined by your agreement with that CSP.
35+
:::
36+
37+
:::{dropdown} Does using Cloud Connect consume ECU?
38+
39+
$$$cc-ecu$$$
40+
41+
No. The currently available functionality does not consume ECU, but this may change when more features are introduced.
42+
:::
43+
44+
:::{dropdown} Will my data be safe when using Cloud Connect?
45+
46+
$$$cc-data$$$
47+
48+
Yes. For AutoOps, {{agent}} only sends cluster metrics to {{ecloud}}, not the underlying data within your cluster. Learn more in [](/deploy-manage/monitor/autoops/cc-cloud-connect-autoops-faq.md).
49+
:::
50+
51+
:::{dropdown} Are more services going to be available with Cloud Connect?
52+
53+
$$$cc-more-services$$$
54+
55+
Yes. AutoOps is the first of many cloud connected services to come. The next planned service is the Elastic Inference Service (EIS), which will provide GPU-powered inference for use cases like semantic search and text embeddings.
56+
:::
57+

‎deploy-manage/deploy/cloud-on-k8s.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,10 +69,23 @@ This section outlines the supported Kubernetes and {{stack}} versions for ECK. C
6969

7070
ECK is compatible with the following Kubernetes distributions and related technologies:
7171

72+
::::{tab-set}
73+
74+
:::{tab-item} ECK 3.1
75+
* Kubernetes 1.29-1.33
76+
* OpenShift 4.15-4.19
77+
* Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), and Amazon Elastic Kubernetes Service (EKS)
78+
* Helm: {{eck_helm_minimum_version}}+
79+
:::
80+
81+
:::{tab-item} ECK 3.0
7282
* Kubernetes 1.28-1.32
7383
* OpenShift 4.14-4.18
7484
* Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), and Amazon Elastic Kubernetes Service (EKS)
7585
* Helm: {{eck_helm_minimum_version}}+
86+
:::
87+
88+
::::
7689

7790
ECK should work with all conformant **installers** listed in these [FAQs](https://github.com/cncf/k8s-conformance/blob/master/faq.md#what-is-a-distribution-hosted-platform-and-an-installer). Distributions include source patches and so may not work as-is with ECK.
7891

‎deploy-manage/deploy/cloud-on-k8s/configuration-fleet.md‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,8 +146,23 @@ By default, every reference targets all instances in your {{es}}, {{kib}} and {{
146146

147147
## Customize {{agent}} configuration [k8s-elastic-agent-fleet-configuration-custom-configuration]
148148

149-
In contrast to {{agents}} in standalone mode, the configuration is managed through {{fleet}}, and it cannot be defined through `config` or `configRef` elements.
149+
In contrast to {{agents}} in standalone mode, the configuration is managed through {{fleet}}, and it cannot be defined through `config` or `configRef` elements with a few exceptions.
150150

151+
One of those exceptions is the configuration of providers as described in [advanced Agent configuration managed by Fleet](/reference/fleet/advanced-kubernetes-managed-by-fleet.md). When {{agent}} is managed by {{fleet}} and is orchestrated by ECK, the configuration of providers can simply be done through the `.spec.config` element in the Agent resource as of {applies_to}`stack: ga 8.13`:
152+
153+
```yaml
154+
apiVersion: agent.k8s.elastic.co/v1alpha1
155+
kind: Agent
156+
metadata:
157+
name: elastic-agent
158+
spec:
159+
config:
160+
fleet:
161+
enabled: true
162+
providers.kubernetes:
163+
add_resource_metadata:
164+
deployment: true
165+
```
151166

152167
## Upgrade the {{agent}} specification [k8s-elastic-agent-fleet-configuration-upgrade-specification]
153168

‎deploy-manage/deploy/cloud-on-k8s/propagate-labels-annotations.md‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
---
2+
applies_to:
3+
deployment:
4+
eck: preview 3.1
5+
products:
6+
- id: cloud-kubernetes
7+
---
8+
9+
# Propagate Labels and Annotations [k8s-propagate-labels-annotations]
10+
11+
Starting with version `3.1.0`, {{eck}} supports propagating labels and annotations from the parent resource to the child resources it creates. This can be used on all custom resources managed by ECK, such as {{eck_resources_list}}.
12+
13+
The example below demonstrates how to use this feature on a {{es}} cluster, however, as mentioned above, this can be also applied to any custom resource managed by {{eck}}.
14+
15+
```yaml
16+
apiVersion: elasticsearch.k8s.elastic.co/v1
17+
kind: Elasticsearch
18+
metadata:
19+
annotations:
20+
# Some custom annotations to be propagated to resources created by the operator.
21+
my-annotation1: "my-annotation1-value"
22+
my-annotation2: "my-annotation2-value"
23+
# Instructions for the operator to propagate these annotations and labels to resources it creates.
24+
eck.k8s.alpha.elastic.co/propagate-annotations: "my-annotation1, my-annotation2"
25+
eck.k8s.alpha.elastic.co/propagate-labels: "my-label1, my-label2"
26+
labels:
27+
# Some custom labels to be propagated to resources created by the operator.
28+
my-label1: "my-label1-value"
29+
my-label2: "my-label2-value"
30+
name: elasticsearch-sample
31+
spec:
32+
version: 9.1.0
33+
nodeSets:
34+
- name: default
35+
config:
36+
# this allows ES to run on nodes even if their vm.max_map_count has not been increased, at a performance cost
37+
node.store.allow_mmap: false
38+
count: 1
39+
```
40+
41+
The custom labels and annotations specified in the `metadata` section of the parent resource will be propagated to all child resources created by {{eck}}, such as StatefulSets, Pods, Services, and Secrets. This ensures that all resources have consistent metadata, which can be useful for filtering, monitoring, and managing resources in Kubernetes:
42+
43+
```sh
44+
kubectl get sts,pods,svc -l my-label1=my-label1-value,my-label2=my-label2-value
45+
```
46+
47+
```sh
48+
NAME READY AGE
49+
statefulset.apps/elasticsearch-sample-es-default 1/1 4m10s
50+
51+
NAME READY STATUS RESTARTS AGE
52+
pod/elasticsearch-sample-es-default-0 1/1 Running 0 4m9s
53+
54+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
55+
service/elasticsearch-sample-es-default ClusterIP None <none> 9200/TCP 4m12s
56+
service/elasticsearch-sample-es-http ClusterIP XX.XX.XX.XX <none> 9200/TCP 4m14s
57+
service/elasticsearch-sample-es-internal-http ClusterIP XX.XX.XX.XX <none> 9200/TCP 4m14s
58+
service/elasticsearch-sample-es-transport ClusterIP None <none> 9300/TCP 4m14s
59+
```
60+
61+
It is possible to use `*` as a wildcard to propagate all labels and annotations from the parent resource to the child resources. For example:
62+
63+
```yaml
64+
apiVersion: elasticsearch.k8s.elastic.co/v1
65+
kind: Elasticsearch
66+
metadata:
67+
annotations:
68+
# Instructions for the operator to propagate all the annotations and labels to resources it creates.
69+
eck.k8s.alpha.elastic.co/propagate-annotations: "*"
70+
eck.k8s.alpha.elastic.co/propagate-labels: "*"
71+
name: elasticsearch-sample
72+
spec:
73+
version: 9.1.0
74+
nodeSets:
75+
- name: default
76+
config:
77+
# this allows ES to run on nodes even if their vm.max_map_count has not been increased, at a performance cost
78+
node.store.allow_mmap: false
79+
count: 1
80+
```
81+
82+
::::{note}
83+
Note the following considerations when using this feature:
84+
* Propagated labels and annotations are not automatically deleted. If you want to remove them from the child resources, you need to do so manually or use a cleanup script.
85+
* To prevent conflicts, some labels and annotations reserved for internal use by ECK or Kubernetes are not propagated. This is the case for labels and annotations that match `*.k8s.*.elastic.co/` and also `kubectl.kubernetes.io/last-applied-configuration`.
86+
::::

‎deploy-manage/monitor/_snippets/disconnect-cluster.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
:::{warning}
2+
If you disconnect a cluster, its connection to your {{ecloud}} account will end and all metrics and AutoOps data will be permanently deleted.
3+
:::

‎deploy-manage/monitor/_snippets/single-cloud-org.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Users can only belong to one {{ecloud}} organization at a time. If you want to create or join another organization, you must [leave the previous one](/cloud-account/join-or-leave-an-organization.md#ec-leave-organization) or use a different email address.

‎deploy-manage/monitor/autoops/cc-autoops-as-cloud-connected.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
---
2+
applies_to:
3+
deployment:
4+
self:
5+
ece:
6+
eck:
7+
navigation_title: AutoOps for self-managed clusters
8+
---
9+
10+
# AutoOps for self-managed clusters
11+
12+
Use [AutoOps](/deploy-manage/monitor/autoops.md) with your self-managed cluster through [Cloud Connect](/deploy-manage/cloud-connect.md).
13+
14+
Cloud Connect enables users of self-managed clusters to use {{ecloud}} services. This means you can take advantage of the simplified cluster monitoring, real-time issue detection, and performance recommendations of AutoOps without having to run and manage the underlying infrastructure.
15+
16+
## Section overview
17+
18+
In this section, you'll find the following information:
19+
20+
* How to [connect your self-managed cluster to AutoOps](/deploy-manage/monitor/autoops/cc-connect-self-managed-to-autoops.md)
21+
* How to [manage users of your connected clusters](/deploy-manage/monitor/autoops/cc-manage-users.md)
22+
* [FAQs](/deploy-manage/monitor/autoops/cc-cloud-connect-autoops-faq.md) about AutoOps for self-managed clusters
23+
* A [troubleshooting guide](/deploy-manage/monitor/autoops/cc-cloud-connect-autoops-troubleshooting.md) to help you with any issues you may encounter

‎deploy-manage/monitor/autoops/cc-cloud-connect-autoops-faq.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
---
2+
applies_to:
3+
deployment:
4+
self:
5+
ece:
6+
eck:
7+
navigation_title: FAQ
8+
---
9+
10+
# AutoOps for self-managed clusters FAQ
11+
12+
Find answers to your questions about AutoOps for self-managed clusters.
13+
14+
:::{dropdown} Why should I use AutoOps for my self-managed cluster?
15+
16+
AutoOps simplifies the operation of your {{es}} clusters by providing real-time monitoring, performance insights, and issue detection. It helps you identify and resolve problems like ingestion bottlenecks and unbalanced shards, reducing manual effort and preventing performance issues.
17+
18+
When you need support, AutoOps gives the Elastic team real-time visibility into your cluster, leading to faster resolutions.
19+
20+
Using AutoOps for your self-managed cluster lets you access all these features without the operational overhead of managing their infrastructure.
21+
:::
22+
23+
:::{dropdown} Which versions of {{es}} does AutoOps support?
24+
25+
AutoOps is compatible with all [supported {{es}} versions](https://www.elastic.co/support/eol).
26+
:::
27+
28+
:::{dropdown} Which deployment types can be connected to AutoOps?
29+
30+
You can connect to AutoOps on a standalone {{stack}}, ECE ({{ece}}), or ECK ({{eck}}) deployment.
31+
:::
32+
33+
:::{dropdown} Can I use AutoOps for self-managed clusters if my environment is air-gapped?
34+
35+
Not at this time. AutoOps is currently only available as a cloud service and you need an internet connection to send metrics to {{ecloud}}. For air-gapped environments, we plan to offer a locally deployable version in the future.
36+
:::
37+
38+
:::{dropdown} Do I have to define an Elastic IP address to enable the agent to send data to {{ecloud}}?
39+
40+
You may need to define an IP address if your organization’s settings will block the agent from sending out data.
41+
42+
To enable IP ranges, {{ecloud}} offers a selection of static IP addresses. All traffic directed to {{ecloud}} deployments, whether originating from the public internet, your private cloud network through the public internet, or your on-premise network through the public internet utilizes Ingress Static IPs as the network destination.
43+
44+
For more information, refer to [](/deploy-manage/security/elastic-cloud-static-ips.md).
45+
:::
46+
47+
:::{dropdown} Where are AutoOps metrics stored, and does it cost extra to ship metrics to {{ecloud}}?
48+
49+
You can choose the CSP and region in which your cluster metrics will be stored from a list of [available regions](/deploy-manage/monitor/autoops/ec-autoops-regions.md).
50+
51+
Shipping metrics to {{ecloud}} may come at an additional cost. For example, when sending metrics data from your cluster in a CSP region to {{ecloud}}, shipping costs will be determined by your agreement with that CSP.
52+
:::
53+
54+
:::{dropdown} What information does {{agent}} extract from my cluster?
55+
56+
{{agent}} only extracts and sends cluster metrics to {{ecloud}}, not the underlying data within your cluster. The following metrics are collected:
57+
58+
| API | Description | Collected data |
59+
| --- | --- | --- |
60+
| [_cat/shards](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-shards) | Returns detailed information about the shards within the cluster | Shard states, node allocation, index names, sizes, and replica information |
61+
| [_nodes/stats](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-stats) | Retrieves statistics from cluster nodes including JVM, OS, process, and transport metrics | CPU usage, memory utilization, thread pools, file system stats |
62+
| [_cluster/settings](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings) | Returns the settings configured for the cluster | Persistent and transient settings such as cluster-wide configurations |
63+
| [_cluster/health](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-health) | Provides information about the overall health of the cluster | Status (green/yellow/red), number of nodes, number of shards |
64+
| [_cat/template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-templates) | Lists all index templates in the cluster | Template names, patterns, and basic settings |
65+
| [_index_template](/manage-data/data-store/templates.md) | Retrieves composable index templates | Index settings, mappings, and aliases |
66+
| [_component_template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-component-template) | Fetches component templates used for building index templates | Metadata for re-usable mappings and settings |
67+
| [_tasks](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-tasks) | Displays information about currently running tasks on the cluster | Task descriptions, start times, running nodes, and execution details |
68+
| [_template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-template) | Retrieves legacy index templates | Similar to composable index templates but in older format |
69+
| [_resolve/index/*](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-resolve-index) | Resolves index, data stream, and alias names to their current definitions | Mappings between names and underlying data objects |
70+
:::
71+
72+

0 commit comments

Comments
 (0)