diff --git a/deploy-manage/deploy/elastic-cloud.md b/deploy-manage/deploy/elastic-cloud.md index 1052fd4639..835c3963fc 100644 --- a/deploy-manage/deploy/elastic-cloud.md +++ b/deploy-manage/deploy/elastic-cloud.md @@ -33,23 +33,9 @@ Some of the unique benefits of {{ecloud}} include: For more information, refer to [](/deploy-manage/cloud-organization.md). -## Differences between serverless projects and hosted deployments[general-what-is-serverless-elastic-differences-between-serverless-projects-and-hosted-deployments-on-ecloud] - -You can have multiple hosted deployments and serverless projects in the same {{ecloud}} organization, and each deployment type has its own specificities. - - -| Option | Serverless | Hosted | -| --- | --- | --- | -| **Cluster management** | Fully managed by Elastic. | You provision and manage your hosted clusters. Shared responsibility with Elastic. | -| **Scaling** | Autoscales out of the box. | Manual scaling or autoscaling available for you to enable. | -| **Upgrades** | Automatically performed by Elastic. | You choose when to upgrade. | -| **Pricing** | Individual per project type and based on your usage. | Based on deployment size and subscription level. | -| **Performance** | Autoscales based on your usage. | Manual scaling. | -| **Solutions** | Single solution per project. | Full Elastic Stack per deployment. | -| **User management** | Elastic Cloud-managed users. | Elastic Cloud-managed users and native Kibana users. | -| **API support** | Subset of [APIs](https://www.elastic.co/docs/api). | All Elastic APIs. | -| **Backups** | Projects automatically backed up by Elastic. | Your responsibility with Snapshot & Restore. | -| **Data retention** | Editable on data streams. | Index Lifecycle Management. | +## Compare {{ech}} and {{serverless-short}} [general-what-is-serverless-elastic-differences-between-serverless-projects-and-hosted-deployments-on-ecloud] + +Refer to [Compare Cloud Hosted and Serverless](/deploy-manage/deploy/elastic-cloud/differences-from-other-elasticsearch-offerings.md) for a detailed comparison between {{ech}} deployments and {{serverless-short}} projects. ## APIs diff --git a/deploy-manage/deploy/elastic-cloud/differences-from-other-elasticsearch-offerings.md b/deploy-manage/deploy/elastic-cloud/differences-from-other-elasticsearch-offerings.md index 875b39f662..f4a74c97f6 100644 --- a/deploy-manage/deploy/elastic-cloud/differences-from-other-elasticsearch-offerings.md +++ b/deploy-manage/deploy/elastic-cloud/differences-from-other-elasticsearch-offerings.md @@ -1,81 +1,181 @@ --- -navigation_title: "Serverless differences" +navigation_title: "Compare Cloud Hosted and Serverless" mapped_pages: - https://www.elastic.co/guide/en/serverless/current/elasticsearch-differences.html applies_to: serverless: + ess: --- +# Compare Elastic Cloud Hosted and Serverless [elasticsearch-differences] +This guide compares Elastic Cloud Hosted deployments with Elastic Cloud Serverless projects, highlighting key features and capabilities across different project types. Use this information to understand what's available in each deployment option or to plan migrations between platforms. -# Differences from other Elasticsearch offerings [elasticsearch-differences] +:::{note} +The information below reflects our strategic goals, plans and objectives and includes estimated release dates, anticipated features and functions, and proposed descriptions for commercial features. All details are for information only and are subject to change in our discretion. Information may be updated, added, or removed from this document as features or products become available, canceled, or postponed. +::: +## Architectural differences -[{{es-serverless}}](../../../solutions/search.md) handles all the infrastructure management for you, providing a fully managed {{es}} service. +Elastic Cloud Serverless takes a fundamentally different approach to running the Elastic Stack compared to Elastic Cloud Hosted: -If you’ve used {{es}} before, you’ll notice some differences in how you work with the service on {{serverless-full}}, because a number of APIs and settings are not required for serverless projects. - -This guide helps you understand what’s different, what’s available, and how to work effectively when running {{es}} on {{serverless-full}}. - - -## Fully managed infrastructure [elasticsearch-differences-serverless-infrastructure-management] - -{{es-serverless}} manages all infrastructure automatically, including: +| **Functionality** | Elastic Cloud Hosted | Elastic Cloud Serverless | +|--------|----------------------|--------------------------| +| **Management model** | Self-service infrastructure | Fully managed service | +| **Project organization** | Single deployments with multiple capabilities | Separate projects for Elasticsearch, Observability, and Security | +| **Scaling** | Manual or automated with configuration | Fully automated | +| **Infrastructure decisions** | User manages capacity | Automatically managed by Elastic | +| **Pricing model** | Based on provisioned resources | Based on usage | +| **Cloud providers** | AWS, Azure, GCP | AWS, Azure (in preview), GCP (in preview) | +| **Upgrades** | User-controlled timing | Automatically performed by Elastic | +| **User management** | Elastic Cloud-managed and deployment-local users | Elastic Cloud-managed users only. Serverless users are managed at the organization level with SAML authentication support. | +| **Backups** | User-managed with Snapshot & Restore | Automatically backed up by Elastic | +| **Solutions** | Full Elastic Stack per deployment | Single solution per project | +In Serverless, Elastic automatically manages: * Cluster scaling and optimization * Node management and allocation * Shard distribution and replication * Resource utilization and monitoring - -This fully managed approach means many traditional {{es}} infrastructure APIs and settings are not available to end users, as detailed in the following sections. - - -## Index size guidelines [elasticsearch-differences-serverless-index-size] - -To ensure optimal performance, follow these recommendations for sizing individual indices on {{es-serverless}}: - -| Use case | Maximum index size | Project configuration | +* High availability and disaster recovery strategies + +## Compare features [elasticsearch-differences-serverless-infrastructure-management] + +$$$elasticsearch-differences-serverless-feature-categories$$$ +$$$elasticsearch-differences-serverless-features-replaced$$$ +$$$elasticsearch-differences-serverless-feature-planned$$$ + +### Core platform capabilities + +This table compares the core platform capabilities between Elastic Cloud Hosted deployments and Serverless projects: + +| **Feature** | Elastic Cloud Hosted | Serverless projects| Notes | +|----------|----------------------|--------------------|--------| +| **Audit logging** | ✅ | **Planned** | Anticipated in a future release | +| **Authentication realms** | ✅ | ✅ | Managed at organization level in Serverless; deployment level in Hosted | +| **BYO-Key for Encryption at Rest** | ✅ | **Planned** | Anticipated in a future release; data in Serverless is stored on cloud-provider encrypted object storage | +| **Cloud provider support** | - AWS
- GCP
- Azure | - AWS
- Azure (in preview)
- GCP (in preview) | - [Elastic Cloud Hosted regions](cloud://reference/cloud-hosted/regions.md)
- [Serverless regions](/deploy-manage/deploy/elastic-cloud/regions.md) | +| **Cluster scaling** | Manual with autoscaling option | Managed | Automatic scaling eliminates capacity planning - [Learn more](https://www.elastic.co/blog/elastic-serverless-architecture) | +| **Custom plugins and bundles** | ✅ | ❌ | Not available in Serverless | +| **Custom roles** | ✅ | ✅ | Not available in Serverless Observability projects. | +| **Deployment health monitoring** | AutoOps or monitoring cluster | Managed by Elastic | - No monitoring cluster required
- Automatically handled by Elastic | +| **Deployment model** | Single deployments with multiple solutions | Separate projects for specific use cases | Fundamental architectural difference - [Learn more](https://www.elastic.co/blog/elastic-serverless-architecture) | +| **Deployment monitoring** | AutoOps or monitoring cluster | Managed | Monitoring is handled by Elastic | +| **Hardware configuration** | Limited control | Managed | Hardware choices are managed by Elastic | +| **High availability** | ✅ | ✅ | Automatic resilience | +| **Network security** | Public IP traffic filtering, private connectivity (VPCs, PrivateLink) | **Planned** | - Traffic filtering anticipated in a future release
- Private connectivity options anticipated in a future release | +| **Node management** | User-controlled | Managed | No node configuration access by design | +| **Snapshot/restore** | ✅ | **Planned** | User-initiated snapshots are anticipated in a future release | + +:::{note} +The [Elastic Cloud Serverless roadmap](https://www.elastic.co/cloud/serverless/roadmap) primarily focuses on platform capabilities rather than project-specific features. Use the following project-specific tables for information about features for each project type. +::: + +### Elasticsearch + +This table compares Elasticsearch capabilities between Elastic Cloud Hosted deployments and Serverless projects: + +| **Feature** | Elastic Cloud Hosted | Serverless Elasticsearch projects | Serverless notes | +|---------|----------------------|-----------------------------------|------------------| +| [**AI Assistant**](/solutions/observability/observability-ai-assistant.md) | ✅ | ✅ | | +| **Behavioral analytics** | ❌ (deprecated in 9.0) | ❌ | Not available in Serverless | +| [**Clone index API**](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-clone) | ✅ | **Planned** | Anticipated in a future release | +| [**Cross-cluster replication**](/deploy-manage/tools/cross-cluster-replication.md) | ✅ | **Planned** | Anticipated in a future release | +| [**Cross-cluster search**](/solutions/search/cross-cluster-search.md) | ✅ | **Planned** | Anticipated in a future release | +| **Data lifecycle management** | - [ILM](/manage-data/lifecycle/index-lifecycle-management.md)
- [Data stream lifecycle](/manage-data/lifecycle/data-stream.md) | [Data stream lifecycle](/manage-data/lifecycle/data-stream.md) only | - No data tiers in Serverless
- Optimized for common lifecycle management needs | +| **Elastic connectors (for search)** | ❌ (Managed connectors discontinued with Enterprise Search in 9.0) | Self-managed only | - Managed connectors not available
- Use [**self-managed connectors**](elasticsearch://reference/search-connectors/self-managed-connectors.md) | +| [**Elasticsearch for Apache Hadoop**](https://www.elastic.co/elasticsearch/hadoop) | ✅ | ❌ | Not available in Serverless | +| **Enterprise Search (App Search & Workplace Search)** | ❌ (discontinued in 9.0) | ❌ | Not available in Serverless | +| [**Kibana Alerts**](/deploy-manage/monitor/monitoring-data/configure-stack-monitoring-alerts.md) | ✅ | ✅ | | +| [**Reindexing from remote**](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-reindex) | ✅ | **Planned** | Anticipated in a future release | +| **Repository management** | ✅ | Managed | Automatically managed by Elastic | +| [**Scripted metric aggregations**](elasticsearch://reference/aggregations/search-aggregations-metrics-scripted-metric-aggregation.md) | ✅ | ❌ | Not available in Serverless
The alternative for this in Serverless is [ES|QL](/explore-analyze/query-filter/languages/esql.md) | +| [**Search applications**](https://www.elastic.co/guide/en/elasticsearch/reference/8.18/search-application-overview.html) | - UI and APIs
- Maintenance mode (beta) | - API-only
- Maintenance mode (beta) | UI not available in Serverless | +| **Shard management** | User-configurable | Managed by Elastic | No manual shard allocation in Serverless | +| [**Watcher**](/explore-analyze/alerts-cases/watcher.md) | ✅ | ❌ | Use **Kibana Alerts** instead, which provides rich integrations across use cases | +| **Web crawler** | ❌ (Managed Elastic Crawler discontinued with Enterprise Search in 9.0) | Self-managed only | Use [**self-managed crawler**](https://github.com/elastic/crawler) | + +### Observability + +This table compares Observability capabilities between Elastic Cloud Hosted deployments and Serverless projects: + +| **Feature** | Elastic Cloud Hosted | Serverless Observability projects | Serverless notes | +|---------|----------------------|-----------------------------------|------------------| +| [**AI Assistant**](/solutions/observability/observability-ai-assistant.md) | ✅ | ✅ | | +| **APM integration** | ✅ | ✅ | Use **Managed Intake Service** (supports Elastic APM and OTLP protocols) | +| [**APM Agent Central Configuration**](/solutions/observability/apps/apm-agent-central-configuration.md) | ✅ | ❌ | Not available in Serverless | +| [**APM Tail-based sampling**](/solutions/observability/apps/transaction-sampling.md#apm-tail-based-sampling) | ✅ | ❌ | - Not available in Serverless
- Consider **OpenTelemetry** tail sampling processor as an alternative | +| [**Android agent/SDK instrumentation**](apm-agent-android://reference/index.md) | ✅ | ❌ | Not available in Serverless | +| [**AWS Firehose integration**](/solutions/observability/cloud/monitor-amazon-web-services-aws-with-amazon-data-firehose.md) | ✅ | ✅ | | +| **Custom roles for Kibana Spaces** | ✅ | **Planned** | Anticipated in a future release | +| [**Data stream lifecycle**](/manage-data/lifecycle/data-stream.md) | ✅ | ✅ | Primary lifecycle management method in Serverless | +| **[Elastic Serverless Forwarder](elastic-serverless-forwarder://reference/index.md)** | ✅ | ❌ | | +| **[Elastic Synthetics Private Locations](/solutions/observability/apps/monitor-resources-on-private-networks.md#synthetics-private-location-add)** | ✅ | ✅ | | +| **[Fleet Agent policies](/reference/fleet/agent-policy.md)** | ✅ | ✅ | | +| **[Fleet server](/reference/fleet/fleet-server.md)** | - Self-hosted
- Hosted | ✅ | Fully managed by Elastic | +| [**Index lifecycle management**](/manage-data/lifecycle/index-lifecycle-management.md) | ✅ | ❌ | Use [**Data stream lifecycle**](/manage-data/lifecycle/data-stream.md) instead | +| **[iOS agent/SDK instrumentation](apm-agent-ios://reference/index.md)** | ✅ | ❌ | Not available in Serverless | +| **[Kibana Alerts](/deploy-manage/monitor/monitoring-data/configure-stack-monitoring-alerts.md)** | ✅ | ✅ | | +| **[LogsDB index mode](/manage-data/data-store/data-streams/logs-data-stream.md)** | ✅ | ✅ | - Reduces storage footprint
- Enabled by default
- Cannot be disabled | +| **[Logs management](/solutions/observability/logs.md)** | ✅ | ✅ | | +| **[Metrics monitoring](/solutions/observability/apps/metrics.md)** | ✅ | ✅ | | +| **[Observability SLO](/solutions/observability/incident-management/service-level-objectives-slos.md)** | ✅ | ✅ | | +| [**Real User Monitoring (RUM)**](/solutions/observability/apps/real-user-monitoring-user-experience.md) | ✅ | **Planned** | Anticipated in a future release | +| **[Universal Profiling](/solutions/observability/infra-and-hosts/get-started-with-universal-profiling.md)** | ✅ | ❌ | Not available in Serverless | +| **Uptime monitoring** | ❌ | ❌ | - Deprecated in all deployment types
- Use [**Synthetics app**](/solutions/observability/apps/synthetic-monitoring.md) instead | + +### Security + +This table compares Security capabilities between Elastic Cloud Hosted deployments and Serverless projects: + +| **Feature** | Elastic Cloud Hosted | Serverless Security projects | Serverless notes | +|---------|---------------------|------------------------------|------------------| +| **[Advanced Entity Analytics](/solutions/security/advanced-entity-analytics.md)** | ✅ | ✅ | | +| **[AI Assistant](/solutions/security/ai/ai-assistant.md)** | ✅ | ✅ | | +| **API keys** | ✅ | ✅ | | +| **[Cloud Security](/solutions/security/cloud.md)** | ✅ | ✅ | | +| [**Defend for Containers integration**](https://www.elastic.co/guide/en/security/8.18/d4c-overview.html) | ✅ (deprecated in 9.0) | ❌ | Not available in Serverless | +| **[Endpoint security](/solutions/security/configure-elastic-defend.md)** | ✅ | ✅ | | +| **[Kibana Alerts](/deploy-manage/monitor/monitoring-data/configure-stack-monitoring-alerts.md)** | ✅ | ✅ | | +| **Kibana navigation** | Standard layout | Different layout | UI differences in Security projects | +| **[LogsDB](/manage-data/data-store/data-streams/logs-data-stream.md)** | Optional | ✅ | - Enabled by default
- Cannot be disabled | +| **Role-based access control** | ✅ | Limited | Core RBAC functionality supported | +| **SIEM capabilities** | ✅ | ✅ | Core functionality supported | + +## Elasticsearch index sizing guidelines [elasticsearch-differences-serverless-index-size] + +To ensure optimal performance in Serverless Elasticsearch projects, follow these sizing recommendations: + +| **Use case** | Maximum index size | Project configuration | | --- | --- | --- | -| Vector search | 150GB | Vector optimized | -| General search (non data-stream) | 300GB | General purpose | -| Other uses (non data-stream) | 600GB | General purpose | +| **Vector search** | 150GB | Vector optimized | +| **General search (non data-stream)** | 300GB | General purpose | +| **Other uses (non data-stream)** | 600GB | General purpose | -For large datasets that exceed the recommended maximum size for a single index, consider splitting your data across smaller indices and using an alias to search them collectively. +For large datasets that exceed the recommended maximum size, consider splitting your data across smaller indices and using an alias to search them collectively. -These recommendations do not apply to indices using better binary quantization (BBQ). Refer to [vector quantization](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md#dense-vector-quantization) in the core {{es}} docs for more information. +These recommendations do not apply to indices using better binary quantization (BBQ). Refer to [vector quantization](elasticsearch://reference/elasticsearch/mapping-reference/dense-vector.md#dense-vector-quantization) for more information. +## Available {{es}} APIs [elasticsearch-differences-serverless-apis-availability] -## API availability [elasticsearch-differences-serverless-apis-availability] - -Because {{es-serverless}} manages infrastructure automatically, certain APIs are not available, while others remain fully accessible. - -::::{tip} -Refer to the [{{es-serverless}} API reference](https://www.elastic.co/docs/api/doc/elasticsearch-serverless) for a complete list of available APIs. - -:::: - - -The following categories of operations are unavailable: +Because Elastic Cloud Serverless manages infrastructure automatically, certain Elasticsearch APIs are not available: Infrastructure operations : * All `_nodes/*` operations * All `_cluster/*` operations * Most `_cat/*` operations, except for index-related operations such as `/_cat/indices` and `/_cat/aliases` - Storage and backup : * All `_snapshot/*` operations * Repository management operations - Index management : * `indices/close` operations * `indices/open` operations * Recovery and stats operations * Force merge operations - -When attempting to use an unavailable API, you’ll receive a clear error message: +When attempting to use an unavailable API, you'll receive this error: ```json { @@ -91,20 +191,22 @@ When attempting to use an unavailable API, you’ll receive a clear error messag } ``` +::::{tip} +Refer to the [{{es-serverless}} API reference](https://www.elastic.co/docs/api/doc/elasticsearch-serverless) for a complete list of available APIs. +:::: -## Settings availability [elasticsearch-differences-serverless-settings-availability] +## Available {{es}} settings [elasticsearch-differences-serverless-settings-availability] -In {{es-serverless}}, you can only configure [index-level settings](elasticsearch://reference/elasticsearch/index-settings/index.md). Cluster-level settings and node-level settings are not required by end users and the `elasticsearch.yml` file is fully managed by Elastic. +In Elastic Cloud Serverless Elasticsearch projects, you can only configure [index-level settings](elasticsearch://reference/elasticsearch/index-settings/index.md). Cluster-level settings and node-level settings are fully managed by Elastic. Available settings -: **Index-level settings**: Settings that control how {{es}} documents are processed, stored, and searched are available to end users. These include: +: **Index-level settings**: Settings that control how documents are processed, stored, and searched are available to end users. These include: * Analysis configuration * Mapping parameters * Search/query settings * Indexing settings such as `refresh_interval` - Managed settings : **Infrastructure-related settings**: Settings that affect cluster resources or data distribution are not available to end users. These include: @@ -113,47 +215,12 @@ Managed settings * Shard allocation * Resource management +## Learn more - -## Feature availability [elasticsearch-differences-serverless-feature-categories] - -Some features that are available in Elastic Cloud Hosted and self-managed offerings are not available in {{es-serverless}}. These features have either been replaced by a new feature, are planned to be released in future, or are not applicable in the new serverless architecture. - - -### Replaced features [elasticsearch-differences-serverless-features-replaced] - -These features have been replaced by a new feature and are therefore not available on {{es-serverless}}: - -* **Index lifecycle management ({{ilm-init}})** is not available, in favor of [**data stream lifecycle**](../../../manage-data/data-store/index-basics.md). - - In an Elastic Cloud Hosted or self-managed environment, {{ilm-init}} lets you automatically transition indices through data tiers according to your performance needs and retention requirements. This allows you to balance hardware costs with performance. {{es-serverless}} eliminates this complexity by optimizing your cluster performance for you. - - Data stream lifecycle is an optimized lifecycle tool that lets you focus on the most common lifecycle management needs, without unnecessary hardware-centric concepts like data tiers. - -* **Watcher** is not available, in favor of [**Alerts**](../../../explore-analyze/alerts-cases/alerts.md#rules-alerts). - - Kibana Alerts allows rich integrations across use cases like APM, metrics, security, and uptime. Prepackaged rule types simplify setup and hide the details of complex, domain-specific detections, while providing a consistent interface across Kibana. - - - -### Planned features [elasticsearch-differences-serverless-feature-planned] - -The following features are planned for future support in all {{serverless-full}} projects: - -* Reindexing from remote clusters -* Cross-project search and replication -* Snapshot and restore -* Migrations from non-serverless deployments -* Audit logging -* Clone index API -* Traffic filtering and VPCs - -### Unplanned features [elasticsearch-differences-serverless-feature-unavailable] - -The following features are not available in {{es-serverless}} and are not planned for future support: - -* [Custom plugins and bundles](/deploy-manage/deploy/elastic-cloud/upload-custom-plugins-bundles.md) -* [{{es}} for Apache Hadoop](elasticsearch-hadoop://reference/index.md) -* [Scripted metric aggregations](elasticsearch://reference/aggregations/search-aggregations-metrics-scripted-metric-aggregation.md) -* Managed web crawler: You can use the [self-managed web crawler](https://github.com/elastic/crawler) instead. -* Managed Search connectors: You can use [self-managed Search connectors](elasticsearch://reference/search-connectors/self-managed-connectors.md) instead. \ No newline at end of file +- [Elastic Cloud Serverless roadmap](https://www.elastic.co/cloud/serverless/roadmap): See upcoming features and development plans for the Serverless platform +- [Elasticsearch Serverless API reference](https://www.elastic.co/docs/api/doc/elasticsearch-serverless): Check out the complete list of available APIs in Elastic Cloud Serverless +- [Project settings](/deploy-manage/deploy/elastic-cloud/project-settings.md): Configure project settings in Elastic Cloud Serverless +- [Serverless regions](/deploy-manage/deploy/elastic-cloud/regions.md): Choose the right region for your Elastic Cloud Serverless project +- [Elastic Cloud pricing](https://www.elastic.co/pricing/): Understand pricing for Elastic Cloud Hosted and Serverless projects + - [Serverless project billing](/deploy-manage/cloud-organization/billing/serverless-project-billing-dimensions.md): Understand billing dimensions for Serverless projects + - [Elastic Cloud Hosted billing](/deploy-manage/cloud-organization/billing/cloud-hosted-deployment-billing-dimensions.md): Understand billing dimensions for Elastic Cloud Hosted deployments \ No newline at end of file diff --git a/deploy-manage/toc.yml b/deploy-manage/toc.yml index e8f8d53615..01f5b7cf4e 100644 --- a/deploy-manage/toc.yml +++ b/deploy-manage/toc.yml @@ -6,6 +6,7 @@ toc: - file: deploy/deployment-comparison.md - file: deploy/elastic-cloud.md children: + - file: deploy/elastic-cloud/differences-from-other-elasticsearch-offerings.md - file: deploy/elastic-cloud/create-an-organization.md children: - file: deploy/elastic-cloud/subscribe-from-marketplace.md @@ -29,7 +30,6 @@ toc: - file: deploy/elastic-cloud/heroku-reference-regions.md - file: deploy/elastic-cloud/serverless.md children: - - file: deploy/elastic-cloud/differences-from-other-elasticsearch-offerings.md - file: deploy/elastic-cloud/create-serverless-project.md - file: deploy/elastic-cloud/regions.md - file: deploy/elastic-cloud/project-settings.md