DOC-608 | APM-660 | Release notes for memory accounting metrics and memory usage improvements (#346)

* Release notes for memory accounting metrics and memory usage improvements

* Review

Author: Simran-B

site/content/3.12/release-notes/version-3.12/api-changes-in-3-12.md

@@ -160,6 +160,22 @@ ongoing dumps, ArangoSearch parallelism and used file descriptors:
 - `arangodb_search_execution_threads_demand`
 - `arangodb_search_file_descriptors`
 
+The following new metrics for improved memory observability have been added:
+
+- `arangodb_agency_node_memory_usage`
+- `arangodb_index_estimates_memory_usage`
+- `arangodb_internal_cluster_info_memory_usage`
+- `arangodb_requests_memory_usage`
+- `arangodb_revision_tree_buffered_memory_usage`
+- `arangodb_scheduler_queue_memory_usage`
+- `arangodb_scheduler_stack_memory_usage`
+- `arangodb_search_consolidations_memory_usage`
+- `arangodb_search_mapped_memory`
+- `arangodb_search_readers_memory_usage`
+- `arangodb_search_writers_memory_usage`
+- `arangodb_transactions_internal_memory_usage`
+- `arangodb_transactions_rest_memory_usage`
+
 ---
 
 <small>Introduced in: v3.11.2</small>

site/content/3.12/release-notes/version-3.12/incompatible-changes-in-3-12.md

@@ -79,7 +79,27 @@ The second option is the recommended one, as it signals the intent more clearly,
 and makes the cache behave "as expected", i.e. use up to the configured
 memory limit and not just 56% of it.
 
-#### Adjustable Stream Transaction size
+## Higher reported memory usage for AQL queries
+
+Due to the [improved memory accounting in v3.12](whats-new-in-3-12.md#improved-memory-accounting),
+certain AQL queries may now get aborted because they exceed the defined
+memory limit but didn't get killed in previous versions. This is because of the
+more accurate memory tracking that reports a higher (actual) usage now. It allows
+ArangoDB to more reliably detect and kill queries that go over the per-query and
+global query memory limit, potentially preventing out-of-memory crashes of
+_arangod_ processes.
+
+In particular, AQL queries that perform write operations now report a
+significantly higher `peakMemoryUsage` than before. This is also
+reflected in the `arangodb_aql_global_memory_usage` metric. Memory used for
+ArangoSearch `SEARCH` operations is now also accounted for in the metric.
+
+You may need to adjust affected queries to use less memory or increase the
+per-query limit with the [`memoryLimit` query option](../../aql/how-to-invoke-aql/with-arangosh.md#memorylimit)
+or its default using the `--query.memory-limit` startup option. You can adjust
+the global limit with the `--query.global-memory-limit` startup option.
+
+## Adjustable Stream Transaction size
 
 [Stream Transactions](../../develop/transactions/stream-transactions.md) may
 now be limited to smaller transaction sizes because the maximum transaction size
@@ -119,7 +139,7 @@ server:
   - `x-http-method-override`
   - `x-method-override`
 
-   This functionaltiy posed a potential security risk and was thus removed.
+   This functionality posed a potential security risk and was thus removed.
    Previously, it was only enabled when explicitly starting the 
    server with the `--http.allow-method-override` startup option.
    The functionality has now been removed and setting the startup option does

site/content/3.12/release-notes/version-3.12/whats-new-in-3-12.md

@@ -54,6 +54,39 @@ for details.
 ## Analyzers
 
 
+## Improved memory accounting and usage
+
+Version 3.12 features multiple improvements to observability of ArangoDB
+deployments. Memory usage is more accurately tracked and additional metrics have
+been added for monitoring the memory consumption.
+
+AQL queries may now report a higher memory usage and thus run into memory limits
+sooner, see [Higher reported memory usage for AQL queries](incompatible-changes-in-3-12.md#higher-reported-memory-usage-for-aql-queries).
+
+Furthermore, the memory usage of some subsystems has been optimized. When
+dropping a database, all contained collections are now marked as dropped
+immediately. Ongoing operations on these collections can be stopped earlier, and
+memory for the underlying collections and indexes can be reclaimed sooner.
+Memory used for index selectively estimates is now also released early.
+ArangoSearch has a smaller memory footprint for removal operations now.
+
+The following new metrics have been added for memory observability:
+
+| Label | Description |
+|:------|:------------|
+| `arangodb_agency_node_memory_usage` | Memory used by Agency store/cache. |
+| `arangodb_index_estimates_memory_usage` | Total memory usage of all index selectivity estimates. |
+| `arangodb_internal_cluster_info_memory_usage` | Amount of memory spent in ClusterInfo. |
+| `arangodb_requests_memory_usage` | Memory consumed by incoming, queued, and currently processed requests. |
+| `arangodb_revision_tree_buffered_memory_usage` | Total memory usage of buffered updates for all revision trees. |
+| `arangodb_scheduler_queue_memory_usage` | Number of bytes allocated for tasks in the scheduler queue. |
+| `arangodb_scheduler_stack_memory_usage` | Approximate stack memory usage of worker threads. |
+| `arangodb_search_consolidations_memory_usage` | Amount of memory in bytes that is used for consolidating an ArangoSearch index. |
+| `arangodb_search_mapped_memory` | Amount of memory in bytes that is mapped for an ArangoSearch index. |
+| `arangodb_search_readers_memory_usage` | Amount of memory in bytes that is used for reading from an ArangoSearch index. |
+| `arangodb_search_writers_memory_usage` | Amount of memory in bytes that is used for writing to an ArangoSearch index. |
+| `arangodb_transactions_internal_memory_usage` | Total memory usage of internal transactions. |
+| `arangodb_transactions_rest_memory_usage` | Total memory usage of user transactions (excluding top-level AQL queries). |
 
 ## Web interface