Documentation cleanup (#3486)

* Split upgrade guide into different pages
* Move some metrics under data stores
* Make some nested paths more consistent

Author: elahrvivaz

docs/user/appendix/metrics.rst

@@ -1,149 +1,8 @@
-.. _geomesa_metrics:
-
 Micrometer Metrics
 ==================
 
-GeoMesa has initial support for `Micrometer <https://docs.micrometer.io/micrometer/reference/>`__ metrics. Currently,
-metrics can integrate with either Prometheus or Cloudwatch. Creating registries is idempotent, as long as the
-configuration does not change.
-
-Most GeoMesa data stores support the parameters ``geomesa.metrics.registry`` and ``geomesa.metrics.registry.config``, which are
-used to start up a registry for publishing metrics. Configuration is described below.
-
-When enabled, various GeoMesa components will publish metrics, including the :ref:`kafka_index` and :ref:`converters`.
-
-Prometheus Registry
--------------------
-
-The Prometheus registry can be configured through the following environment variables:
-
-+--------------------------------------+---------+----------------------------------------------+
-| Environment Variable                 | Default | Description                                  |
-+======================================+=========+==============================================+
-| ``RENAME_PROMETHEUS_METRICS``        | true    | `Rename`__ metrics                           |
-+--------------------------------------+---------+----------------------------------------------+
-| ``METRICS_APPLICATION_NAME``         | geomesa | Add a tag for ``application`` to all metrics |
-+--------------------------------------+---------+----------------------------------------------+
-| ``METRICS_PORT``                     | 9090    | Set the port used to host metrics            |
-+--------------------------------------+---------+----------------------------------------------+
-
-__ https://docs.micrometer.io/micrometer/reference/implementations/prometheus.html#_the_prometheus_rename_filter
-
-Instead of environment variables, the registry can be fully configured using using the data store parameter
-``geomesa.metrics.registry.config``, which takes `Lightbend Config <https://github.com/lightbend/config/tree/main>`__:
-
-::
-
-    # port used to serve metrics - not used if push-gateway is defined
-    port = 9090
-    # tags applied to all metrics, may be any key-value string pairs
-    common-tags = { "application" = "my-app" }
-    # use prometheus "standard" names - see https://docs.micrometer.io/micrometer/reference/implementations/prometheus.html#_the_prometheus_rename_filter
-    rename = false
-    # additional config - can also be done via sys props, see https://prometheus.github.io/client_java/config/config/
-    properties = {}
-    # optional - enable pushgateway for short-lived jobs, instead of the standard metrics server for scraping
-    push-gateway = {
-      host = "localhost:9091"
-      job = "my-job"
-      scheme = "http"
-      format = "PROMETHEUS_PROTOBUF" # or PROMETHEUS_TEXT
-    }
-
-If using Pushgateway, the following dependency is required:
-
-.. code-block:: xml
-
-    <dependency>
-      <groupId>io.prometheus</groupId>
-      <artifactId>prometheus-metrics-exporter-pushgateway</artifactId>
-      <version>{{prometheus_version}}</version>
-    </dependency>
-
-Cloudwatch Registry
--------------------
-
-The Cloudwatch registry can be configured through the following environment variables:
-
-+--------------------------------------+---------+----------------------------------------------+
-| Environment Variable                 | Default | Description                                  |
-+======================================+=========+==============================================+
-| ``METRICS_NAMESPACE``                | geomesa | Cloudwatch namespace                         |
-+--------------------------------------+---------+----------------------------------------------+
-
-Instead of environment variables, the registry can be fully configured using using the data store parameter
-``geomesa.metrics.registry.config``, which takes `Lightbend Config <https://github.com/lightbend/config/tree/main>`__:
-
-::
-
-    # cloudwatch namespace
-    namespace = "geomesa"
-    # properties for the cloudwatch client
-    properties = {}
-
-For Cloudwatch, the following dependency is required:
-
-.. code-block:: xml
-
-    <dependency>
-      <groupId>io.micrometer</groupId>
-      <artifactId>micrometer-registry-cloudwatch2</artifactId>
-      <version>{{micrometer_version}}</version>
-    </dependency>
-
-Instrumentations
-----------------
-
-When adding a registry, additional JVM metrics can be exposed with the following environment variables:
-
-+--------------------------------------+---------+----------------------------------------------+
-| Environment Variable                 | Default | Description                                  |
-+======================================+=========+==============================================+
-| ``METRICS_INSTRUMENTATIONS_ENABLED`` | true    | Enable all JVM instrumentations              |
-+--------------------------------------+---------+----------------------------------------------+
-
-Individual instrumentations can be enabled or disabled through the following environment variables:
-
-+--------------------------------------+---------+----------------------------------------------+
-| Environment Variable                 | Default | Description                                  |
-+======================================+=========+==============================================+
-| ``METRICS_INSTRUMENT_CLASSLOADER``   |         | Enable metrics on JVM class loading          |
-+--------------------------------------+---------+----------------------------------------------+
-| ``METRICS_INSTRUMENT_MEMORY``        |         | Enable metrics on JVM memory usage           |
-+--------------------------------------+---------+----------------------------------------------+
-| ``METRICS_INSTRUMENT_GC``            |         | Enable metrics on JVM garbage collection     |
-+--------------------------------------+---------+----------------------------------------------+
-| ``METRICS_INSTRUMENT_PROCESSOR``     |         | Enable metrics on processor usage            |
-+--------------------------------------+---------+----------------------------------------------+
-| ``METRICS_INSTRUMENT_THREADS``       |         | Enable metrics on JVM thread usage           |
-+--------------------------------------+---------+----------------------------------------------+
-| ``METRICS_INSTRUMENT_COMMONS_POOL``  |         | Enable metrics on Apache commons-pool pools  |
-+--------------------------------------+---------+----------------------------------------------+
-| ``METRICS_INSTRUMENT_POSTGRES``      |         | Enable metrics on PostgreSQL                 |
-+--------------------------------------+---------+----------------------------------------------+
-
-Or, when using ``geomesa.metrics.registry.config``, use the following keys:
-
-::
-
-    # tags are key-value string pairs added to metrics from the given instrumentation
-    classloader  = { enabled = true, tags = {} }
-    memory       = { enabled = true, tags = {} }
-    gc           = { enabled = true, tags = {} }
-    processor    = { enabled = true, tags = {} }
-    threads      = { enabled = true, tags = {} }
-    commons-pool = { enabled = true, tags = {} }
-    postgres     = { enabled = true, tags = {} }
-
-.. note::
-
-    PostgreSQL metrics are only available in the :ref:`postgis_index_page`.
-
-Standalone Usage
-----------------
-
-Registries can also be managed programmatically. This allows for greater flexibility and usage outside the normal GeoMesa
-workflows. Configuration can be managed as detailed above.
+GeoMesa provide built-in :ref:`geomesa_metrics` integrations. However, registries can also be managed programmatically, which
+allows for greater flexibility and usage outside the normal GeoMesa workflows. Configuration is detailed in :ref:`geomesa_metrics`.
 
 Prometheus
 ^^^^^^^^^^
@@ -183,7 +42,7 @@ To idempotently create a Prometheus registry and attach it to the global registr
         import com.typesafe.config.ConfigFactory;
         import org.locationtech.geomesa.metrics.micrometer.prometheus.PrometheusFactory;
 
-        // call close() on the result to shut down the server
+        // call close() on the result to deregister the server
         // this call uses environment variables to configure the registry
         Closeable registry = PrometheusFactory.register();
         // alternatively, use the method that accepts configuration directly
@@ -199,7 +58,7 @@ To idempotently create a Prometheus registry and attach it to the global registr
         import com.typesafe.config.ConfigFactory
         import org.locationtech.geomesa.metrics.micrometer.prometheus.PrometheusFactory
 
-        // call close() on the result to shut down the server
+        // call close() on the result to deregister the server
         // this call uses environment variables to configure the registry
         val registry: Closeable = PrometheusFactory.register()
         // alternatively, use the method that accepts configuration directly
@@ -279,7 +138,7 @@ To idempotently start a Cloudwatch registry and attach it to the global registry
         import com.typesafe.config.ConfigFactory;
         import org.locationtech.geomesa.metrics.micrometer.cloudwatch.CloudwatchFactory;
 
-        // call close() on the result to shut down the server
+        // call close() on the result to deregister the server
         // this call uses environment variables to configure the registry (see below)
         Closeable registry = CloudwatchFactory.register();
         // or, for full control pass in a configuration
@@ -290,7 +149,7 @@ To idempotently start a Cloudwatch registry and attach it to the global registry
         import com.typesafe.config.ConfigFactory
         import org.locationtech.geomesa.metrics.micrometer.cloudwatch.CloudwatchFactory
 
-        // call close() on the result to shut down the server
+        // call close() on the result to deregister the server
         // this call uses environment variables to configure the registry (see below)
         val registry: Closeable = CloudwatchFactory.register()
         // or, for full control pass in a configuration
@@ -315,9 +174,3 @@ pooling. First, ensure that the ``commons-pool`` instrumentation is enabled (abo
         import org.locationtech.geomesa.metrics.micrometer.dbcp2.MetricsDataSource
 
         val dataSource = new MetricsDataSource()
-
-Custom Registries
------------------
-
-If the provided registries are not sufficient, metrics can be exposed by programmatically adding any Micrometer registry to the
-`global registry <https://docs.micrometer.io/micrometer/reference/concepts/registry.html#_global_registry>`__.

docs/user/datastores/index.rst

@@ -16,7 +16,6 @@ Not sure where to begin? Take a look at :doc:`/user/getting_started`.
     json
     index_overview
     index_basics
-    index_ext
     index_config
     runtime_config
     query_planning
@@ -26,6 +25,7 @@ Not sure where to begin? Take a look at :doc:`/user/getting_started`.
     analytic_queries
     security
     auditing
+    metrics
     data
     reserved_words
 

docs/user/datastores/index_basics.rst

@@ -106,7 +106,7 @@ To prioritize certain attributes over others, see :ref:`attribute_cardinality`.
 .. _index_versioning:
 
 Index Versioning
-================
+----------------
 
 In order to ensure cross-compatibility, each index created by GeoMesa has a version number that identifies
 the layout of data on disk, which is fixed at the time of creation. Updating GeoMesa versions
@@ -215,3 +215,32 @@ back-end-specific index version numbers.
 
 Note that GeoMesa versions prior to 1.2.2 included a geohash index. That index has been replaced with
 the Z indices and is no longer supported.
+
+Additional Index Implementations
+--------------------------------
+
+GeoMesa supports additional index formats at runtime, using
+`Java service providers <https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html>`__. To add a new
+index, implement ``org.locationtech.geomesa.index.api.GeoMesaFeatureIndexFactory`` and register your class under
+``META-INF/services``, as described in the
+`Java documentation <https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html>`__.
+
+Once an index is registered, it can be enabled through the ``SimpleFeatureType`` user data, as described in
+:ref:`customizing_index_creation`.
+
+Some additional indices are provided out-of-the-box:
+
+S2/S3 Index
+^^^^^^^^^^^
+
+The S2 and S3 indices are based on the `S2 library <https://github.com/google/s2geometry>`__:
+
+.. pull-quote::
+
+  The S2 library represents all data on a three-dimensional sphere (similar to a globe). This makes it
+  possible to build a worldwide geographic database with no seams or singularities, using a single coordinate
+  system, and with low distortion everywhere compared to the true shape of the Earth.
+
+The S2 index is a spatial index, identified by the name ``s2``. The S3 index is a composite spatial and time index,
+identified by the name ``s3``. The default time period for the S3 index can be configured through the user data
+key ``geomesa.s3.interval`` (see :ref:`customizing_z_index` for reference).