Enable index pruning by default (#5759)

The permissions its probes need were added to ppl_full_access in
opensearch-project/security#6471.

Signed-off-by: Chen Dai <daichen@amazon.com>

Author: dai-chen

docs/user/admin/settings.rst

@@ -214,22 +214,22 @@ Version
 Description
 -----------
 
-Prunes a wildcard index expression down to the concrete indices that can hold data in the query's ``@timestamp`` range, so fewer indices and shards are touched. The primary use currently is to avoid exhausting the open point-in-time (PIT) context limit when a query would otherwise open a reader context over many indices.
+Prunes a wildcard index expression down to the concrete indices that can hold data in the query's ``@timestamp`` range, so fewer indices and shards are touched. The primary use currently is to avoid exhausting the open point-in-time (PIT) context limit when a query would otherwise open a reader context over many indices. Enabled by default.
 
-Pruning only applies to a wildcard expression whose query filters on a ``@timestamp`` range; anything else is left untouched, and any failure while probing the cluster falls back to querying the full expression. Weigh these limitations before enabling it:
+Pruning only applies to a wildcard expression whose query filters on a ``@timestamp`` range; anything else is left untouched, and any failure while probing the cluster falls back to querying the full expression. Weigh these limitations before turning it off:
 
 1. An index whose shards are all unavailable is pruned rather than reported, because ``_field_caps`` does not surface per-index failures. Such a query returns fewer rows instead of an error.
 2. Pruning fixes the list of index names, so an index created or deleted between pruning and PIT creation, by a rollover or retention policy for instance, is missed or fails the query. The interval between the two is short, so this is unlikely in practice.
 3. An expression that matches an alias or a data stream is never pruned, because a filtered alias contributes a filter and routing that are resolved from the expression itself and so would be silently dropped.
-4. Pruning probes the cluster with the ``indices:admin/resolve/index`` and ``indices:data/read/field_caps`` actions. A principal lacking either permission falls back to querying the full expression silently, so pruning simply never takes effect.
+4. Pruning probes the cluster with the ``indices:admin/resolve/index`` and ``indices:data/read/field_caps*`` actions, both granted by the ``ppl_full_access`` role of the security plugin since 3.9. A principal lacking either permission falls back to querying the full expression silently, so pruning simply never takes effect.
 
 Pruning is also skipped when it would not reduce the read, that is when no index is excluded. The query then uses the original wildcard expression and reads exactly the same indices.
 
-Enable it with::
+Disable it with::
 
 	>> curl -H 'Content-Type: application/json' -X PUT localhost:9200/_plugins/_query/settings -d '{
 	  "transient" : {
-	    "plugins.query.pruning.enabled" : true
+	    "plugins.query.pruning.enabled" : false
 	  }
 	}'
 
@@ -242,7 +242,7 @@ Result set::
         "plugins" : {
           "query" : {
             "pruning" : {
-              "enabled" : "true"
+              "enabled" : "false"
             }
           }
         }
@@ -251,7 +251,7 @@ Result set::
 
 Settings:
 
-1. The default value is false.
+1. The default value is true.
 2. This setting is node scope.
 3. This setting can be updated dynamically.
 

integ-test/src/yamlRestTest/resources/rest-api-spec/test/ppl/index_pruning.yml

@@ -21,7 +21,6 @@ setup:
             plugins.calcite.enabled: true
             # Without this a Calcite failure is silently answered by the v2 engine, which never prunes.
             plugins.calcite.fallback.allowed: false
-            plugins.query.pruning.enabled: true
   - do:
       indices.put_index_template:
         name: pruning-it

opensearch/src/main/java/org/opensearch/sql/opensearch/setting/OpenSearchSettings.java

@@ -220,7 +220,7 @@ public class OpenSearchSettings extends Settings {
   public static final Setting<?> QUERY_PRUNING_ENABLED_SETTING =
       Setting.boolSetting(
           Key.QUERY_PRUNING_ENABLED.getKeyValue(),
-          false,
+          true,
           Setting.Property.NodeScope,
           Setting.Property.Dynamic);
 

opensearch/src/test/java/org/opensearch/sql/opensearch/setting/OpenSearchSettingsTest.java

@@ -125,6 +125,18 @@ void testPplValuesMaxLimitSetting() {
     assertEquals(5000, newLimit);
   }
 
+  @Test
+  void testQueryPruningEnabledSetting() {
+    when(clusterSettings.get(ClusterName.CLUSTER_NAME_SETTING)).thenReturn(ClusterName.DEFAULT);
+    when(clusterSettings.get(not((eq(ClusterName.CLUSTER_NAME_SETTING))))).thenReturn(null);
+    OpenSearchSettings settings = new OpenSearchSettings(clusterSettings);
+
+    assertEquals(true, settings.getSettingValue(Settings.Key.QUERY_PRUNING_ENABLED));
+
+    settings.new Updater(Settings.Key.QUERY_PRUNING_ENABLED).accept(false);
+    assertEquals(false, settings.getSettingValue(Settings.Key.QUERY_PRUNING_ENABLED));
+  }
+
   @Test
   void testDeserializationStructuralLimitSettings() {
     when(clusterSettings.get(ClusterName.CLUSTER_NAME_SETTING)).thenReturn(ClusterName.DEFAULT);