Add code snippets inclusion tool

Author: swallez

docs/reference/api-conventions/blocking-async.md

@@ -15,10 +15,9 @@ ElasticsearchTransport transport = ...
 :::{include} {doc-tests-src}/api_conventions/ApiConventionsTest.java[blocking-and-async]
 ```
 -->
-
+% :::include::start -- do not remove
 ```java
 ElasticsearchTransport transport = ...
-
 // Synchronous blocking client
 ElasticsearchClient client = new ElasticsearchClient(transport);
 
@@ -40,6 +39,7 @@ asyncClient
         }
     });
 ```
+% :::include::end -- do not remove
 
 Although we won’t go in deeper details on asynchronous programming in Java, remember to handle failures of asynchronous tasks. It’s easy to overlook them and have errors go unnoticed.
 

docs/reference/api-conventions/building-objects.md

@@ -10,6 +10,13 @@ mapped_pages:
 
 All data types in the Java API Client are immutable. Object creation uses the [builder pattern](https://www.informit.com/articles/article.aspx?p=1216151&seqNum=2) that was popularized in **Effective Java** in 2008.
 
+<!-- :::include
+```java
+ElasticsearchClient client = ...
+:::{include} {doc-tests-src}/api_conventions/ApiConventionsTest.java[builders]
+```
+-->
+% :::include::start -- do not remove
 ```java
 ElasticsearchClient client = ...
 CreateIndexResponse createResponse = client.indices().create(
@@ -21,6 +28,7 @@ CreateIndexResponse createResponse = client.indices().create(
         .build()
 );
 ```
+% :::include::end -- do not remove
 
 Note that a builder should not be reused after its `build()` method has been called.
 
@@ -29,6 +37,13 @@ Note that a builder should not be reused after its `build()` method has been cal
 
 Although this works nicely, having to instantiate builder classes and call the `build()` method is a bit verbose. So every property setter in the Java API Client also accepts a lambda expression that takes a newly created builder as a parameter and returns a populated builder. The snippet above can also be written as:
 
+<!-- :::include
+```java
+ElasticsearchClient client = ...
+:::{include} {doc-tests-src}/api_conventions/ApiConventionsTest.java[builder-lambdas]
+```
+-->
+% :::include::start -- do not remove
 ```java
 ElasticsearchClient client = ...
 CreateIndexResponse createResponse = client.indices()
@@ -39,11 +54,19 @@ CreateIndexResponse createResponse = client.indices()
         )
     );
 ```
+% :::include::end -- do not remove
 
 This approach allows for much more concise code, and also avoids importing classes (and even remembering their names) since types are inferred from the method parameter signature.
 
 Note in the above example that builder variables are only used to start a chain of property setters. The names of these variables are therefore unimportant and can be shortened to improve readability:
 
+<!-- :::include
+```java
+ElasticsearchClient client = ...
+:::{include} {doc-tests-src}/api_conventions/ApiConventionsTest.java[builder-lambdas-short]
+```
+-->
+% :::include::start -- do not remove
 ```java
 ElasticsearchClient client = ...
 CreateIndexResponse createResponse = client.indices()
@@ -54,11 +77,19 @@ CreateIndexResponse createResponse = client.indices()
         )
     );
 ```
+% :::include::end -- do not remove
 
 Builder lambdas become particularly useful with complex nested queries like the one below, taken from the [intervals query API documentation](elasticsearch://reference/query-languages/query-dsl/query-dsl-intervals-query.md).
 
 This example also highlights a useful naming convention for builder parameters in deeply nested structures. For lambda expressions with a single argument, Kotlin provides the implicit `it` parameter and Scala allows use of `_`. This can be approximated in Java by using an underscore or a single letter prefix followed by a number representing the depth level (i.e. `_0`, `_1`, or `b0`, `b1` and so on). Not only does this remove the need to create throw-away variable names, but it also improves code readability. Correct indentation also allows the structure of the query to stand out.
 
+<!-- :::include
+```java
+ElasticsearchClient client = ...
+:::{include} {doc-tests-src}/api_conventions/ApiConventionsTest.java[builder-intervals]
+```
+-->
+% :::include::start -- do not remove
 ```java
 ElasticsearchClient client = ...
 SearchResponse<SomeApplicationData> results = client
@@ -92,9 +123,10 @@ SearchResponse<SomeApplicationData> results = client
                 )
             )
         ),
-    SomeApplicationData.class <1>
+    SomeApplicationData.class // <1>
 );
 ```
+% :::include::end -- do not remove
 
 1. Search results will be mapped to `SomeApplicationData` instances to be readily available to the application.
 

docs/reference/api-conventions/lists-maps.md

@@ -12,6 +12,12 @@ Properties of type `List` and `Map` are exposed by object builders as a set of o
 
 Object builders create immutable objects, and this also applies to list and map properties that are made immutable at object construction time.
 
+<!-- :::include
+```java
+:::{include} {doc-tests-src}/api_conventions/ApiConventionsTest.java[collections]
+```
+-->
+% :::include::start -- do not remove
 ```java
 // Prepare a list of index names
 List<String> names = Arrays.asList("idx-a", "idx-b", "idx-c");
@@ -47,7 +53,7 @@ SearchRequest search = SearchRequest.of(r -> r
         a -> a.histogram(h -> h.field("price")))
 );
 ```
-
+% :::include::end -- do not remove
 
 ## List and map values are never `null` [_list_and_map_values_are_never_null]
 
@@ -57,6 +63,12 @@ For lists and maps however, applications often only care about whether they’re
 
 If you ever need to distinguish between a missing (undefined) optional collection and an effectively-empty collection returned by {{es}}, the `ApiTypeHelper` class provides a utility method to distinguish them:
 
+<!-- :::include
+```java
+:::{include} {doc-tests-src}/api_conventions/ApiConventionsTest.java[optional-collections]
+```
+-->
+% :::include::start -- do not remove
 ```java
 NodeStatistics stats = NodeStatistics.of(b -> b
     .total(1)
@@ -72,6 +84,7 @@ assertEquals(0, stats.failures().size());
 // - and if needed we can know it was actually not defined
 assertFalse(ApiTypeHelper.isDefined(stats.failures()));
 ```
+% :::include::end -- do not remove
 
 :::{include} /reference/_snippets/doc-tests-blurb.md
 :::

docs/reference/api-conventions/loading-json.md

@@ -31,17 +31,24 @@ Consider a resource file `some-index.json` containing an index definition:
 
 You can create an index from that definition as follows:
 
+<!-- :::include
+```java
+:::{include} {doc-tests-src}/api_conventions/LoadingJsonTest.java[load-index]
+```
+-->
+% :::include::start -- do not remove
 ```java
 InputStream input = this.getClass()
-    .getResourceAsStream("some-index.json"); <1>
+    .getResourceAsStream("some-index.json"); // <1>
 
 CreateIndexRequest req = CreateIndexRequest.of(b -> b
     .index("some-index")
-    .withJson(input) <2>
+    .withJson(input) // <2>
 );
 
 boolean created = client.indices().create(req).acknowledged();
 ```
+% :::include::end -- do not remove
 
 1. open an input stream for the JSON resource file.
 2. populate the index creation request with the resource file contents.
@@ -52,10 +59,16 @@ boolean created = client.indices().create(req).acknowledged();
 
 Similarly, you can read documents to be stored in {{es}} from data files:
 
+<!-- :::include
+```java
+:::{include} {doc-tests-src}/api_conventions/LoadingJsonTest.java[ingest-data]
+```
+-->
+% :::include::start -- do not remove
 ```java
 FileReader file = new FileReader(new File(dataDir, "document-1.json"));
 
-IndexRequest<JsonData> req; <1>
+IndexRequest<JsonData> req; // <1>
 
 req = IndexRequest.of(b -> b
     .index("some-index")
@@ -64,6 +77,7 @@ req = IndexRequest.of(b -> b
 
 client.index(req);
 ```
+% :::include::end -- do not remove
 
 1. when calling `withJson()` on data structures that have generic type parameters, these generic types will be considered to be `JsonData`.
 
@@ -73,6 +87,12 @@ client.index(req);
 
 You can combine `withJson()` with regular calls to setter methods. The example below loads the query part of a search request from a `String` and programmatically adds an aggregation.
 
+<!-- :::include
+```java
+:::{include} {doc-tests-src}/api_conventions/LoadingJsonTest.java[query]
+```
+-->
+% :::include::start -- do not remove
 ```java
 Reader queryJson = new StringReader(
     "{" +
@@ -86,8 +106,8 @@ Reader queryJson = new StringReader(
     "}");
 
 SearchRequest aggRequest = SearchRequest.of(b -> b
-    .withJson(queryJson) <1>
-    .aggregations("max-cpu", a1 -> a1 <2>
+    .withJson(queryJson) // <1>
+    .aggregations("max-cpu", a1 -> a1 // <2>
         .dateHistogram(h -> h
             .field("@timestamp")
             .calendarInterval(CalendarInterval.Hour)
@@ -100,9 +120,10 @@ SearchRequest aggRequest = SearchRequest.of(b -> b
 );
 
 Map<String, Aggregate> aggs = client
-    .search(aggRequest, Void.class) <3>
+    .search(aggRequest, Void.class) // <3>
     .aggregations();
 ```
+% :::include::end -- do not remove
 
 1. loads the query from the JSON string.
 2. adds the aggregation.
@@ -114,6 +135,12 @@ Map<String, Aggregate> aggs = client
 
 The `withJson()` methods are partial deserializers: the properties loaded from the JSON will set property values or replace the previous ones, but will not reset other properties not found in the JSON input. You can use this to combine multiple JSON snippets to build complex search requests. In the example below, we combine separate definitions of a query that selects some documents and an aggregation that is run on the results of this query.
 
+<!-- :::include
+```java
+:::{include} {doc-tests-src}/api_conventions/LoadingJsonTest.java[query-and-agg]
+```
+-->
+% :::include::start -- do not remove
 ```java
 Reader queryJson = new StringReader(
     "{" +
@@ -124,12 +151,12 @@ Reader queryJson = new StringReader(
     "      }" +
     "    }" +
     "  }," +
-    "  \"size\": 100" + <1>
+    "  \"size\": 100" + // <1>
     "}");
 
 Reader aggregationJson = new StringReader(
     "{" +
-    "  \"size\": 0, " + <2>
+    "  \"size\": 0, " + // <2>
     "  \"aggregations\": {" +
     "    \"hours\": {" +
     "      \"date_histogram\": {" +
@@ -148,15 +175,16 @@ Reader aggregationJson = new StringReader(
     "}");
 
 SearchRequest aggRequest = SearchRequest.of(b -> b
-    .withJson(queryJson) <3>
-    .withJson(aggregationJson) <4>
-    .ignoreUnavailable(true) <5>
+    .withJson(queryJson) // <3>
+    .withJson(aggregationJson) // <4>
+    .ignoreUnavailable(true) // <5>
 );
 
 Map<String, Aggregate> aggs = client
     .search(aggRequest, Void.class)
     .aggregations();
 ```
+% :::include::end -- do not remove
 
 1. set max number of returned document to 100 for queries.
 2. we do not want any matching document in aggregations.