Simplify generated code module structure (#138)

Remove the `generated` directory and the various re-exports.

- namespace docs have been extracted from the respective `mod.rs` to
  `.md` files in api_generator/docs, and are inserted in the generated
  source file.
- namespace sources are generated directly in the `src` directory
- the generator can now insert generated sections in regular source
  files based on `GENERATE-BEGIN` and `GENERATE-END` markers. This is
  used to generate the namespace module list in the main `lib.rs` and
  merge generated with manual code in `params.rs`
- generated and merged source files are listed in `src/.generated.toml`
  so that the generator can cleanup on start.

Before considering merging generated code with special markers we
experimented with the `include!()` macro, but this didn't work well:
- rustfmt will not reformat include files
- `mod` statement look for files relative to the source file, so
  include files have to be located in the same directory as their
  includer, while locating them in a dedicated directory would have
  provided a nicer organization.

Merging generated sections provides a more natural source code layout.

Author: swallez

api_generator/Cargo.toml

@@ -16,6 +16,7 @@ globset = "~0.4"
 Inflector = "0.11.4"
 indicatif = "0.12.0"
 lazy_static = "1.4.0"
+path-slash = "0.1.3"
 quote = "~0.3"
 reduce = "0.1.2"
 regex = "1.3.1"
@@ -26,5 +27,9 @@ serde_json = "~1"
 serde_derive = "~1"
 syn = { version = "~0.11", features = ["full"] }
 tar = "~0.4"
+toml = "0.5.6"
 url = "2.1.1"
-void = "1.0.2"
+void = "1.0.2"
+
+[dev-dependencies]
+tempfile = "3.1.0"

elasticsearch/docs/Elasticsearch.fn.scroll.md renamed to api_generator/docs/functions/Elasticsearch.scroll.md

@@ -0,0 +1,5 @@
+Async Search APIs
+
+[Async search APIs](https://www.elastic.co/guide/en/elasticsearch/reference/master/async-search.html)
+let you asynchronously execute a search request, monitor its progress, and retrieve
+partial results as they become available.

elasticsearch/docs/Indices.fn.create.md renamed to api_generator/docs/functions/Indices.create.md

@@ -0,0 +1,94 @@
+Cat APIs
+
+The [Cat APIs](https://www.elastic.co/guide/en/elasticsearch/reference/master/cat.html) aim to
+meet the needs of humans when looking at data returned from Elasticsearch,
+formatting it as compact, column aligned text, making it easier on human eyes.
+
+# Plain text responses
+
+By default, all Cat APIs are configured to send requests with `text/plain` content-type
+and accept headers, returning plain text responses
+
+```rust,no_run
+# use elasticsearch::{Elasticsearch, Error, SearchParts};
+# use url::Url;
+# use elasticsearch::auth::Credentials;
+# use serde_json::{json, Value};
+# async fn doc() -> Result<(), Box<dyn std::error::Error>> {
+# let client = Elasticsearch::default();
+let response = client
+    .cat()
+    .nodes()
+    .send()
+    .await?;
+
+let response_body = response.text().await?;
+# Ok(())
+# }
+```
+
+# JSON responses
+
+JSON responses can be returned from Cat APIs either by using `.format("json")`
+
+```rust,no_run
+# use elasticsearch::{Elasticsearch, Error, SearchParts};
+# use url::Url;
+# use elasticsearch::auth::Credentials;
+# use serde_json::{json, Value};
+# async fn doc() -> Result<(), Box<dyn std::error::Error>> {
+# let client = Elasticsearch::default();
+let response = client
+    .cat()
+    .nodes()
+    .format("json")
+    .send()
+    .await?;
+
+let response_body = response.json::<Value>().await?;
+# Ok(())
+# }
+```
+
+Or by setting an accept header using `.headers()`
+
+```rust,no_run
+# use elasticsearch::{Elasticsearch, Error, SearchParts, http::headers::{HeaderValue, DEFAULT_ACCEPT, ACCEPT}};
+# use url::Url;
+# use serde_json::{json, Value};
+# async fn doc() -> Result<(), Box<dyn std::error::Error>> {
+# let client = Elasticsearch::default();
+let response = client
+    .cat()
+    .nodes()
+    .header(ACCEPT, HeaderValue::from_static(DEFAULT_ACCEPT))
+    .send()
+    .await?;
+
+let response_body = response.json::<Value>().await?;
+# Ok(())
+# }
+```
+
+# Column Headers
+
+The column headers to return can be controlled with `.h()`
+
+```rust,no_run
+# use elasticsearch::{Elasticsearch, Error, SearchParts};
+# use url::Url;
+# use serde_json::{json, Value};
+# async fn doc() -> Result<(), Box<dyn std::error::Error>> {
+# let client = Elasticsearch::default();
+let response = client
+    .cat()
+    .nodes()
+    .h(&["ip", "port", "heapPercent", "name"])
+    .send()
+    .await?;
+
+let response_body = response.json::<String>().await?;
+# Ok(())
+# }
+```
+

elasticsearch/docs/Indices.fn.put_mapping.md renamed to api_generator/docs/functions/Indices.put_mapping.md

@@ -0,0 +1,7 @@
+Cross-cluster Replication APIs
+
+[Enable replication of indices in remote clusters to a local cluster](https://www.elastic.co/guide/en/elasticsearch/reference/master/xpack-ccr.html).
+This functionality can be used in some common production use cases:
+
+- Disaster recovery in case a primary cluster fails. A secondary cluster can serve as a hot backup
+- Geo-proximity so that reads can be served locally

api_generator/docs/namespaces/async_search.md

@@ -0,0 +1,4 @@
+Cluster APIs
+
+[Manage settings](https://www.elastic.co/guide/en/elasticsearch/reference/master/cluster.html),
+perform operations, and retrieve information about an Elasticsearch cluster.

api_generator/docs/namespaces/cat.md

@@ -0,0 +1,8 @@
+Dangling Index APIs
+
+If Elasticsearch encounters index data that is absent from the current cluster state,
+those indices are considered to be _dangling_. For example, this can happen if you delete
+more than `cluster.indices.tombstones.size` number of indices while an Elasticsearch node
+is offline.
+
+The dangling indices APIs can list, import and delete dangling indices.

api_generator/docs/namespaces/ccr.md

@@ -0,0 +1,6 @@
+Enrich APIs
+
+Manage [enrich policies](https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest-enriching-data.html#enrich-policy)
+that can be used by the [enrich processor](https://www.elastic.co/guide/en/elasticsearch/reference/master/enrich-processor.html)
+as part of an [ingest pipeline](../ingest/index.html), to add data from your existing indices
+to incoming documents during ingest.