Merge pull request #910 from Lorak-mmk/book-preprocessor

Docs: Add a book preprocessor

Author: piodul

.github/workflows/book.yml

@@ -33,7 +33,5 @@ jobs:
       run: cargo build --verbose --examples
     - name: Build the book
       run: mdbook build docs
-    - name: Build the book using the script
-      run: python3 docs/build_book.py
     - name: Run book tests
       run: mdbook test -L target/debug/deps docs

CONTRIBUTING.md

@@ -72,17 +72,20 @@ But they are removed when building the book
 cargo install mdbook
 ```
 
-Build the book (simple method, contains `Sphinx` artifacts):
+Book build process uses preprocessor to remove Sphinx artifacts.
+Due to limitation of mdbook, it can only be built either from main directory,
+using `mdbook X docs` or from `docs` directory, using `mdbook X`, where
+`X` is mdbook command such as `build` / `serve` / `test` etc.
+
+If the book is built from another directory (e.g. scylla, using `mdbook build ../docs`),
+preprocessor won't be found, so the result will contain Sphinx artifacts.
+
+Build the book.
 ```bash
 mdbook build docs
 # HTML will be in docs/book
 ```
 
-To build the release version use a script which automatically removes `Sphinx` chunks:
-```bash
-python3 docs/build_book.py
-# HTML will be in docs/book/scriptbuild/book
-```
 
 Or serve it on a local http server (automatically refreshes on changes)
 ```bash

Makefile

@@ -46,7 +46,7 @@ build:
 
 .PHONY: docs
 docs:
-	./docs/build_book.py
+	mdbook build docs
 
 .PHONY: up
 up:

docs/book.toml

@@ -11,4 +11,13 @@ edition = "2021"
 
 [output.html]
 default-theme = "ayu"
-git-repository-url = "https://github.com/scylladb/scylla-rust-driver"
+git-repository-url = "https://github.com/scylladb/scylla-rust-driver"
+
+[preprocessor.sphinx]
+command = '''bash -c '
+if test -f "./docs/sphinx_preprocessor.py"; then
+    python ./docs/sphinx_preprocessor.py "$@" 
+else 
+    python ./sphinx_preprocessor.py "$@"
+fi' run_preprocessor
+'''

docs/build_book.py

@@ -0,0 +1,34 @@
+import json
+import sys
+
+def remove_sphinx_markdown(md_file_text):
+    text_split = md_file_text.split("```eval_rst")
+    
+    result = text_split[0]
+
+    for i in range(1, len(text_split)):
+        cur_chunk = text_split[i]
+        result += cur_chunk[cur_chunk.find("```") + 3:]
+
+    return result
+
+def process_section(section):
+    if 'Chapter' in section:
+        section['Chapter']['content'] = remove_sphinx_markdown(section['Chapter']['content'])
+        for s in section['Chapter']['sub_items']:
+            process_section(s)
+
+if __name__ == '__main__':
+    if len(sys.argv) > 1: # we check if we received any argument
+        if sys.argv[1] == "supports": 
+            # then we are good to return an exit status code of 0, since the other argument will just be the renderer's name
+            sys.exit(0)
+
+    # load both the context and the book representations from stdin
+    context, book = json.load(sys.stdin)
+
+    for section in book['sections']:
+        process_section(section)
+
+    # we are done with the book's modification, we can just print it to stdout
+    print(json.dumps(book))