Merge branch 'main' of https://github.com/ClickHouse/clickhouse-docs into Blargian-patch-33

Author: Blargian

.github/workflows/badges-notify.yml

@@ -0,0 +1,57 @@
+name: Weekly Badge Monitor
+
+on:
+  schedule:
+    # Run every Monday at 10:00 AM Amsterdam time (UTC+1/+2)
+    # Using 8:00 UTC to account for Amsterdam time (adjusts for DST)
+    - cron: '0 8 * * 1'
+  workflow_dispatch: # Allow manual triggering
+
+jobs:
+  monitor-badges:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Make script executable
+        run: chmod +x scripts/badger.sh
+
+      - name: Run badge finder script
+        id: find_badges
+        run: |
+          # Run the script and capture output
+          SCRIPT_OUTPUT=$(scripts/badger.sh)
+          
+          # Escape the output for JSON and store in GitHub output
+          # Replace newlines with \n and escape quotes
+          ESCAPED_OUTPUT=$(echo "$SCRIPT_OUTPUT" | sed 's/\\/\\\\/g' | sed 's/"/\\"/g' | sed ':a;N;$!ba;s/\n/\\n/g')
+          
+          echo "badges_output<<EOF" >> $GITHUB_OUTPUT
+          echo "$ESCAPED_OUTPUT" >> $GITHUB_OUTPUT
+          echo "EOF" >> $GITHUB_OUTPUT
+
+      - name: Send webhook
+        env:
+          WEBHOOK_URL: ${{ secrets.DOCS_BADGER }}
+          BADGES_OUTPUT: ${{ steps.find_badges.outputs.badges_output }}
+        run: |
+          # Create JSON payload
+          JSON_PAYLOAD=$(cat <<EOF
+          {
+            "badges_list": "$BADGES_OUTPUT"
+          }
+          EOF
+          )
+          
+          # Send webhook
+          curl -X POST \
+            -H "Content-Type: application/json" \
+            -d "$JSON_PAYLOAD" \
+            "$WEBHOOK_URL"
+
+      - name: Log completion
+        run: |
+          echo "Badge monitoring completed successfully"
+          echo "Webhook sent to configured endpoint"

.gitignore

@@ -55,16 +55,22 @@ static/knowledgebase_toc.json
 .frontmatter-validation-failed
 logs/
 
+# Landing page files with autogenerated table of contents
+docs/cloud/reference/release-notes-index.md
+docs/cloud/manage/api/api-reference-index.md
+docs/getting-started/index.md
+docs/data-modeling/projections/index.md
+docs/cloud/manage/jan2025_faq/index.md
+docs/chdb/guides/index.md
+docs/use-cases/AI_ML/index.md
+docs/use-cases/AI_ML/MCP/index.md
+docs/use-cases/AI_ML/MCP/ai_agent_libraries/index.md
+docs/integrations/data-ingestion/clickpipes/kafka/index.md
+
 .vscode
 .aspell.en.prepl
 *.md.bak
 
 **.translated
 **.translate
 /ClickHouse/
-
-# Ignore table of contents files
-docs/cloud/reference/release-notes-index.md
-docs/whats-new/changelog/index.md
-docs/cloud/manage/api/api-reference-index.md
-docs/getting-started/index.md

README.md

@@ -1,8 +1,8 @@
 <div align=center>
 
-![Website](https://img.shields.io/website?up_message=AVAILABLE&down_message=DOWN&url=https%3A%2F%2Fclickhouse.com%2Fdocs&style=for-the-badge)
+[![Website](https://img.shields.io/website?up_message=AVAILABLE&down_message=DOWN&url=https%3A%2F%2Fclickhouse.com%2Fdocs&style=for-the-badge)](https://clickhouse.com)
 [![CC BY-NC-SA 4.0 License](https://img.shields.io/badge/license-CC-blueviolet?style=for-the-badge)](http://creativecommons.org/licenses/by-nc-sa/4.0/)
-![Checks](https://img.shields.io/github/actions/workflow/status/clickhouse/clickhouse-docs/debug.yml?style=for-the-badge&label=Checks)
+[![Checks](https://img.shields.io/github/actions/workflow/status/clickhouse/clickhouse-docs/debug.yml?style=for-the-badge&label=Checks)](https://github.com/ClickHouse/clickhouse-docs/actions)
 
 <picture align=center>
     <source media="(prefers-color-scheme: dark)" srcset="https://github.com/ClickHouse/clickhouse-docs/assets/9611008/4ef9c104-2d3f-4646-b186-507358d2fe28">
@@ -18,19 +18,19 @@
 
 ClickHouse is blazing fast, but understanding ClickHouse and using it effectively is a journey. The documentation is your source for gaining the knowledge you need to be successful with your ClickHouse projects and applications. [Head over to clickhouse.com/docs to learn more →](https://clickhouse.com/)
 
-## Table of contents {#table-of-contents}
+## Table of contents
 
 - [About this repo](#about-this-repo)
 - [Run locally](#run-locally)
 - [Contributing](#contributing)
 - [Issues](#issues)
 - [License](#license)
 
-## About this repo {#about-this-repo}
+## About this repo
 
 This repository manages the documentation for [ClickHouse](https://clickhouse.com/docs). The content is built with [Docusaurus](https://docusaurus.io/) and hosted on [Vercel](https://vercel.com). Documentation content is written in Markdown and is held in the `/docs` directory.
 
-## Run locally {#run-locally}
+## Run locally
 
 You can run a copy of this website locally within a few steps. Some folks find this useful when contributing so they can see precisely what their changes will look like on the production site.
 
@@ -116,23 +116,19 @@ To check spelling and markdown is correct locally run:
 yarn check-style
 ```
 
-### Notes {#notes}
+### Notes
 
 Here are some things to keep in mind when building a local copy of the ClickHouse docs site.
 
-#### Build-time {#build-time}
-
-Due to the complex structure of this repo, the docs site can take some time to build locally. As a benchmark, it takes ~3 minutes to build on an M1 Macbook with 8GB RAM.
-
-#### Redirects {#redirects}
+#### Redirects
 
 Due to how the local server is built, redirects will not work. For example, visiting `clickhouse.com/docs` on the production site will lead you to `clickhouse.com/docs/intro`. However, on a local copy of the site, you will see a 404 page if you try to visit `localhost:8000/docs`.
 
-## Contributing {#contributing}
+## Contributing
 
 Want to help out? Contributions are always welcome! If you want to help out but aren't sure where to start, check out the [issues board](https://github.com/clickhouse/clickhouse-docs/issues).
 
-### Pull requests {#pull-requests}
+### Pull requests
 
 Please assign any pull request (PR) against an issue; this helps the docs team track who is working on what and what each PR is meant to address. If there isn't an issue for the specific _thing_ you want to work on, quickly create one and comment so that it can be assigned to you. One of the repository maintainers will add you as an assignee.
 
@@ -153,7 +149,7 @@ yarn check-style
 For an overview of how reference documentation such as settings, system tables
 and functions are generated from the source code, see ["Generating documentation from source code"](/contribute/autogenerated-documentation-from-source.md)
 
-### Tests and CI/CD {#tests-and-cicd}
+### Tests and CI/CD
 
 There are five workflows that run against PRs in this repo:
 
@@ -165,45 +161,45 @@ There are five workflows that run against PRs in this repo:
 | [Scheduled Vercel build](https://github.com/ClickHouse/clickhouse-docs/blob/main/.github/workflows/scheduled-vercel-build.yml) | Builds the site every day at 00:10 UTC and hosts the build on Vercel. |
 | [Trigger build](https://github.com/ClickHouse/clickhouse-docs/blob/main/.github/workflows/trigger-build.yml) | Uses the [peter-evans/repository-dispatch@v2](https://github.com/peter-evans/repository-dispatch) workflow to create a repository dispatch. |
 
-### Quick contributions {#quick-contributions}
+### Quick contributions
 
 Have you noticed a typo or found some wonky formatting? For small contributions like these, it's usually faster and easier to make your changes directly in GitHub. Here's a quick guide to show you how the GitHub editor works:
 
 1. Each page in Clickhouse.com/docs has an **Edit this page** link at the top:
 
-    ![The ClickHouse Docs website with the edit button highlighted.](./static/images/contribute/readme-edit-this-page.png)
+    ![The ClickHouse Docs website with the edit button highlighted.](./static/images/contributing/readme-edit-this-page.png)
 
     Click this button to edit this page in GitHub.
 
 1. Once you're in GitHub, click the pencil icon to edit this page:
 
-    ![README Pencil Icon](./static/images/contribute/readme-pencil-icon.png)
+    ![README Pencil Icon](./static/images/contributing/readme-pencil-icon.png)
 
 1. GitHub will _fork_ the repository for you. This creates a copy of the `clickhouse-docs` repository on your personal GitHub account.
 1. Make your changes in the textbox. Once you're done, click **Commit changes**:
 
-    ![README Commit Changes](./static/images/contribute/readme-commit-changes.png)
+    ![README Commit Changes](./static/images/contributing/readme-commit-changes.png)
 
 1. In the **Propose changes** popup, enter a descriptive title to explain the changes you just made. Keep this title to 10 words or less. If your changes are fairly complex and need further explanation, enter your comments into the **Extended description** field.
 1. Make sure **Create a new branch** is selected, and click **Propose changes**:
 
-    ![README Propose Changes](./static/images/contribute/readme-propose-changes.png)
+    ![README Propose Changes](./static/images/contributing/readme-propose-changes.png)
 
 1. A new page should open with a new pull request. Double-check that the title and description are accurate.
 1. If you've spoken to someone on the docs team about your changes, tag them into the **Reviewers** section:
 
-    ![README Create Pull Request](./static/images/contribute/readme-create-pull-request.png)
+    ![README Create Pull Request](./static/images/contributing/readme-create-pull-request.png)
 
     If you haven't mentioned your changes to anyone yet, leave the **Reviewers** section blank.
 
 1. Click **Create pull request**.
 
 At this point, your pull request will be handed over to the docs team, who will review it and suggest or make changes where necessary.
 
-## Issues {#issues}
+## Issues
 
 Found a problem with the Clickhouse docs site? [Please raise an issue](https://github.com/clickhouse/clickhouse-docs/issues/new). Be as specific and descriptive as possible; screenshots help!
 
-## License {#license}
+## License
 
 This work is licensed under a [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-nc-sa/4.0/).

contribute/autogenerated-documentation-from-source.md

@@ -1,15 +1,18 @@
 # Generating documentation from source code
 
 A [script in our docs repo](https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/settings/autogenerate-settings.sh) 
-extracts setting names, descriptions, default values, etc. from ClickHouse's source code.
+is used to run SQL queries which generate markdown documentation from the system tables in ClickHouse and insert it 
+into the appropriate pages before build time of the docs.
 
 ## Session settings
 
 Documentation for session settings is autogenerated from source file [`src/Core/Settings.cpp`](https://github.com/ClickHouse/ClickHouse/blob/master/src/Core/Settings.cpp).
+[SQL](../scripts/settings/session-settings.sql)
 
 ## Format settings
 
 Documentation for session settings is autogenerated from source file [`src/Core/FormatFactorySettings.h`](https://github.com/ClickHouse/ClickHouse/blob/master/src/Core/FormatFactorySettings.h).
+[SQL](../scripts/settings/format-settings.sql)
 
 ## Server settings
 
@@ -43,14 +46,33 @@ If it is, then please edit the setting description in the source code documentat
 in [`ServerSettings.cpp`](https://github.com/ClickHouse/ClickHouse/blob/master/src/Core/ServerSettings.cpp) 
 or else edit `_server_settings_outside_source.md`.**
 
+[SQL](../scripts/settings/global-server-settings.sql)
+
 ## MergeTree settings
 
 Documentation for MergeTree settings is autogenerated from [MergeTreeSettings.cpp](https://github.com/ClickHouse/ClickHouse/blob/master/src/Storages/MergeTree/MergeTreeSettings.cpp)
 
+[SQL](../scripts/settings/mergetree-settings.sql)
+
 ## System tables
 
-TO DO. ETA April.
+TO DO
 
 ## Functions
 
-TO DO. ETA April/May.
+Documentation for functions is autogenerated from the documentation in system tables (`system.functions`) which is 
+registered along with the function registration in the C++ code.
+
+To add a new page:
+- add SQL to generate the markdown from system tables
+- update `autogenerate-settings.sh` in the section which specifies which markdown file contents to
+  copy to which file in between the `<!--AUTOGENERATED_START-->` and `<!--AUTOGENERATED_END-->`
+
+The following pages are autogenerated from the source code for functions:
+- [Arithmetic](/sql-reference/functions/arithmetic-functions) ([SQL](../scripts/settings/arithmetic-functions.sql))
+- [Arrays](/sql-reference/functions/array-functions) ([SQL](../scripts/settings/array-functions.sql))
+- [Bit](/sql-reference/functions/bit-functions) ([SQL](../scripts/settings/bit-functions.sql))
+- [Bitmap](/sql-reference/functions/bitmap-functions) ([SQL](../scripts/settings/bitmap-functions.sql))
+- [Comparison](/sql-reference/functions/comparison-functions) ([SQL](../scripts/settings/comparison-functions.sql))
+- [Conditional](/sql-reference/functions/conditional-functions) ([SQL](../scripts/settings/conditional-functions.sql))
+

contribute/style-guide.md

@@ -340,7 +340,7 @@ When using URL parameters to control which version of documentation is displayed
 there are conventions to follow for reliable functionality. 
 Here's how the `?v=v08` parameter relates to the snippet selection:
 
-#### How It Works
+#### How it works
 
 The URL parameter acts as a selector that matches against the `version` property 
 in your component configuration. For example:
@@ -393,3 +393,22 @@ show_related_blogs: true
 
 This will show it on the page, assuming there is a matching blog. If there is no
 match then it remains hidden.
+
+## Vale
+
+Vale is a command-line tool that brings code-like linting to prose.
+We have a number of rules set up to ensure that our documentation is
+consistent in style.
+
+The style rules are located at `/styles/ClickHouse`, and largely based
+off of the Google styleset, with some ClickHouse specific adaptions.
+If you want to check only a specific rule locally, you
+can run:
+
+```bash
+vale --filter='.Name == "ClickHouse.Headings"' docs/integrations
+```
+
+This will run only the rule named `Headings` on
+the `docs/integrations` directory. Specifying a specific markdown
+file is also possible.

docs/_snippets/_GCS_authentication_and_bucket.md

@@ -19,7 +19,7 @@ import Image from '@theme/IdealImage';
 
 <Image size="md" img={GCS_bucket_2} alt="Creating a GCS bucket in US East 4" border />
 
-### Generate an Access key {#generate-an-access-key}
+### Generate an access key {#generate-an-access-key}
 
 ### Create a service account HMAC key and secret {#create-a-service-account-hmac-key-and-secret}
 

docs/_snippets/_add_superset_detail.md

@@ -13,7 +13,7 @@ There are a few tasks to be done before running `docker compose`:
 The commands below are to be run from the  top level of the GitHub repo, `superset`.
 :::
 
-## Official ClickHouse Connect driver {#official-clickhouse-connect-driver}
+## Official ClickHouse connect driver {#official-clickhouse-connect-driver}
 
 To make the ClickHouse Connect driver available in the Superset deployment add it to the local requirements file:
 

docs/_snippets/_users-and-roles-common.md

@@ -269,7 +269,7 @@ Roles are used to define groups of users for certain privileges instead of manag
    Verify that only the above two rows are returned, rows with the value `B` in `column1` should be excluded.
    :::
 
-## Modifying Users and Roles {#modifying-users-and-roles}
+## Modifying users and roles {#modifying-users-and-roles}
 
 Users can be assigned multiple roles for a combination of privileges needed. When using multiple roles, the system will combine the roles to determine privileges, the net effect will be that the role permissions will be cumulative.
 

docs/about-us/beta-and-experimental-features.md

@@ -14,19 +14,19 @@ Due to the uncertainty of when features are classified as generally available, w
 
 The sections below explicitly describe the properties of **Beta** and **Experimental** features:
 
-## Beta Features {#beta-features}
+## Beta features {#beta-features}
 
 - Under active development to make them generally available (GA)
 - Main known issues can be tracked on GitHub
 - Functionality may change in the future
 - Possibly enabled in ClickHouse Cloud
 - The ClickHouse team supports beta features
 
-The following features are considered Beta in ClickHouse Cloud and are available for use in ClickHouse Cloud Services, even though they may be currently under a ClickHouse SETTING named ```allow_experimental_*```:
+You can find below the features considered Beta in ClickHouse Cloud and are available for use in your ClickHouse Cloud Services.
 
 Note: please be sure to be using a current version of the ClickHouse [compatibility](/operations/settings/settings#compatibility) setting to be using a recently introduced feature.
 
-## Experimental Features {#experimental-features}
+## Experimental features {#experimental-features}
 
 - May never become GA
 - May be removed