Add dataflow pubsub template docs#6295
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
4 Skipped Deployments
|
|
Docs PR opened: ClickHouse/mintlify-docs-dev#87 Added documentation for the new Pub/Sub to ClickHouse Google Dataflow template and linked it from related pages. |
chernser
approved these changes
May 27, 2026
…into add-dataflow-pubsub-template-docs
Blargian
approved these changes
May 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add documentation to our new Dataflow template from PubSub to ClickHouse
Note
Low Risk
Documentation-only changes with no runtime or security impact.
Overview
Adds a full Pub/Sub to ClickHouse Dataflow template guide and wires it into the docs site navigation and template index.
The new page documents the streaming pipeline end to end: prerequisites, template parameters (subscription, ClickHouse HTTP endpoint, batching via
windowSeconds/batchRowCount,ClickHouseIOoptions), JSON-to-column schema mapping and type coercion, dead-letter handling (ClickHouse table schema and/or Pub/Sub topic), console run steps, custom metrics, and troubleshooting (memory limits, DLQ-only traffic, no rows).Index and cross-links are updated so predefined templates mention both BigQuery batch and Pub/Sub streaming; the templates list promotes Pub/Sub from “coming soon” to a live doc and drops the old issue link. The overview
dataflow.mdblurb now calls out streaming from Pub/Sub. BigQuery template source code section now points to both Google’s upstreamDataflowTemplatesrepo and ClickHouse’s fork (same pattern on the new Pub/Sub page). Sidebar adds the new doc under Dataflow templates.Reviewed by Cursor Bugbot for commit 2f91560. Bugbot is set up for automated code reviews on this repo. Configure here.