Docs: Hybrid Runner Doc Updation #113
Conversation
There was a problem hiding this comment.
Pull request overview
This PR reorganizes the Hybrid Runner documentation by consolidating content and removing several detailed external orchestration guides (Airflow, MWAA, GCP Composer, GitHub Actions). The main content from the Hybrid Ingestion Runner has been moved to the index page, and a new Local Ingestion Agent documentation file has been added.
- Simplified hybrid deployment documentation structure
- Added Local Ingestion Agent documentation for quick laptop-based setup
- Removed external orchestration guides (Airflow, MWAA, GitHub Actions, GCP Composer, credentials management)
Reviewed changes
Copilot reviewed 12 out of 15 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| getting-started/index.mdx | Updated link to point to hybrid-saas overview instead of specific hybrid-ingestion-runner section |
| getting-started/day-1/hybrid-saas/index.mdx | Consolidated Hybrid Ingestion Runner content, now serves as main overview with secrets management instructions |
| getting-started/day-1/hybrid-saas/local-ingestion-agent.mdx | New documentation for Local Ingestion Agent setup with Docker-based workflow |
| getting-started/day-1/hybrid-saas/hybrid-ingestion-runner.mdx | Deleted - content moved to index.mdx |
| getting-started/day-1/hybrid-saas/mwaa.mdx | Deleted - AWS MWAA ingestion guide removed |
| getting-started/day-1/hybrid-saas/github-actions.mdx | Deleted - GitHub Actions ingestion guide removed |
| getting-started/day-1/hybrid-saas/gcp-composer.mdx | Deleted - GCP Composer ingestion guide removed |
| getting-started/day-1/hybrid-saas/credentials.mdx | Deleted - credentials management guide removed |
| getting-started/day-1/hybrid-saas/airflow.mdx | Deleted - Airflow ingestion guide removed |
| docs.json | Updated navigation to reflect new structure and added collate-ai entries |
| collate-ai/sql-agent.mdx | Added setup instructions with screenshots |
| collate-ai/index.mdx | Added SQL Agent card to AI features overview |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| <img noZoom src="/public/images/collate-ai/collate-ai-sql-agent-feature.png" alt="update or suggest tiers" /> | ||
|
|
||
| ## Setup Instructions |
There was a problem hiding this comment.
This section appears to be duplicated. Lines 17-56 contain setup instructions with screenshots, but lines 55-57 begin the same "Setup Instructions" section again with the same image and heading. This creates redundant and confusing documentation structure.
| <Card icon="/public/images/icons/qualityagent.svg" title="Collate AI Quality Agent" href="/collate-ai/quality-agent" horizontal> | ||
| Automatically create Data Quality tests to ensure health and stability. | ||
| </Card> | ||
| <Card icon='/public/images/icons/queryrunner.svg' title="Collate AI SQL Agent" href="/collate-ai/quality-agent" horizontal> |
There was a problem hiding this comment.
The link href points to "/collate-ai/quality-agent" but the card title says "Collate AI SQL Agent". This appears to be a copy-paste error - the href should point to "/collate-ai/sql-agent" instead.
| <Card icon='/public/images/icons/queryrunner.svg' title="Collate AI SQL Agent" href="/collate-ai/quality-agent" horizontal> | |
| <Card icon='/public/images/icons/queryrunner.svg' title="Collate AI SQL Agent" href="/collate-ai/sql-agent" horizontal> |
| #### Set Up the Connection: | ||
| - Enter your Collate platform URL (e.g., `https://<your-company>.collate.com/api`). | ||
| - Add the ingestion bot token from the Collate settings under **Settings > Bots > Ingestion Bot**. | ||
| When executing workflows on your Hybrid environment, you have to use your existing cloud provider's Secrets Manager to store sensitive credentials (like passwords or token), and reference them securely in Collate via the Hybrid Runner. |
There was a problem hiding this comment.
Grammatical error: "token" should be plural "tokens" to match the plural form of "passwords" in the parenthetical example. The sentence should read "like passwords or tokens".
| When executing workflows on your Hybrid environment, you have to use your existing cloud provider's Secrets Manager to store sensitive credentials (like passwords or token), and reference them securely in Collate via the Hybrid Runner. | |
| When executing workflows on your Hybrid environment, you have to use your existing cloud provider's Secrets Manager to store sensitive credentials (like passwords or tokens), and reference them securely in Collate via the Hybrid Runner. |
| @@ -0,0 +1,76 @@ | |||
| --- | |||
| title: ocal Ingestion Runner | Secure Metadata Ingestion for Collate | |||
There was a problem hiding this comment.
There's a spelling error in the title. "ocal" should be "Local". The title is missing the letter "L" at the beginning.
| title: ocal Ingestion Runner | Secure Metadata Ingestion for Collate | |
| title: Local Ingestion Runner | Secure Metadata Ingestion for Collate |
|
|
||
| The Local Ingestion Agent is designed to facilitate metadata ingestion for hybrid deployments, allowing organizations to securely push metadata from their infrastructure into the Collate platform without exposing their internal systems. It provides a secure and efficient way for running ingestion workflows while maintaining full control over data processing within your network. | ||
|
|
||
| The main difference between the Local Ingestion Agent and the Hybrid Runner[link to docs Hybrid SaaS page] is that the Local Ingestion Agent can quickly be set up in your own laptop, without relying on any other infrastructure. |
There was a problem hiding this comment.
The link text "[link to docs Hybrid SaaS page]" appears to be a placeholder that was left in the documentation. It should either be converted to an actual markdown link or the text should be revised to properly reference the Hybrid SaaS documentation page.
| The main difference between the Local Ingestion Agent and the Hybrid Runner[link to docs Hybrid SaaS page] is that the Local Ingestion Agent can quickly be set up in your own laptop, without relying on any other infrastructure. | |
| The main difference between the Local Ingestion Agent and the Hybrid Runner is that the Local Ingestion Agent can quickly be set up on your own laptop, without relying on any other infrastructure. |
| You need to be able to run Docker images on your laptop. | ||
| Run the following commands: | ||
|
|
||
| 1. Log in to Docker with the credentials provided by Collate to authenticate. You can reach out to [email protected] to your credentials. |
There was a problem hiding this comment.
Missing space between the period and "You" in the sentence. It should be "credentials. You" instead of "credentials.You".
| 1. Log in to Docker with the credentials provided by Collate to authenticate. You can reach out to [email protected] to your credentials. | |
| 1. Log in to Docker with the credentials provided by Collate to authenticate. You can reach out to [email protected] to get your credentials. |
| #### 5. Verify Ingested Data | ||
|
|
||
| 1. Return to the Collate platform at https://(your-company).getcollate.io and refresh the database services page. Your new service will appear there now. | ||
| 2.Verify that the ingested metadata, including schemas, tables, and column details, is available. |
There was a problem hiding this comment.
Missing space between "2." and "Verify" on this line. The period after the step number should be followed by a space before the text begins.
| 2.Verify that the ingested metadata, including schemas, tables, and column details, is available. | |
| 2. Verify that the ingested metadata, including schemas, tables, and column details, is available. |
| "collate-ai/documentation-agent", | ||
| "/collate-ai/quality-agent" |
There was a problem hiding this comment.
There's a duplicate entry for "collate-ai/quality-agent" in the pages array. Line 1135 contains "/collate-ai/quality-agent" which duplicates the entry on line 1133. This duplicate should be removed.
| "collate-ai/documentation-agent", | |
| "/collate-ai/quality-agent" |
3 participants
I worked on Hybrid Runner Doc Updation.