Migrate studio documentation to datachain #1394

amritghimire · 2025-10-10T11:00:33Z

Migrate Studio documentation from DVC.org to the DataChain repository, updating all content for DataChain branding and features.

This PR reintroduces comprehensive Studio documentation, including user guides, API references, and self-hosting guides, which were previously removed from DVC.org. The content has been adapted to reference "DataChain Studio" and "studio.datachain.ai" and integrated into the docs/studio/ section of this repository.

Completely generated by AI.

Summary by Sourcery

Migrate and integrate the full DataChain Studio documentation into the DataChain repository, rebranding and adapting content previously hosted on DVC.org and updating the mkdocs configuration to include Studio’s user guides, API reference, webhooks, self-hosting and troubleshooting materials.

Build:

Update mkdocs.yml to include the Studio documentation section and adjust mkdocstrings configuration (inventories and show_submodules settings).

Documentation:

Add comprehensive DataChain Studio user guide covering account management, datasets, jobs, Git connections, authentication, team collaboration and troubleshooting.
Introduce API reference and webhooks guide for Studio under docs/studio.
Add self-hosting documentation with installation, configuration, upgrade and troubleshooting guides.
Integrate new Studio documentation structure into mkdocs navigation (docs/studio).

(Closes https://github.com/iterative/itops/issues/5861)

Migrate comprehensive Studio documentation from the DVC.org repository (removed in iterative/dvc.org#5446) to DataChain documentation under the Studio section. ## Changes ### New Documentation Structure - **studio/index.md**: DataChain Studio overview and introduction - **studio/user-guide/**: Complete user guide with sections for: - Account management and authentication (SSO, OpenID Connect) - Datasets (create, explore, share, visualize) - Jobs (create, run, monitor) - Git connections (GitHub App, GitLab) - Team collaboration and troubleshooting - **studio/api/index.md**: Comprehensive REST API documentation - **studio/self-hosting/**: Self-hosting guides and configuration ### Content Adaptations - Updated all references from "DVC Studio"/"Iterative Studio" to "DataChain Studio" - Adapted content for DataChain workflows (datasets and jobs vs experiments) - Updated URLs from studio.iterative.ai to studio.datachain.ai - Revised feature descriptions to match DataChain Studio capabilities ### Navigation Updates - Added comprehensive Studio navigation structure to mkdocs.yml - Organized documentation into logical sections with proper hierarchy - Ensured all links are properly structured for the new layout ### Technical Changes - Fixed mkdocstrings configuration for compatibility - Updated navigation paths to match new file structure - Maintained existing webhooks.md with minor updates ## Files Added - 20 new Studio documentation files - Complete user guide covering all major Studio features - Self-hosting documentation for enterprise deployments - API documentation adapted for DataChain Studio ## Validation - Tested mkdocs build in strict mode - Validated navigation structure and internal links - Ensured proper markdown formatting and compatibility This migration provides DataChain users with comprehensive Studio documentation while maintaining the existing structure and adding DataChain-specific adaptations.

sourcery-ai · 2025-10-10T11:00:39Z

Reviewer's Guide

This PR migrates the Studio documentation into the DataChain repository by updating the MkDocs configuration and adding a complete Studio section under docs/studio, including an overview page, a comprehensive user guide, API and Webhooks references, and self-hosting instructions—all rebranded and structured for DataChain Studio.

File-Level Changes

Change	Details	Files
Extended site navigation and plugin settings in mkdocs.yml for Studio docs	Added a new "🔗 Studio" nav section with nested entries for overview, user guide, API, webhooks, and self-hosting Replaced mkdocstrings Python handler `import` key with `inventories` and updated plugin options	`mkdocs.yml`
Created root overview page for Studio documentation	Introduced docs/studio/index.md as the landing page for DataChain Studio Configured links to user guide, API reference, webhooks, and self-hosting sections	`docs/studio/index.md`
Reintroduced comprehensive Studio user guide content	Added account management and authentication guides Included detailed dataset workflows (create, explore, share, visualize) Added job management guides (create, run, monitor) Provided Git connections and team collaboration overviews Included a troubleshooting guide for common Studio issues	`docs/studio/user-guide/account-management.md` `docs/studio/user-guide/authentication/single-sign-on.md` `docs/studio/user-guide/authentication/openid-connect.md` `docs/studio/user-guide/create-dataset.md` `docs/studio/user-guide/explore-datasets.md` `docs/studio/user-guide/share-dataset.md` `docs/studio/user-guide/visualize-and-compare.md` `docs/studio/user-guide/jobs/create-and-run.md` `docs/studio/user-guide/jobs/monitor-jobs.md` `docs/studio/user-guide/git-connections/index.md` `docs/studio/user-guide/git-connections/github-app.md` `docs/studio/user-guide/git-connections/custom-gitlab-server.md` `docs/studio/user-guide/team-collaboration.md` `docs/studio/user-guide/troubleshooting.md`
Reintroduced Studio API reference and Webhooks documentation	Added docs/studio/api/index.md for API reference Added docs/studio/webhooks.md for event notifications	`docs/studio/api/index.md` `docs/studio/webhooks.md`
Integrated self-hosting instructions under Studio docs	Added self-hosting overview and system requirements Provided installation guides for AWS AMI and Kubernetes (Helm) Included configuration, upgrading, and troubleshooting pages	`docs/studio/self-hosting/index.md` `docs/studio/self-hosting/installation/index.md` `docs/studio/self-hosting/configuration/index.md` `docs/studio/self-hosting/upgrading/index.md` `docs/studio/self-hosting/troubleshooting/index.md`

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

cursor · 2025-10-10T11:00:45Z

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
_{Learn more about Cursor Agents}

cloudflare-workers-and-pages · 2025-10-10T11:01:08Z

Deploying datachain-documentation with Cloudflare Pages

Latest commit:	`208166a`
Status:	✅ Deploy successful!
Preview URL:	https://66c4d776.datachain-documentation.pages.dev
Branch Preview URL:	https://cursor-migrate-studio-docume.datachain-documentation.pages.dev

View logs

for more information, see https://pre-commit.ci

codecov · 2025-10-10T11:10:29Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.77%. Comparing base (e24c2d5) to head (208166a).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1394   +/-   ##
=======================================
  Coverage   87.77%   87.77%           
=======================================
  Files         160      160           
  Lines       15161    15161           
  Branches     2173     2173           
=======================================
  Hits        13307    13307           
  Misses       1351     1351           
  Partials      503      503

Flag	Coverage Δ
datachain	`87.72% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Create all missing self-hosting documentation files to resolve mkdocs strict mode warnings: ## Files Added - installation/aws-ami.md - AWS AMI installation guide - installation/k8s-helm.md - Kubernetes Helm installation guide - configuration/index.md - Main configuration overview - configuration/ssl-tls.md - SSL/TLS certificate configuration - configuration/ca-certificates.md - Custom CA certificate setup - configuration/git-forges/*.md - Git forge integration guides - upgrading/*.md - Upgrade procedures (regular and airgap) - troubleshooting/*.md - Troubleshooting guides and support bundle ## Content Features - Comprehensive installation guides for AWS AMI and Kubernetes - Detailed configuration documentation with examples - Complete Git forge integration (GitHub, GitLab, Bitbucket) - Step-by-step upgrade procedures for both connected and air-gapped environments - Troubleshooting guides for common issues like 502 errors - Support bundle generation for diagnostic information ## Technical Validation - ✅ mkdocs builds successfully in strict mode (no warnings) - ✅ All navigation links resolve correctly - ✅ Content adapted for DataChain Studio terminology - ✅ Internal links and references properly structured This completes the comprehensive Studio documentation migration with full self-hosting support for enterprise deployments.

sourcery-ai

Sorry @amritghimire, your pull request is larger than the review limit of 150000 diff characters

shcheklein · 2025-10-10T16:50:05Z

@amritghimire some tests are failing, PTAL

shcheklein · 2025-10-10T16:53:06Z

It works on top of DataChain and Git, maintaining Git as the single-source-of-truth for your data, jobs and datasets.

just remove it, or better mention that it also has DVC + Git integration for experiments, and model registry.

Simplify data processing job tracking, visualization, and collaboration on top of Git.

find mentions of Git - remove them when context is not about DVC please

shcheklein · 2025-10-10T16:54:37Z

@amritghimire I see that you removed the whole thing about DVC and experiments? Can we keep it as a separate section? It is still good to have I think.

Co-authored-by: cursor <[email protected]>

amritghimire · 2025-10-13T04:00:34Z

pre-commit.ci autofix

Co-authored-by: cursor <[email protected]>

for more information, see https://pre-commit.ci

Address feedback on Git workflow mentions and restore DVC capabilities: ## Key Improvements ### Content Clarifications - **Dual Workflow Support**: Updated index to clearly show Studio supports both: - DataChain workflows for unstructured data processing - DVC + Git workflows for ML experiment tracking and model registry - **Context-Specific Git References**: Updated Git workflow mentions to be specific to DVC-based projects where appropriate - **Architecture Description**: Changed 'Git Integration' to 'Repository Integration' for broader accuracy ### DVC Experiments Section Added - **New experiments/index.md**: Comprehensive guide for DVC experiment tracking - **Navigation Updated**: Added Experiments (DVC) section to documentation structure - **Feature Coverage**: Documents experiment tracking, model registry, visualization - **Integration Guidance**: Shows how DataChain and DVC workflows complement each other - **Migration Guide**: Helps users transition from standalone DVC to Studio ### Workflow Clarity - **Separated Concerns**: Clear distinction between DataChain jobs and DVC experiments - **Use Case Guidance**: When to use each workflow type - **Hybrid Workflows**: How to use both approaches together - **Best Practices**: Integration patterns for teams using both systems ## Technical Validation - ✅ mkdocs builds successfully in strict mode (0 warnings) - ✅ All internal links resolve correctly - ✅ Pre-commit hooks pass (trailing whitespace fixed) - ✅ Navigation structure properly updated This maintains the comprehensive nature of the documentation while providing clear guidance on both DataChain and DVC capabilities.

amritghimire · 2025-10-13T07:58:10Z

pre-commit.ci autofix

for more information, see https://pre-commit.ci

…hain-0028

amritghimire · 2025-10-13T14:11:22Z

@sourcery-ai review

sourcery-ai · 2025-10-13T14:11:27Z

Sorry @amritghimire, your pull request is larger than the review limit of 150000 diff characters

SourceryAI · 2025-10-13T14:12:14Z

Sorry @amritghimire, your pull request is larger than the review limit of 150000 diff characters

Copilot

Pull Request Overview

This PR migrates the comprehensive DataChain Studio documentation from DVC.org to the DataChain repository. The content has been rebranded from DVC Studio to DataChain Studio and integrated into the docs/studio/ section, providing complete documentation for user guides, API references, webhooks, self-hosting, and troubleshooting.

Adds complete DataChain Studio documentation covering all aspects from user guides to enterprise self-hosting
Updates mkdocs configuration to include the Studio documentation navigation structure
Migrates and rebrands content to reference "DataChain Studio" and "studio.datachain.ai"

Reviewed Changes

Copilot reviewed 38 out of 38 changed files in this pull request and generated 2 comments.

File	Description
pyproject.toml	Adds duplicate mkdocs-section-index dependency version
mkdocs.yml	Extensive navigation updates and mkdocstrings configuration changes
Multiple docs/studio/ files	Complete Studio documentation migration with user guides, API, webhooks, and self-hosting content

Comments suppressed due to low confidence (1)

mkdocs.yml:1

The show_submodules configuration appears to be duplicated between the old rendering section and the new options section. The old rendering section should be completely removed to avoid conflicts.

site_name: 'DataChain'

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

pyproject.toml

mkdocs.yml

amritghimire · 2025-10-18T03:14:05Z

@sourcery-ai review

sourcery-ai · 2025-10-18T03:14:11Z

Sorry @amritghimire, your pull request is larger than the review limit of 150000 diff characters

amritghimire · 2025-10-18T03:14:29Z

I have reviewed the code and think this is good enough for first pass. PTAL cc. @shcheklein

SourceryAI · 2025-10-18T03:15:06Z

Sorry @amritghimire, your pull request is larger than the review limit of 150000 diff characters

Copilot

Pull Request Overview

Copilot reviewed 46 out of 46 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (2)

pyproject.toml:1

The mkdocs-section-index package is listed twice with conflicting version constraints. Keep a single entry (prefer the newer >=0.3.10) to avoid resolver ambiguity and keep dependencies clear.

[build-system]

docs/studio/user-guide/model-registry/use-models.md:1

The and elements are not recognized by MkDocs/Material; they will not render as interactive tabs. Replace with Material tabs syntax (e.g., === \"CLI (DVC)\" sections) or remove the UI constructs.

# Use models

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

mkdocs.yml

docs/studio/user-guide/troubleshooting.md

docs/studio/user-guide/team-collaboration.md

docs/studio/user-guide/model-registry/use-models.md

docs/studio/user-guide/team-collaboration.md

docs/studio/user-guide/model-registry/view-and-compare-models.md

…hain-0028

shcheklein

@amritghimire check GItHub AI reviews, to make sure everything works fine.

@dmpetrov @miclee13 please review this as well.

Co-authored-by: Copilot <[email protected]>

…hain-0028

amritghimire · 2025-10-28T15:57:01Z

@shcheklein Updated the pull request. Lets merge this and pick up on followup if we need to change anything.

[pre-commit.ci] auto fixes from pre-commit.com hooks

e9888ee

for more information, see https://pre-commit.ci

amritghimire and others added 2 commits October 10, 2025 18:09

INstall mkdocs section index

66255e5

amritghimire marked this pull request as ready for review October 10, 2025 12:47

sourcery-ai bot reviewed Oct 10, 2025

View reviewed changes

Checkpoint before follow-up message

7518ac4

Co-authored-by: cursor <[email protected]>

cursoragent and others added 3 commits October 13, 2025 04:01

Checkpoint before follow-up message

7319284

Co-authored-by: cursor <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

e9ddcd6

for more information, see https://pre-commit.ci

pre-commit-ci bot and others added 2 commits October 13, 2025 07:58

[pre-commit.ci] auto fixes from pre-commit.com hooks

82c1256

for more information, see https://pre-commit.ci

Merge branch 'main' into cursor/migrate-studio-documentation-to-datac…

a733847

…hain-0028

amritghimire requested a review from Copilot October 13, 2025 14:10

Copilot AI reviewed Oct 13, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

mkdocs.yml Show resolved Hide resolved

amritghimire self-assigned this Oct 16, 2025

Changes the documentation manually

806b383

amritghimire requested a review from Copilot October 18, 2025 03:13

Copilot AI reviewed Oct 18, 2025

View reviewed changes

amritghimire requested review from a team October 20, 2025 00:23

Merge branch 'main' into cursor/migrate-studio-documentation-to-datac…

b768adc

…hain-0028

shcheklein approved these changes Oct 21, 2025

View reviewed changes

amritghimire and others added 4 commits October 28, 2025 20:42

Apply suggestion from @Copilot

91029b4

Co-authored-by: Copilot <[email protected]>

Apply suggestion from @Copilot

94a1c51

Co-authored-by: Copilot <[email protected]>

Apply suggestions

6b74557

Merge branch 'main' into cursor/migrate-studio-documentation-to-datac…

208166a

…hain-0028

amritghimire merged commit 7500174 into main Oct 28, 2025
37 checks passed

amritghimire deleted the cursor/migrate-studio-documentation-to-datachain-0028 branch October 28, 2025 15:57

0x2b3bfa0 mentioned this pull request Oct 29, 2025

Fix AMI name in aws-ami.md #1433

Open

Migrate studio documentation to datachain #1394

Migrate studio documentation to datachain #1394

Uh oh!

Conversation

amritghimire commented Oct 10, 2025 • edited by 0x2b3bfa0 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by Sourcery

Uh oh!

sourcery-ai bot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

File-Level Changes

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

cursor bot commented Oct 10, 2025

Uh oh!

cloudflare-workers-and-pages bot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying datachain-documentation with Cloudflare Pages

Uh oh!

codecov bot commented Oct 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

shcheklein commented Oct 10, 2025

Uh oh!

shcheklein commented Oct 10, 2025

Uh oh!

shcheklein commented Oct 10, 2025

Uh oh!

amritghimire commented Oct 13, 2025

Uh oh!

amritghimire commented Oct 13, 2025

Uh oh!

amritghimire commented Oct 13, 2025

Uh oh!

sourcery-ai bot commented Oct 13, 2025

Uh oh!

SourceryAI commented Oct 13, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

amritghimire commented Oct 18, 2025

Uh oh!

sourcery-ai bot commented Oct 18, 2025

Uh oh!

amritghimire commented Oct 18, 2025

Uh oh!

SourceryAI commented Oct 18, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shcheklein left a comment

Choose a reason for hiding this comment

Uh oh!

amritghimire commented Oct 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

amritghimire commented Oct 10, 2025 •

edited by 0x2b3bfa0

Loading

sourcery-ai bot commented Oct 10, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Oct 10, 2025 •

edited

Loading

codecov bot commented Oct 10, 2025 •

edited

Loading