Skip to content

Allow code blocks to import code from files or URLs #4236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

Blargian
Copy link
Member

@Blargian Blargian commented Aug 6, 2025

Summary

Per request, code for code blocks should be able to be imported from files or URLs.

This change will allow the user to specify file= or url= on code snippets. For example:

file=code_snippets/ClickStack/config-unstructured-logs-with-processor.yaml
url=https://raw.githubusercontent.com/ClickHouse/clickhouse-connect/refs/heads/main/examples/pandas_examples.py

The code from the file or URL will be inserted as text into the code block in the markdown when yarn build is run. The changes show as diffs (i.e the markdown files are updated with the changes) so we can commit the latest change permanently for human or LLM readers of the markdown. The advantage of this is that we can test these snippets externally and pull them from other repositories.

Checklist

Copy link

vercel bot commented Aug 6, 2025

@Blargian is attempting to deploy a commit to the ClickHouse Team on Vercel.

A member of the Team first needs to authorize it.

Copy link

vercel bot commented Aug 6, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
clickhouse-docs ❌ Failed (Inspect) Aug 11, 2025 0:12am

@Blargian Blargian changed the title Allow code blocks to import code from files Allow code blocks to import code from files or remotely Aug 8, 2025
@Blargian Blargian marked this pull request as ready for review August 8, 2025 21:36
@Blargian Blargian requested a review from a team as a code owner August 8, 2025 21:36
@Blargian Blargian requested a review from gingerwizard August 8, 2025 21:36
@Blargian Blargian changed the title Allow code blocks to import code from files or remotely Allow code blocks to import code from files or URLs Aug 8, 2025
@Blargian Blargian linked an issue Aug 8, 2025 that may be closed by this pull request
@Blargian
Copy link
Member Author

CC @mshustov, I remember you mentioned it would be nice to test code snippets in the docs. This change will allow you to import code into docs from integrations repos using url. Would this API be suitable for your use case as well, or are there some improvements we can make?

@mshustov
Copy link
Member

@Blargian This is an incredible improvement, thank you for taking a look at it!

or are there some improvements we can make?

We might need a way to specify not just a file, but specific rows to incluse not the whole file, but a snippet. That's how GitHub does it:
https://github.com/ClickHouse/clickhouse-docs/blob/main/package.json#L39-L43

It adds some complexity to the source file, though, since we'd need to ensure that either the lines are static or the docs are updated if the lines in the snippet move or content changed.

@mshustov
Copy link
Member

Per request, code for code blocks should be able to be imported from files or URLs.
This change will allow the user to specify file= or url= on code snippets. For example:
file=code_snippets/ClickStack/config-unstructured-logs-with-processor.yaml
url=https://raw.githubusercontent.com/ClickHouse/clickhouse-connect/refs/heads/main/examples/pandas_examples.py

May I ask you to include this information in the contribution guide, please? https://github.com/ClickHouse/clickhouse-docs/tree/main/contribute

@Blargian
Copy link
Member Author

@Blargian This is an incredible improvement, thank you for taking a look at it!

or are there some improvements we can make?

We might need a way to specify not just a file, but specific rows to incluse not the whole file, but a snippet. That's how GitHub does it: https://github.com/ClickHouse/clickhouse-docs/blob/main/package.json#L39-L43

It adds some complexity to the source file, though, since we'd need to ensure that either the lines are static or the docs are updated if the lines in the snippet move or content changed.

@mshustov good point. Maybe we could add to the example files comments something like #docs-begin and #docs-end (or //docs-begin, //docs-end etc depending on language of the file). The processing script can then extract only lines between those comments if they are present, otherwise the entire file contents. Advantage of this approach will be that it's only necessary to change the files with example code, and not the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Import code for code snippets from files
3 participants