Merge pull request #10 from MrPunyapal/feat/private-source-auth

Support private repositories in remote sources

Author: MrPunyapal

CHANGELOG.md

@@ -3,6 +3,11 @@
 > Changes in **0.1.4 and after** — every release from `0.1.4` through `0.2.1`.
 > (`chore: regenerate docs` and merge commits are omitted.)
 
+## Unreleased
+
+### Features
+- **Private remote sources** — sync from private Git repositories by adding `'token' => '${ENV_VAR}'` (and optionally `'username'`) to a `docsmith.sources.php` entry. Tokens resolve from the environment at sync time; without an explicit token, `DOCSMITH_TOKEN` is used for any host and `GITHUB_TOKEN` / `GH_TOKEN` only for github.com hosts (never sent to third-party hosts). Requires [mrpunyapal/git-reader](https://github.com/MrPunyapal/git-reader) 0.2.0.
+
 ## 0.3.0-beta.1 — 2026-08-22
 
 ### Features

composer.json

@@ -6,7 +6,7 @@
     "require": {
         "php": "^8.3",
         "league/commonmark": "^2.7",
-        "mrpunyapal/git-reader": "^0.1",
+        "mrpunyapal/git-reader": "^0.2",
         "phiki/phiki": "^2.2"
     },
     "require-dev": {

docs/export/docs.md

@@ -596,6 +596,8 @@ Plain builds remain fully deterministic and offline. If no `docsmith.sources.php
 | `ref` | yes | Branch name, tag name, or an advertised tip SHA. Annotated tags resolve to their commit. |
 | `path` | no | Subdirectory to extract. Empty or `/` means the whole tree. `..` is rejected. |
 | `target` | yes | Directory name under the markdown root (`[A-Za-z0-9._-]+`). Must be unique across sources. |
+| `token` | no | Access token for private repositories. `'${ENV_VAR}'` reads the named environment variable (recommended). |
+| `username` | no | Username used with the token. Defaults to `x-access-token`. |
 
 ### Caching and determinism
 
@@ -612,7 +614,27 @@ Materialization is hardened by default:
 - Per-file (20 MB), total-size (200 MB), and file-count (20 000) budgets guard against oversized or hostile repositories.
 - Extraction writes to a staging directory and swaps atomically, so failures never leave half-updated targets.
 
-Private repositories are **not** supported yet; authentication attempts fail with a clear message.
+## Private repositories
+
+Private repositories are supported — pass a token in `docsmith.sources.php`:
+
+```php
+return [
+    [
+        'repository' => 'https://github.com/acme/private-docs.git',
+        'ref' => 'main',
+        'path' => 'docs',
+        'target' => 'private-docs',
+        'token' => '${ACME_PAT}',   // read from the ACME_PAT environment variable
+        'username' => 'doc-bot',    // optional; defaults to x-access-token
+    ],
+];
+```
+
+- **`'token' => '${ENV_VAR_NAME}'`** is the recommended form: DocSmith reads the variable from the environment at sync time and fails with a clear message if it is unset. A literal token string also works, but hardcoding secrets in a committed file is discouraged.
+- **Automatic fallbacks** — if no `token` key is present, DocSmith uses `DOCSMITH_TOKEN` for any HTTPS host, and `GITHUB_TOKEN` / `GH_TOKEN` only for repositories on github.com. GitHub tokens are never sent to third-party hosts, and fallback tokens are never attached to plain-HTTP URLs.
+- **Never commit tokens.** Keep them in your shell profile or `.env`, and let CI inject them via repository secrets.
+
 
 ## Programmatic use
 
@@ -1175,6 +1197,8 @@ jobs:
       # Syncs remote sources (incremental when docsmith.sources.lock.json
       # matches) and builds in one step.
       - run: php bin/docsmith build --sync
+        env:
+          ACME_PAT: ${{ secrets.ACME_PAT }}   # only needed for private sources
 
       - uses: actions/upload-pages-artifact@v3
         with:

docs/llms-full.txt

@@ -115,6 +115,16 @@ <h2 id="source-options">Source options</h2>
 <td>yes</td>
 <td>Directory name under the markdown root (<code>[A-Za-z0-9._-]+</code>). Must be unique across sources.</td>
 </tr>
+<tr>
+<td><code>token</code></td>
+<td>no</td>
+<td>Access token for private repositories. <code>'${ENV_VAR}'</code> reads the named environment variable (recommended).</td>
+</tr>
+<tr>
+<td><code>username</code></td>
+<td>no</td>
+<td>Username used with the token. Defaults to <code>x-access-token</code>.</td>
+</tr>
 </tbody>
 </table></div>
 <h3 id="caching-and-determinism">Caching and determinism</h3>
@@ -128,7 +138,24 @@ <h2 id="safety">Safety</h2>
 <li>Per-file (20 MB), total-size (200 MB), and file-count (20 000) budgets guard against oversized or hostile repositories.</li>
 <li>Extraction writes to a staging directory and swaps atomically, so failures never leave half-updated targets.</li>
 </ul>
-<p>Private repositories are <strong>not</strong> supported yet; authentication attempts fail with a clear message.</p>
+<h2 id="private-repositories">Private repositories</h2>
+<p>Private repositories are supported — pass a token in <code>docsmith.sources.php</code>:</p>
+<pre class="phiki language-php github-light phiki-themes github-dark" data-language="php" style="background-color: #fff;color: #24292e;--phiki-dark-background-color: #24292e;--phiki-dark-color: #e1e4e8;"><code><span class="line"><span class="token" style="color: #d73a49;--phiki-dark-color: #f97583;">return</span><span class="token"> </span><span class="token">[</span><span class="token">
+</span></span><span class="line"><span class="token">    </span><span class="token">[</span><span class="token">
+</span></span><span class="line"><span class="token">        </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">repository</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token"> </span><span class="token" style="color: #d73a49;--phiki-dark-color: #f97583;">=&gt;</span><span class="token"> </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">https://github.com/acme/private-docs.git</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token">,</span><span class="token">
+</span></span><span class="line"><span class="token">        </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">ref</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token"> </span><span class="token" style="color: #d73a49;--phiki-dark-color: #f97583;">=&gt;</span><span class="token"> </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">main</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token">,</span><span class="token">
+</span></span><span class="line"><span class="token">        </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">path</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token"> </span><span class="token" style="color: #d73a49;--phiki-dark-color: #f97583;">=&gt;</span><span class="token"> </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">docs</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token">,</span><span class="token">
+</span></span><span class="line"><span class="token">        </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">target</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token"> </span><span class="token" style="color: #d73a49;--phiki-dark-color: #f97583;">=&gt;</span><span class="token"> </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">private-docs</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token">,</span><span class="token">
+</span></span><span class="line"><span class="token">        </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">token</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token"> </span><span class="token" style="color: #d73a49;--phiki-dark-color: #f97583;">=&gt;</span><span class="token"> </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">${ACME_PAT}</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token">,</span><span class="token">   </span><span class="token" style="color: #6a737d;--phiki-dark-color: #6a737d;">//</span><span class="token" style="color: #6a737d;--phiki-dark-color: #6a737d;"> read from the ACME_PAT environment variable</span><span class="token" style="color: #6a737d;--phiki-dark-color: #6a737d;">
+</span></span><span class="line"><span class="token">        </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">username</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token"> </span><span class="token" style="color: #d73a49;--phiki-dark-color: #f97583;">=&gt;</span><span class="token"> </span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">doc-bot</span><span class="token" style="color: #032f62;--phiki-dark-color: #9ecbff;">'</span><span class="token">,</span><span class="token">    </span><span class="token" style="color: #6a737d;--phiki-dark-color: #6a737d;">//</span><span class="token" style="color: #6a737d;--phiki-dark-color: #6a737d;"> optional; defaults to x-access-token</span><span class="token" style="color: #6a737d;--phiki-dark-color: #6a737d;">
+</span></span><span class="line"><span class="token">    </span><span class="token">]</span><span class="token">,</span><span class="token">
+</span></span><span class="line"><span class="token">]</span><span class="token">;</span><span class="token">
+</span></span></code></pre>
+<ul>
+<li><strong><code>'token' =&gt; '${ENV_VAR_NAME}'</code></strong> is the recommended form: DocSmith reads the variable from the environment at sync time and fails with a clear message if it is unset. A literal token string also works, but hardcoding secrets in a committed file is discouraged.</li>
+<li><strong>Automatic fallbacks</strong> — if no <code>token</code> key is present, DocSmith uses <code>DOCSMITH_TOKEN</code> for any HTTPS host, and <code>GITHUB_TOKEN</code> / <code>GH_TOKEN</code> only for repositories on github.com. GitHub tokens are never sent to third-party hosts, and fallback tokens are never attached to plain-HTTP URLs.</li>
+<li><strong>Never commit tokens.</strong> Keep them in your shell profile or <code>.env</code>, and let CI inject them via repository secrets.</li>
+</ul>
 <h2 id="programmatic-use">Programmatic use</h2>
 <pre class="phiki language-php github-light phiki-themes github-dark" data-language="php" style="background-color: #fff;color: #24292e;--phiki-dark-background-color: #24292e;--phiki-dark-color: #e1e4e8;"><code><span class="line"><span class="token" style="color: #d73a49;--phiki-dark-color: #f97583;">use</span><span class="token"> </span><span class="token" style="color: #005cc5;--phiki-dark-color: #79b8ff;">Docsmith</span><span class="token" style="color: #005cc5;--phiki-dark-color: #79b8ff;">\</span><span class="token" style="color: #005cc5;--phiki-dark-color: #79b8ff;">RemoteSources</span><span class="token" style="color: #005cc5;--phiki-dark-color: #79b8ff;">\</span><span class="token" style="color: #005cc5;--phiki-dark-color: #79b8ff;">RemoteSources</span><span class="token">;</span><span class="token">
 </span></span><span class="line"><span class="token">
@@ -147,7 +174,7 @@ <h2 id="programmatic-use">Programmatic use</h2>
             </article>
             <nav class="pager" aria-label="Page navigation"><a class="pager-link" href="../docs-hub/"><span>Previous</span><strong>Docs Hub</strong></a><a class="pager-link pager-link-next" href="../workflows/"><span>Next</span><strong>Workflows</strong></a></nav>
         </main>
-        <aside class="toc-sidebar" data-docsmith-toc><p class="toc-title">On this page</p><nav class="toc-links"><a class="toc-link toc-link-level-2" href="#works-with-any-build" data-docsmith-toc-link="works-with-any-build">Works with any build</a><a class="toc-link toc-link-level-2" href="#how-it-works" data-docsmith-toc-link="how-it-works">How it works</a><a class="toc-link toc-link-level-2" href="#commands" data-docsmith-toc-link="commands">Commands</a><a class="toc-link toc-link-level-2" href="#source-options" data-docsmith-toc-link="source-options">Source options</a><a class="toc-link toc-link-level-3" href="#caching-and-determinism" data-docsmith-toc-link="caching-and-determinism">Caching and determinism</a><a class="toc-link toc-link-level-2" href="#safety" data-docsmith-toc-link="safety">Safety</a><a class="toc-link toc-link-level-2" href="#programmatic-use" data-docsmith-toc-link="programmatic-use">Programmatic use</a></nav></aside>
+        <aside class="toc-sidebar" data-docsmith-toc><p class="toc-title">On this page</p><nav class="toc-links"><a class="toc-link toc-link-level-2" href="#works-with-any-build" data-docsmith-toc-link="works-with-any-build">Works with any build</a><a class="toc-link toc-link-level-2" href="#how-it-works" data-docsmith-toc-link="how-it-works">How it works</a><a class="toc-link toc-link-level-2" href="#commands" data-docsmith-toc-link="commands">Commands</a><a class="toc-link toc-link-level-2" href="#source-options" data-docsmith-toc-link="source-options">Source options</a><a class="toc-link toc-link-level-3" href="#caching-and-determinism" data-docsmith-toc-link="caching-and-determinism">Caching and determinism</a><a class="toc-link toc-link-level-2" href="#safety" data-docsmith-toc-link="safety">Safety</a><a class="toc-link toc-link-level-2" href="#private-repositories" data-docsmith-toc-link="private-repositories">Private repositories</a><a class="toc-link toc-link-level-2" href="#programmatic-use" data-docsmith-toc-link="programmatic-use">Programmatic use</a></nav></aside>
     </div>
     <div class="search-overlay" data-docsmith-search-overlay hidden role="dialog" aria-label="Search documentation">
     <div class="search-overlay-backdrop" data-docsmith-search-overlay-close></div>