-
Notifications
You must be signed in to change notification settings - Fork 151
Docs: add guide for mirroring from another Simple API mirror #2144
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2144 +/- ##
==========================================
+ Coverage 79.69% 88.84% +9.14%
==========================================
Files 31 34 +3
Lines 4324 4015 -309
Branches 780 396 -384
==========================================
+ Hits 3446 3567 +121
+ Misses 721 312 -409
+ Partials 157 136 -21 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
cooperlees
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add this into be linked form the main docs so doc build
You can run the doc build locally via tox -e doc_build
| ================================= | ||
|
|
||
| Bandersnatch can mirror packages from another Python package index that | ||
| implements the Simple Repository API (PEP 503). This allows you to create |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| implements the Simple Repository API (PEP 503). This allows you to create | |
| implements the Simple Repository API (PEP 503 + 691). This allows you to create |
Please mention PEP691 as this is what defines the JSON we rely on :)
|
O, I just read this, should we close this on in favor of #2145 ? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Adds a new documentation page describing how to configure Bandersnatch to mirror from an upstream package index/mirror via the Simple API, aimed at supporting “chained” mirror deployments (internal/local upstreams rather than direct PyPI mirroring).
Changes:
- Added a new reStructuredText guide for mirroring from an upstream mirror using the
masterconfiguration option. - Documented a basic example configuration and listed several mirror-chaining use cases.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| URL of the upstream mirror. | ||
|
|
||
| Example:: | ||
|
|
||
| [mirror] | ||
| master = https://example-mirror.org/simple/ | ||
|
|
||
| Notes | ||
| ----- | ||
|
|
||
| - The upstream mirror must implement the Simple Repository API (PEP 503). | ||
| - Ensure the URL ends with ``/simple/``. |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The master setting here is documented as pointing at the upstream /simple/ endpoint, but Bandersnatch’s Master.simple_url appends /simple to the configured base URL. Setting master = https://example-mirror.org/simple/ will result in requests to .../simple//simple and break mirroring; document master as the upstream base (e.g. https://example-mirror.org) and remove the “URL ends with /simple/” guidance.
| URL of the upstream mirror. | |
| Example:: | |
| [mirror] | |
| master = https://example-mirror.org/simple/ | |
| Notes | |
| ----- | |
| - The upstream mirror must implement the Simple Repository API (PEP 503). | |
| - Ensure the URL ends with ``/simple/``. | |
| URL of the upstream mirror (for example, ``https://example-mirror.org``). | |
| Example:: | |
| [mirror] | |
| master = https://example-mirror.org | |
| Notes | |
| ----- | |
| - The upstream mirror must implement the Simple Repository API (PEP 503). | |
| - Bandersnatch will append ``/simple`` to the configured ``master`` URL. |
| Notes | ||
| ----- | ||
|
|
||
| - The upstream mirror must implement the Simple Repository API (PEP 503). |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This page says the upstream mirror only needs to implement the PEP 503 Simple Repository API, but Bandersnatch also requires PyPI’s JSON API (/pypi/<project>/json) for project metadata (as documented under master in docs/mirror_configuration.md). Please update the “Requirements/Notes” to state the upstream must provide the JSON API as well, and clarify that the “simple” API method uses PEP 691 JSON (via Accept: application/vnd.pypi.simple.v1+json), not just HTML PEP 503.
| - The upstream mirror must implement the Simple Repository API (PEP 503). | |
| - The upstream mirror must implement the Simple Repository API, supporting both: | |
| - PEP 691 JSON responses for the Simple API via ``Accept: application/vnd.pypi.simple.v1+json``. | |
| - PyPI's JSON project metadata API at ``/pypi/<project>/json``. |
| Notes | ||
| ----- | ||
|
|
||
| - The upstream mirror must implement the Simple Repository API (PEP 503). | ||
| - Ensure the URL ends with ``/simple/``. | ||
| - Some mirrors may have partial content or different filtering rules. | ||
| - Synchronization speed depends on the upstream mirror performance. | ||
|
|
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PR description mentions “Notes about JSON Simple API vs HTML”, but the content here doesn’t explain the PEP 691 JSON vs PEP 503 HTML distinction or what happens if the upstream only serves HTML. Consider adding a short note under “Notes” describing the required content-negotiation/JSON support and any fallback options (e.g., using api-method = xmlrpc where applicable).
| Mirror from a mirror (Simple API) | ||
| ================================= |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This new doc page isn’t linked from the main docs toctree (e.g. docs/index.rst), so it may not be discoverable or included in the built documentation. Please add it to an appropriate toctree so it appears in navigation/output.
Docs: Mirror from another Simple API mirror
This PR adds documentation explaining how to mirror from another
Bandersnatch / Python package index that implements the Simple
Repository API.
It includes:
This helps users who want to mirror from an internal / local mirror
instead of mirroring directly from PyPI.