Skip to content

Conversation

@vani2009
Copy link

@vani2009 vani2009 commented Feb 6, 2026

Docs: Mirror from another Simple API mirror

This PR adds documentation explaining how to mirror from another
Bandersnatch / Python package index that implements the Simple
Repository API.

It includes:

  • Requirements for the upstream mirror
  • Example configuration
  • Notes about JSON Simple API vs HTML
  • Common use case: chaining mirrors

This helps users who want to mirror from an internal / local mirror
instead of mirroring directly from PyPI.

@codecov
Copy link

codecov bot commented Feb 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.84%. Comparing base (4d020e8) to head (c256d6a).
⚠️ Report is 472 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2144      +/-   ##
==========================================
+ Coverage   79.69%   88.84%   +9.14%     
==========================================
  Files          31       34       +3     
  Lines        4324     4015     -309     
  Branches      780      396     -384     
==========================================
+ Hits         3446     3567     +121     
+ Misses        721      312     -409     
+ Partials      157      136      -21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

@cooperlees cooperlees left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add this into be linked form the main docs so doc build

You can run the doc build locally via tox -e doc_build

=================================

Bandersnatch can mirror packages from another Python package index that
implements the Simple Repository API (PEP 503). This allows you to create
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
implements the Simple Repository API (PEP 503). This allows you to create
implements the Simple Repository API (PEP 503 + 691). This allows you to create

Please mention PEP691 as this is what defines the JSON we rely on :)

@cooperlees
Copy link
Contributor

O, I just read this, should we close this on in favor of #2145 ?

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new documentation page describing how to configure Bandersnatch to mirror from an upstream package index/mirror via the Simple API, aimed at supporting “chained” mirror deployments (internal/local upstreams rather than direct PyPI mirroring).

Changes:

  • Added a new reStructuredText guide for mirroring from an upstream mirror using the master configuration option.
  • Documented a basic example configuration and listed several mirror-chaining use cases.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +12 to +23
URL of the upstream mirror.

Example::

[mirror]
master = https://example-mirror.org/simple/

Notes
-----

- The upstream mirror must implement the Simple Repository API (PEP 503).
- Ensure the URL ends with ``/simple/``.
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The master setting here is documented as pointing at the upstream /simple/ endpoint, but Bandersnatch’s Master.simple_url appends /simple to the configured base URL. Setting master = https://example-mirror.org/simple/ will result in requests to .../simple//simple and break mirroring; document master as the upstream base (e.g. https://example-mirror.org) and remove the “URL ends with /simple/” guidance.

Suggested change
URL of the upstream mirror.
Example::
[mirror]
master = https://example-mirror.org/simple/
Notes
-----
- The upstream mirror must implement the Simple Repository API (PEP 503).
- Ensure the URL ends with ``/simple/``.
URL of the upstream mirror (for example, ``https://example-mirror.org``).
Example::
[mirror]
master = https://example-mirror.org
Notes
-----
- The upstream mirror must implement the Simple Repository API (PEP 503).
- Bandersnatch will append ``/simple`` to the configured ``master`` URL.

Copilot uses AI. Check for mistakes.
Notes
-----

- The upstream mirror must implement the Simple Repository API (PEP 503).
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This page says the upstream mirror only needs to implement the PEP 503 Simple Repository API, but Bandersnatch also requires PyPI’s JSON API (/pypi/<project>/json) for project metadata (as documented under master in docs/mirror_configuration.md). Please update the “Requirements/Notes” to state the upstream must provide the JSON API as well, and clarify that the “simple” API method uses PEP 691 JSON (via Accept: application/vnd.pypi.simple.v1+json), not just HTML PEP 503.

Suggested change
- The upstream mirror must implement the Simple Repository API (PEP 503).
- The upstream mirror must implement the Simple Repository API, supporting both:
- PEP 691 JSON responses for the Simple API via ``Accept: application/vnd.pypi.simple.v1+json``.
- PyPI's JSON project metadata API at ``/pypi/<project>/json``.

Copilot uses AI. Check for mistakes.
Comment on lines +19 to +26
Notes
-----

- The upstream mirror must implement the Simple Repository API (PEP 503).
- Ensure the URL ends with ``/simple/``.
- Some mirrors may have partial content or different filtering rules.
- Synchronization speed depends on the upstream mirror performance.

Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description mentions “Notes about JSON Simple API vs HTML”, but the content here doesn’t explain the PEP 691 JSON vs PEP 503 HTML distinction or what happens if the upstream only serves HTML. Consider adding a short note under “Notes” describing the required content-negotiation/JSON support and any fallback options (e.g., using api-method = xmlrpc where applicable).

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +2
Mirror from a mirror (Simple API)
=================================
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new doc page isn’t linked from the main docs toctree (e.g. docs/index.rst), so it may not be discoverable or included in the built documentation. Please add it to an appropriate toctree so it appears in navigation/output.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants