-
-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Add linkcheck_ignore_case configuration option #14046
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
FazeelUsmani
wants to merge
27
commits into
sphinx-doc:master
Choose a base branch
from
FazeelUsmani:linkcheck_case_insensitive
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+161
−1
Open
Changes from 20 commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
63e108c
Update configuration.rst
FazeelUsmani caae7eb
Add linkcheck_ignore_case config option
FazeelUsmani 9e6dd40
Update i18n.py
FazeelUsmani eccd6d7
fixed the failing test test_numfig_disabled_warn
FazeelUsmani 6300483
Enable case-insensitive URL and anchor checking for linkcheck builder
FazeelUsmani b61366c
strip ANSI color codes from stderr before assertion
FazeelUsmani 7ea45c6
fixed the failing test test_connect_to_selfsigned_fails
FazeelUsmani 99a5dc0
Update test_build_linkcheck.py
FazeelUsmani f99651f
Merge branch 'master' into linkcheck_case_insensitive
FazeelUsmani ac12d63
Update linkcheck.py
FazeelUsmani 1a0d9ed
Update test_build_linkcheck.py
FazeelUsmani d115b1e
Update test_build_linkcheck.py
FazeelUsmani 0075419
fix ruff check linkcheck.py
FazeelUsmani 4eceef2
fix ruff check test_build_linkcheck.py
FazeelUsmani e772df9
Update configuration.rst
FazeelUsmani 14ded5b
Update configuration.rst
FazeelUsmani 386d4ac
Update configuration.rst
FazeelUsmani 53a47e3
Update doc/usage/configuration.rst
FazeelUsmani 3e545f3
Update i18n.py (reert \)
FazeelUsmani d9940da
Use .casefold() for case-insensitive URL comparison
FazeelUsmani 322fcf5
Update test_build_linkcheck.py (revert)
FazeelUsmani cfcbef2
Update test_build_linkcheck.py (revert)
FazeelUsmani 2c4567d
restore original pytest markers
FazeelUsmani c18d573
Removed the duplicate @pytest.mark.sphinx
FazeelUsmani 07b1795
Removed test_linkcheck_anchors_remain_case_sensitive
FazeelUsmani bc8fa7c
Rename linkcheck_ignore_case to linkcheck_case_insensitive and update…
FazeelUsmani 029a720
Fix ruff format check
FazeelUsmani File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -886,6 +886,10 @@ def test_invalid_ssl(get_request, app): | |||||||||||||||||
| 'linkcheck', | ||||||||||||||||||
| testroot='linkcheck-localserver-https', | ||||||||||||||||||
| freshenv=True, | ||||||||||||||||||
| confoverrides={ | ||||||||||||||||||
| 'linkcheck_timeout': 10, | ||||||||||||||||||
| 'linkcheck_report_timeouts_as_broken': True, | ||||||||||||||||||
| }, | ||||||||||||||||||
FazeelUsmani marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||
| ) | ||||||||||||||||||
| def test_connect_to_selfsigned_fails(app: SphinxTestApp) -> None: | ||||||||||||||||||
| with serve_application(app, OKHandler, tls_enabled=True) as address: | ||||||||||||||||||
|
|
@@ -897,7 +901,12 @@ def test_connect_to_selfsigned_fails(app: SphinxTestApp) -> None: | |||||||||||||||||
| assert content['filename'] == 'index.rst' | ||||||||||||||||||
| assert content['lineno'] == 1 | ||||||||||||||||||
| assert content['uri'] == f'https://{address}/' | ||||||||||||||||||
| assert '[SSL: CERTIFICATE_VERIFY_FAILED]' in content['info'] | ||||||||||||||||||
| # Accept either SSL certificate error or timeout (both indicate connection failure) | ||||||||||||||||||
| assert ( | ||||||||||||||||||
| '[SSL: CERTIFICATE_VERIFY_FAILED]' in content['info'] | ||||||||||||||||||
| or 'timed out' in content['info'].lower() | ||||||||||||||||||
| or 'timeout' in content['info'].lower() | ||||||||||||||||||
| ) | ||||||||||||||||||
FazeelUsmani marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| @pytest.mark.sphinx( | ||||||||||||||||||
|
|
@@ -1439,3 +1448,148 @@ def test_linkcheck_exclude_documents(app: SphinxTestApp) -> None: | |||||||||||||||||
| 'uri': 'https://www.sphinx-doc.org/this-is-another-broken-link', | ||||||||||||||||||
| 'info': 'br0ken_link matched br[0-9]ken_link from linkcheck_exclude_documents', | ||||||||||||||||||
| } in content | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| class CaseSensitiveHandler(BaseHTTPRequestHandler): | ||||||||||||||||||
| """Handler that returns URLs with uppercase in the redirect location.""" | ||||||||||||||||||
|
|
||||||||||||||||||
| protocol_version = 'HTTP/1.1' | ||||||||||||||||||
|
|
||||||||||||||||||
| def do_HEAD(self): | ||||||||||||||||||
| # Simulate a server that returns URLs with different case | ||||||||||||||||||
| if self.path == '/path': | ||||||||||||||||||
| # Return the path with uppercase | ||||||||||||||||||
| self.send_response(200, 'OK') | ||||||||||||||||||
| # Simulate the response URL being in uppercase | ||||||||||||||||||
| self.send_header('Content-Length', '0') | ||||||||||||||||||
| self.end_headers() | ||||||||||||||||||
| elif self.path == '/anchor.html': | ||||||||||||||||||
| self.send_response(200, 'OK') | ||||||||||||||||||
| self.send_header('Content-Length', '0') | ||||||||||||||||||
| self.end_headers() | ||||||||||||||||||
|
Comment on lines
+1457
to
+1460
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd recommend removing most/all unused code paths from the test server -- so, for example, this |
||||||||||||||||||
| else: | ||||||||||||||||||
| self.send_response(404, 'Not Found') | ||||||||||||||||||
| self.send_header('Content-Length', '0') | ||||||||||||||||||
| self.end_headers() | ||||||||||||||||||
|
|
||||||||||||||||||
| def do_GET(self): | ||||||||||||||||||
| if self.path == '/path': | ||||||||||||||||||
| content = b'ok\n\n' | ||||||||||||||||||
| self.send_response(200, 'OK') | ||||||||||||||||||
| self.send_header('Content-Length', str(len(content))) | ||||||||||||||||||
| self.end_headers() | ||||||||||||||||||
| self.wfile.write(content) | ||||||||||||||||||
| elif self.path == '/anchor.html': | ||||||||||||||||||
| # HTML with anchor in mixed case | ||||||||||||||||||
| doc = '<!DOCTYPE html><html><body><a id="MyAnchor"></a></body></html>' | ||||||||||||||||||
| content = doc.encode('utf-8') | ||||||||||||||||||
| self.send_response(200, 'OK') | ||||||||||||||||||
| self.send_header('Content-Length', str(len(content))) | ||||||||||||||||||
| self.end_headers() | ||||||||||||||||||
| self.wfile.write(content) | ||||||||||||||||||
| else: | ||||||||||||||||||
| self.send_response(404, 'Not Found') | ||||||||||||||||||
| self.send_header('Content-Length', '0') | ||||||||||||||||||
| self.end_headers() | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| @pytest.mark.sphinx( | ||||||||||||||||||
| 'linkcheck', | ||||||||||||||||||
| testroot='linkcheck-localserver', | ||||||||||||||||||
| freshenv=True, | ||||||||||||||||||
| confoverrides={'linkcheck_ignore_case': False}, | ||||||||||||||||||
| ) | ||||||||||||||||||
| def test_linkcheck_case_sensitive(app: SphinxTestApp) -> None: | ||||||||||||||||||
| """Test that case-sensitive checking is the default behavior.""" | ||||||||||||||||||
| with serve_application(app, CaseSensitiveHandler) as address: | ||||||||||||||||||
| # Monkey-patch the session to change the response URL to uppercase | ||||||||||||||||||
| # to simulate a case-insensitive server | ||||||||||||||||||
| from unittest.mock import patch | ||||||||||||||||||
|
|
||||||||||||||||||
| original_request = requests._Session.request | ||||||||||||||||||
|
|
||||||||||||||||||
| def mock_request(self, method, url, **kwargs): | ||||||||||||||||||
| response = original_request(self, method, url, **kwargs) | ||||||||||||||||||
| # Change the URL to uppercase to simulate server behavior | ||||||||||||||||||
| if '/path' in str(response.url).lower(): | ||||||||||||||||||
| response.url = str(response.url).replace('/path', '/PATH') | ||||||||||||||||||
| return response | ||||||||||||||||||
|
|
||||||||||||||||||
| with patch.object(requests._Session, 'request', mock_request): | ||||||||||||||||||
| app.build() | ||||||||||||||||||
|
|
||||||||||||||||||
| content = (app.outdir / 'output.json').read_text(encoding='utf8') | ||||||||||||||||||
| rows = [json.loads(x) for x in content.splitlines()] | ||||||||||||||||||
| rowsby = {row['uri']: row for row in rows} | ||||||||||||||||||
|
|
||||||||||||||||||
| # With case-sensitive checking, a URL that redirects to different case | ||||||||||||||||||
| # should be marked as redirected | ||||||||||||||||||
| lowercase_uri = f'http://{address}/path' | ||||||||||||||||||
| if lowercase_uri in rowsby: | ||||||||||||||||||
| # Should be redirected because case doesn't match | ||||||||||||||||||
| assert rowsby[lowercase_uri]['status'] == 'redirected' | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| @pytest.mark.sphinx( | ||||||||||||||||||
| 'linkcheck', | ||||||||||||||||||
| testroot='linkcheck-localserver', | ||||||||||||||||||
| freshenv=True, | ||||||||||||||||||
| confoverrides={'linkcheck_ignore_case': True}, | ||||||||||||||||||
| ) | ||||||||||||||||||
| def test_linkcheck_case_insensitive(app: SphinxTestApp) -> None: | ||||||||||||||||||
| """Test that linkcheck_ignore_case=True ignores case differences in URLs.""" | ||||||||||||||||||
| with serve_application(app, CaseSensitiveHandler) as address: | ||||||||||||||||||
| # Monkey-patch the session to change the response URL to uppercase | ||||||||||||||||||
| from unittest.mock import patch | ||||||||||||||||||
|
|
||||||||||||||||||
| original_request = requests._Session.request | ||||||||||||||||||
|
|
||||||||||||||||||
| def mock_request(self, method, url, **kwargs): | ||||||||||||||||||
| response = original_request(self, method, url, **kwargs) | ||||||||||||||||||
| # Change the URL to uppercase to simulate server behavior | ||||||||||||||||||
| if '/path' in str(response.url).lower(): | ||||||||||||||||||
| response.url = str(response.url).replace('/path', '/PATH') | ||||||||||||||||||
| return response | ||||||||||||||||||
|
|
||||||||||||||||||
| with patch.object(requests._Session, 'request', mock_request): | ||||||||||||||||||
| app.build() | ||||||||||||||||||
|
|
||||||||||||||||||
| content = (app.outdir / 'output.json').read_text(encoding='utf8') | ||||||||||||||||||
| rows = [json.loads(x) for x in content.splitlines()] | ||||||||||||||||||
| rowsby = {row['uri']: row for row in rows} | ||||||||||||||||||
|
|
||||||||||||||||||
| # With case-insensitive checking, a URL that differs only in case | ||||||||||||||||||
| # should be marked as working | ||||||||||||||||||
| lowercase_uri = f'http://{address}/path' | ||||||||||||||||||
| if lowercase_uri in rowsby: | ||||||||||||||||||
| # Should be working because case is ignored | ||||||||||||||||||
| assert rowsby[lowercase_uri]['status'] == 'working' | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| @pytest.mark.sphinx( | ||||||||||||||||||
| 'linkcheck', | ||||||||||||||||||
| testroot='linkcheck-localserver-anchor', | ||||||||||||||||||
| freshenv=True, | ||||||||||||||||||
| confoverrides={'linkcheck_ignore_case': True}, | ||||||||||||||||||
| ) | ||||||||||||||||||
| def test_linkcheck_anchors_remain_case_sensitive(app: SphinxTestApp) -> None: | ||||||||||||||||||
FazeelUsmani marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||||||||||||||
| """Test that anchors remain case-sensitive even with linkcheck_ignore_case=True.""" | ||||||||||||||||||
| with serve_application(app, CaseSensitiveHandler) as address: | ||||||||||||||||||
| # Create a document with an anchor in lowercase that doesn't match HTML | ||||||||||||||||||
| index = app.srcdir / 'index.rst' | ||||||||||||||||||
| index.write_text( | ||||||||||||||||||
| f'* `Link with wrong case anchor <http://{address}/anchor.html#myanchor>`_\n', | ||||||||||||||||||
| encoding='utf-8', | ||||||||||||||||||
| ) | ||||||||||||||||||
| app.build() | ||||||||||||||||||
|
|
||||||||||||||||||
| content = (app.outdir / 'output.json').read_text(encoding='utf8') | ||||||||||||||||||
| rows = [json.loads(x) for x in content.splitlines()] | ||||||||||||||||||
|
|
||||||||||||||||||
| # The HTML has "MyAnchor" but we request "myanchor" | ||||||||||||||||||
| # Even with linkcheck_ignore_case=True, anchors should be case-sensitive | ||||||||||||||||||
| # so this should be broken | ||||||||||||||||||
| assert len(rows) == 1 | ||||||||||||||||||
| assert rows[0]['status'] == 'broken' | ||||||||||||||||||
| assert rows[0]['uri'] == f'http://{address}/anchor.html#myanchor' | ||||||||||||||||||
| assert "Anchor 'myanchor' not found" in rows[0]['info'] | ||||||||||||||||||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.