Skip to content

linkcheck: Ignore URLs that respond with non-Unicode content #12197

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jul 14, 2024
Merged

linkcheck: Ignore URLs that respond with non-Unicode content #12197

merged 15 commits into from
Jul 14, 2024

Conversation

jayaddison
Copy link
Contributor

@jayaddison jayaddison commented Mar 24, 2024

Feature or Bugfix

  • Bugfix

Purpose

  • Improve robustness during anchor-checks, when the linkcheck decodes response content.
  • Begin to add more validation of parsed HTML.

Detail

  • Do not attempt to check for anchors when UTF8 decoding fails; report affected hyperlinks as ignored.
  • Report hyperlinks as broken when their content fails validation checks, but allow this to be overridden using URI-match patterns.

Relates

@AA-Turner AA-Turner changed the title [linkcheck] Add additional anchor-checking safety checks. linkcheck: Improve robustness during anchor checks Apr 24, 2024
@jayaddison
Copy link
Contributor Author

@AA-Turner is it OK if we pause on this one for a while? I think/hope it's a good idea, but I'm slightly concerned that the distinction between lenient and strict parsing might be a bit arbitrary. I should perhaps move this back into a draft status.

@AA-Turner AA-Turner marked this pull request as draft April 24, 2024 18:50
@jayaddison
Copy link
Contributor Author

Another way to phrase that: I'd like to canvas more (and perhaps leaning skeptical) opinions about this before proceeding.

@jayaddison
Copy link
Contributor Author

Thanks!

…king

Conflicts:
	sphinx/builders/linkcheck.py
@jayaddison jayaddison marked this pull request as ready for review July 13, 2024 16:58
@jayaddison jayaddison changed the title linkcheck: Improve robustness during anchor checks linkcheck: ignore URLs that respond with non-unicode content. Jul 13, 2024
@jayaddison jayaddison changed the title linkcheck: ignore URLs that respond with non-unicode content. [linkcheck] ignore URLs that respond with non-unicode content. Jul 13, 2024
@AA-Turner AA-Turner changed the title [linkcheck] ignore URLs that respond with non-unicode content. linkcheck: ignore URLs that respond with non-unicode content. Jul 14, 2024
@AA-Turner AA-Turner changed the title linkcheck: ignore URLs that respond with non-unicode content. linkcheck: Ignore URLs that respond with non-Unicode content. Jul 14, 2024
@AA-Turner AA-Turner changed the title linkcheck: Ignore URLs that respond with non-Unicode content. linkcheck: Ignore URLs that respond with non-Unicode content Jul 14, 2024
@AA-Turner AA-Turner merged commit 35e7bfc into sphinx-doc:master Jul 14, 2024
21 checks passed
@jayaddison jayaddison deleted the issue-11041/linkcheck-stricter-anchor-checking branch July 14, 2024 11:42
@jayaddison
Copy link
Contributor Author

Thanks @AA-Turner!

@AA-Turner AA-Turner added this to the 7.4.0 milestone Jul 15, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[linkcheck] PDF anchor (...pdf#anchor) leads to 'utf-8' codec can't decode byte ...
2 participants