Skip to content

Conversation

@ranguard
Copy link
Member

@ranguard ranguard commented Jun 21, 2025

So that bots/crawlers do not try large resultsets

It _was_ already in robots.txt :(
@ranguard ranguard requested a review from haarg June 21, 2025 21:09
@ranguard ranguard force-pushed the leo/less_pagination branch 2 times, most recently from 7892479 to b7ed8fc Compare June 22, 2025 07:47
@ranguard ranguard force-pushed the leo/less_pagination branch from b7ed8fc to 5f435d3 Compare June 22, 2025 07:49
@codecov
Copy link

codecov bot commented Jun 22, 2025

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 73.13%. Comparing base (316311e) to head (5f435d3).
Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
lib/MetaCPAN/Web/Role/Request.pm 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##           master    #3387   +/-   ##
=======================================
  Coverage   73.13%   73.13%           
=======================================
  Files          67       67           
  Lines        2345     2345           
  Branches      331      331           
=======================================
  Hits         1715     1715           
  Misses        502      502           
  Partials      128      128           
Files with missing lines Coverage Δ
lib/MetaCPAN/Web/Role/Request.pm 79.16% <0.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ranguard ranguard requested a review from oalders June 22, 2025 07:57
@ranguard ranguard merged commit 212a283 into master Jun 22, 2025
10 of 11 checks passed
@ranguard ranguard deleted the leo/less_pagination branch June 22, 2025 07:58
@guest20
Copy link

guest20 commented Jun 22, 2025

Don't higher per-page results reduce the per-overhead?

@ranguard
Copy link
Member Author

It is more about the number of page variations that bots end up crawling - I didn't change the defaults but might be worth doing that as well... and going from 20-50 or something - not sure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants