Skip to content

Conversation

@nisbet-hubbard
Copy link
Contributor

Summary

Currently, Cloudflare always caches robots.txt no matter what header or cache rule is set. And the only way to have the file updated is to purge it. This PR improves the existing wording to reflect the behaviour.

Screenshots (optional)

Documentation checklist

Copy link
Contributor

@hyperlint-ai hyperlint-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 files reviewed, 1 total issue(s) found.

@github-actions github-actions bot added size/xs product:cache Issues or PRs related to Cache labels Sep 25, 2024
@tkornhammar
Copy link
Contributor

Hi @nisbet-hubbard. Thank you for taking the time to identify and file a change request to the robots.txt behavior. Instead of updating the docs I've created a feature request to allow users to configure the edge TTL for robots.txt that would give users more flexibility. If this would have a longer lead time then we'll update the docs first.

@nisbet-hubbard
Copy link
Contributor Author

@tkornhammar that'd be ideal! Is there an issue I can track or is this an internal feature request?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

product:cache Issues or PRs related to Cache size/xs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants