AI agent instructions ignore basePath, causing `/llms.txt` to 404 on subpath-hosted docs

[hey-august]

Component

Docs

Priority

P2 - Medium (Would be helpful)

SDK Language(s)

No response

Bug Description

Summary

Fern Docs server-renders a hidden discovery hint for AI agents into every page:

<div class="sr-only" aria-hidden="true">For AI agents: a documentation index is available at the root level at /llms.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.</div>

On docs hosted under a custom domain subpath (like ours, signalwire.com/docs), the "root level at /llms.txt" statement is wrong. Fern only serves under /docs, and the domain root belongs to a separate site. An agent that follows the hint gets a 404 and concludes no index exists, which is worse than having no hint at all, since this div exists precisely so non-JS fetchers (Claude, GPT crawlers, etc.) can discover llms.txt.

Reproduction

$ curl -sL https://signalwire.com/docs/swml/ -A "Mozilla/5.0" | grep -o 'For AI agents[^<]*'
For AI agents: a documentation index is available at the root level at /llms.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.

$ curl -s -o /dev/null -w '%{http_code}\n' https://signalwire.com/llms.txt
404

$ curl -s -o /dev/null -w '%{http_code}\n' https://signalwire.com/docs/llms.txt
200

The other two statements in the div work fine on subpath instances: <page-url>/llms.txt and <page-url>.md both resolve. Only the root-level pointer is broken.

Expected behavior

The directive should point to the instance's actual llms.txt location. In other words, it should be basePath-aware: …available at /docs/llms.txt.

Alternatively (or additionally), let docs.yml override the div text. agents.page-directive already customizes the directive prepended to .md/markdown output. We already set it to the correct absolute URL, which works great there:

agents:
  page-directive: >-
    For a complete index of all SignalWire documentation pages, fetch
    https://signalwire.com/docs/llms.txt

Either honoring page-directive in the HTML div or adding a sibling key would solve this too.

Impact

Fern customers hosting sites on subpaths of custom domains are misdirecting AI agents on every page. At SignalWire, we noticed because Claude was consistently failing to locate SignalWire docs content. Pointing it manually at /docs/llms.txt fixes it.

Versions

5.67.1

Workaround

No response

Are you interested in contributing a fix?

No

[ellisonantoniafelicia-star]

Component

Docs

Priority

P2 - Medium (Would be helpful)

SDK Language(s)

No response

Bug Description

Summary

Fern Docs server-renders a hidden discovery hint for AI agents into every page:

For AI agents: a documentation index is available at the root level at /llms.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.

On docs hosted under a custom domain subpath (like ours, signalwire.com/docs), the "root level at /llms.txt" statement is wrong. Fern only serves under `/docs`, and the domain root belongs to a separate site. An agent that follows the hint gets a 404 and concludes no index exists, which is worse than having no hint at all, since this div exists precisely so non-JS fetchers (Claude, GPT crawlers, etc.) can discover `llms.txt`.

Reproduction

$ curl -sL https://signalwire.com/docs/swml/ -A "Mozilla/5.0" | grep -o 'For AI agents[^<]*'
For AI agents: a documentation index is available at the root level at /llms.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.

$ curl -s -o /dev/null -w '%{http_code}\n' https://signalwire.com/llms.txt
404

$ curl -s -o /dev/null -w '%{http_code}\n' https://signalwire.com/docs/llms.txt
200
The other two statements in the div work fine on subpath instances: <page-url>/llms.txt and <page-url>.md both resolve. Only the root-level pointer is broken.

Expected behavior

The directive should point to the instance's actual llms.txt location. In other words, it should be basePath-aware: …available at /docs/llms.txt.

Alternatively (or additionally), let docs.yml override the div text. agents.page-directive already customizes the directive prepended to .md/markdown output. We already set it to the correct absolute URL, which works great there:

agents:
page-directive: >-
For a complete index of all SignalWire documentation pages, fetch
https://signalwire.com/docs/llms.txt
Either honoring page-directive in the HTML div or adding a sibling key would solve this too.

Impact

Fern customers hosting sites on subpaths of custom domains are misdirecting AI agents on every page. At SignalWire, we noticed because Claude was consistently failing to locate SignalWire docs content. Pointing it manually at /docs/llms.txt fixes it.

Versions

5.67.1

Workaround

No response

Are you interested in contributing a fix?

Best practices for preventing data leaks in your organization

Learn guidance and recommendations to help you avoid private or sensitive data present in your organization from being exposed.

About this guide

As an organization owner, preventing exposure of private or sensitive data should be a top priority. Whether intentional or accidental, data leaks can cause substantial risk to the parties involved. While GitHub takes measures to help protect you against data leaks, you are also responsible for administering your organization to harden security.

There are several key components when it comes to defending against data leaks:

• Taking a proactive approach towards prevention
• Early detection of possible leaks
• Maintaining a mitigation plan when an incident occurs

The best approach will depend on the type of organization you're managing. For example, an organization that focuses on open source development might require looser controls than a fully commercial organization, to allow for external collaboration. This article provide high level guidance on the GitHub features and settings to consider, which you should implement according to your needs.

Secure accounts

Protect your organization's repositories and settings by implementing security best practices, including enabling 2FA and requiring it for all members, and establishing strong password guidelines.

• Requiring organization members, outside collaborators, and billing managers to enable 2FA for their personal accounts, making it harder for malicious actors to access an organization's repositories and settings. For more information, see Requiring two-factor authentication in your organization.

• Encouraging your users to create strong passwords and secure them appropriately, by following GitHub’s recommended password guidelines. For more information, see Creating a strong password.

• Encouraging your users to keep push protection for users enabled in their personal account settings, so that no matter which public repository they push to, they are protected. For more information, see Managing push protection for users.

• Establishing an internal security policy in GitHub, so users know the appropriate steps to take and who to contact if an incident is suspected. For more information, see Adding a security policy to your repository.

For more detailed information about securing accounts, see Best practices for securing accounts.

Prevent data leaks

As an organization owner, you should limit and review access as appropriate for the type of your organization. Consider the following settings for tighter control:

Recommendation	More information
Disable the ability to fork repositories.	Managing the forking policy for your repository
Disable changing repository visibility.	Restricting repository visibility changes in your organization
Restrict repository creation to private or internal.	Restricting repository creation in your organization
Disable repository deletion and transfer.	Setting permissions for deleting or transferring repositories
Scope personal access tokens to the minimum permissions necessary.	None
Secure your code by converting public repositories to private whenever appropriate. You can alert the repository owners of this change automatically using a GitHub App.	Prevent-Public-Repos in GitHub Marketplace
Confirm your organization’s identity by verifying your domain and restricting email notifications to only verified email domains.	Verifying or approving a domain for your organization
Ensure your organization has upgraded to the GitHub Customer Agreement instead of using the Standard Terms of Service.	Upgrading to the GitHub Customer Agreement
Prevent contributors from making accidental commits.	Removing sensitive data from a repository

Detect data leaks

No matter how well you tighten your organization to prevent data leaks, some may still occur, and you can respond by using secret scanning, the audit log, and branch protection rules.

Use secret scanning

Secret scanning helps secure code and keep secrets safe across organizations and repositories by scanning and detecting secrets that were accidentally committed over the full Git history of every branch in GitHub repositories. Any strings that match patterns provided by secret scanning partners, by other service providers, or defined by you or your organization, are reported as alerts in the Security and quality tab of repositories.

There are two forms of secret scanning available: Secret scanning alerts for partners and Secret scanning alerts for users.

• Secret scanning alerts for partners: These are enabled by default and automatically run on all public repositories and public npm packages.

• Secret scanning alerts for users: To get additional scanning capabilities for your organization, you need to enable secret scanning alerts for users.

  When enabled, secret scanning alerts for users can be detected on the following types of repository:

  • Public repositories owned by personal accounts on GitHub.com
  • Public repositories owned by organizations
  • Private and internal repositories owned by organizations using GitHub Team or GitHub Enterprise Cloud, with a license for GitHub Code Security

Tip

Regardless of the enablement status of secret scanning and push protection, organizations on GitHub Team and GitHub Enterprise can run a free report to scan the code in the organization for leaked secrets. See Secret security with GitHub.

For more information about secret scanning, see Secret scanning.

You can also enable secret scanning as a push protection for a repository or an organization. When you enable this feature, secret scanning prevents contributors from pushing code with a detected secret. For more information, see Push protection. Finally, you can also extend the detection to include custom secret string structures. For more information, see Defining custom patterns for secret scanning.

Review the audit log for your organization

You can also proactively secure IP and maintain compliance for your organization by leveraging your organization's audit log, along with the GraphQL Audit Log API. For more information, see Reviewing the audit log for your organization and Enterprise administration.

Set up branch protection rules

To ensure that all code is properly reviewed prior to being merged into the default branch, you can enable branch protection. By setting branch protection rules, you can enforce certain workflows or requirements before a contributor can push changes. For more information, see About protected branches.

As an alternative to branch protection rules, you can create rulesets. Rulesets have a few advantages over branch protection rules, such as statuses, and better discoverability without requiring admin access. You can also apply multiple rulesets at the same time. For more information, see About rulesets.

Mitigate data leaks

If a user pushes sensitive data, ask them to remove it by using the git filter-repo tool. For more information, see Removing sensitive data from a repository. Also, if the sensitive data has not been pushed yet, you can just undo those changes locally; for more information, see the GitHub Blog (but note that git revert is not a valid way to undo the addition of sensitive data as it leaves the original sensitive commit in Git history).

If you're unable to coordinate directly with the repository owner to remove data that you're confident you own, you can fill out a DMCA takedown notice form and tell GitHub Support. Make sure to include the problematic commit hashes. For more information, see DMCA takedown notice.

Note

If one of your repositories has been taken down due to a false claim, you should fill out a DMCA
counter notice form and alert GitHub Support. For more information, see DMCA counter notice.

Revoke exposed tokens

If credentials have been exposed in a GitHub repository, GitHub secret scanning can be used to report and revoke the credentials. For more information, see Resolving alerts from secret scanning.

You can also revoke exposed credentials that you do not own and have been exposed outside of GitHub repositories. By doing this, you are contributing to the overall security of the GitHub community and can quickly limit the impact of these credentials. The API supports revoking:

• Personal access tokens (classic) with the ghp_ prefix
• Fine-grained personal access tokens with the github_pat_ prefix
• OAuth app tokens with the gho_ prefix
• GitHub App user-to-server tokens with the ghu_ prefix
• GitHub App refresh tokens with the ghr_ prefix

If you find any exposed tokens either on GitHub or elsewhere, you can submit a revocation request using the REST API. See Revocation for the complete and authoritative list of supported credential types.

Next steps

• Best practices for securing code in your supply chain
• Best practices for securing your build systemsuggest unit tests for the OpenBrowser behavior.