feat: add custom 404 page with Rudderstack docs_404 tracking#191
Merged
Conversation
Overrides Starlight's built-in 404 to fire a `docs_404` track event via Rudderstack with the originally-requested broken URL and referrer. Without this, every 404 hit logs https://docs.warp.dev/404/ as the page URL in analytics — hiding which paths are actually broken. The new event captures `broken_url` (window.location.href) and `referrer` (document.referrer) so we can identify the top missing redirects and fix them systematically. The script uses the RS queue pattern (is:inline + window.rudderanalytics) so it fires reliably before the async SDK finishes loading. Part of the broken-redirects fix plan. Co-Authored-By: Oz <oz-agent@warp.dev>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
I'm starting a first review of this pull request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
petradonka
approved these changes
Jun 8, 2026
![oz-for-oss[bot]](https://avatars.githubusercontent.com/in/3450139?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Overview
This PR adds a custom src/pages/404.astro page that uses Starlight chrome and emits a docs_404 RudderStack event for missing docs URLs. No approved spec context was available, so this review is based on the attached PR description and annotated diff.
Concerns
- The new telemetry payload sends the full requested URL and full referrer to analytics, which can include query strings or fragments containing sensitive values. Capture only the redirect-monitoring data needed, or redact sensitive components before tracking.
Security
- Full 404 URLs and referrers can contain tokens, email addresses, OAuth codes, or other private values; avoid sending them unredacted to RudderStack.
Verdict
Found: 0 critical, 1 important, 0 suggestions
Request changes
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
Co-authored-by: oz-for-oss[bot] <277970191+oz-for-oss[bot]@users.noreply.github.com>
rachaelrenk
added a commit
that referenced
this pull request
Jun 8, 2026
Adds a new Oz skill + helper script for a recurring Monday morning agent that surfaces broken docs.warp.dev URLs, diffs them against vercel.json, and posts a Slack summary. The skill (SKILL.md) defines the full runbook: - Queries stg_website_events for docs_404 track events (past 7 days) - Loads vercel.json redirect sources from disk or GitHub raw URL - Identifies uncovered broken URLs and computes week-over-week delta - Posts a Block Kit Slack summary to the docs team channel - Writes a CSV artifact to data/404-reports/ (run artifact, not committed) - Handles no-data state (when PR #191 hasn't been live long enough) - Handles failure gracefully with Slack error notices The helper (run_404_report.py) implements the BigQuery queries via Metabase API, diff logic, and CSV writing. Outputs JSON summary to stdout for the agent to use when constructing the Slack message. Schedule: every Monday 9am PT (cron: 0 17 * * 1 UTC). Deploy via oz.warp.dev once METABASE_API_KEY, SLACK_BOT_TOKEN, and SLACK_CHANNEL_ID are sSLACK_CHANNEL_ID are sSLACK_CHANNEL_ID a Part of the broken-redirecPart of the broken-redirecPart of the broken-regent@warp.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a custom
src/pages/404.astropage that overrides Starlight's built-in 404 to fire a Rudderstackdocs_404track event with the originally-requested broken URL.Why this matters: Every 404 visit currently logs
https://docs.warp.dev/404/as the page URL in analytics. This means we have ~3,800 broken-URL visits/week but zero visibility into which URLs are broken. With this change, the event capturesbroken_url(the actual path the visitor tried to reach) andreferrer, enabling the weekly automated 404 monitoring agent (Step 5 of the redirect-fix plan) to surface the top missing redirects.Changes
src/pages/404.astro(new file)StarlightPageto inherit full site chrome (nav, custom Head, Rudderstack snippet, Vercel Analytics)is:inlinescript that fireswindow.rudderanalytics.track('docs_404', { broken_url, referrer })via the existing RS queue pattern — works reliably before the async SDK finishes loadingTesting
npm run buildpasses (337 pages built)Part of
Broken redirects fix plan — Step 1 (instrumentation prerequisite for all other steps).
Co-Authored-By: Oz oz-agent@warp.dev