[Workers] [Astro] Noindex value for specific pages (#19818)

Author: kodster28

src/components/overrides/Head.astro

@@ -144,6 +144,19 @@ if (currentSection) {
 	}
 }
 
+// Add noindex tag if present in frontmatter
+
+if (Astro.props.entry.data.noindex) {
+	Astro.props.entry.data.head.push({
+		tag: "meta",
+		attrs: {
+			name: "robots",
+			content: "noindex",
+		},
+		content: "",
+	});
+}
+
 // Adding metadata used for reporting and search indexing
 // content type
 if (Astro.props.entry.data.pcx_content_type) {

src/content/docs/workers/configuration/sites/configuration.mdx

@@ -1,12 +1,12 @@
 ---
 pcx_content_type: configuration
 title: Workers Sites configuration
+noindex: true
 sidebar:
   order: 4
-
 ---
 
-import { Render, WranglerConfig } from "~/components"
+import { Render, WranglerConfig } from "~/components";
 
 <Render file="workers_sites" />
 
@@ -16,29 +16,23 @@ Workers Sites require the latest version of [Wrangler](https://github.com/cloudf
 
 There are a few specific configuration settings for Workers Sites in your Wrangler file:
 
+- `bucket` required
 
+  - The directory containing your static assets, path relative to your `wrangler.toml / wrangler.json` file. Example: `bucket = "./public"`.
 
-* `bucket` required
-
-  * The directory containing your static assets, path relative to your `wrangler.toml / wrangler.json` file. Example: `bucket = "./public"`.
-
-* `include` optional
-
-  * A list of gitignore-style patterns for files or directories in `bucket` you exclusively want to upload. Example: `include = ["upload_dir"]`.
-
-* `exclude` optional
-  * A list of gitignore-style patterns for files or directories in `bucket` you want to exclude from uploads. Example: `exclude = ["ignore_dir"]`.
+- `include` optional
 
+  - A list of gitignore-style patterns for files or directories in `bucket` you exclusively want to upload. Example: `include = ["upload_dir"]`.
 
+- `exclude` optional
+  - A list of gitignore-style patterns for files or directories in `bucket` you want to exclude from uploads. Example: `exclude = ["ignore_dir"]`.
 
 To learn more about the optional `include` and `exclude` fields, refer to [Ignoring subsets of static assets](#ignoring-subsets-of-static-assets).
 
 :::note
 
-
 If your project uses [environments](/workers/wrangler/environments/), make sure to place `site` above any environment-specific configuration blocks.
 
-
 :::
 
 Example of a `wrangler.toml / wrangler.json` file:
@@ -80,8 +74,6 @@ This means that you should use gitignore semantics when declaring which director
 
 If you want to include only a certain set of files or directories in your `bucket`, you can add an `include` field to your `[site]` section of your Wrangler file:
 
-
-
 <WranglerConfig>
 
 ```toml
@@ -98,8 +90,6 @@ Wrangler will only upload files or directories matching the patterns in the `inc
 
 If you want to exclude files or directories in your `bucket`, you can add an `exclude` field to your `[site]` section of your Wrangler file:
 
-
-
 <WranglerConfig>
 
 ```toml
@@ -120,9 +110,9 @@ If you provide both `include` and `exclude` fields, the `include` field will be
 
 Wrangler will always ignore:
 
-* `node_modules`
-* Hidden files and directories
-* Symlinks
+- `node_modules`
+- Hidden files and directories
+- Symlinks
 
 #### More about include/exclude patterns
 

src/content/docs/workers/configuration/sites/index.mdx

@@ -3,45 +3,50 @@ pcx_content_type: navigation
 title: Workers Sites
 head: []
 description: Use [Workers Static Assets](/workers/static-assets/) to host full-stack applications instead of Workers Sites. Do not use Workers Sites for new projects.
+noindex: true
 ---
 
-import { Render } from "~/components"
-import { LinkButton } from "~/components"
+import { Render } from "~/components";
+import { LinkButton } from "~/components";
 
 <Render file="workers_sites" />
 
 Workers Sites enables developers to deploy static applications directly to Workers. It can be used for deploying applications built with static site generators like [Hugo](https://gohugo.io) and [Gatsby](https://www.gatsbyjs.org), or front-end frameworks like [Vue](https://vuejs.org) and [React](https://reactjs.org).
 
 To deploy with Workers Sites, select from one of these three approaches depending on the state of your target project:
 
-***
+---
 
 ## 1. Start from scratch
 
 If you are ready to start a brand new project, this quick start guide will help you set up the infrastructure to deploy a HTML website to Workers.
 
-<LinkButton href="/workers/configuration/sites/start-from-scratch/">Start from scratch</LinkButton>
+<LinkButton href="/workers/configuration/sites/start-from-scratch/">
+	Start from scratch
+</LinkButton>
 
-***
+---
 
 ## 2. Deploy an existing static site
 
 If you have an existing project or static assets that you want to deploy with Workers, this quick start guide will help you install Wrangler and configure Workers Sites for your project.
 
-<LinkButton href="/workers/configuration/sites/start-from-existing/">Start from an existing static site</LinkButton>
+<LinkButton href="/workers/configuration/sites/start-from-existing/">
+	Start from an existing static site
+</LinkButton>
 
-***
+---
 
 ## 3. Add static assets to an existing Workers project
 
 If you already have a Worker deployed to Cloudflare, this quick start guide will show you how to configure the existing codebase to use Workers Sites.
 
-<LinkButton href="/workers/configuration/sites/start-from-worker/">Start from an existing Worker</LinkButton>
+<LinkButton href="/workers/configuration/sites/start-from-worker/">
+	Start from an existing Worker
+</LinkButton>
 
 :::note
 
-
 Workers Sites is built on Workers KV, and usage rates may apply. Refer to [Pricing](/workers/platform/pricing/) to learn more.
 
-
 :::

src/content/docs/workers/configuration/sites/start-from-existing.mdx

@@ -1,6 +1,7 @@
 ---
 pcx_content_type: how-to
 title: Start from existing
+noindex: true
 sidebar:
   order: 1
 ---
@@ -126,8 +127,6 @@ async function handleEvent(event) {
 
 6. Deploy your site to a [custom domain](/workers/configuration/routing/custom-domains/) that you own and have already attached as a Cloudflare zone. Add a `route` property to the Wrangler file.
 
-
-
    <WranglerConfig>
 
    ```toml

src/content/docs/workers/configuration/sites/start-from-scratch.mdx

@@ -1,6 +1,7 @@
 ---
 pcx_content_type: how-to
 title: Start from scratch
+noindex: true
 sidebar:
   order: 2
 ---
@@ -60,8 +61,6 @@ The template project contains the following files and directories:
 
 - Consider updating`compatibility_date` to today's date to get access to the most recent Workers features:
 
-
-
   <WranglerConfig>
 
   ```toml
@@ -72,8 +71,6 @@ The template project contains the following files and directories:
 
 - Deploy your site to a [custom domain](/workers/configuration/routing/custom-domains/) that you own and have already attached as a Cloudflare zone:
 
-
-
   <WranglerConfig>
 
   ```toml

src/content/docs/workers/configuration/sites/start-from-worker.mdx

@@ -1,6 +1,7 @@
 ---
 pcx_content_type: how-to
 title: Start from Worker
+noindex: true
 sidebar:
   order: 3
 ---

src/schemas/base.ts

@@ -73,6 +73,7 @@ export const baseSchema = z.object({
 	summary: z.string().optional(),
 	goal: z.string().array().optional(),
 	operation: z.string().array().optional(),
+	noindex: z.boolean().optional(),
 	sidebar: z
 		.object({
 			order: z.number().optional(),