Create setting-custom-user-agent.mdx (cloudflare#25073)

* Create setting-custom-user-agent.mdx

* Update content-endpoint.mdx

add partial

* Update json-endpoint.mdx

* Update links-endpoint.mdx

* Update markdown-endpoint.mdx

* Update pdf-endpoint.mdx

* Update scrape-endpoint.mdx

* Update screenshot-endpoint.mdx

* Update snapshot.mdx

* Update src/content/partials/browser-rendering/setting-custom-user-agent.mdx

Co-authored-by: ToriLindsay <[email protected]>

* Update src/content/partials/browser-rendering/setting-custom-user-agent.mdx

Co-authored-by: ToriLindsay <[email protected]>

---------

Co-authored-by: ToriLindsay <[email protected]>

Author: kathayl

src/content/docs/browser-rendering/rest-api/content-endpoint.mdx

@@ -5,7 +5,7 @@ sidebar:
   order: 2
 ---
 
-import { Tabs, TabItem } from "~/components";
+import { Tabs, TabItem, Render } from "~/components";
 
 The `/content` endpoint instructs the browser to navigate to a website and capture the fully rendered HTML of a page, including the `head` section, after JavaScript execution. This is ideal for capturing content from JavaScript-heavy or interactive websites.
 
@@ -58,3 +58,8 @@ curl -X POST 'https://api.cloudflare.com/client/v4/accounts/<accountId>/browser-
 ```
 
 Many more options exist, like setting HTTP headers using `setExtraHTTPHeaders`, setting `cookies`, and using `gotoOptions` to control page load behaviour - check the endpoint [reference](/api/resources/browser_rendering/subresources/content/methods/create/) for all available parameters.
+
+<Render
+  file="setting-custom-user-agent"
+  product="browser-rendering"
+/>

src/content/docs/browser-rendering/rest-api/json-endpoint.mdx

@@ -7,7 +7,7 @@ sidebar:
   order: 9
 ---
 
-import { Tabs, TabItem } from "~/components";
+import { Tabs, TabItem, Render } from "~/components";
 
 The `/json` endpoint extracts structured data from a webpage. You can specify the expected output using either a `prompt` or a `response_format` parameter which accepts a JSON schema. The endpoint returns the extracted data in JSON format. By default, this endpoint leverages [Workers AI](/workers-ai/). If you would like to specify your own AI model for the extraction, you can use the `custom_ai` parameter.
 
@@ -349,3 +349,8 @@ In this example, Browser Rendering first calls Anthropic's Claude Sonnet 4 model
   }
 ]
 ```
+
+<Render
+  file="setting-custom-user-agent"
+  product="browser-rendering"
+/>

src/content/docs/browser-rendering/rest-api/links-endpoint.mdx

@@ -5,7 +5,7 @@ sidebar:
   order: 10
 ---
 
-import { Tabs, TabItem } from "~/components";
+import { Tabs, TabItem, Render } from "~/components";
 
 The `/links` endpoint retrieves all links from a webpage. It can be used to extract all links from a page, including those that are hidden.
 
@@ -212,3 +212,8 @@ curl -X POST 'https://api.cloudflare.com/client/v4/accounts/<accountId>/browser-
 	]
 }
 ```
+
+<Render
+  file="setting-custom-user-agent"
+  product="browser-rendering"
+/>

src/content/docs/browser-rendering/rest-api/markdown-endpoint.mdx

@@ -5,7 +5,7 @@ sidebar:
   order: 10
 ---
 
-import { Tabs, TabItem } from "~/components";
+import { Tabs, TabItem, Render } from "~/components";
 
 The `/markdown` endpoint retrieves a webpage's content and converts it into Markdown format. You can specify a URL and optional parameters to refine the extraction process.
 
@@ -93,6 +93,11 @@ curl -X 'POST' 'https://api.cloudflare.com/client/v4/accounts/<accountId>/browse
 }
 ```
 
+<Render
+  file="setting-custom-user-agent"
+  product="browser-rendering"
+/>
+
 ## Potential use-cases
 
 1. **Content extraction:** Convert a blog post or article into Markdown format for storage or further processing.

src/content/docs/browser-rendering/rest-api/pdf-endpoint.mdx

@@ -5,7 +5,7 @@ sidebar:
   order: 5
 ---
 
-import { Tabs, TabItem } from "~/components";
+import { Tabs, TabItem, Render } from "~/components";
 
 The `/pdf` endpoint instructs the browser to generate a PDF of a webpage or custom HTML using Cloudflare's headless browser rendering service.
 
@@ -137,3 +137,8 @@ curl -X POST https://api.cloudflare.com/client/v4/accounts/<acccountID>/browser-
 }' \
   --output "cloudflare.pdf"
 ```
+
+<Render
+  file="setting-custom-user-agent"
+  product="browser-rendering"
+/>

src/content/docs/browser-rendering/rest-api/scrape-endpoint.mdx

@@ -5,7 +5,7 @@ sidebar:
   order: 7
 ---
 
-import { Tabs, TabItem } from "~/components";
+import { Tabs, TabItem, Render } from "~/components";
 
 The `/scrape` endpoint extracts structured data from specific elements on a webpage, returning details such as element dimensions and inner HTML.
 
@@ -101,3 +101,10 @@ Many more options exist, like setting HTTP credentials using `authenticate`, set
     - `html` _(string)_ - Inner HTML of the element.
     - `attributes` _(array of objects)_ - List of extracted attributes such as `href` for links.
     - `height`, `width`, `top`, `left` _(number)_ - Position and dimensions of the element.
+
+## Advanced Usage
+
+<Render
+  file="setting-custom-user-agent"
+  product="browser-rendering"
+/>

src/content/docs/browser-rendering/rest-api/screenshot-endpoint.mdx

@@ -5,7 +5,7 @@ sidebar:
   order: 3
 ---
 
-import { Tabs, TabItem } from "~/components";
+import { Tabs, TabItem, Render } from "~/components";
 
 The `/screenshot` endpoint renders the webpage by processing its HTML and JavaScript, then captures a screenshot of the fully rendered page.
 
@@ -152,3 +152,8 @@ curl -X POST 'https://api.cloudflare.com/client/v4/accounts/<accountId>/browser-
 ```
 
 Many more options exist, like setting HTTP credentials using `authenticate`, setting `cookies`, and using `gotoOptions` to control page load behaviour - check the endpoint [reference](/api/resources/browser_rendering/subresources/screenshot/methods/create/) for all available parameters.
+
+<Render
+  file="setting-custom-user-agent"
+  product="browser-rendering"
+/>

src/content/docs/browser-rendering/rest-api/snapshot.mdx

@@ -5,7 +5,7 @@ sidebar:
   order: 6
 ---
 
-import { Tabs, TabItem } from "~/components";
+import { Tabs, TabItem, Render } from "~/components";
 
 The `/snapshot` endpoint captures both the HTML content and a screenshot of the webpage in one request. It returns the HTML as a text string and the screenshot as a Base64-encoded image.
 
@@ -102,5 +102,9 @@ curl -X POST 'https://api.cloudflare.com/client/v4/accounts/<accountId>/browser-
 	}
 }
 ```
+<Render
+  file="setting-custom-user-agent"
+  product="browser-rendering"
+/>
 
 Many more options exist, like setting HTTP credentials using `authenticate`, setting `cookies`, and using `gotoOptions` to control page load behaviour - check the endpoint [reference](/api/resources/browser_rendering/subresources/snapshot/) for all available parameters.

src/content/partials/browser-rendering/setting-custom-user-agent.mdx

@@ -0,0 +1,7 @@
+### Set a custom user agent
+
+You can change the user agent at the page level by passing `userAgent` as a top-level parameter in the JSON body. This is useful if the target website serves different content based on the user agent.
+
+:::note
+The `userAgent` parameter does not bypass bot protection. Requests from Browser Rendering will always be identified as a bot. 
+:::