docs: bump Docusaurus to v3.9.1 (#1428)

This PR updates the website's Docusaurus installation from v3.8.1 to
v3.9.1 and applies the configuration changes from
[apify/crawlee#3175](apify/crawlee#3175) to fix
search modal styles after the docsearch v4 update.

## Changes Made

### Package Updates
- Updated all Docusaurus packages from `3.8.1` to `3.9.1`:
  - `@docusaurus/core`
  - `@docusaurus/faster`
  - `@docusaurus/mdx-loader`
  - `@docusaurus/plugin-client-redirects`
  - `@docusaurus/preset-classic`
  - `@docusaurus/theme-mermaid`
  - `@docusaurus/module-type-aliases`
  - `@docusaurus/types`
- Added `@mermaid-js/layout-elk@^0.1.5` dependency for enhanced Mermaid
diagram support

### Configuration Updates
Updated `docusaurus.config.js` to match the new v3.9.1 configuration
structure:

- **Simplified broken link handling**: Removed TypeScript type
annotations from `onBrokenLinks` configuration
- **Restructured markdown configuration**: Moved `onBrokenMarkdownLinks`
from top-level to `markdown.hooks.onBrokenMarkdownLinks`
- **Enhanced markdown options**: Maintained mermaid support while
adopting the new hooks-based configuration

### CSS Fixes
Applied DocSearch styling fixes in `custom.css`:

- Added `.DocSearch-Clear` color styling for better visibility
- Removed `flex-direction: row-reverse` from `.DocSearch-Form` to fix
search input layout issues

## Testing

The website builds successfully with the new Docusaurus version and all
functionality is preserved. The search modal styling issues present in
the previous version have been resolved.

## Impact

This update brings the website up to the latest stable Docusaurus
version, improving performance and fixing known issues with the search
functionality while maintaining full backward compatibility.

<!-- START COPILOT CODING AGENT SUFFIX -->



<details>

<summary>Original prompt</summary>

> update docusaurus to v3.9.1 and modify the config the same way as in
@apify/crawlee/pull/3175, and add `@mermaid-js/layout-elk` to the
website dependencies
> 
> do this inside the `website` folder, where the docusaurus project
lives


</details>


<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.

---------

Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: B4nan <[email protected]>

Author: Copilot

website/docusaurus.config.js

@@ -73,16 +73,16 @@ module.exports = {
         gaGtag: true,
         repoUrl: 'https://github.com/apify/crawlee-python',
     },
+    onBrokenLinks: 'throw',
     markdown: {
         mermaid: true,
+        hooks: {
+            onBrokenMarkdownLinks: 'throw',
+        },
     },
     themes: [
         '@docusaurus/theme-mermaid',
     ],
-    onBrokenLinks:
-    /** @type {import('@docusaurus/types').ReportingSeverity} */ ('throw'),
-    onBrokenMarkdownLinks:
-    /** @type {import('@docusaurus/types').ReportingSeverity} */ ('throw'),
     presets: /** @type {import('@docusaurus/types').PresetConfig[]} */ ([
         [
             '@docusaurus/preset-classic',

website/package.json

@@ -18,8 +18,8 @@
     "devDependencies": {
         "@apify/eslint-config-ts": "^0.4.0",
         "@apify/tsconfig": "^0.1.0",
-        "@docusaurus/module-type-aliases": "3.8.1",
-        "@docusaurus/types": "3.8.1",
+        "@docusaurus/module-type-aliases": "3.9.1",
+        "@docusaurus/types": "3.9.1",
         "@types/react": "^19.0.0",
         "@typescript-eslint/eslint-plugin": "8.44.0",
         "@typescript-eslint/parser": "8.44.0",
@@ -36,14 +36,15 @@
     "dependencies": {
         "@apify/docusaurus-plugin-typedoc-api": "^4.4.7",
         "@apify/utilities": "^2.8.0",
-        "@docusaurus/core": "3.8.1",
-        "@docusaurus/faster": "3.8.1",
-        "@docusaurus/mdx-loader": "3.8.1",
-        "@docusaurus/plugin-client-redirects": "3.8.1",
-        "@docusaurus/preset-classic": "3.8.1",
-        "@docusaurus/theme-mermaid": "3.8.1",
+        "@docusaurus/core": "3.9.1",
+        "@docusaurus/faster": "3.9.1",
+        "@docusaurus/mdx-loader": "3.9.1",
+        "@docusaurus/plugin-client-redirects": "3.9.1",
+        "@docusaurus/preset-classic": "3.9.1",
+        "@docusaurus/theme-mermaid": "3.9.1",
         "@giscus/react": "^3.0.0",
         "@mdx-js/react": "^3.0.1",
+        "@mermaid-js/layout-elk": "^0.1.5",
         "@signalwire/docusaurus-plugin-llms-txt": "^1.2.1",
         "axios": "^1.5.0",
         "buffer": "^6.0.3",

website/src/css/custom.css

@@ -761,6 +761,10 @@ div[class^="announcementBar_"] button {
         }
     }
 
+    .DocSearch-Clear {
+        color: var(--color-text-subtle);
+    }
+
     .DocSearch-Form {
         border-radius: 6px;
         border-radius: var(--Radius-6, 6px);
@@ -769,7 +773,6 @@ div[class^="announcementBar_"] button {
         box-shadow: none;
         height: 40px;
         padding: 8px 12px;
-        flex-direction: row-reverse;
     }
 
     .DocSearch-Input {