[Portal] Separate LLM content extraction from search data extraction

[joaquim-verges]

---

PR-Codex overview

This PR focuses on enhancing the extraction of LLM content from HTML files for documentation purposes and refactoring related functionalities.

Detailed summary

• Added extractLLMData.ts to extract LLM content and write it to files.
• Updated extractSearchData.ts to remove LLM content extraction.
• Modified package.json to include a new script for LLM content extraction.
• Refactored extractContent in index.ts to exclude LLM content.
• Introduced extractContentForLLM in llm-extract.ts to handle LLM extraction logic.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
docs-v2	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 25, 2025 0:29am

4 Skipped Deployments

Name	Status	Preview	Comments	Updated (UTC)
login	⬜️ Skipped (Inspect)			Apr 25, 2025 0:29am
thirdweb_playground	⬜️ Skipped (Inspect)			Apr 25, 2025 0:29am
thirdweb-www	⬜️ Skipped (Inspect)			Apr 25, 2025 0:29am
wallet-ui	⬜️ Skipped (Inspect)			Apr 25, 2025 0:29am

[changeset-bot]

⚠️ No Changeset found

Latest commit: bcd6f08

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[joaquim-verges]

• [Portal] Separate LLM content extraction from search data extraction #6854 👈 (View in Graphite)
• main

---

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• merge-queue - adds this PR to the back of the merge queue
• hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

size-limit report 📦

Path	Size	Loading time (3g)	Running time (snapdragon)	Total time
thirdweb (esm)	51.16 KB (0%)	1.1 s (0%)	228 ms (+143.03% 🔺)	1.3 s
thirdweb (cjs)	143.77 KB (0%)	2.9 s (0%)	381 ms (+27.55% 🔺)	3.3 s
thirdweb (minimal + tree-shaking)	5.66 KB (0%)	114 ms (0%)	46 ms (+513.64% 🔺)	159 ms
thirdweb/chains (tree-shaking)	514 B (0%)	11 ms (0%)	17 ms (+299.02% 🔺)	27 ms
thirdweb/react (minimal + tree-shaking)	19.33 KB (0%)	387 ms (0%)	122 ms (+573.81% 🔺)	509 ms

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 55.31%. Comparing base (68aa693) to head (bcd6f08).
Report is 8 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #6854   +/-   ##
=======================================
  Coverage   55.31%   55.31%           
=======================================
  Files         896      896           
  Lines       57023    57024    +1     
  Branches     3971     3968    -3     
=======================================
+ Hits        31541    31542    +1     
  Misses      25385    25385           
  Partials       97       97           

Flag	Coverage Δ
packages	55.31% <ø> (+<0.01%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.