Skip to content

Commit 36aa38a

Browse files
committed
image & tokens comparison
1 parent fcacae1 commit 36aa38a

File tree

2 files changed

+18
-1
lines changed

2 files changed

+18
-1
lines changed
49.7 KB
Loading

src/content/docs/style-guide/how-we-docs/ai-consumability.mdx

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ meta:
55
title: AI consumability | How we docs
66
---
77

8-
import { Tabs, TabItem } from "~/components";
8+
import { Tabs, TabItem, Width } from "~/components";
99

1010
We have various approaches for making our content visible to AI as well as making sure it's easily consumed in a plain-text format.
1111

@@ -25,6 +25,11 @@ We have implemented `llms.txt`, `llms-full.txt` and also created per-page Markdo
2525

2626
In the top right of this page, you will see a `Page options` button where you can copy the current page as Markdown that can be given to your LLM of choice.
2727

28+
<Width size="medium">
29+
![Page options
30+
button](~/assets/images/style-guide/how-we-docs/page-options.png)
31+
</Width>
32+
2833
## Textual representation of interactive elements
2934

3035
HTML is easily parsed - after all, the browser has to parse it to decide how to render the page you're reading now - it tends to not be very _portable_.
@@ -73,3 +78,15 @@ For example, take a look at our Markdown test fixture (or any page by appending
7378

7479
- [`/style-guide/fixtures/markdown/`](/style-guide/fixtures/markdown/)
7580
- [`/style-guide/fixtures/markdown/index.md`](/style-guide/fixtures/markdown/index.md)
81+
82+
### Saving on tokens
83+
84+
Most AI pricing is around input & output tokens and our approach greatly reduces the amount of input tokens required.
85+
86+
For example, let's take a look at the amount of tokens required for the [Workers Get Started](/workers/get-started/guide/) using [OpenAI's tokenizer](https://platform.openai.com/tokenizer):
87+
88+
- HTML: 15,229 tokens
89+
- turndown: 3,401 tokens (4.48x less than HTML)
90+
- index.md: 2,110 tokens (7.22x less than HTML)
91+
92+
When providing our content to AI, we can see a real-world ~7x saving in input tokens cost.

0 commit comments

Comments
 (0)