claude: Refine how to read API docs a bit#2135
Open
gnprice wants to merge 3 commits intozulip:mainfrom
Open
Conversation
When working in this codebase, one regularly wants to read the Zulip API docs, which is at URLs under https://zulip.com/api/ . Give Claude permission to do that, so that it doesn't cause a permissions prompt each time. Unfortunately, the permissions language is limited so that we can do this only by giving it permission to fetch arbitrary URLs on the domain. (I tried writing a "skill" that would enable only this more specific set of pages, but couldn't find a way to do it that would be airtight to a prompt injection causing an equally broad range of fetches. I believe something airtight could be done by putting this functionality inside an MCP server... but setting up an MCP server seems to be significant overhead.) Fortunately, though, there's nothing sensitive on that domain: it's all public information pages, even outside the API docs, with nothing that accepts credentials in order to take any sort of privileged action. So allowing the broader set of fetches is acceptable.
The API changelog is very long (over 250 kB). As a result, Claude's normal WebFetch tool tries to summarize it with a cheap model before passing the result to anything else. These summaries make it impossible to get accurate answers about what happened at a given feature level or range of feature levels. To do much of anything useful with the changelog, we therefore need to get the full text and let Claude extract relevant snippets from it verbatim with tools like grep, rather than any LLM. Claude will naturally do this once it has the text as a local file. We could do that by telling Claude to fetch the page from the web, with a prompt fragment like: Use curl to fetch the changelog page and save the file. That works. But in this case we can do a bit better still: assuming the developer has a Zulip server/web worktree at ../zulip , we can read the page's Markdown source file there. It has all the same information as the web page, with a bit less clutter. This does cause a permission prompt. (The "additionalDirectories" setting looks like it should offer a way to avoid that; but I tried, and couldn't get that setting to help.) Fetching with curl would do that too, though.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Discussion: #mobile-team > Claude: reading API changelog @ 💬
Selected commit messages
9506e72 claude: Allow fetching from zulip.com, aiming at API docs
When working in this codebase, one regularly wants to read the
Zulip API docs, which is at URLs under https://zulip.com/api/ .
Give Claude permission to do that, so that it doesn't cause a
permissions prompt each time.
Unfortunately, the permissions language is limited so that we can
do this only by giving it permission to fetch arbitrary URLs on the
domain. (I tried writing a "skill" that would enable only this
more specific set of pages, but couldn't find a way to do it that
would be airtight to a prompt injection causing an equally broad
range of fetches. I believe something airtight could be done by
putting this functionality inside an MCP server... but setting up
an MCP server seems to be significant overhead.)
Fortunately, though, there's nothing sensitive on that domain: it's
all public information pages, even outside the API docs, with
nothing that accepts credentials in order to take any sort of
privileged action. So allowing the broader set of fetches is
acceptable.
279eeea claude: Get API changelog from local source, rather than web
The API changelog is very long (over 250 kB). As a result, Claude's
normal WebFetch tool tries to summarize it with a cheap model before
passing the result to anything else. These summaries make it
impossible to get accurate answers about what happened at a given
feature level or range of feature levels.
To do much of anything useful with the changelog, we therefore need
to get the full text and let Claude extract relevant snippets from
it verbatim with tools like grep, rather than any LLM. Claude will
naturally do this once it has the text as a local file.
We could do that by telling Claude to fetch the page from the web,
with a prompt fragment like:
Use curl to fetch the changelog page and save the file.
That works. But in this case we can do a bit better still: assuming
the developer has a Zulip server/web worktree at ../zulip , we can
read the page's Markdown source file there. It has all the same
information as the web page, with a bit less clutter.
This does cause a permission prompt. (The "additionalDirectories"
setting looks like it should offer a way to avoid that; but I tried,
and couldn't get that setting to help.) Fetching with curl would do
that too, though.