Skip to content

Commit bde0b05

Browse files
committed
Gate paywalled searches on VPN confirmation before booting Chrome
The LLM-as-agent path was running searches that involved IEEE / Scholar / paywalled publishers without first checking whether the user had VPN or institutional access. Without VPN: Scholar captchas within a few requests, IEEE serves abstract-only / 403 for the PDF stage, and the user spends minutes watching a useless Chrome window. CLAUDE.md HARD RULE now requires the agent to confirm VPN status (recall from conversation, or AskUserQuestion) BEFORE running 'python -m autopapertoppt -q ...' or any scripts/llm_*.py invocation. When the user says no VPN, the source mix is restricted to open sources (arxiv,openalex,pubmed,crossref,dblp,openaire). compliance-auditor: the in-practice list now leads with the VPN confirmation step before the WebRunner-vs-httpx rules. paper-summary-author: source-level browser-automation rule prefaced with a VPN gate paragraph naming AskUserQuestion as the canonical asking mechanism.
1 parent bf4e088 commit bde0b05

3 files changed

Lines changed: 33 additions & 7 deletions

File tree

.claude/agents/compliance-auditor.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -75,11 +75,12 @@ A subset of upstreams reject anonymous `httpx` outright (TLS / JS fingerprint, A
7575

7676
**In practice:**
7777

78-
1. `sources/ieee/fetcher.py:_scrape_search` tries WebRunner first. The httpx `POST /rest/search` branch is a CI / no-Chrome safety net, **not** the production path. On a user machine with VPN, silent fall-through to httpx is a bug — surface it instead of trusting the results.
79-
2. Never propose `--source` lists that exclude `ieee` "to avoid the slow browser boot." VPN access is precisely why the user wants the browser path.
80-
3. LLM-as-agent paywalled PDF fetch: drive a one-off Bash + Selenium script in the shape of `scripts/llm_driven_search.py``webrunner_browser.make_driver()` for visible Chrome, `driver.get(...)`, `wait_for_captcha_solved(...)`, capture `driver.page_source` or trigger a real download. Never paste a publisher URL into `httpx` / `urllib` / `subprocess curl` and call it equivalent. Never call `mcp__webrunner__webrunner_run_actions` — that tool is not exposed by the MCP server registered here.
81-
4. The visible Chrome window is a feature (CAPTCHA + SSO). Don't suppress it — no `--headless`, no `options.add_argument("--headless")`.
82-
5. Debugging: look for the `IEEE (scrape) returned N papers …` INFO log emitted by `sources/ieee/fetcher.py`. If results came back in under ~5 seconds without that log line AND without a Chrome window appearing, WebRunner threw and httpx silently fired — flag it.
78+
1. **Confirm VPN access BEFORE running any search that involves IEEE / Scholar / paywalled-PDF flows.** When the user requests a paper search ("搜尋 X" / "search X" / "find papers on X"), the LLM's first action is NOT to invoke the search — it is to check VPN status. Either recall from the conversation, or ask via `AskUserQuestion` ("Do you have VPN / institutional access for IEEE / ACM / Springer for this topic? Affects whether I include them as sources."). Without VPN: Scholar gets captcha'd within a few requests, IEEE returns abstract-only / 403 for the PDF stage, and the user wastes minutes staring at a useless Chrome window. Same gate applies before invoking `scripts/llm_driven_search.py` or `scripts/llm_download_pdfs.py`. When the user confirms NO VPN, restrict the source mix to open sources (`arxiv,openalex,pubmed,crossref,dblp,openaire`) and skip `ieee,scholar`.
79+
2. `sources/ieee/fetcher.py:_scrape_search` tries WebRunner first. The httpx `POST /rest/search` branch is a CI / no-Chrome safety net, **not** the production path. On a user machine with VPN, silent fall-through to httpx is a bug — surface it instead of trusting the results.
80+
3. Never propose `--source` lists that exclude `ieee` "to avoid the slow browser boot." VPN access is precisely why the user wants the browser path — but only after step 1 confirmed they have it.
81+
4. LLM-as-agent paywalled PDF fetch: drive a one-off Bash + Selenium script in the shape of `scripts/llm_driven_search.py``webrunner_browser.make_driver()` for visible Chrome, `driver.get(...)`, `wait_for_captcha_solved(...)`, capture `driver.page_source` or trigger a real download. Never paste a publisher URL into `httpx` / `urllib` / `subprocess curl` and call it equivalent. Never call `mcp__webrunner__webrunner_run_actions` — that tool is not exposed by the MCP server registered here.
82+
5. The visible Chrome window is a feature (CAPTCHA + SSO). Don't suppress it — no `--headless`, no `options.add_argument("--headless")`.
83+
6. Debugging: look for the `IEEE (scrape) returned N papers …` INFO log emitted by `sources/ieee/fetcher.py`. If results came back in under ~5 seconds without that log line AND without a Chrome window appearing, WebRunner threw and httpx silently fired — flag it.
8384

8485
**Audit checks for this category:**
8586
- Grep changed files for `headless`, `--headless`, `add_argument("--headless")`.

.claude/agents/paper-summary-author.md

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,18 @@ The xlsx has columns `# | Title | Authors | Year | Source | Indexed via | DOI |
2323

2424
## Source-level browser-automation rule (read before anything else)
2525

26+
**VPN gate (applies to SEARCH, not just PDF download).** Before invoking
27+
any search that includes IEEE / Scholar / paywalled-publisher domains
28+
— including the parent agent's `python -m autopapertoppt -q ...` or
29+
`scripts/llm_driven_search.py` — confirm the user's VPN / institutional
30+
access status. If unknown, ask via `AskUserQuestion` ("Do you have VPN
31+
for IEEE / ACM / Springer for this topic? Affects whether I include
32+
them."). Without VPN: Scholar gets captcha'd quickly and IEEE returns
33+
abstract-only / 403 PDFs, making the run a waste of the user's time
34+
watching Chrome boot. When the user confirms NO VPN, restrict the
35+
search to open sources (`arxiv,openalex,pubmed,crossref,dblp,openaire`)
36+
and skip `ieee,scholar`.
37+
2638
Even before you touch a PDF: if the user's run included IEEE (default on),
2739
Scholar (opt-in), or any paywalled publisher CDN, the canonical path is
2840
**visible Chrome via WebRunner**, never direct httpx. The IEEE plugin's
@@ -31,8 +43,8 @@ branch is only a safety net for machines without Chrome. If you reviewed
3143
a previous run and IEEE returned a result set in under ~5 seconds without
3244
a Chrome window appearing, WebRunner silently fell through to httpx —
3345
flag this to the user, do not treat those results as authoritative for
34-
summary authoring. (Full rule in `CLAUDE.md` "Browser Automation Is
35-
Mandatory for Publisher Domains".)
46+
summary authoring. (Full rule in `CLAUDE.md` "IEEE / Publisher CDN:
47+
Browser Automation Is Mandatory".)
3648

3749
## When the CLI couldn't download a paywalled PDF (LLM-driven Bash + Selenium)
3850

CLAUDE.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,19 @@ later" is not allowed.
6969

7070
## IEEE / Publisher CDN: Browser Automation Is Mandatory (HARD RULE)
7171

72+
**Before triggering ANY search that involves IEEE / Scholar / paywalled
73+
publishers, the LLM in this session MUST confirm the user's VPN /
74+
institutional access status first** — either by recalling a recent
75+
statement, or by asking via `AskUserQuestion` ("Do you have VPN for
76+
IEEE / ACM / Springer for this topic?"). Without VPN, Scholar gets
77+
captcha'd quickly and IEEE returns abstract-only / 403 PDFs, so the
78+
run wastes the user's time watching a useless Chrome window. When the
79+
user says no VPN, restrict the search to open sources
80+
(`arxiv,openalex,pubmed,crossref,dblp,openaire`) and skip
81+
`ieee,scholar`. This gate applies BEFORE running `python -m
82+
autopapertoppt -q …`, before `scripts/llm_driven_search.py`, and
83+
before any `scripts/llm_download_*pdf*.py` invocation.
84+
7285
IEEE search, IEEE document fetch, Google Scholar search, and any paywalled-PDF
7386
download from publisher CDNs (ieeexplore.ieee.org, dl.acm.org, link.springer.com,
7487
sciencedirect.com, wiley/oup/nature/science/…) MUST go through **visible Chrome**.

0 commit comments

Comments
 (0)