Commit 2bd852f
authored
feat(tools): add WebScrapeExecutor with scrape-core integration (#99)
* feat(tools): add WebScrapeExecutor with scrape-core integration (#57)
Implement safe HTML scraping via scrape-core with CSS selectors:
- WebScrapeExecutor: fetch HTTPS URLs, parse HTML with Soup, extract
data using CSS selectors (text, html, attr modes) in spawn_blocking
- CompositeExecutor<A, B>: generic first-match-wins executor chaining
that preserves Agent's single ToolExecutor generic parameter
- SSRF protection: HTTPS-only, IPv4/IPv6 private IP blocking including
link-local, unique local, IPv4-mapped, broadcast, unspecified
- Bundled web-scrape skill with CSS selector examples
- Shared extract_fenced_blocks() utility (DRY refactor from shell.rs)
- Config: [tools.scrape] with timeout (15s) and max_body_bytes (1MB)
Closes #57 Phase 2
* chore: bump version to 0.7.1, update CHANGELOG and README for web scrape1 parent a247425 commit 2bd852f
File tree
16 files changed
+1203
-50
lines changed- config
- crates
- zeph-core/src
- zeph-tools
- src
- skills/web-scrape
- src
16 files changed
+1203
-50
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
9 | 22 | | |
10 | 23 | | |
11 | 24 | | |
| |||
0 commit comments