Skip to content

feat: add Tavily search alongside Sogou in search_and_scrape_webpage#150

Open
tavily-integrations wants to merge 3 commits intoMiroMindAI:mainfrom
Tavily-FDE:feat/tavily-migration/miroflow-sogou-tavily-supplement
Open

feat: add Tavily search alongside Sogou in search_and_scrape_webpage#150
tavily-integrations wants to merge 3 commits intoMiroMindAI:mainfrom
Tavily-FDE:feat/tavily-migration/miroflow-sogou-tavily-supplement

Conversation

@tavily-integrations
Copy link
Copy Markdown

Summary

  • Added tavily_search MCP tool to search_and_scrape_webpage.py as a configurable alternative for global/non-Chinese queries
  • Reads TAVILY_API_KEY from environment; tool is only available when the key is set
  • Forwarded TAVILY_API_KEY through the search_and_scrape_webpage server env dict in settings.py
  • Added tavily-python>=0.5.0 to libs/miroflow-tools/pyproject.toml
  • Existing Sogou/Tencent and Serper search tools remain fully intact (additive migration)

Files changed

  • libs/miroflow-tools/src/miroflow_tools/dev_mcp_servers/search_and_scrape_webpage.py — new tavily_search tool + TAVILY_API_KEY env read
  • apps/miroflow-agent/src/config/settings.pyTAVILY_API_KEY env var + forwarded in server env dict
  • libs/miroflow-tools/pyproject.toml — added tavily-python dependency

Dependency changes

  • Added tavily-python>=0.5.0 to libs/miroflow-tools/pyproject.toml

Environment variable changes

  • Added TAVILY_API_KEY reference in settings.py and search_and_scrape_webpage.py

Notes for reviewers

  • The tavily_search tool follows the same guard pattern as google_search (checks for API key, returns error JSON if missing)
  • Banned URL filtering is applied to Tavily results just like other search tools
  • The Tavily client is imported inside the tool function to avoid import errors when the package is not installed

Automated Review

  • Passed after 3 attempt(s)
  • Final review: The implementation correctly adds a tavily_search MCP tool as an additive parallel supplement to sogou_search and google_search in search_and_scrape_webpage.py. All imports are valid (tavily.errors.TimeoutError confirmed present in v0.7.23). The AsyncTavilyClient is correctly initialized at module level with a None guard, and the TAVILY_API_KEY check in tavily_search ensures make_tavily_request is never called when the client is None. Retry logic is correctly scoped to transient exceptions only. TAVILY_API_KEY is properly threaded through settings.py (module level, MCP server env forwarding, and get_env_info). The pyproject.toml dependency addition is correct. Code style is consistent with the existing Serper/Sogou patterns. One minor issue: the search_depth allowlist omits two valid Tavily values, and .env.example is not updated in this branch (covered by prerequisite unit).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant