Skip to content

Commit 88b4bc7

Browse files
veithlyveithly
authored
Merge pull request #13 from helloissariel/main
core-concepts: Short-term memory and Toolkit
2 parents 436238d + 473daa4 commit 88b4bc7

File tree

22 files changed

+1658
-1
lines changed

22 files changed

+1658
-1
lines changed

‎Toolkit/crypto/data-tools.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
2+
3+
`spoon_toolkits.crypto.crypto_data_tools` is SpoonAI’s Bitquery-centric toolbox for price discovery, liquidity intelligence, lending-rate aggregation, and wallet forensics. Every class in this package plugs directly into the Spoon tool runtime, so agents can reuse the same `BaseTool` lifecycle (validation, structured outputs, telemetry) without additional glue code.
4+
5+
6+
## Environment & Dependencies
7+
8+
```bash
9+
export BITQUERY_API_KEY=your_rest_key # legacy modules
10+
export BITQUERY_CLIENT_ID=your_oauth_client_id # required by Bitquery OAuth
11+
export BITQUERY_CLIENT_SECRET=your_oauth_secret
12+
export RPC_URL=https://eth.llamarpc.com
13+
```
14+
15+
- `Get*`/`Alert*` tools depend on `web3` (for Uniswap) and respect `RPC_URL`. Provide a reliable mainnet endpoint to avoid public-rate limits.
16+
- `PredictPrice` requires `pandas`, `scikit-learn`, and `numpy`. Install toolkit extras via `pip install -r requirements.txt` at the project root or install modules individually.
17+
- `LendingRateMonitorTool` performs concurrent HTTP requests using `aiohttp`; event loops must be active (use `nest_asyncio` in notebooks if necessary).
18+
19+
## Tool Catalog
20+
21+
### Spot & Historical Pricing
22+
- `GetTokenPriceTool` resolves ERC-20 pairs to the canonical Uniswap v3 pool, fetches slot0, and converts ticks into human prices.
23+
- `Get24hStatsTool` and `GetKlineDataTool` expose the same provider stack for downstream analytics (volatility, TWAP, etc.).
24+
25+
### Alerting & Monitoring
26+
- `PriceThresholdAlertTool` compares live prices with a configurable ±% drift versus prior day.
27+
- `LpRangeCheckTool` reads Uniswap positions, current ticks, and warns when LPs approach range boundaries (`buffer_ticks`).
28+
- `SuddenPriceIncreaseTool` scans Bitquery datasets for large-cap, high-volume tokens with rapid appreciation—ideal for whale or listing alerts.
29+
- `CryptoMarketMonitor` abstracts “set-and-forget” monitors by POSTing well-formed payloads to the monitoring daemon you run (defaults to `localhost:8888` but can be swapped).
30+
31+
### Wallet Intelligence
32+
- `WalletAnalysis`, `TokenHolders`, and `TradingHistory` share the Bitquery GraphQL templates embedded in their modules so you always know the dataset (`combined` vs `archive`) and filters before execution.
33+
- Pagination is driven by the `limit` clauses inside the template; adjust the class constants if you need bigger windows.
34+
35+
### DeFi Rates & Liquidity
36+
- `LendingRateMonitorTool` merges DeFiLlama pools with Aave/Compound/Morpho APIs, derives utilization, and wraps the result in `ToolResult` for consumption by governance agents.
37+
- `UniswapLiquidity` emits the latest Mint/Burn events so you can approximate real-time liquidity deltas without running a listener yourself.
38+
- `PredictPrice` is intentionally heavyweight (builds a RandomForest and scaler); cache the fitted estimator in your application if you call it frequently.
39+
40+
## Usage Patterns
41+
42+
### Synchronous price lookup
43+
```python
44+
from spoon_toolkits.crypto.crypto_data_tools import GetTokenPriceTool
45+
46+
tool = GetTokenPriceTool()
47+
result = await tool.execute(symbol="ETH-USDC")
48+
49+
print(result.output["price"]) # structured response
50+
print(result.diagnostic["pool"]) # extra context for logging
51+
```
52+
53+
### Long-running alert inside an agent
54+
```python
55+
import asyncio
56+
from spoon_toolkits.crypto.crypto_data_tools import PriceThresholdAlertTool
57+
58+
alerts = PriceThresholdAlertTool()
59+
60+
async def monitor():
61+
while True:
62+
tool_result = await alerts.execute(symbol="ETH-USDC", threshold_percent=7)
63+
if tool_result.output.get("exceeded"):
64+
await send_notification(tool_result.output["message"])
65+
await asyncio.sleep(60)
66+
67+
asyncio.run(monitor())
68+
```
69+
70+
Tips:
71+
- Most tools return dictionaries; when a class subclasses `DexBaseTool`, the `ToolResult.output` mirrors the Bitquery JSON fragment shown in the GraphQL template.
72+
- Async utilities (`PriceThresholdAlertTool`, `LendingRateMonitorTool`) already shield you from rate spikes with short sleeps; avoid spawning excessive parallel loops unless you also raise Bitquery limits.
73+
74+
## Operational Notes
75+
76+
- Centralize credential management: add the Bitquery OAuth keys to your `.env` and load them before instantiating tools to prevent runtime `ValueError`s from `DexBaseTool.oAuth()`.
77+
- Because Uniswap calls hit your RPC, failures there do **not** imply Bitquery downtime—inspect `ToolResult.diagnostic` for the precise failure domain.
78+
- The monitoring helper in `blockchain_monitor.py` assumes a scheduler reachable at `http://localhost:8888/monitoring/tasks`. Override `api_url` in the module before deploying if your infra differs.
79+
- `PredictPrice` fetches up to ~1000 rows per query. If Bitquery throttles you, lower the template limit or use a paid API plan.

‎Toolkit/crypto/evm.md‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
2+
3+
`spoon_toolkits.crypto.evm` gathers the core on-chain primitives agents need—native transfers, ERC-20 operations, swaps, quotes, bridges, and balance lookups—without shelling out to the JS plugin. Each class inherits `BaseTool`, so input validation, diagnostics, and async receipt waiting all behave consistently across Spoon agents.
4+
5+
## Environment & Dependencies
6+
7+
```bash
8+
export EVM_PROVIDER_URL=https://mainnet.infura.io/v3/...
9+
export EVM_PRIVATE_KEY=0xyourSignerKey
10+
export EVM_CHAIN_ID=1 # optional; inferred via RPC when omitted
11+
# Optional global fallback used by other crypto toolkits
12+
export RPC_URL=https://eth.llamarpc.com
13+
```
14+
15+
- Every tool accepts `rpc_url`/`private_key` overrides per call; defaults resolve to `EVM_PROVIDER_URL` and `EVM_PRIVATE_KEY`, then `RPC_URL`.
16+
- `web3.py` is lazily imported inside each execute path, so include it (and `requests`) in your runtime environment.
17+
- Aggregator-backed tools call public REST APIs (Bebop or LiFi). If you operate behind a proxy, ensure outbound HTTPS is allowed.
18+
19+
## Quick Start – Native Transfer
20+
21+
```python
22+
from spoon_toolkits.crypto.evm import EvmTransferTool
23+
24+
transfer = EvmTransferTool()
25+
tx = await transfer.execute(
26+
to_address="0xrecipient...",
27+
amount_ether="0.05",
28+
data="0x", # optional calldata
29+
)
30+
31+
print(tx.output["hash"])
32+
```
33+
34+
Receipts are awaited with reasonable timeouts; if the transaction reverts, `ToolResult.error` includes the tx hash so you can inspect it with an explorer.
35+
36+
## Toolkit Overview
37+
38+
| Tool | Module | Purpose |
39+
| --- | --- | --- |
40+
| `EvmTransferTool` | `transfer.py` | Send native tokens with optional data payloads, auto gas estimation, and nonce management. |
41+
| `EvmErc20TransferTool` | `erc20.py` | Transfer ERC-20 tokens (balanceOf/decimals aware) with gas estimation and optional gas price overrides. |
42+
| `EvmBalanceTool` | `balance.py` | Read native or ERC-20 balances for any address. |
43+
| `EvmSwapTool` | `swap.py` | Execute same-chain swaps through the Bebop aggregator, including approvals when needed. |
44+
| `EvmSwapQuoteTool` | `quote.py` | Fetch swap quotes without execution; compares Bebop and LiFi outputs when requested. |
45+
| `EvmBridgeTool` | `bridge.py` | Bridge assets across chains via LiFi advanced routes and step transactions. |
46+
47+
## Tool Notes
48+
49+
### `EvmTransferTool`
50+
- Defaults to `EVM_PROVIDER_URL`/`EVM_PRIVATE_KEY` but also checks `RPC_URL` so the tool can run inside broader Spoon stacks.
51+
- Automatically estimates gas; if estimation fails it falls back to 21k for plain transfers or 100k when `data` is present.
52+
- Explicit overrides exist for `gas_limit`, `gas_price_gwei`, and `nonce` so advanced workflows (batched transactions, EOA abstraction) remain possible.
53+
54+
### `EvmErc20TransferTool`
55+
- Resolves token decimals via the contract; if the call fails it assumes 18 decimals, so pass `amount` accordingly.
56+
- Builds and signs a `transfer` call, then waits for completion. Emits `{hash, token, amount, decimals}` in `output`.
57+
- Accepts `gas_price_gwei`; otherwise uses the RPC’s `gas_price`.
58+
59+
### `EvmBalanceTool`
60+
- Returns native balances (Ether-equivalent) when `token_address` is omitted, or ERC-20 balances plus decimals when provided.
61+
- Useful for guardrails before making transfers or swaps.
62+
63+
### `EvmSwapTool`
64+
- Calls Bebop’s router for supported chains (`1`, `10`, `137`, `42161`, `8453`, `59144`). Unsupported IDs return a descriptive error.
65+
- Handles ERC-20 approvals automatically by checking allowance against `approvalTarget` and submitting an `approve` tx when required.
66+
- Accepts `gas_price_gwei` overrides; otherwise reuses the aggregator-suggested gas price or the RPC default.
67+
- `slippage_bps` is currently informational because Bebop does not accept slippage; expect fills to match aggregator routing.
68+
69+
### `EvmSwapQuoteTool`
70+
- Works in quote-only contexts where you want to compare outputs without signing anything.
71+
- Resolve chain ID via `chain_id` parameter or by pointing `rpc_url` at the target network.
72+
- `aggregator="both"` (default) fetches Bebop and LiFi; response includes `quotes` plus a precalculated `best` entry ranked by min output amount.
73+
- Requires RPC access for ERC-20 decimals when quoting token sales; native sells use zero address.
74+
75+
### `EvmBridgeTool`
76+
- Wraps LiFi’s `/advanced/routes` + `/advanced/stepTransaction`. After selecting the recommended path, it executes the first step locally and waits for the on-chain receipt.
77+
- Performs ERC-20 approvals before bridging when necessary; approvals and bridge txs both honor `gas_price_gwei`.
78+
- `from_address` is implied from the signer, while `to_address` defaults to the same account but can target any destination address.
79+
- Returns hashes plus source/destination chain IDs so orchestrators can track the bridge until completion on the far chain.
80+
81+
## Operational Tips
82+
83+
- Centralize signer management: store hot keys in a secure vault and inject them as environment variables just before launching the agent process.
84+
- Throttle swap/bridge calls if you expect to fire many approvals quickly; both Bebop and LiFi enforce rate limits.
85+
- Log the `ToolResult` payloads—especially `output["hash"]`—so you can reconcile actions if an agent crashes mid-run.
86+
- When rotating RPC endpoints, prefer passing `rpc_url` explicitly to keep observability clear (environment-based fallbacks can mask misconfiguration).

‎Toolkit/crypto/powerdata.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
2+
3+
`spoon_toolkits.crypto.crypto_powerdata` fuses CCXT-powered CEX feeds, OKX Web3 DEX data, TA-Lib/enhanced indicators, and an MCP server that can stream results over stdio or SSE. Use it when agents need richer analytics than simple price lookups.
4+
5+
## Environment & Settings
6+
7+
```bash
8+
export OKX_API_KEY=...
9+
export OKX_SECRET_KEY=...
10+
export OKX_API_PASSPHRASE=...
11+
export OKX_PROJECT_ID=...
12+
export OKX_BASE_URL=https://web3.okx.com/api/v5/ # optional override
13+
14+
# Optional overrides (defaults shown)
15+
export RATE_LIMIT_REQUESTS_PER_SECOND=10
16+
export MAX_RETRIES=3
17+
export RETRY_DELAY=1.0
18+
export TIMEOUT_SECONDS=30
19+
```
20+
21+
`data_provider.Settings` ingests these variables (plus indicator defaults such as SMA/EMA periods). Missing OKX keys raise immediately before any HTTP call, so configure them centrally—either via environment or by passing `env_vars` into the MCP helpers.
22+
23+
## What’s Inside the Toolkit
24+
25+
<table>
26+
<colgroup>
27+
<col style={{ width: "22%" }} />
28+
<col style={{ width: "22%" }} />
29+
<col style={{ width: "56%" }} />
30+
</colgroup>
31+
<thead>
32+
<tr>
33+
<th>Component</th>
34+
<th>File(s)</th>
35+
<th>Purpose</th>
36+
</tr>
37+
</thead>
38+
<tbody>
39+
<tr>
40+
<td>`CryptoPowerDataCEXTool`</td>
41+
<td>`tools.py`</td>
42+
<td>Pull OHLCV candles from 100+ CCXT exchanges and pipe them through the enhanced indicator stack.</td>
43+
</tr>
44+
<tr>
45+
<td>`CryptoPowerDataDEXTool`</td>
46+
<td>`tools.py`</td>
47+
<td>Hit OKX Web3 DEX APIs for on-chain pairs specified by `chain_index` + token address.</td>
48+
</tr>
49+
<tr>
50+
<td>`CryptoPowerDataPriceTool`</td>
51+
<td>`tools.py`</td>
52+
<td>Lightweight spot price snapshot (CEX or DEX) without fetching an entire candle set.</td>
53+
</tr>
54+
<tr>
55+
<td>`CryptoPowerDataIndicatorsTool`</td>
56+
<td>`tools.py`</td>
57+
<td>Enumerate every indicator name/parameter accepted by the enhanced TA registry (TA-Lib + custom extras).</td>
58+
</tr>
59+
<tr>
60+
<td>`Settings`, `OKXDEXClient`, `TechnicalAnalysis`</td>
61+
<td>`data_provider.py`</td>
62+
<td>Central place for rate limiting, retries, authenticated OKX calls, and TA-Lib helpers.</td>
63+
</tr>
64+
<tr>
65+
<td>MCP server runners (`start_crypto_powerdata_mcp_*`)</td>
66+
<td>`server.py`, `dual_transport_server.py`</td>
67+
<td>Start stdio or HTTP/SSE transports so UI agents can subscribe to continuous feeds.</td>
68+
</tr>
69+
<tr>
70+
<td>Analytics core</td>
71+
<td>`main.py`, `enhanced_indicators.py`, `talib_registry.py`</td>
72+
<td>Parse indicator configs, register TA functions, and expose them via FastMCP tools.</td>
73+
</tr>
74+
</tbody>
75+
</table>
76+
77+
All tools inherit `CryptoPowerDataBaseTool`, which lazily initializes global settings and reuses throttled clients; you rarely need to micromanage sessions yourself.
78+
79+
## Indicator Configuration Cheatsheet
80+
81+
- Accepts either JSON strings (most MCP clients) or native dicts. Double-encoded JSON like `"\"{\\\"ema\\\": ...}\""` is auto-decoded.
82+
- Mix-and-match multiple parameters per indicator:
83+
`{"ema": [{"timeperiod": 12}, {"timeperiod": 26}], "macd": [{"fastperiod": 12, "slowperiod": 26, "signalperiod": 9}]}`
84+
- Enhanced registry supports 150+ TA-Lib functions plus custom composites (VWAP, BB width/position, Aroon oscillators, etc.).
85+
- Validation errors bubble back as descriptive `ToolResult.error` messages so you can surface them directly to users.
86+
87+
## Usage Patterns
88+
89+
### CEX candles + indicators
90+
91+
```python
92+
from spoon_toolkits.crypto.crypto_powerdata import CryptoPowerDataCEXTool
93+
94+
tool = CryptoPowerDataCEXTool()
95+
result = await tool.execute(
96+
exchange="binance",
97+
symbol="BTC/USDT",
98+
timeframe="1h",
99+
limit=200,
100+
indicators_config='{"ema": [{"timeperiod": 12}, {"timeperiod": 26}], "rsi": [{"timeperiod": 14}]}',
101+
)
102+
103+
ohlcv_rows = result.output["data"]
104+
metadata = result.output.get("metadata")
105+
```
106+
107+
### DEX analytics on OKX Web3
108+
109+
```python
110+
from spoon_toolkits.crypto.crypto_powerdata import CryptoPowerDataDEXTool
111+
112+
dex = CryptoPowerDataDEXTool()
113+
result = await dex.execute(
114+
chain_index="1", # Ethereum
115+
token_address="0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2", # WETH
116+
timeframe="1H",
117+
limit=150,
118+
indicators_config='{"macd": [{"fastperiod": 12, "slowperiod": 26, "signalperiod": 9}], "bb": [{"period": 20, "std": 2}]}',
119+
)
120+
121+
candles = result.output["data"]
122+
```
123+
124+
### Real-time price snapshot
125+
126+
```python
127+
from spoon_toolkits.crypto.crypto_powerdata import CryptoPowerDataPriceTool
128+
129+
price_tool = CryptoPowerDataPriceTool()
130+
btc = await price_tool.execute(source="cex", exchange="okx", symbol="BTC/USDT")
131+
dex_price = await price_tool.execute(source="dex", chain_index="42161", token_address="0xFF970A61A04b1cA14834A43f5de4533eBDDB5CC8")
132+
```
133+
134+
### Discover supported indicators
135+
136+
```python
137+
from spoon_toolkits.crypto.crypto_powerdata import CryptoPowerDataIndicatorsTool
138+
139+
catalog = await CryptoPowerDataIndicatorsTool().execute()
140+
print(catalog.output["indicators"]) # list of indicator metadata with defaults
141+
```
142+
143+
## MCP Server & Streaming
144+
145+
- `start_crypto_powerdata_mcp_stdio(env_vars=...)` spins up a FastMCP stdio server; pass `background=True` if you need it alongside other async work.
146+
- `start_crypto_powerdata_mcp_sse(host, port, env_vars)` exposes identical tools over HTTP + Server-Sent Events (see `dual_transport_server.py` for endpoints `/mcp` and `/health`).
147+
- `start_crypto_powerdata_mcp_auto` picks stdio vs SSE automatically based on environment; handy for container images.
148+
- `CryptoPowerDataMCPServer` keeps track of running threads so you can query `status()` or stop everything on shutdown.
149+
- `mcp_bridge.py` wires FastMCP methods into the dual transport so CLI agents and browser extensions consume the same tool definitions.
150+
151+
When you need streaming OHLCV updates (vs. periodic `execute` calls), run the SSE server and subscribe to `/mcp` with a persistent session ID. The same OKX rate limiting (`rate_limit_requests_per_second`) and retry envelopes apply regardless of transport.

0 commit comments

Comments
 (0)