Skip to content

Commit 4bcf892

Browse files
author
Matthias Zimmermann
committed
feat add async providers to ProviderBuilder
1 parent 2cff34b commit 4bcf892

File tree

3 files changed

+425
-9
lines changed

3 files changed

+425
-9
lines changed

‎STEP1_ASYNC_PROVIDER_SUMMARY.md‎

Lines changed: 237 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,237 @@
1+
# Step 1: Async Provider Support - Implementation Summary
2+
3+
## Overview
4+
5+
Extended `ProviderBuilder` with an `async_mode()` modifier that controls provider creation at the `build()` point, maintaining backward compatibility while enabling async provider support.
6+
7+
## Changes Made
8+
9+
### 1. Updated Imports (`provider.py`)
10+
11+
Added async provider imports:
12+
```python
13+
from web3.providers import AsyncHTTPProvider, HTTPProvider, WebSocketProvider
14+
from web3.providers.async_base import AsyncBaseProvider
15+
from web3.providers.base import BaseProvider
16+
```
17+
18+
### 2. Added `_is_async` State Flag
19+
20+
Added to `ProviderBuilder.__init__()`:
21+
```python
22+
self._is_async: bool = False # Default to sync providers
23+
```
24+
25+
**Default behavior preserved**: Sync mode by default (backward compatible).
26+
27+
### 3. New `async_mode()` Method
28+
29+
```python
30+
def async_mode(self) -> ProviderBuilder:
31+
"""
32+
Enable async provider mode.
33+
34+
When enabled, build() will return async-compatible providers:
35+
- HTTP transport → AsyncHTTPProvider
36+
- WebSocket transport → WebSocketProvider (inherently async)
37+
38+
By default (async mode disabled), build() returns sync providers:
39+
- HTTP transport → HTTPProvider
40+
- WebSocket transport → WebSocketProvider (inherently async)
41+
"""
42+
self._is_async = True
43+
return self
44+
```
45+
46+
**Key characteristics**:
47+
- Chainable (returns `self`)
48+
- Can be called at any point in the builder chain
49+
- Toggles async mode flag
50+
51+
### 4. Updated `build()` Method
52+
53+
Modified return type and logic:
54+
55+
```python
56+
def build(self) -> BaseProvider | AsyncBaseProvider:
57+
# ... URL resolution logic (unchanged) ...
58+
59+
# Build provider based on transport and async mode
60+
if self._transport == HTTP:
61+
if self._is_async:
62+
return AsyncHTTPProvider(url)
63+
else:
64+
return HTTPProvider(url)
65+
else: # WebSocket
66+
# WebSocketProvider is always async
67+
return cast(AsyncBaseProvider, WebSocketProvider(url))
68+
```
69+
70+
**Behavior matrix**:
71+
72+
| Transport | async_mode() | Provider Created | Base Class |
73+
|-----------|--------------|------------------|------------|
74+
| HTTP | ❌ No (default) | `HTTPProvider` | `BaseProvider` |
75+
| HTTP | ✅ Yes | `AsyncHTTPProvider` | `AsyncBaseProvider` |
76+
| WebSocket | ❌ No | `WebSocketProvider` | `AsyncBaseProvider` |
77+
| WebSocket | ✅ Yes | `WebSocketProvider` | `AsyncBaseProvider` |
78+
79+
**Important**: WebSocket providers are **always async** (inherit from `AsyncBaseProvider`), regardless of `async_mode()` flag.
80+
81+
## Test Coverage
82+
83+
Added comprehensive test class `TestProviderBuilderAsyncMode` with 12 tests:
84+
85+
1. ✅ `test_async_mode_sets_correct_state` - Verifies `_is_async` flag
86+
2. ✅ `test_default_is_sync_mode` - Confirms sync default
87+
3. ✅ `test_async_mode_with_http_creates_async_http_provider` - AsyncHTTPProvider creation
88+
4. ✅ `test_async_mode_with_ws_creates_websocket_provider` - WebSocket with async_mode()
89+
5. ✅ `test_sync_mode_with_http_creates_http_provider` - Default sync behavior
90+
6. ✅ `test_sync_mode_with_ws_creates_websocket_provider` - WebSocket always async
91+
7. ✅ `test_async_mode_with_kaolin_http` - Async with Kaolin network
92+
8. ✅ `test_async_mode_with_kaolin_ws` - WebSocket with Kaolin
93+
9. ✅ `test_async_mode_with_custom_url` - Async with custom URLs
94+
10. ✅ `test_async_mode_chaining` - Chainable at different positions
95+
11. ✅ `test_async_mode_with_node` - Async with ArkivNode
96+
12. ✅ `test_async_mode_return_type_annotation` - Type annotation validation
97+
98+
**All 44 provider tests passing** (32 existing + 12 new).
99+
100+
## Usage Examples
101+
102+
### Sync HTTP Provider (Default - Unchanged)
103+
```python
104+
provider = ProviderBuilder().localhost().build()
105+
# Returns: HTTPProvider (BaseProvider)
106+
```
107+
108+
### Async HTTP Provider (NEW)
109+
```python
110+
provider = ProviderBuilder().localhost().async_mode().build()
111+
# Returns: AsyncHTTPProvider (AsyncBaseProvider)
112+
```
113+
114+
### WebSocket Provider (Always Async)
115+
```python
116+
# Without async_mode()
117+
provider = ProviderBuilder().localhost().ws().build()
118+
# Returns: WebSocketProvider (AsyncBaseProvider)
119+
120+
# With async_mode() - same result
121+
provider = ProviderBuilder().localhost().ws().async_mode().build()
122+
# Returns: WebSocketProvider (AsyncBaseProvider)
123+
```
124+
125+
### Chaining Examples
126+
```python
127+
# async_mode() can be anywhere in chain
128+
ProviderBuilder().async_mode().localhost().http().build()
129+
ProviderBuilder().localhost().async_mode().http().build()
130+
ProviderBuilder().localhost().http().async_mode().build()
131+
# All return: AsyncHTTPProvider
132+
```
133+
134+
### With Different Networks
135+
```python
136+
# Kaolin async HTTP
137+
ProviderBuilder().kaolin().async_mode().build()
138+
# Returns: AsyncHTTPProvider("https://kaolin.hoodi.arkiv.network/rpc")
139+
140+
# Custom URL async HTTP
141+
ProviderBuilder().custom("https://my-rpc.io").async_mode().build()
142+
# Returns: AsyncHTTPProvider("https://my-rpc.io")
143+
```
144+
145+
### With ArkivNode
146+
```python
147+
with ArkivNode() as node:
148+
# Async HTTP
149+
provider = ProviderBuilder().node(node).async_mode().build()
150+
# Returns: AsyncHTTPProvider(node.http_url)
151+
152+
# Async WebSocket
153+
provider = ProviderBuilder().node(node).ws().async_mode().build()
154+
# Returns: WebSocketProvider(node.ws_url)
155+
```
156+
157+
## Backward Compatibility
158+
159+
✅ **100% Backward Compatible**
160+
161+
- Default behavior unchanged (sync providers)
162+
- All existing code continues to work
163+
- No breaking changes to API
164+
- Existing tests all pass
165+
166+
## Design Decisions
167+
168+
### 1. Why `async_mode()` instead of `async()`?
169+
170+
- `async` is a Python keyword and cannot be used as method name
171+
- `async_mode()` is explicit and self-documenting
172+
- Alternatives considered: `enable_async()`, `use_async()`, `asynchronous()`
173+
174+
### 2. Why is `_is_async` a separate flag?
175+
176+
- Clear separation of concerns: transport vs. execution mode
177+
- Allows future extensions (e.g., different async strategies)
178+
- Makes state management explicit and testable
179+
180+
### 3. Why does WebSocket ignore `async_mode()`?
181+
182+
- Web3.py's `WebSocketProvider` inherits from `AsyncBaseProvider`
183+
- WebSocket is inherently async in the library
184+
- No sync WebSocket provider exists
185+
- Being explicit would be misleading
186+
187+
## Type Safety
188+
189+
Updated return type annotation:
190+
```python
191+
def build(self) -> BaseProvider | AsyncBaseProvider:
192+
```
193+
194+
Type checkers (mypy/pyright) can now distinguish:
195+
```python
196+
sync_provider: BaseProvider = ProviderBuilder().localhost().build()
197+
async_provider: AsyncBaseProvider = ProviderBuilder().localhost().async_mode().build()
198+
```
199+
200+
## Next Steps (Step 2)
201+
202+
With `async_mode()` in place, we can now:
203+
204+
1. Create `AsyncArkiv` client class
205+
2. Add provider validation for async providers
206+
3. Ensure sync `Arkiv` rejects async providers
207+
4. Ensure async `AsyncArkiv` rejects sync providers
208+
5. Update tests accordingly
209+
210+
## Files Modified
211+
212+
1. **`src/arkiv/provider.py`**:
213+
- Added imports: `AsyncHTTPProvider`, `AsyncBaseProvider`
214+
- Added `_is_async` flag to `__init__()`
215+
- Added `async_mode()` method
216+
- Updated `build()` logic and return type
217+
218+
2. **`tests/test_provider.py`**:
219+
- Added imports: `AsyncHTTPProvider`, `AsyncBaseProvider`
220+
- Added `TestProviderBuilderAsyncMode` test class (12 tests)
221+
222+
## Performance Considerations
223+
224+
- No performance impact on sync path (default)
225+
- Async providers only created when explicitly requested
226+
- No runtime overhead for flag check (single boolean comparison)
227+
228+
## Documentation Updates Needed
229+
230+
- Update `ProviderBuilder` docstring examples ✅ (done)
231+
- Update README.md with async examples (pending Step 2)
232+
- Create migration guide for async adoption (pending)
233+
234+
---
235+
236+
**Status**: ✅ Step 1 Complete - All tests passing (44/44)
237+
**Ready for**: Step 2 - AsyncArkiv client implementation

‎src/arkiv/provider.py‎

Lines changed: 69 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@
55
import logging
66
from typing import TYPE_CHECKING, Literal, cast
77

8-
from web3.providers import HTTPProvider, WebSocketProvider
8+
from web3.providers import AsyncHTTPProvider, HTTPProvider, WebSocketProvider
9+
from web3.providers.async_base import AsyncBaseProvider
910
from web3.providers.base import BaseProvider
1011

1112
if TYPE_CHECKING:
@@ -45,12 +46,25 @@ class ProviderBuilder:
4546
Fluent builder for Web3 providers with Arkiv presets.
4647
4748
Examples:
48-
- ProviderBuilder().localhost().build() # http://127.0.0.1:8545
49-
- ProviderBuilder().localhost(9000).ws().build() # ws://127.0.0.1:9000
49+
- ProviderBuilder().localhost().build() # http://127.0.0.1:8545 (HTTPProvider)
50+
- ProviderBuilder().localhost(9000).ws().build() # ws://127.0.0.1:9000 (WebSocketProvider, async)
51+
- ProviderBuilder().localhost().async_mode().build() # http://127.0.0.1:8545 (AsyncHTTPProvider)
5052
- ProviderBuilder().kaolin().build() # https://kaolin.hoodi.arkiv.network/rpc
5153
- ProviderBuilder().custom("https://my-rpc.io").build() # https://my-rpc.io
5254
- ProviderBuilder().node().build() # Auto-creates and starts ArkivNode
53-
- ProviderBuilder().node(my_node).ws().build() # Use existing node with WebSocket
55+
- ProviderBuilder().node(my_node).ws().build() # Use existing node with WebSocket (async)
56+
57+
Note:
58+
For best practice, call async_mode() at the end of your builder chain, just before build():
59+
>>> ProviderBuilder().localhost().http().async_mode().build()
60+
61+
Defaults:
62+
- Transport: HTTP (sync HTTPProvider)
63+
- Network: localhost:8545
64+
- Async mode: False (sync providers)
65+
66+
WebSocket transport (ws()) always returns async providers (AsyncBaseProvider),
67+
regardless of async_mode() setting, as WebSocketProvider is inherently async.
5468
"""
5569

5670
def __init__(self) -> None:
@@ -60,6 +74,7 @@ def __init__(self) -> None:
6074
self._port: int | None = DEFAULT_PORT # Set default port for localhost
6175
self._url: str | None = None
6276
self._node: ArkivNode | None = None
77+
self._is_async: bool = False # Default to sync providers
6378

6479
def localhost(self, port: int | None = None) -> ProviderBuilder:
6580
"""
@@ -163,19 +178,59 @@ def ws(self) -> ProviderBuilder:
163178
"""
164179
Use WebSocket transport.
165180
181+
Note: WebSocket providers are always async (AsyncBaseProvider).
182+
166183
Returns:
167184
Self for method chaining
168185
"""
169186
self._transport = cast(TransportType, WS)
170187
return self
171188

172-
def build(self) -> BaseProvider:
189+
def async_mode(self, async_provider=True) -> ProviderBuilder:
190+
"""
191+
Sets the async provider mode.
192+
193+
When enabled, build() will return async-compatible providers:
194+
- HTTP transport → AsyncHTTPProvider
195+
- WebSocket transport → WebSocketProvider (inherently async)
196+
197+
By default (async mode disabled), build() returns sync providers:
198+
- HTTP transport → HTTPProvider
199+
- WebSocket transport → WebSocketProvider (inherently async)
200+
201+
Returns:
202+
Self for method chaining
203+
204+
Examples:
205+
Async HTTP provider:
206+
>>> provider = ProviderBuilder().localhost().async_mode().build()
207+
>>> # Returns AsyncHTTPProvider
208+
209+
Async WebSocket provider:
210+
>>> provider = ProviderBuilder().localhost().ws().async_mode().build()
211+
>>> # Returns WebSocketProvider (always async)
212+
213+
Sync HTTP provider (default):
214+
>>> provider = ProviderBuilder().localhost().build() # or .async_mode(False)
215+
>>> # Returns HTTPProvider
216+
"""
217+
self._is_async = async_provider
218+
return self
219+
220+
def build(self) -> BaseProvider | AsyncBaseProvider:
173221
"""
174222
Build and return the Web3 provider.
175223
176224
Returns:
177225
Configured Web3 provider instance.
178-
Uses HTTPProvider or WebSocketProvider based on specified transport (HTTP being default).
226+
227+
By default (sync mode):
228+
- HTTP transport → HTTPProvider
229+
- WebSocket transport → WebSocketProvider (inherently async)
230+
231+
With async_mode() enabled:
232+
- HTTP transport → AsyncHTTPProvider
233+
- WebSocket transport → WebSocketProvider (inherently async)
179234
180235
Raises:
181236
ValueError: If no URL has been configured or if transport is not available
@@ -217,7 +272,13 @@ def build(self) -> BaseProvider:
217272
"No URL or network configured. Use localhost(), kaolin(), or custom()."
218273
)
219274

275+
# Build provider based on transport
220276
if self._transport == HTTP:
221-
return HTTPProvider(url)
277+
# Consider async mode
278+
if self._is_async:
279+
return AsyncHTTPProvider(url)
280+
else:
281+
return HTTPProvider(url)
282+
# Web socket transport (always async)
222283
else:
223-
return cast(BaseProvider, WebSocketProvider(url))
284+
return cast(AsyncBaseProvider, WebSocketProvider(url))

0 commit comments

Comments
 (0)