Skip to content

Commit ad7bbbb

Browse files
TexasCodingTexasCoding
committed
update
1 parent b2d0079 commit ad7bbbb

File tree

1 file changed

+234
-0
lines changed

1 file changed

+234
-0
lines changed

‎async_refactoring_issue.md‎

Lines changed: 234 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,234 @@
1+
# Async/Await Refactoring Plan for project-x-py
2+
3+
## Summary
4+
5+
This issue outlines a comprehensive plan to refactor the project-x-py SDK from synchronous to asynchronous operations, enabling better performance, resource utilization, and natural integration with real-time trading workflows.
6+
7+
## Motivation
8+
9+
The current synchronous architecture has several limitations:
10+
11+
1. **Blocking I/O**: HTTP requests block the thread, preventing concurrent operations
12+
2. **Inefficient for Real-time**: SignalR/WebSocket connections naturally fit async patterns
13+
3. **Resource Utilization**: Can't efficiently handle multiple market data streams or order operations concurrently
14+
4. **Modern Python Standards**: Async is the standard for I/O-heavy Python applications, especially in financial/trading contexts
15+
16+
## Benefits of Async Migration
17+
18+
- **Concurrent Operations**: Execute multiple API calls simultaneously (e.g., fetch positions while placing orders)
19+
- **Non-blocking Real-time**: Process WebSocket events without blocking other operations
20+
- **Better Resource Usage**: Single thread can handle many concurrent connections
21+
- **Improved Responsiveness**: UI/strategy code won't freeze during API calls
22+
- **Natural Event Handling**: Async/await patterns match event-driven trading systems
23+
24+
## Technical Analysis
25+
26+
### Current Architecture
27+
28+
1. **HTTP Client**: Uses `requests` library with session pooling
29+
2. **WebSocket**: Uses `signalrcore` for SignalR connections
30+
3. **Blocking Pattern**: All API calls block until completion
31+
4. **Managers**: OrderManager, PositionManager, etc. all use synchronous methods
32+
33+
### Proposed Async Architecture
34+
35+
1. **HTTP Client**: Migrate to `httpx` (supports both sync/async)
36+
2. **WebSocket**: Use `python-signalrcore-async` or create async wrapper for SignalR
37+
3. **Async Pattern**: All public APIs become `async def` methods
38+
4. **Managers**: Convert to async with proper concurrency handling
39+
40+
## Implementation Plan
41+
42+
### Phase 1: Foundation (Week 1-2)
43+
44+
- [ ] Add async dependencies to `pyproject.toml`:
45+
- `httpx[http2]` for async HTTP with HTTP/2 support
46+
- `python-signalrcore-async` or evaluate alternatives
47+
- Update `pytest-asyncio` for testing
48+
- [ ] Create async base client class (`AsyncProjectX`)
49+
- [ ] Implement async session management and connection pooling
50+
- [ ] Design async error handling and retry logic
51+
52+
### Phase 2: Core Client Migration (Week 2-3)
53+
54+
- [ ] Convert authentication methods to async
55+
- [ ] Migrate account management endpoints
56+
- [ ] Convert market data methods (get_bars, get_instrument)
57+
- [ ] Implement async caching mechanisms
58+
- [ ] Add async rate limiting
59+
60+
### Phase 3: Manager Migration (Week 3-4)
61+
62+
- [ ] Convert OrderManager to async
63+
- [ ] Convert PositionManager to async
64+
- [ ] Convert RealtimeDataManager to async
65+
- [ ] Update OrderBook for async operations
66+
- [ ] Ensure managers can share async ProjectXRealtimeClient
67+
68+
### Phase 4: SignalR/WebSocket Integration (Week 4-5)
69+
70+
- [ ] Research SignalR async options:
71+
- Option A: `python-signalrcore-async` (if mature enough)
72+
- Option B: Create async wrapper around current `signalrcore`
73+
- Option C: Use `aiohttp` with custom SignalR protocol implementation
74+
- [ ] Implement async event handling
75+
- [ ] Convert callback system to async-friendly pattern (async generators?)
76+
- [ ] Test reconnection logic with async
77+
78+
### Phase 5: Testing & Documentation (Week 5-6)
79+
80+
- [ ] Convert all tests to async using `pytest-asyncio`
81+
- [ ] Add integration tests for concurrent operations
82+
- [ ] Update all examples to use async/await
83+
- [ ] Document migration guide for users
84+
- [ ] Performance benchmarks (sync vs async)
85+
86+
## API Design Decisions
87+
88+
### Option 1: Pure Async (Recommended per CLAUDE.md)
89+
```python
90+
# All methods become async
91+
client = AsyncProjectX(api_key, username)
92+
await client.authenticate()
93+
positions = await client.get_positions()
94+
```
95+
96+
### Option 2: Dual API (Not recommended due to complexity)
97+
```python
98+
# Both sync and async clients
99+
sync_client = ProjectX(api_key, username)
100+
async_client = AsyncProjectX(api_key, username)
101+
```
102+
103+
Given the CLAUDE.md directive for "No Backward Compatibility" and "Clean Code Priority", **Option 1 (Pure Async) is recommended**.
104+
105+
## Breaking Changes
106+
107+
This refactoring will introduce breaking changes:
108+
109+
1. All public methods become `async`
110+
2. Clients must use `async with` for proper cleanup
111+
3. Event handlers must be async functions
112+
4. Example code and integrations need updates
113+
114+
## Migration Guide (Draft)
115+
116+
```python
117+
# Old (Sync)
118+
client = ProjectX(api_key, username)
119+
client.authenticate()
120+
positions = client.get_positions()
121+
122+
# New (Async)
123+
async with AsyncProjectX(api_key, username) as client:
124+
await client.authenticate()
125+
positions = await client.get_positions()
126+
```
127+
128+
## Technical Considerations
129+
130+
### SignalR Compatibility
131+
132+
ProjectX requires SignalR for real-time connections. Options:
133+
134+
1. **python-signalrcore-async**: Check maturity and compatibility
135+
2. **Async Wrapper**: Create async wrapper around sync signalrcore
136+
3. **Custom Implementation**: Use aiohttp with SignalR protocol (complex but most control)
137+
138+
### Connection Management
139+
140+
- Use async context managers for resource cleanup
141+
- Implement proper connection pooling for HTTP/2
142+
- Handle WebSocket reconnection in async context
143+
144+
### Performance Targets
145+
146+
- Concurrent API calls should show 3-5x throughput improvement
147+
- WebSocket event processing latency < 1ms
148+
- Memory usage should remain comparable to sync version
149+
150+
## Dependencies to Add
151+
152+
```toml
153+
[project.dependencies]
154+
httpx = ">=0.27.0"
155+
# SignalR async solution (TBD based on research)
156+
157+
[project.optional-dependencies.dev]
158+
pytest-asyncio = ">=0.23.0"
159+
aioresponses = ">=0.7.6" # For mocking async HTTP
160+
```
161+
162+
## Example: Async Trading Bot
163+
164+
```python
165+
import asyncio
166+
from project_x_py import AsyncProjectX, create_async_trading_suite
167+
168+
async def trading_bot():
169+
async with AsyncProjectX(api_key, username) as client:
170+
await client.authenticate()
171+
172+
# Create async trading suite
173+
suite = await create_async_trading_suite(
174+
instrument="MGC",
175+
project_x=client,
176+
jwt_token=client.session_token,
177+
account_id=client.account_info.id
178+
)
179+
180+
# Concurrent operations
181+
positions_task = asyncio.create_task(
182+
suite["position_manager"].get_positions()
183+
)
184+
market_data_task = asyncio.create_task(
185+
suite["data_manager"].get_bars("1m", 100)
186+
)
187+
188+
positions, market_data = await asyncio.gather(
189+
positions_task, market_data_task
190+
)
191+
192+
# Real-time event handling
193+
async for tick in suite["data_manager"].stream_ticks():
194+
await process_tick(tick)
195+
196+
async def process_tick(tick):
197+
# Async tick processing
198+
pass
199+
200+
if __name__ == "__main__":
201+
asyncio.run(trading_bot())
202+
```
203+
204+
## Timeline
205+
206+
- **Total Duration**: 5-6 weeks
207+
- **Testing Phase**: Additional 1 week
208+
- **Documentation**: Ongoing throughout
209+
210+
## Success Criteria
211+
212+
1. All tests pass with async implementation
213+
2. Performance benchmarks show improvement
214+
3. Real-time SignalR connections work reliably
215+
4. Clean async API without sync remnants
216+
5. Comprehensive documentation and examples
217+
218+
## Open Questions
219+
220+
1. Which SignalR async library to use?
221+
2. Should we provide any sync compatibility layer?
222+
3. How to handle existing users during transition?
223+
4. Performance benchmarking methodology?
224+
225+
## References
226+
227+
- [ProjectX SignalR Documentation](https://gateway.docs.projectx.com/docs/realtime/)
228+
- [httpx Documentation](https://www.python-httpx.org/)
229+
- [Python Async Best Practices](https://docs.python.org/3/library/asyncio-task.html)
230+
- [SignalR Protocol Specification](https://github.com/dotnet/aspnetcore/tree/main/src/SignalR/docs/specs)
231+
232+
---
233+
234+
**Note**: This refactoring aligns with the CLAUDE.md directive for "No Backward Compatibility" and "Clean Code Priority" during active development.

0 commit comments

Comments
 (0)