-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
Description
Current script needs the following improvements:
-
Docker/Proxy Integration:
- Fix URL handling for host.docker.internal vs localhost
- Handle proxy authentication if needed
- Better error messages for proxy connection failures
- Add proxy status to telemetry output
-
Streaming Output:
- Ensure proper ndjson handling
- Handle chunked transfer encoding correctly
- Add progress indicators for long responses
-
Telemetry Enhancements:
- Add network latency metrics
- Calculate tokens/second rate
- Track connection retries
- Add timing for proxy vs direct connections
-
Error Handling:
- Better curl error parsing
- Retries with exponential backoff
- Timeout handling
- Signal handling (CTRL+C)
-
Documentation:
- Add detailed usage examples
- Document mitmproxy integration
- Add configuration guide
- Document metrics format
-
Testing:
- Add test cases
- Mock responses
- Proxy failure scenarios
- Various model scenarios
-
CI Integration:
- Add GitHub action
- Automated tests
- Shellcheck integration
- Version tagging
Context:
The script was created as part of the OpenTelemetry integration effort. It currently supports basic streaming and telemetry but needs hardening for production use.