Skip to content

[BUGFIX] response text returns junk#30

Merged
arman-bd merged 4 commits intodevelopfrom
29-response-text-returns-junk
Nov 10, 2025
Merged

[BUGFIX] response text returns junk#30
arman-bd merged 4 commits intodevelopfrom
29-response-text-returns-junk

Conversation

@arman-bd
Copy link
Owner

No description provided.

This commit addresses multiple security issues discovered during edge case
analysis and improves test infrastructure for better reliability.

Security Fixes:
- Fix HTTP/1.1 body reallocation bug causing data loss (#1)
  * Modified realloc_body_buffer() to use current_data_size parameter
  * Fixes issue where response->body_len was 0 during receive
  * Prevents data loss when buffer needs to grow during receive

- Add integer overflow protection in 8 critical locations (#7, #8)
  * HTTP/2 data callback buffer doubling (http2_logic.c:140)
  * HTTP/1.1 body buffer reallocation (http1.c:417, 549, 606)
  * Gzip decompression buffer expansion (compression.c:55)
  * Response header array growth (response.c:123)
  * Request header array growth (request.c:112)
  * Async request array growth (async_request_manager.c:171)
  * All checks use SIZE_MAX/2 to prevent integer overflow

- Fix memory leak in DNS cache deep copy (#13)
  * Added proper cleanup on allocation failures in addrinfo_deep_copy()
  * Prevents memory leaks when malloc/strdup fails mid-operation

Async HTTP Proxy Improvements:
- Fix async HTTP proxy to use absolute URI for proxy requests
- Add Proxy-Authorization header support for authenticated HTTP proxies
- Properly distinguish between HTTP (uses absolute URI) and HTTPS (uses path)

Test Infrastructure:
- Add comprehensive edge case security tests (25 test cases)
  * Integer overflow protection tests
  * Memory leak prevention tests
  * Thread safety tests
  * Boundary condition tests

- Add buffer reallocation regression tests (11 test cases)
  * Large response handling
  * Gzip decompression
  * Chunked transfer encoding
  * Multiple buffer doubling scenarios

- Update proxy tests to use httpmorph-bin.bytetunnels.com
  * Added fixtures for both HTTP and HTTPS testing
  * HTTPS uses verify=False for self-signed certificates
  * Improved test reliability by using dedicated test server

Results: All 371 tests pass with 14 expected skips
@arman-bd arman-bd self-assigned this Nov 10, 2025
Fix ModuleNotFoundError in CI environments where python-dotenv is not installed.

Changes:
- Wrap dotenv import in try/except block in test_buffer_reallocation.py
- Wrap dotenv import in try/except block in test_edge_cases_security.py
- Follow same pattern as conftest.py for optional dependency handling

Impact:
- Tests now work in CI without requiring python-dotenv installation
- Local development still benefits from .env file loading when dotenv is available
- Environment variables can be set directly in CI/CD pipelines

Fixes CI failures across all workflows with:
  ModuleNotFoundError: No module named 'dotenv'
Add TEST_HTTPBIN_HOST environment variable to CI workflows to fix test failures.

Changes:
- Add TEST_HTTPBIN_HOST to workflow secrets in _test.yml
- Pass TEST_HTTPBIN_HOST to test environment in _test.yml
- Pass TEST_HTTPBIN_HOST from ci.yml to _test.yml workflow

Impact:
- Edge case security tests can now access httpmorph-bin test server in CI
- Buffer reallocation tests can run in CI environment
- Fixes collection errors: "TEST_HTTPBIN_HOST environment variable is not set"

Related:
- Works together with previous commit making dotenv import optional
- TEST_HTTPBIN_HOST must be configured as repository secret in GitHub
@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@arman-bd arman-bd merged commit 9c87891 into develop Nov 10, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants