Commit 608decb
authored
[FEATURE] Enhance TLS/HTTP fingerprinting for Chrome 127-143 (#36)
* feat: Enhance TLS/HTTP fingerprinting for Chrome 127-143
- Added support for multiple Chrome profiles (127-143) with exact JA4 fingerprint matches.
- Implemented Brotli and zlib decompression for the compress_certificate extension.
- Updated SSL_CTX configuration to prevent redundant setups for browser profiles.
- Improved session creation logic to initialize cookie jars and connection pools.
- Enhanced tests to validate Chrome 143 fingerprint accuracy, including user-agent checks and TLS version support.
- Updated browser profile aliases to default to Chrome 143.
* updated pyproject.toml
* fix: Add brotli CI dependencies and bump version to 0.2.8
- Add brotli dependencies to Linux CI (brotli-devel/libbrotli-dev)
- Add brotli to macOS CI (brew install brotli)
- Add brotli:x64-windows to Windows CI (vcpkg)
- Update setup.py with Windows brotli path detection and linking
- Fix flaky test_simple_get to use local httpbin_server fixture
* fix: Build brotli from source on macOS for correct deployment target
Homebrew brotli has MACOSX_DEPLOYMENT_TARGET=15.0 which causes
delocate-wheel to fail when building wheels targeting macOS 11.0.
- Add brotli build to scripts/darwin/setup_vendors.sh
- Build with CMAKE_OSX_DEPLOYMENT_TARGET=11.0 for wheel compatibility
- Update setup.py to prefer vendor brotli over Homebrew on macOS
- Remove brotli from brew install in CI (built from source instead)
* fix: Use correct brotli library names (without -static suffix)
CMake builds brotli as libbrotlidec.a and libbrotlicommon.a,
not libbrotlidec-static.a and libbrotlicommon-static.a.
* feat: Enable HTTP/2 by default for Session class
Chrome browsers use HTTP/2 by default, so httpmorph should too.
This ensures correct JA4 fingerprint (h2 instead of h1) and
Akamai HTTP/2 fingerprint matching out of the box.
* feat: Perfect Chrome 143 fingerprint matching with documentation sync
- Add Chrome-like HTTP/2 priority (weight=256, exclusive=1) in http2_logic.c
- Remove zlib from certificate compression, use Brotli only (per Chrome behavior)
- Add Chrome default headers to Session (sec-ch-ua, sec-fetch-*, priority)
- Update all documentation to reflect Chrome 143 as default profile
- Sync README and ReadTheDocs with actual code behavior:
- Client and Session both default to http2=True
- Certificate compression is Brotli only
- Updated fingerprint characteristics (JA4, JA3N, Peetprint, Akamai)
- Fix test names to match http2=True default behavior
* fix: Restore zlib certificate decompression for server compatibility
Chrome 143 advertises brotli in the compress_certificate extension,
but some servers still send zlib-compressed certificates. Keep both
decompression handlers registered for compatibility.
Fixes TLS handshake failures with CERT_DECOMPRESSION_FAILED error
on servers using zlib certificate compression.
* fix: Match Chrome 143 TLS fingerprint exactly
Changes to match Chrome 143's actual TLS fingerprint:
1. Certificate compression (compress_certificate extension 27):
- Only advertise brotli (2), not zlib (1)
- Chrome only supports brotli certificate compression
- Servers compliant with RFC 8879 will only send brotli
2. Application settings (ALPS extension 17613):
- Only advertise "h2" protocol, not "http/1.1"
- Chrome only includes h2 in application_settings
3. Extension permutation:
- Enable SSL_CTX_set_permute_extensions() to randomize extension order
- Chrome randomizes extension order in each ClientHello
- JA4 sorts extensions alphabetically so order doesn't affect JA4
Matching fingerprints:
- Cipher hash (JA4 part 2): 8daaf6152771 ✓
- Akamai HTTP/2: 1:65536;2:0;4:6291456;6:262144|15663105|0|m,a,s,p ✓
- compress_certificate: brotli only ✓
- application_settings: h2 only ✓
Note: Chrome fingerprint shows 17 extensions vs httpmorph's 16 because
Chrome capture was from a resumed session with pre_shared_key (41).
Fresh connections don't include pre_shared_key, which is expected behavior.
* fix: Prevent heap corruption in HTTP/2 response body buffer cleanup on Windows
The HTTP/2 code was replacing the response body buffer (originally from
the buffer pool) with a newly malloc'd buffer, but not properly handling
the cleanup:
1. The original buffer from the pool was never returned to the pool
2. _buffer_pool was still pointing to the pool
3. _body_actual_size wasn't updated for the new buffer
When httpmorph_response_destroy() was later called, it tried to return
the malloc'd buffer to the buffer pool using the stale _body_actual_size,
causing heap corruption (especially visible on Windows with error code
0xc0000374).
Fix: Before assigning the new malloc'd buffer to response->body:
- Return the original buffer to the pool (if from pool) or free it
- Clear _buffer_pool since the new buffer is not from the pool
- Set _body_actual_size to track the actual allocation size
Fixes Windows CI heap corruption crash in Session.__del__1 parent 4bf6a49 commit 608decb
File tree
22 files changed
+825
-340
lines changed- .github/workflows
- docs/source
- include
- scripts/darwin
- src
- bindings
- core
- internal
- httpmorph
- tls
- tests
22 files changed
+825
-340
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | | - | |
| 60 | + | |
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
| |||
131 | 131 | | |
132 | 132 | | |
133 | 133 | | |
134 | | - | |
| 134 | + | |
135 | 135 | | |
136 | | - | |
| 136 | + | |
137 | 137 | | |
138 | 138 | | |
139 | 139 | | |
| |||
158 | 158 | | |
159 | 159 | | |
160 | 160 | | |
161 | | - | |
| 161 | + | |
162 | 162 | | |
163 | 163 | | |
164 | 164 | | |
| |||
254 | 254 | | |
255 | 255 | | |
256 | 256 | | |
257 | | - | |
| 257 | + | |
258 | 258 | | |
259 | 259 | | |
260 | 260 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
17 | | - | |
18 | | - | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
| |||
74 | 75 | | |
75 | 76 | | |
76 | 77 | | |
77 | | - | |
| 78 | + | |
78 | 79 | | |
79 | 80 | | |
80 | | - | |
81 | | - | |
| 81 | + | |
| 82 | + | |
82 | 83 | | |
83 | 84 | | |
84 | | - | |
| 85 | + | |
85 | 86 | | |
86 | 87 | | |
87 | 88 | | |
| |||
111 | 112 | | |
112 | 113 | | |
113 | 114 | | |
114 | | - | |
| 115 | + | |
115 | 116 | | |
116 | | - | |
| 117 | + | |
117 | 118 | | |
118 | | - | |
119 | | - | |
| 119 | + | |
120 | 120 | | |
121 | | - | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
122 | 124 | | |
123 | | - | |
| 125 | + | |
124 | 126 | | |
125 | | - | |
| 127 | + | |
126 | 128 | | |
127 | 129 | | |
128 | 130 | | |
129 | 131 | | |
130 | 132 | | |
131 | 133 | | |
132 | 134 | | |
133 | | - | |
134 | | - | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
135 | 138 | | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
| 139 | + | |
140 | 140 | | |
141 | 141 | | |
142 | | - | |
| 142 | + | |
143 | 143 | | |
144 | 144 | | |
145 | 145 | | |
| |||
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
151 | | - | |
152 | | - | |
| 151 | + | |
| 152 | + | |
153 | 153 | | |
154 | 154 | | |
155 | 155 | | |
156 | | - | |
157 | | - | |
| 156 | + | |
158 | 157 | | |
159 | 158 | | |
160 | 159 | | |
161 | | - | |
162 | | - | |
163 | | - | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
164 | 163 | | |
165 | 164 | | |
166 | 165 | | |
| |||
417 | 416 | | |
418 | 417 | | |
419 | 418 | | |
420 | | - | |
| 419 | + | |
421 | 420 | | |
422 | 421 | | |
423 | 422 | | |
| |||
439 | 438 | | |
440 | 439 | | |
441 | 440 | | |
442 | | - | |
443 | | - | |
| 441 | + | |
| 442 | + | |
444 | 443 | | |
445 | 444 | | |
446 | 445 | | |
447 | 446 | | |
448 | 447 | | |
449 | 448 | | |
450 | | - | |
451 | | - | |
| 449 | + | |
| 450 | + | |
452 | 451 | | |
453 | 452 | | |
454 | 453 | | |
| |||
457 | 456 | | |
458 | 457 | | |
459 | 458 | | |
460 | | - | |
| 459 | + | |
461 | 460 | | |
462 | 461 | | |
463 | 462 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
56 | | - | |
| 56 | + | |
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
93 | | - | |
| 93 | + | |
94 | 94 | | |
95 | | - | |
| 95 | + | |
96 | 96 | | |
97 | 97 | | |
98 | 98 | | |
99 | | - | |
| 99 | + | |
100 | 100 | | |
101 | 101 | | |
102 | 102 | | |
| |||
127 | 127 | | |
128 | 128 | | |
129 | 129 | | |
130 | | - | |
| 130 | + | |
131 | 131 | | |
132 | | - | |
| 132 | + | |
133 | 133 | | |
134 | 134 | | |
135 | 135 | | |
136 | | - | |
| 136 | + | |
137 | 137 | | |
138 | | - | |
| 138 | + | |
139 | 139 | | |
140 | 140 | | |
141 | 141 | | |
| |||
423 | 423 | | |
424 | 424 | | |
425 | 425 | | |
426 | | - | |
427 | | - | |
| 426 | + | |
| 427 | + | |
428 | 428 | | |
429 | | - | |
| 429 | + | |
430 | 430 | | |
431 | 431 | | |
432 | 432 | | |
433 | | - | |
434 | | - | |
435 | | - | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
436 | 437 | | |
437 | 438 | | |
438 | | - | |
| 439 | + | |
439 | 440 | | |
440 | | - | |
| 441 | + | |
| 442 | + | |
441 | 443 | | |
442 | 444 | | |
443 | 445 | | |
444 | | - | |
445 | | - | |
446 | | - | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
447 | 449 | | |
448 | 450 | | |
449 | 451 | | |
450 | 452 | | |
451 | 453 | | |
452 | | - | |
| 454 | + | |
453 | 455 | | |
454 | 456 | | |
455 | | - | |
456 | | - | |
| 457 | + | |
| 458 | + | |
457 | 459 | | |
458 | 460 | | |
459 | 461 | | |
460 | 462 | | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
461 | 474 | | |
462 | 475 | | |
463 | 476 | | |
464 | | - | |
| 477 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
102 | 102 | | |
103 | 103 | | |
104 | 104 | | |
105 | | - | |
| 105 | + | |
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
109 | | - | |
110 | | - | |
| 109 | + | |
| 110 | + | |
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
121 | | - | |
122 | | - | |
123 | | - | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
124 | 125 | | |
125 | 126 | | |
126 | 127 | | |
| |||
344 | 345 | | |
345 | 346 | | |
346 | 347 | | |
347 | | - | |
| 348 | + | |
348 | 349 | | |
349 | 350 | | |
350 | 351 | | |
351 | | - | |
352 | | - | |
| 352 | + | |
| 353 | + | |
353 | 354 | | |
| 355 | + | |
354 | 356 | | |
355 | | - | |
356 | | - | |
| 357 | + | |
357 | 358 | | |
| 359 | + | |
358 | 360 | | |
359 | | - | |
360 | | - | |
361 | | - | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
362 | 364 | | |
363 | 365 | | |
364 | 366 | | |
| |||
0 commit comments