Commit 3d018b2
authored
fix: Fix
### Description
- Use `top_domain_under_public_suffix` instead of just `domain` when
comparing domains in `same-domain` strategy.
### Issues
- Closes: #1571
### Testing
- Added unit test
### Checklist
- [x] CI passedsame-domain strategy ignoring public suffix (#1572)1 parent 50163c3 commit 3d018b2
File tree
2 files changed
+6
-5
lines changed- src/crawlee/crawlers/_basic
- tests/unit/crawlers/_basic
2 files changed
+6
-5
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1043 | 1043 | | |
1044 | 1044 | | |
1045 | 1045 | | |
1046 | | - | |
1047 | | - | |
| 1046 | + | |
| 1047 | + | |
1048 | 1048 | | |
1049 | 1049 | | |
1050 | 1050 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
347 | 347 | | |
348 | 348 | | |
349 | 349 | | |
| 350 | + | |
350 | 351 | | |
351 | 352 | | |
352 | 353 | | |
| |||
401 | 402 | | |
402 | 403 | | |
403 | 404 | | |
404 | | - | |
| 405 | + | |
405 | 406 | | |
406 | 407 | | |
407 | 408 | | |
| |||
411 | 412 | | |
412 | 413 | | |
413 | 414 | | |
414 | | - | |
| 415 | + | |
415 | 416 | | |
416 | 417 | | |
417 | 418 | | |
| |||
421 | 422 | | |
422 | 423 | | |
423 | 424 | | |
424 | | - | |
| 425 | + | |
425 | 426 | | |
426 | 427 | | |
427 | 428 | | |
| |||
0 commit comments