v3.13.7

3.13.7 (2025-06-06)

Bug Fixes

• use merged cookies correctly in GotScrapingHttpClient (#3000) (a2985f2), closes #2991

v3.13.6

3.13.6 (2025-06-05)

Bug Fixes

• enable full cookie support for ImpitHttpClient (#2991) (120f0a7)
• ensure PlaywrightGotoOptions won't result in unknown when playwright is not installed (#2995) (93eba38), closes #2994
• extract only body from iframe elements (#2986) (c36166e), closes #2979

v3.13.5

3.13.5 (2025-05-20)

Features

• add MinimumSpeedStream and ByteCounterStream helpers (#2970) (921c4ee)
• Allow the AdaptivePlaywrightCrawler result comparator to signal an inconclusive result (#2975) (7ba8906)

v3.13.4

3.13.4 (2025-05-14)

Bug Fixes

• core: respect systemInfoV2 in snapshotter (#2961) (4100eab), closes #2958
• core: use short timeouts for periodic KVS.setRecord calls (#2962) (d31d90e)
• core: optimize request unlocking to get rid of unnecessary unlock calls (#2963) (a433037)
• social: extract emails from each text node separately (#2952) (799afc1)

Features

• impit-client: bump impit version to v0.4 (#2954) (7a1bf10)

v3.13.3

3.13.3 (2025-05-05)

Bug Fixes

• await _createPageForBrowser in browser pool (#2950) (27ba74b), closes #2789
• convert @apilink to @link on build (#2949) (abe1dee), closes #2717
• disable default fingerprints in the Camoufox template (#2928) (6dffa00)
• Fix trailing slash removal in BrowserPool (#2921) (c1fc439), closes #2878
• Fix useState behavior in adaptive crawler (#2941) (5282381)
• Persist SitemapRequestList state periodically (#2923) (e6e7a9f), closes #2897
• respect autoscaledPoolOptions.isTaskReadyFunction option (#2948) (fe2d206), closes #2922
• statistics: track actual request.retryCount in Statistics (#2940) (c9f7f54)

Features

• pass response to FileDownload.streamHandler (#2930) (008c4c7)

v3.13.2

3.13.2 (2025-04-08)

Features

• add onSkippedRequest option (#2916) (764f992), closes #2910

v3.13.1

3.13.1 (2025-04-07)

Bug Fixes

• don't double increment session usage count in BrowserCrawler (#2908) (3107e55), closes #2851
• rename RobotsFile to RobotsTxtFile (#2913) (3160f71), closes #2910
• treat 406 as other 4xx status codes in HttpCrawler (#2907) (b0e6f6d), closes #2892

Features

• add respectRobotsTxtFile crawler option (#2910) (0eabed1)

v3.13.0

3.13.0 (2025-03-04)

Bug Fixes

• cheerio: don't decode HTML entities in context.body (#2838) (32d6d0e), closes #2401
• install browser in the camoufox template correctly (#2864) (a9d008c), closes #2863
• Make log message in RequestQueue.isFinished more accurate (#2848) (3d124ae)
• Simplified RequestQueueV2 implementation (#2775) (d1a094a), closes #2767 #2700

Features

• Camoufox-based crawler template (#2842) (7f08de4), closes #2836
• improved cross platform metric collection (#2834) (e41b2f7), closes #2771
• playwright: add handleCloudflareChallenge helper (#2865) (9a1725f)
• remove old docker CI (#2831) (7f09d56)
• use native impit streaming (#2833) (af2fe23), closes #2756

v3.12.2

3.12.2 (2025-01-27)

Bug Fixes

• core: type definition of Dataset.reduce (#2774) (59bc6d1), closes #2773
• destructure CrawlerRunOptions before passing them to addRequests (#2803) (02a598c), closes #2802
• graceful BasicCrawler tidy-up on CriticalError (#2817) (53331e8), closes #2807

Features

• impit-based HttpClient implementation (#2787) (61d7ffa)
• add support for parsing comma-separated list environment variables (#2765) (4e50c47)
• stopping the crawlers gracefully with BasicCrawler.stop() (#2792) (af2966f), closes #2777

v3.12.1

3.12.1 (2024-12-04)

Bug Fixes

• log status message timeouts to debug level (55ee44a)
• social: support new URL formats for Facebook, YouTube and X (#2758) (4c95847), closes #525

Features

• tieredProxyUrls accept null for switching the proxy off (#2743) (82f4ea9), closes #2740