v2.2.0

• 52fc13e - add custom httpResponseHandler
• 4b92006 - add onResourceSaved and onResourceError callbacks
• 423792d - update readme structure
• c2fb99d - update author name in license and package.json

v2.1.1

• c28bf7c - Support picture srcset attr, save .webp files to images dir

v2.1.0

• b2a18e3 decode url-based filenames
• caf894e correctly handle errors which occur on downloading main urls
• 1af3274 refactor code duplicates

v2.0.0

Breaking changes

• 3aacd8e - drop nodejs < 4 support
• 23e76c0 - export function instead of object
• c10a4d2 - rework css handling
• 3e698b8 - use mime-types to determine resource's type
• bfb55d7 - rename assets to children in result object
• 40043ba - ignore errors by default

Non-breaking changes

• 7600e64 - correctly handle different URI-schemas (mailto:, skype:, etc.)
• 3bb47ea - handle svg external links by default
• a2b5c56 - send referer in each request
• a2b5c56 - decode html entities in url found in html resource

Migrate from v1.*

• Call exported function instead of scrape method. Example:

// old usage
var scraper = require('website-scraper');
scraper.scrape({/*options*/});

// new usage
var scrape = require('website-scraper');
scrape({/*options*/});

• Css text will be handled by default but if you use source option and want to keep previous behavior need to add next objects to sources.

{ selector: 'style' },
{ selector: '[style]', attr: 'style' }

v1.2.3

• 8b094bc - update debug version (prevent using broken [email protected])

v1.2.2

• 589deb6 Fix error handling (#151)

v1.2.1

• a8546f9 - fix error for urls with emails (#140)

v1.2.0

• 137413a - add ignoreErrors option
• 001c231 - fix too long filename error, shorten filename if needed
• 70d0225 - add logger
• eeb0914 - save woff2 to fonts directory by defaults

v1.1.1

• a74927d - Fix deadlock in css handler

v1.1.0

• 479b8ec - Improve mechanism for re-using already loaded / loading resources