Breaking changes
- 3aacd8e - drop nodejs < 4 support
- 23e76c0 - export function instead of object
- c10a4d2 - rework css handling
- 3e698b8 - use mime-types to determine resource's type
- bfb55d7 - rename
assets to children in result object
- 40043ba - ignore errors by default
Non-breaking changes
- 7600e64 - correctly handle different URI-schemas (mailto:, skype:, etc.)
- 3bb47ea - handle svg external links by default
- a2b5c56 - send referer in each request
- a2b5c56 - decode html entities in url found in html resource
Migrate from v1.*
- Call exported function instead of scrape method. Example:
// old usage
var scraper = require('website-scraper');
scraper.scrape({/*options*/});
// new usage
var scrape = require('website-scraper');
scrape({/*options*/});
- Css text will be handled by default but if you use
source option and want to keep previous behavior need to add next objects to sources.
{ selector: 'style' },
{ selector: '[style]', attr: 'style' }