You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Makes request to `url` and saves all files found with `sources` to `directory`.
39
+
Makes requests to `urls` and saves all files found with `sources` to `directory`.
37
40
38
41
**options** - object containing next options:
39
42
40
43
-`urls:` array of urls to load and filenames for them *(required, see example below)*
41
44
-`directory:` path to save loaded files *(required)*
42
-
-`log:` boolean indicates whether to write the log to console *(optional, default: false)*
43
45
-`defaultFilename:` filename for index page *(optional, default: 'index.html')*
44
-
-`sources:` array of objects to load, specifies selectors and attribute values to select files for loading *(optional, see default value in `lib/defaults.js`)*
46
+
-`sources:` array of objects to load, specifies selectors and attribute values to select files for loading *(optional, see default value in `lib/config/defaults.js`)*
45
47
-`subdirectories:` array of objects, specifies subdirectories for file extensions. If `null` all files will be saved to `directory`*(optional, see example below)*
46
48
-`request`: object, custom options for [request](https://github.com/request/request#requestoptions-callback)*(optional, see example below)*
47
49
48
50
49
51
**callback** - callback function *(optional)*, includes following parameters:
50
52
51
-
-`error:` if error - `Error object`, if success - `null`
53
+
-`error:` if error - `Error` object, if success - `null`
52
54
-`result:` if error - `null`, if success - array if objects containing:
53
55
-`url:` url of loaded page
54
-
-`filename:`absolute filename where page was saved
56
+
-`filename:` filename where page was saved (relative to `directory`)
55
57
56
58
57
59
## Examples
@@ -68,6 +70,7 @@ and separate files into directories:
68
70
-`css` for .css (full path `/path/to/save/css`)
69
71
70
72
```javascript
73
+
var scraper =require('website-scraper');
71
74
scraper.scrape({
72
75
urls: [
73
76
'http://nodejs.org/', // Will be saved with default filename 'index.html'
0 commit comments