Skip to content

Commit 2598a80

Browse files
committed
add s3 documentation
1 parent b3a4e69 commit 2598a80

File tree

2 files changed

+35
-11
lines changed

2 files changed

+35
-11
lines changed

FAQ.md

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -50,22 +50,41 @@ If you would like to retrieve these posts, then you should use full mode: `--ful
5050

5151
This behavior may change in the future.
5252

53-
## How do I use the `--upload` flag and depot?
53+
## How do I use the `--bucket` flag and S3?
54+
1. Create an S3 bucket. Find help [here](https://docs.aws.amazon.com/AmazonS3/latest/gsg/CreatingABucket.html).
55+
2. Configure your AWS credentials. Find help [here](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-shared.html).
56+
1. Ensure you can write to S3 with the credentials you're using.
57+
3. Use instamancer like so:
58+
59+
```
60+
instamancer ... -d --bucket=BUCKET_NAME
61+
```
62+
63+
Where `BUCKET_NAME` is the name of the bucket.
64+
65+
Example:
66+
67+
```
68+
instamancer hashtag puppies -c10 -d --bucket=instagram-puppies
69+
```
70+
71+
72+
## How do I use the `--depot` flag and depot?
5473
1. Set up [depot](https://github.com/ScriptSmith/depot)
5574
1. Set up basic access authentication if you're using a public server
5675
2. Generate a UUIDv4
5776
3. Use instamancer like so:
5877

5978
```
60-
instamancer ... -d --upload=http://127.0.0.1:8080/jobs/UUID/
79+
instamancer ... -d --depot=http://127.0.0.1:8080/jobs/UUID/
6180
```
6281

6382
Where `UUID` is the UUID you generated.
6483

6584
Example:
6685

6786
```
68-
instamancer hashtag puppies -c10 -d --upload=https://depot:[email protected]/jobs/4cdc21fe-6b35-473a-b26e-66f62ad66c4c/
87+
instamancer hashtag puppies -c10 -d --depot=https://depot:[email protected]/jobs/4cdc21fe-6b35-473a-b26e-66f62ad66c4c/
6988
```
7089

7190
You can use any server that accepts `PUT` requests.

README.md

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Read more about how Instamancer works [here](https://scriptsmith.github.io/insta
2828
- Batch scraping
2929
- Search hashtags, users, and locations
3030
- API response validation
31-
- Stream files to [depot](https://github.com/ScriptSmith/depot)
31+
- Upload files to [S3](https://github.com/ScriptSmith/instamancer/blob/master/FAQ.md#how-do-i-use-the---bucket-flag-and-s3) and [depot](https://github.com/ScriptSmith/instamancer/blob/master/FAQ.md#how-do-i-use-the---depot-flag-and-depot)
3232
- [Plugins](plugins)
3333

3434
### Data
@@ -120,20 +120,24 @@ Configuration
120120
--sleep, -s Seconds to sleep between interactions [number] [default: 2]
121121
--graft, -g Enable grafting [boolean] [default: true]
122122
--browser, -b Browser path. Defaults to the puppeteer version [string]
123+
--sameBrowser Use a single browser when grafting [boolean] [default: false]
123124
124125
Download
125126
--download, -d Save images from posts [boolean] [default: false]
126127
--downdir Download path [default: "downloads/[endpoint]/[id]"]
127128
--video, -v Download videos (requires full) [boolean] [default: false]
128-
--upload, -u Upload files to a URL with a PUT request [string]
129129
--sync Force download between requests [boolean] [default: false]
130-
--threads, -k Parallel download / upload threads [number] [default: 4]
130+
--threads, -k Parallel download / depot threads [number] [default: 4]
131131
--waitDownload, -w Download media after scraping [boolean] [default: false]
132132
133+
Upload
134+
--bucket Upload files to an AWS S3 bucket [string]
135+
--depot Upload files to a URL with a PUT request (depot) [string]
136+
133137
Output
134-
--filename, --file, -o Name of the output file [string] [default: "[id]"]
135-
--filetype, --type, -t [choices: "csv", "json", "both"] [default: "json"]
136-
--mediaPath, -m Add filepaths to _mediaPath [boolean] [default: false]
138+
--file, -o Output filename. '-' for stdout [string] [default: "[id]"]
139+
--type, -t Filetype [choices: "csv", "json", "both"] [default: "json"]
140+
--mediaPath, -m Add filepaths to _mediaPath [boolean] [default: false]
137141
138142
Display
139143
--visible Show browser on the screen [boolean] [default: false]
@@ -156,11 +160,12 @@ Options:
156160
Examples:
157161
instamancer hashtag instagood -fvd Download all the available posts,
158162
and their media from #instagood
159-
instamancer user arianagrande Download Ariana Grande's posts to a
160-
--filetype=csv --logging=info --visible CSV file with a non-headless
163+
instamancer user arianagrande --type=csv Download Ariana Grande's posts to a
164+
--logging=info --visible CSV file with a non-headless
161165
browser, and log all events
162166
163167
Source code available at https://github.com/ScriptSmith/instamancer
168+
164169
```
165170

166171
### Module

0 commit comments

Comments
 (0)