wiki/s3.qmd at main · ruralinnovation/wiki · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
---
title: "S3 Simple Storage Service"
execute:
  echo: true
  output: true
  message: false
  warning: false
format:
  html:
    code-fold: true
reference-location: margin
citation-location: margin
---


## S3 storage:

- [ ] csv (R read into memory)

- [ ] pmtiles ???

- [ ] does it need to download to your computer vs accessing directly to the cloud?

- [ ] parquet

- handling permissions / credentials in R
    * [ ] listing
    * [ ] reading
    * [ ] writing

- [ ] targets?

- [ ] duckDB <-> s3

- [ ] back end: arrow vs duckdb vs polars?

### R option to interact with S3

- [Pins](https://pins.rstudio.com/index.html): can write parquet / arrow etc, dev. by the Posit team, I have not a clear view of their dependencies ..

- [arrow's FileSystem classes](https://github.com/apache/arrow/blob/main/r/man/FileSystem.Rd) or better `help("FileSystem", package = "arrow")`

- [paws](https://www.paws-r-sdk.com/) package for the full suite of AWS services, wrapper of SDK. For s3 see [here](https://www.paws-r-sdk.com/examples/s3/).