A storr driver using TileDB Embedded storage engine.
What is storr?
A key-value store with content-addressable storage for R objects with optional in-memory caching layer.
The interface is provided by storr
package and defines a set of common operations (set, get, del methods) across a range of different
storage drivers (DBI, LMDB, Redis, RDS local file, R's environment). The storr package is
written by Rich FitzJohn.
What is storr.tiledb?
storr.tiledb is an R package that extends the storr ecosystem by providing
a TileDB-based driver. Additionally, it includes storr-like classes that leverages
the advantages of TileDB backend with new features: ability to add notes
and expiration time-stamps next to key-namespace pairs and data snapshots support
(time-travel).
Warning
The package is in experimental status. Currently, the driver is complete and fully functional in that it passes the auto test specification defined by storr package. More testing is needed and additional features to be implemented before it is moved to mature state. Feedback is welcomed.
-
Key-value operations use driver's methods for speed and efficiency
-
Key-value metadata such as notes and timestamps for data expiration (time-to-live, TTL)
-
Asynchronous key-value operations or in parallel through mirai framework
-
Hash tables (hashtab) for in memory caching layers instead of environments
-
Cloud storage support (AWS S3, Azure Blob, Google Cloud Storage)
-
Data version (time-travelling functionality)
-
Encryption support
Development version from GitHub:
# pak
pak::pkg_install("cgiachalis/storr.tiledb")
# remotes
remotes::install_github("cgiachalis/storr.tiledb")library(storr.tiledb)
# Temp URI path
uri <- tempfile()
# Set up storr
sto <- storr_tiledb(uri, init = TRUE)
# Set
sto$set("mykey1", list(a = 1))
sto$set("mykey2", "abc")
# Get
sto$get("mykey2")
[1] "abc"
# List all keys
sto$list()
[1] "mykey1" "mykey2"
# Del
sto$del("mykey1")
- storr_enviroment() in memory storage using R enviroment
- storr_rds() on disk storage using RDS file format
- storr_dbi() using DBI interface
- storr_redis_api() using Redis through redux
- storr_thor() - using LMDB Lightning Memory-Mapped Database through thor