|
3 | 3 | [](https://juliahub.com/ui/Packages/DataSets/4adr3)
|
4 | 4 | [](https://juliahub.com/docs/DataSets)
|
5 | 5 |
|
6 |
| -DataSets.jl exists to help manage data and reduce the amount of data wrangling |
7 |
| -code you need to write. It's annoying to write |
8 |
| -* Command line wrappers which deal with paths to data storage |
9 |
| -* Code to load and save from various *data storage systems* (eg, local |
10 |
| - filesystem data; local git data, downloaders for remote data over various |
11 |
| - protocols, etc) |
12 |
| -* Code to load the same data model from various serializations (eg, text: |
13 |
| - plain/compressed, property tree: toml/json/msgpack/bson/... |
14 |
| - tabular: csv/csv.gz/parquet/sqlite/...) |
15 |
| -* Code to deal with data lifecycle; versions, provenance, etc |
| 6 | +DataSets helps make data wrangling code more reusable. |
16 | 7 |
|
17 |
| -DataSets provides scaffolding to make this kind of code more reusable. We want |
18 |
| -to make it easy to *relocate* an algorithm between different data environments |
19 |
| -without code changes. For example from your laptop to the cloud, to another |
20 |
| -user's machine, or to an HPC system. |
| 8 | +* We want to make it easy to *relocate* an algorithm between different **data |
| 9 | + storage environments** without code changes. For example from your laptop to |
| 10 | + the cloud, to another user's machine, or to an HPC system. |
| 11 | +* We want to reduce coupling between data and code, by storing **rich type |
| 12 | + information** in metadata. Metadata bridges the gap between the ad hoc |
| 13 | + implicit type system of data outside your program and the Julia data |
| 14 | + structures within your program. |
21 | 15 |
|
22 |
| -Read [**the latest documentation**](https://juliahub.com/docs/DataSets) more information. |
| 16 | +Watch [**DataSets.jl talk**](https://www.youtube.com/watch?v=PJkf0CO5APs) from |
| 17 | +JuliaCon 2021, or read [**the latest documentation**](https://juliahub.com/docs/DataSets) |
| 18 | +more information. |
23 | 19 |
|
24 | 20 | ### Development
|
25 | 21 |
|
|
0 commit comments