Skip to content

Commit e22572f

Browse files
EvizeroEvizero
committed
fix .gitignore
1 parent fb440cf commit e22572f

File tree

5 files changed

+488
-1
lines changed

5 files changed

+488
-1
lines changed

‎.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1 @@
1-
datasets/
21
sandbox.jl

‎docs/src/datasets/CIFAR10.md

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
# [CIFAR-10](@id CIFAR10)
2+
3+
Description from the [original
4+
website](https://www.cs.toronto.edu/~kriz/cifar.html)
5+
6+
> The CIFAR-10 and CIFAR-100 are labeled subsets of the
7+
> [80 million tiny images](http://people.csail.mit.edu/torralba/tinyimages/)
8+
> dataset. They were collected by Alex Krizhevsky, Vinod Nair,
9+
> and Geoffrey Hinton.
10+
>
11+
> The CIFAR-10 dataset consists of 60000 32x32 colour images in
12+
> 10 classes, with 6000 images per class. There are 50000
13+
> training images and 10000 test images.
14+
15+
## Contents
16+
17+
```@contents
18+
Pages = ["CIFAR10.md"]
19+
Depth = 3
20+
```
21+
22+
## Overview
23+
24+
The `MLDatasets.CIFAR10` sub-module provides a programmatic
25+
interface to download, load, and work with the CIFAR-10 dataset.
26+
27+
```julia
28+
using MLDatasets
29+
30+
# load full training set
31+
train_x, train_y = CIFAR10.traindata()
32+
33+
# load full test set
34+
test_x, test_y = CIFAR10.testdata()
35+
```
36+
37+
The provided functions also allow for optional arguments, such as
38+
the directory `dir` where the dataset is located, or the specific
39+
observation `indices` that one wants to work with. For more
40+
information on the interface take a look at the documentation
41+
(e.g. `?CIFAR10.traindata`).
42+
43+
Function | Description
44+
---------|-------------
45+
`download([dir])` | Trigger interactive download of the dataset
46+
[`classnames()`](@ref CIFAR10.classnames) | Return the class names as a vector of strings
47+
[`traintensor([T], [indices]; [dir])`](@ref CIFAR10.traintensor) | Load the training images as an array of eltype `T`
48+
[`trainlabels([indices]; [dir])`](@ref CIFAR10.trainlabels) | Load the labels for the training images
49+
[`testtensor([T], [indices]; [dir])`](@ref CIFAR10.testtensor) | Load the test images as an array of eltype `T`
50+
[`testlabels([indices]; [dir])`](@ref CIFAR10.testlabels) | Load the labels for the test images
51+
[`traindata([T], [indices]; [dir])`](@ref CIFAR10.traindata) | Load images and labels of the training data
52+
[`testdata([T], [indices]; [dir])`](@ref CIFAR10.testdata) | Load images and labels of the test data
53+
54+
This module also provides utility functions to make working with
55+
the CIFAR-10 dataset in Julia more convenient.
56+
57+
Function | Description
58+
---------|-------------
59+
[`convert2features(array)`](@ref CIFAR10.convert2features) | Convert the CIFAR-10 tensor to a flat feature matrix
60+
[`convert2image(array)`](@ref CIFAR10.convert2image) | Convert the CIFAR-10 tensor/matrix to a colorant array
61+
62+
You can use the function
63+
[`convert2features`](@ref CIFAR10.convert2features) to convert
64+
the given CIFAR-10 tensor to a feature matrix (or feature vector
65+
in the case of a single image). The purpose of this function is
66+
to drop the spatial dimensions such that traditional ML
67+
algorithms can process the dataset.
68+
69+
```julia
70+
julia> CIFAR10.convert2features(CIFAR10.traintensor()) # full training data
71+
3072×50000 Array{N0f8,2}:
72+
[...]
73+
```
74+
75+
To visualize an image or a prediction we provide the function
76+
[`convert2image`](@ref CIFAR10.convert2image) to convert the
77+
given CIFAR10 horizontal-major tensor (or feature matrix) to a
78+
vertical-major `Colorant` array.
79+
80+
```julia
81+
julia> CIFAR10.convert2image(CIFAR10.traintensor(1)) # first training image
82+
32×32 Array{RGB{N0f8},2}:
83+
[...]
84+
```
85+
86+
## API Documentation
87+
88+
### Trainingset
89+
90+
```@docs
91+
CIFAR10.traintensor
92+
CIFAR10.trainlabels
93+
CIFAR10.traindata
94+
```
95+
96+
### Testset
97+
98+
```@docs
99+
CIFAR10.testtensor
100+
CIFAR10.testlabels
101+
CIFAR10.testdata
102+
```
103+
104+
### Utilities
105+
106+
```@docs
107+
CIFAR10.classnames
108+
CIFAR10.convert2features
109+
CIFAR10.convert2image
110+
```
111+
112+
## References
113+
114+
- **Authors**: Alex Krizhevsky, Vinod Nair, Geoffrey Hinton
115+
116+
- **Website**: https://www.cs.toronto.edu/~kriz/cifar.html
117+
118+
- **[Krizhevsky, 2009]** Alex Krizhevsky. ["Learning Multiple Layers of Features from Tiny Images"](https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf), Tech Report, 2009.

‎docs/src/datasets/CIFAR100.md

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# [CIFAR-100](@id CIFAR100)
2+
3+
Description from the [original
4+
website](https://www.cs.toronto.edu/~kriz/cifar.html)
5+
6+
> The CIFAR-10 and CIFAR-100 are labeled subsets of the
7+
> [80 million tiny images](http://people.csail.mit.edu/torralba/tinyimages/)
8+
> dataset. They were collected by Alex Krizhevsky, Vinod Nair,
9+
> and Geoffrey Hinton.
10+
>
11+
> This dataset is just like the CIFAR-10, except it has 100
12+
> classes containing 600 images each. There are 500 training
13+
> images and 100 testing images per class. The 100 classes in the
14+
> CIFAR-100 are grouped into 20 superclasses. Each image comes
15+
> with a "fine" label (the class to which it belongs) and a
16+
> "coarse" label (the superclass to which it belongs).
17+
18+
## Contents
19+
20+
```@contents
21+
Pages = ["CIFAR100.md"]
22+
Depth = 3
23+
```
24+
25+
## Overview
26+
27+
The `MLDatasets.CIFAR100` sub-module provides a programmatic
28+
interface to download, load, and work with the CIFAR-100 dataset.
29+
30+
```julia
31+
using MLDatasets
32+
33+
# load full training set
34+
train_x, train_y_coarse, train_y_fine = CIFAR100.traindata()
35+
36+
# load full test set
37+
test_x, test_y_coarse, test_y_fine = CIFAR100.testdata()
38+
```
39+
40+
The provided functions also allow for optional arguments, such as
41+
the directory `dir` where the dataset is located, or the specific
42+
observation `indices` that one wants to work with. For more
43+
information on the interface take a look at the documentation
44+
(e.g. `?CIFAR100.traindata`).
45+
46+
Function | Description
47+
---------|-------------
48+
`download([dir])` | Trigger interactive download of the dataset
49+
[`classnames_coarse()`](@ref CIFAR100.classnames_coarse) | Return the 20 super-class names as a vector of strings
50+
[`classnames_fine()`](@ref CIFAR100.classnames_fine) | Return the 100 class names as a vector of strings
51+
[`traintensor([T], [indices]; [dir])`](@ref CIFAR100.traintensor) | Load the training images as an array of eltype `T`
52+
[`trainlabels([indices]; [dir])`](@ref CIFAR100.trainlabels) | Load the labels for the training images
53+
[`testtensor([T], [indices]; [dir])`](@ref CIFAR100.testtensor) | Load the test images as an array of eltype `T`
54+
[`testlabels([indices]; [dir])`](@ref CIFAR100.testlabels) | Load the labels for the test images
55+
[`traindata([T], [indices]; [dir])`](@ref CIFAR100.traindata) | Load images and labels of the training data
56+
[`testdata([T], [indices]; [dir])`](@ref CIFAR100.testdata) | Load images and labels of the test data
57+
58+
This module also provides utility functions to make working with
59+
the CIFAR-100 dataset in Julia more convenient.
60+
61+
Function | Description
62+
---------|-------------
63+
[`convert2features(array)`](@ref CIFAR10.convert2features) | Convert the CIFAR-100 tensor to a flat feature matrix
64+
[`convert2image(array)`](@ref CIFAR10.convert2image) | Convert the CIFAR-100 tensor/matrix to a colorant array
65+
66+
You can use the function
67+
[`convert2features`](@ref CIFAR10.convert2features) to convert
68+
the given CIFAR-100 tensor to a feature matrix (or feature vector
69+
in the case of a single image). The purpose of this function is
70+
to drop the spatial dimensions such that traditional ML
71+
algorithms can process the dataset.
72+
73+
```julia
74+
julia> CIFAR100.convert2features(CIFAR100.traintensor()) # full training data
75+
3072×50000 Array{N0f8,2}:
76+
[...]
77+
```
78+
79+
To visualize an image or a prediction we provide the function
80+
[`convert2image`](@ref CIFAR10.convert2image) to convert the
81+
given CIFAR-100 horizontal-major tensor (or feature matrix) to a
82+
vertical-major `Colorant` array.
83+
84+
```julia
85+
julia> CIFAR100.convert2image(CIFAR100.traintensor(1)) # first training image
86+
32×32 Array{RGB{N0f8},2}:
87+
[...]
88+
```
89+
90+
## API Documentation
91+
92+
### Trainingset
93+
94+
```@docs
95+
CIFAR100.traintensor
96+
CIFAR100.trainlabels
97+
CIFAR100.traindata
98+
```
99+
100+
### Testset
101+
102+
```@docs
103+
CIFAR100.testtensor
104+
CIFAR100.testlabels
105+
CIFAR100.testdata
106+
```
107+
108+
### Utilities
109+
110+
See [`CIFAR10.convert2features`](@ref) and
111+
[`CIFAR10.convert2image`](@ref)
112+
113+
```@docs
114+
CIFAR100.classnames_coarse
115+
CIFAR100.classnames_fine
116+
```
117+
118+
## References
119+
120+
- **Authors**: Alex Krizhevsky, Vinod Nair, Geoffrey Hinton
121+
122+
- **Website**: https://www.cs.toronto.edu/~kriz/cifar.html
123+
124+
- **[Krizhevsky, 2009]** Alex Krizhevsky. ["Learning Multiple Layers of Features from Tiny Images"](https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf), Tech Report, 2009.

‎docs/src/datasets/FashionMNIST.md

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
# [Fashion-MNIST](@id FashionMNIST)
2+
3+
Description from the [official website](https://github.com/zalandoresearch/fashion-mnist)
4+
5+
> Fashion-MNIST is a dataset of Zalando's article
6+
> images—consisting of a training set of 60,000 examples and a
7+
> test set of 10,000 examples. Each example is a 28x28 grayscale
8+
> image, associated with a label from 10 classes. We intend
9+
> Fashion-MNIST to serve as a direct drop-in replacement for the
10+
> original MNIST dataset for benchmarking machine learning
11+
> algorithms. It shares the same image size and structure of
12+
> training and testing splits.
13+
14+
## Contents
15+
16+
```@contents
17+
Pages = ["FashionMNIST.md"]
18+
Depth = 3
19+
```
20+
21+
## Overview
22+
23+
The `MLDatasets.FashionMNIST` sub-module provides a programmatic
24+
interface to download, load, and work with the Fashion-MNIST
25+
dataset.
26+
27+
```julia
28+
using MLDatasets
29+
30+
# load full training set
31+
train_x, train_y = FashionMNIST.traindata()
32+
33+
# load full test set
34+
test_x, test_y = FashionMNIST.testdata()
35+
```
36+
37+
The provided functions also allow for optional arguments, such as
38+
the directory `dir` where the dataset is located, or the specific
39+
observation `indices` that one wants to work with. For more
40+
information on the interface take a look at the documentation
41+
(e.g. `?FashionMNIST.traindata`).
42+
43+
Function | Description
44+
---------|-------------
45+
`download([dir])` | Trigger (interactive) download of the dataset
46+
[`classnames()`](@ref FashionMNIST.classnames) | Return the class names as a vector of strings
47+
[`traintensor([T], [indices]; [dir])`](@ref FashionMNIST.traintensor) | Load the training images as an array of eltype `T`
48+
[`trainlabels([indices]; [dir])`](@ref FashionMNIST.trainlabels) | Load the labels for the training images
49+
[`testtensor([T], [indices]; [dir])`](@ref FashionMNIST.testtensor) | Load the test images as an array of eltype `T`
50+
[`testlabels([indices]; [dir])`](@ref FashionMNIST.testlabels) | Load the labels for the test images
51+
[`traindata([T], [indices]; [dir])`](@ref FashionMNIST.traindata) | Load images and labels of the training data
52+
[`testdata([T], [indices]; [dir])`](@ref FashionMNIST.testdata) | Load images and labels of the test data
53+
54+
This module also provides utility functions to make working with
55+
the Fashion-MNIST dataset in Julia more convenient.
56+
57+
Function | Description
58+
---------|-------------
59+
[`convert2features(array)`](@ref FashionMNIST.convert2features) | Convert the Fashion-MNIST tensor to a flat feature matrix
60+
[`convert2image(array)`](@ref FashionMNIST.convert2image) | Convert the Fashion-MNIST tensor/matrix to a colorant array
61+
62+
You can use the function
63+
[`convert2features`](@ref FashionMNIST.convert2features) to
64+
convert the given Fashion-MNIST tensor to a feature matrix (or
65+
feature vector in the case of a single image). The purpose of
66+
this function is to drop the spatial dimensions such that
67+
traditional ML algorithms can process the dataset.
68+
69+
```julia
70+
julia> FashionMNIST.convert2features(FashionMNIST.traintensor()) # full training data
71+
784×60000 Array{N0f8,2}:
72+
[...]
73+
```
74+
75+
To visualize an image or a prediction we provide the function
76+
[`convert2image`](@ref FashionMNIST.convert2image) to convert the
77+
given Fashion-MNIST horizontal-major tensor (or feature matrix)
78+
to a vertical-major `Colorant` array. The values are also color
79+
corrected according to the website's description, which means
80+
that the digits are black on a white background.
81+
82+
```julia
83+
julia> FashionMNIST.convert2image(FashionMNIST.traintensor(1)) # first training image
84+
28×28 Array{Gray{N0f8},2}:
85+
[...]
86+
```
87+
88+
## API Documentation
89+
90+
### Trainingset
91+
92+
```@docs
93+
FashionMNIST.traintensor
94+
FashionMNIST.trainlabels
95+
FashionMNIST.traindata
96+
```
97+
98+
### Testset
99+
100+
```@docs
101+
FashionMNIST.testtensor
102+
FashionMNIST.testlabels
103+
FashionMNIST.testdata
104+
```
105+
106+
### Utilities
107+
108+
See [`MNIST.convert2features`](@ref) and
109+
[`MNIST.convert2image`](@ref)
110+
111+
```@docs
112+
FashionMNIST.classnames
113+
```
114+
115+
## References
116+
117+
- **Authors**: Han Xiao, Kashif Rasul, Roland Vollgraf
118+
119+
- **Website**: https://github.com/zalandoresearch/fashion-mnist
120+
121+
- **[Han Xiao et al. 2017]** Han Xiao, Kashif Rasul, and Roland Vollgraf. "Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms." arXiv:1708.07747

0 commit comments

Comments
 (0)