Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ jobs:
matrix:
version:
- '1'
- 'nightly'
- 'lts'
- 'pre'
os:
- ubuntu-latest
arch:
Expand Down
4 changes: 2 additions & 2 deletions Project.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "ImageNetDataset"
uuid = "d141d81c-f7c4-4296-8313-8866e6f9f78a"
authors = ["Adrian Hill <[email protected]>"]
version = "1.0.0-DEV"
version = "0.1.0"

[deps]
DataAugmentation = "88a5189c-e7ff-4f85-ac6b-e6158070f02e"
Expand All @@ -19,7 +19,7 @@ DataAugmentation = "88a5189c-e7ff-4f85-ac6b-e6158070f02e"
ImageNetDatasetDataAugmentationExt = "DataAugmentation"

[compat]
DataAugmentation = "0.2"
DataAugmentation = "0.3"
DataDeps = "0.7"
ImageCore = "0.10"
JpegTurbo = "0.1"
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ dataset = ImageNet(:val; transform=tfm)
Custom transformations can be implemented by extending `AbstractTransformation`.

To apply a transformation outside of the `ImageNet` dataset,
e.g. to preprocess a single image add a given `path`, run
e.g. to preprocess a single image at a given `path`, run

```julia
transform(tfm, path)
Expand All @@ -82,6 +82,6 @@ dataset = ImageNet(:val; transform=tfm)

> [!NOTE]
> This repository is based on [MLDatasets.jl PR #146](https://github.com/JuliaML/MLDatasets.jl/pull/146)
> and mirrors the MLDatasets API.
> and mirrors the MLDatasets `v0.7` API.
>
> Copyright (c) 2015 Hiroyuki Shindo and contributors.
19 changes: 10 additions & 9 deletions docs/src/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,23 +70,25 @@ Download the following files from the [ILSVRC2012 download page](https://image-n
You can use ImageNetDataset.jl without downloading all splits.

After downloading the data, move and extract the training and validation images to
labeled subfolders by running the following shell script:
labeled subfolders by running the following shell scripts:

**Extract metadata from the devkit:**
```bash
mkdir -p ImageNet/devkit && tar -xvf ILSVRC2012_devkit_t12.tar -C ImageNet/devkit --strip-components=1
```

**Extract the training data:**
```bash
# Extract the training data:
mkdir -p ImageNet/train && tar -xvf ILSVRC2012_img_train.tar -C ImageNet/train

# Unpack all 1000 compressed tar-files, one for each category:
cd ImageNet/train
find . -name "*.tar" | while read NAME ; do mkdir -p "\${NAME%.tar}"; tar -xvf "\${NAME}" -C "\${NAME%.tar}"; rm -f "\${NAME}"; done
```

# Extract the validation data:
cd ../..
**Extract the validation data:**
```bash
mkdir -p ImageNet/val && tar -xvf ILSVRC2012_img_val.tar -C ImageNet/val

# Extract metadata from the devkit:
cd ../..
mkdir -p ImageNet/devkit && tar -xvf ILSVRC2012_devkit_t12.tar.gz -C ImageNet/devkit --strip-components=1
```

And run the following script
Expand All @@ -97,4 +99,3 @@ to create all class directories and move images into corresponding directories:
cd ImageNet/val
wget -qO- https://raw.githubusercontent.com/Julia-XAI/ImageNetDataset.jl/master/docs/src/valprep.sh | bash
```

6 changes: 3 additions & 3 deletions src/imagenet.jl
Original file line number Diff line number Diff line change
Expand Up @@ -131,8 +131,6 @@ function ImageNet(
test_dir="test",
devkit_dir="devkit",
)
@assert split ∈ (:train, :val, :test)

depname = "ImageNet"
root_dir = get_datadep_dir(depname, dir)

Expand All @@ -146,9 +144,11 @@ function ImageNet(
elseif split == :val
paths = get_file_paths(joinpath(root_dir, val_dir))
@assert length(paths) == VALSET_SIZE
else
elseif split == :test
paths = get_file_paths(joinpath(root_dir, test_dir))
@assert length(paths) == TESTSET_SIZE
else
throw(ArgumentError("Unknown split :$split"))
end
targets = [metadata["wnid_to_label"][wnid] for wnid in get_wnids(paths)]
return ImageNet(split, transform, paths, targets, metadata)
Expand Down
2 changes: 1 addition & 1 deletion test/references/DataAugmentations.txt

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion test/test_preprocessing.jl
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ img = inverse_transform(tfm, A)
@test size(img) == (40, 60)

# DataAugmentations.jl
tfm = CenterResizeCrop((60, 40)) |> ImageToTensor() |> Normalize(default_mean, default_std)
tfm = CenterResizeCrop((40, 60)) |> ImageToTensor() |> Normalize(default_mean, default_std)
A = transform(tfm, path)
@test size(A) == (60, 40, 3)
@test_reference "references/DataAugmentations.txt" A
Loading