Skip to content

Commit 886740b

Browse files
Sachin GoyalMJ10
andauthored
DROCC-LF trainer (#198)
* lfoc initial push * working * Readme added for DROCCLF * minor changes * torch version compatibility issus resolved * drocclf trainer changes * update readme * drocc:save best model * add eval to drocc examples * Update README.md * Update README.md * fix model eval * Eval code checked * eval code checked * Update README.md * Update drocc_trainer.py * Update main_cifar.py * Update main_cifar.py * Update README.md Co-authored-by: Moksh Jain <[email protected]>
1 parent eeec40c commit 886740b

File tree

11 files changed

+841
-70
lines changed

11 files changed

+841
-70
lines changed

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -65,12 +65,14 @@ Code for algorithms, applications and tools contributed by:
6565
- [Don Dennis](https://dkdennis.xyz)
6666
- [Yash Gaurkar](https://github.com/mr-yamraj/)
6767
- [Sridhar Gopinath](http://www.sridhargopinath.in/)
68+
- [Sachin Goyal](https://saching007.github.io/)
6869
- [Chirag Gupta](https://aigen.github.io/)
6970
- [Moksh Jain](https://github.com/MJ10)
7071
- [Ashish Kumar](https://ashishkumar1993.github.io/)
7172
- [Aditya Kusupati](https://adityakusupati.github.io/)
7273
- [Chris Lovett](https://github.com/lovettchris)
7374
- [Shishir Patil](https://shishirpatil.github.io/)
75+
- [Oindrila Saha](https://github.com/oindrilasaha)
7476
- [Harsha Vardhan Simhadri](http://harsha-simhadri.org)
7577

7678
[Contributors](https://microsoft.github.io/EdgeML/People) to this project. New contributors welcome.
@@ -81,9 +83,9 @@ If you use software from this library in your work, please use the BibTex entry
8183

8284
```
8385
@software{edgeml03,
84-
author = {{Dennis, Don Kurian and Gaurkar, Yash and Gopinath, Sridhar and Gupta, Chirag and
85-
Jain, Moksh and Kumar, Ashish and Kusupati, Aditya and Lovett, Chris
86-
and Patil, Shishir G and Simhadri, Harsha Vardhan}},
86+
author = {{Dennis, Don Kurian and Gaurkar, Yash and Gopinath, Sridhar and Goyal, Sachin
87+
and Gupta, Chirag and Jain, Moksh and Kumar, Ashish and Kusupati, Aditya and
88+
Lovett, Chris and Patil, Shishir G and Saha, Oindrila and Simhadri, Harsha Vardhan}},
8789
title = {{EdgeML: Machine Learning for resource-constrained edge devices}},
8890
url = {https://github.com/Microsoft/EdgeML},
8991
version = {0.3},

examples/pytorch/DROCC/README.md

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Deep Robust One-Class Classification
2-
In this directory we present examples of how to use the `DROCCTrainer` to replicate results in [paper](https://proceedings.icml.cc/book/4293.pdf).
2+
In this directory we present examples of how to use the `DROCCTrainer` and `DROCCLFTrainer` to replicate results in [paper](https://proceedings.icml.cc/book/4293.pdf).
33

4-
`DROCCTrainer` is part of the `edgeml_pytorch` package. Please install the `edgeml_pytorch` package as follows:
4+
`DROCCTrainer` and `DROCCLFTrainer` are part of the `edgeml_pytorch` package. Please install the `edgeml_pytorch` package as follows:
55
```
66
git clone https://github.com/microsoft/EdgeML
77
cd EdgeML/pytorch
@@ -38,17 +38,17 @@ The output path is referred to as "root_data" in the following section.
3838
### Command to run experiments to reproduce results
3939
#### Arrhythmia
4040
```
41-
python3 main_tabular.py --hd 128 --lr 0.0001 --lamda 1 --gamma 2 --ascent_step_size 0.001 --radius 16 --batch_size 256 --epochs 200 --optim 0 --restore 0 --metric F1 -d "root_data"
41+
python3 main_tabular.py --hd 128 --lr 0.0001 --lamda 1 --gamma 2 --ascent_step_size 0.001 --radius 16 --batch_size 256 --epochs 200 --optim 0 --metric F1 -d "root_data"
4242
```
4343

4444
#### Thyroid
4545
```
46-
python3 main_tabular.py --hd 128 --lr 0.001 --lamda 1 --gamma 2 --ascent_step_size 0.001 --radius 2.5 --batch_size 256 --epochs 100 --optim 0 --restore 0 --metric F1 -d "root_data"
46+
python3 main_tabular.py --hd 128 --lr 0.001 --lamda 1 --gamma 2 --ascent_step_size 0.001 --radius 2.5 --batch_size 256 --epochs 100 --optim 0 --metric F1 -d "root_data"
4747
```
4848

4949
#### Abalone
5050
```
51-
python3 main_tabular.py --hd 128 --lr 0.001 --lamda 1 --gamma 2 --ascent_step_size 0.001 --radius 3 --batch_size 256 --epochs 200 --optim 0 --restore 0 --metric F1 -d "root_data"
51+
python3 main_tabular.py --hd 128 --lr 0.001 --lamda 1 --gamma 2 --ascent_step_size 0.001 --radius 3 --batch_size 256 --epochs 200 --optim 0 --metric F1 -d "root_data"
5252
```
5353

5454

@@ -67,20 +67,26 @@ The output path is referred to as "root_data" in the following section.
6767

6868
### Example Usage for Epilepsy Dataset
6969
```
70-
python3 main_timeseries.py --hd 128 --lr 0.00001 --lamda 0.5 --gamma 2 --ascent_step_size 0.1 --radius 10 --batch_size 256 --epochs 200 --optim 0 --restore 0 --metric AUC -d "root_data"
70+
python3 main_timeseries.py --hd 128 --lr 0.00001 --lamda 0.5 --gamma 2 --ascent_step_size 0.1 --radius 10 --batch_size 256 --epochs 200 --optim 0 --metric AUC -d "root_data"
7171
```
7272

7373
## CIFAR Experiments
7474
```
75-
python3 main_cifar.py --lamda 1 --radius 8 --lr 0.001 --gamma 1 --ascent_step_size 0.001 --batch_size 256 --epochs 40 --optim 0 --normal_class 0
75+
python3 main_cifar.py --lamda 1 --radius 8 --lr 0.001 --gamma 1 --ascent_step_size 0.001 --batch_size 256 --epochs 100 --optim 0 --normal_class 0
7676
```
7777

78+
## DROCC-LF MNIST Experiment
79+
MNIST Digit 0 vs Digit 1 experiment where close negatives are generated by randomly masking the pixels.
80+
```
81+
python3 main_drocclf_mnist.py --lamda 1 --radius 16 --lr 0.0001 --batch_size 256 --epochs 40 --one_class_adv 1 --optim 0 -oce 10 --ascent_num_steps 100 --ascent_step_size 0.1 --normal_class 0
82+
```
7883

7984
### Arguments Detail
8085
normal_class => CIFAR10 class to be considered as normal
8186
lamda => Weightage to the loss from adversarially sampled negative points (\mu in the paper)
82-
radius => radius corresponding to the definition of set N_i(r)
87+
radius => Radius corresponding to the definition of set N_i(r)
8388
hd => LSTM Hidden Dimension
8489
optim => 0: Adam 1: SGD(M)
85-
ascent_step_size => step size for gradient ascent to generate adversarial anomalies
86-
90+
ascent_step_size => Step size for gradient ascent to generate adversarial anomalies
91+
ascent_num_steps => Number of gradient ascent steps
92+
oce => Only Cross Entropy Steps (No adversarial loss is calculated)

examples/pytorch/DROCC/data_process_scripts/process_cifar.py

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -58,18 +58,6 @@ def __init__(self, root: str, normal_class=5):
5858
self.outlier_classes = list(range(0, 10))
5959
self.outlier_classes.remove(normal_class)
6060

61-
# Pre-computed min and max values (after applying GCN) from train data per class
62-
# min_max = [(-28.94083453598571, 13.802961825439636),
63-
# (-6.681770233365245, 9.158067708230273),
64-
# (-34.924463588638204, 14.419298165027628),
65-
# (-10.599172931391799, 11.093187820377565),
66-
# (-11.945022995801637, 10.628045447867583),
67-
# (-9.691969487694928, 8.948326776180823),
68-
# (-9.174940012342555, 13.847014686472365),
69-
# (-6.876682005899029, 12.282371383343161),
70-
# (-15.603507135507172, 15.2464923804279),
71-
# (-6.132882973622672, 8.046098172351265)]
72-
# CIFAR-10 preprocessing: GCN (with L1 norm) and min-max feature scaling to [0,1]
7361
transform = transforms.Compose([transforms.ToTensor(),
7462
transforms.Normalize(mean=[0.4914, 0.4822, 0.4465],
7563
std=[0.247, 0.243, 0.261])])
Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
'''
2+
Code borrowed from https://github.com/lukasruff/Deep-SVDD-PyTorch
3+
'''
4+
from PIL import Image
5+
import numpy as np
6+
from random import sample
7+
from abc import ABC, abstractmethod
8+
import torch
9+
from torch.utils.data import Subset
10+
from torchvision.datasets import MNIST
11+
import torchvision.transforms as transforms
12+
from torch.utils.data import DataLoader
13+
14+
class BaseADDataset(ABC):
15+
"""Anomaly detection dataset base class."""
16+
17+
def __init__(self, root: str):
18+
super().__init__()
19+
self.root = root # root path to data
20+
21+
self.n_classes = 2 # 0: normal, 1: outlier
22+
self.normal_classes = None # tuple with original class labels that define the normal class
23+
self.outlier_classes = None # tuple with original class labels that define the outlier class
24+
25+
self.train_set = None # must be of type torch.utils.data.Dataset
26+
self.test_set = None # must be of type torch.utils.data.Dataset
27+
28+
@abstractmethod
29+
def loaders(self, batch_size: int, shuffle_train=True, shuffle_test=False, num_workers: int = 0) -> (
30+
DataLoader, DataLoader):
31+
"""Implement data loaders of type torch.utils.data.DataLoader for train_set and test_set."""
32+
pass
33+
34+
def __repr__(self):
35+
return self.__class__.__name__
36+
37+
class TorchvisionDataset(BaseADDataset):
38+
"""TorchvisionDataset class for datasets already implemented in torchvision.datasets."""
39+
40+
def __init__(self, root: str):
41+
super().__init__(root)
42+
43+
def loaders(self, batch_size: int, shuffle_train=True, shuffle_test=False, num_workers: int = 0) -> (
44+
DataLoader, DataLoader):
45+
train_loader = DataLoader(dataset=self.train_set, batch_size=batch_size, shuffle=shuffle_train,
46+
num_workers=num_workers)
47+
test_loader = DataLoader(dataset=self.test_set, batch_size=batch_size, shuffle=shuffle_test,
48+
num_workers=num_workers)
49+
return train_loader, test_loader
50+
51+
class MNIST_Dataset(TorchvisionDataset):
52+
53+
def __init__(self, root: str, normal_class=0):
54+
super().__init__(root)
55+
#Loads only the digit 0 and digit 1 data
56+
# for both train and test
57+
self.n_classes = 2 # 0: normal, 1: outlier
58+
self.normal_classes = tuple([0])
59+
self.train_classes = tuple([0,1])
60+
self.test_class = tuple([0,1])
61+
62+
transform = transforms.Compose([transforms.ToTensor(),
63+
transforms.Normalize(mean=[0.1307],
64+
std=[0.3081])])
65+
66+
target_transform = transforms.Lambda(lambda x: int(x in self.normal_classes))
67+
68+
train_set = MyMNIST(root=self.root, train=True, download=True,
69+
transform=transform, target_transform=target_transform)
70+
# Subset train_set to normal class
71+
train_idx_normal = get_target_label_idx(train_set.targets, self.train_classes)
72+
self.train_set = Subset(train_set, train_idx_normal)
73+
74+
test_set = MyMNIST(root=self.root, train=False, download=True,
75+
transform=transform, target_transform=target_transform)
76+
test_idx_normal = get_target_label_idx(test_set.targets, self.test_class)
77+
self.test_set = Subset(test_set, test_idx_normal)
78+
79+
class MyMNIST(MNIST):
80+
"""Torchvision MNIST class with patch of __getitem__ method to also return the index of a data sample."""
81+
82+
def __init__(self, *args, **kwargs):
83+
super(MyMNIST, self).__init__(*args, **kwargs)
84+
85+
def __getitem__(self, index):
86+
"""Override the original method of the MNIST class.
87+
Args:
88+
index (int): Index
89+
Returns:
90+
triple: (image, target, index) where target is index of the target class.
91+
"""
92+
img, target = self.data[index], self.targets[index]
93+
94+
# doing this so that it is consistent with all other datasets
95+
# to return a PIL Image
96+
img = Image.fromarray(img.numpy(), mode='L')
97+
98+
if self.transform is not None:
99+
img = self.transform(img)
100+
101+
if self.target_transform is not None:
102+
target = self.target_transform(target)
103+
104+
return img, target, index # only line changed
105+
106+
107+
def get_target_label_idx(labels, targets):
108+
"""
109+
Get the indices of labels that are included in targets.
110+
:param labels: array of labels
111+
:param targets: list/tuple of target labels
112+
:return: list with indices of target labels
113+
"""
114+
return np.argwhere(np.isin(labels, targets)).flatten().tolist()
115+
116+
117+
def global_contrast_normalization(x: torch.tensor, scale='l2'):
118+
"""
119+
Apply global contrast normalization to tensor, i.e. subtract mean across features (pixels) and normalize by scale,
120+
which is either the standard deviation, L1- or L2-norm across features (pixels).
121+
Note this is a *per sample* normalization globally across features (and not across the dataset).
122+
"""
123+
124+
assert scale in ('l1', 'l2')
125+
126+
n_features = int(np.prod(x.shape))
127+
128+
mean = torch.mean(x) # mean over all features (pixels) per sample
129+
x -= mean
130+
131+
if scale == 'l1':
132+
x_scale = torch.mean(torch.abs(x))
133+
134+
if scale == 'l2':
135+
x_scale = torch.sqrt(torch.sum(x ** 2)) / n_features
136+
137+
x /= x_scale
138+
139+
return x

examples/pytorch/DROCC/main_cifar.py

Lines changed: 18 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -85,19 +85,24 @@ def main():
8585
lr=args.lr)
8686
print("using Adam")
8787

88-
# Training the model
8988
trainer = DROCCTrainer(model, optimizer, args.lamda, args.radius, args.gamma, device)
90-
91-
# Restore from checkpoint
92-
if args.restore == 1:
89+
90+
if args.eval == 0:
91+
# Training the model
92+
trainer.train(train_loader, test_loader, args.lr, adjust_learning_rate, args.epochs,
93+
metric=args.metric, ascent_step_size=args.ascent_step_size, only_ce_epochs = 0)
94+
95+
trainer.save(args.model_dir)
96+
97+
else:
9398
if os.path.exists(os.path.join(args.model_dir, 'model.pt')):
9499
trainer.load(args.model_dir)
95100
print("Saved Model Loaded")
96-
97-
trainer.train(train_loader, test_loader, args.lr, adjust_learning_rate, args.epochs,
98-
metric=args.metric, ascent_step_size=args.ascent_step_size, only_ce_epochs = 0)
99-
100-
trainer.save(args.model_dir)
101+
else:
102+
print('Saved model not found. Cannot run evaluation.')
103+
exit()
104+
score = trainer.test(test_loader, 'AUC')
105+
print('Test AUC: {}'.format(score))
101106

102107
if __name__ == '__main__':
103108
torch.set_printoptions(precision=5)
@@ -111,15 +116,15 @@ def main():
111116
help='number of epochs to train')
112117
parser.add_argument('-oce,', '--only_ce_epochs', type=int, default=50, metavar='N',
113118
help='number of epochs to train with only CE loss')
114-
parser.add_argument('--ascent_num_steps', type=int, default=50, metavar='N',
119+
parser.add_argument('--ascent_num_steps', type=int, default=100, metavar='N',
115120
help='Number of gradient ascent steps')
116121
parser.add_argument('--hd', type=int, default=128, metavar='N',
117122
help='Num hidden nodes for LSTM model')
118123
parser.add_argument('--lr', type=float, default=0.001, metavar='LR',
119124
help='learning rate')
120125
parser.add_argument('--ascent_step_size', type=float, default=0.001, metavar='LR',
121126
help='step size of gradient ascent')
122-
parser.add_argument('--mom', type=float, default=0.99, metavar='M',
127+
parser.add_argument('--mom', type=float, default=0.0, metavar='M',
123128
help='momentum')
124129
parser.add_argument('--model_dir', default='log',
125130
help='path where to save checkpoint')
@@ -131,8 +136,8 @@ def main():
131136
help='Weight to the adversarial loss')
132137
parser.add_argument('--reg', type=float, default=0, metavar='N',
133138
help='weight reg')
134-
parser.add_argument('--restore', type=int, default=0, metavar='N',
135-
help='whether to load a pretrained model, 1: load 0: train from scratch ')
139+
parser.add_argument('--eval', type=int, default=0, metavar='N',
140+
help='whether to load a saved model and evaluate (0/1)')
136141
parser.add_argument('--optim', type=int, default=0, metavar='N',
137142
help='0 : Adam 1: SGD')
138143
parser.add_argument('--gamma', type=float, default=2.0, metavar='N',

0 commit comments

Comments
 (0)