[Question] Huge size of PADIM Weights on custom dataset

[pranjal-joshi-cc]

While training PADIM for the custom dataset, it provides most accurate result for my dataset as compared to other models but the size of weights is one of the big issue.
I am already using following method to reduce the weight size during the training:

model.model.gaussian.inv_covariance=model.model.gaussian.inv_covariance.type(torch.float16)
model.model.gaussian.mean=model.model.gaussian.mean.type(torch.float16)

However I am still getting weights in GBs for my configuration as follow:

Custom image size = 448 x 448
reduced dimensions tested on 550 and 448.

5.0GB -> padim_Wall1_2_float16_448x448_emb448_5G.ckpt
7.4GB -> padim_Wall1_2_float16_448x448_emb550_7.4G.ckpt

Is there any way to further optimize the size of the weight files because it's getting very resource intensive during inference!

CC @roshan-dadlaney

[ORippler]

If you want to stick to PADIM you can:

• Exchange the underlying feature-extractor (EfficientNets have less features than ResNets for example)
• Reduce the number of "layers" where you extract the features from
• Reduce the dimensionality (increase random subsampling rate) of your feature space

If you want to code/implement some stuff, you can:

• "Tie" the covariance matrix across spatial locations, which drastically reduces the memory requirement (see paper here)
• Perform "nPCA" or alternatively "Semi-Orthogonal" projection for a more-prinicpled dimensionality reduction

[lorenzomammana]

@ORippler Any insight to implement the tied covariance? Is it just the covariance computed on the full set of features, an average of the covariances at each location or something else?

[ORippler]

the covariance computed on the full set of features

It's this one, so you just have to collapse your batch & spatial dimensions

[lorenzomammana]

Thanks! Too bad (as expected) performances are way too much affected on my dataset.

[samet-akcay]

I'll move this to Discussions. Feel free to continue the discussion from there. Cheers!

[JohnnyHopp]

Hi, @pranjal-joshi-cc , I implemented a EfficientNetV2 based PaDiM, hope it may help you to some extent.
PaDiM-EfficientNetV2

[SilverStarCoder]

Hi, @pranjal-joshi-cc , I implemented a EfficientNetV2 based PaDiM, hope it may help you to some extent. PaDiM-EfficientNetV2

Do you have an example of how to use it with a custom dataset?

[lweggartner]

Hey all,

I have noticed that the model size does scale with size of the dataset. Is this due to the implementation in anomalib? As far as I understand the publication it shouldn't right?

Src: "Memory complexity. Unlike SPADE [5] and Patch SVDD, the space complexity of our model is independent of the
dataset training size and depends only on the image resolution"

Thanks in advance!

[ORippler]

Hey all,
I have noticed that the model size does scale with size of the dataset. Is this due to the implementation in anomalib? As far as I understand the publication it shouldn't right?
Src: "Memory complexity. Unlike SPADE [5] and Patch SVDD, the space complexity of our model is independent of the
dataset training size and depends only on the image resolution"
Thanks in advance!

Please be more specific in what you mean when you refer to "model size". How did you measure? At what step are you measuring? Etc.

W.r.t. the method itself: It has constant memory-footprint during inference, but not during training (you need to estimate covariance matrix and mean vector of your per-pixel distributions, and this scales with the size of the training data, at least if estimation is done empirically).

W.r.t. implementation: What could be optimized further is to free memory used to store training features after fitting, but you should be able to easily implement this by inheriting from PaDiM and overriding that function (disclaimer: I didn't deep dive into current code state so may very well be this is done somewhere in the logic flow).

Cheers,

[lweggartner]

Training works fine. But I want to save the model after training for later use, which i do with:
torch.save(model, folder_output_path)

Here I noticed when training the model on the entire custom dataset (approx 100k images with 512x512 resolution) the model size is bigger than 400Gb. (size of the file created from torch.save())

[ORippler]

I'd recommend to inspect checkpoint and see what's in there then.

[lweggartner]

Update: I was able to solve this by explicitly only saving the model dictionary.