How to run WinCLIP model on my custom dataset?

[tuguldur102]

I have tried to run the WinCLIP model for my custom dataset under 2 settings, namely zero-shot and few-shot learning. I tried both the high-level Lightning + Engine API and Pytorch implementation of the WinCLIP model, but neither of them succeeded. I installed Anomalib version 2.0.0.

My custom dataset structure:

my_custom_dataset/
├── train/
│   └── good/ # 10 good images (for reference)
├── test/
│   ├── good/ # 10 good images
│   └── bad/ # 10 bad ima

SMALL_REF_GOOD = r"/my_custom_dataset/train/good"
SMALL_TEST_BAD = r"/my_custom_dataset/test/bad"
SMALL_TEST_GOOD = r"/my_custom_dataset/test/good"

1. Zero-Shot approach with high-level Lightning settings

`
from anomalib.engine import Engine
from anomalib.models.image import WinClip

model = WinClip()
engine = Engine(task="classification") # In the new version, it requires "task" argument, I'm not sure why it requires task, where in older versions it was fine

predictions = engine.predict(model, data_path=SMALL_REF_GOOD)
`
Error:

2. Few-Shot approach with Pytorch implementation - WinCLIPMode

I have transformed the reference images and bad images into image tensors with the shape of (10, 3, 240, 240).
`
model = WinClipModel()
model.setup("abnormal", input_images)
model.k_shot # gives 10

model.visual_embeddings[0].shape # gives torch.Size([10, 196, 640])

model.text_embeddings.shape # gives torch.Size([2, 640])

results = model.forward(bad_images)
results
`

Not sure, it's working or not, but the pred_scores for both bad and good images were not distinguishable according to previous image results.

I had found this issue: #2444 before many attempts that I have tried. But this indicates Anomalib version 1.2.0, and when I'm following the instructions, it leads to this error:

From this issue, which I have understood, is it right or wrong?

As well as, it becomes harder to install Anomalib version 1.2.0 with pip install anomalib==1.2.0, then anomalib install command, therefore I've manually installed all of the required libraries.

Thank you.

[samet-akcay]

@tuguldur102, before performing .predict, you first need to validate to compute the threshold metrics on your dataset. This would be something like the following:

from anomalib.data import Visa
from anomalib.engine import Engine
from anomalib.models import WinClip

# Initialize model and data
datamodule = Visa() # Replace this with your Folder dataset
model = WinClip()

# Validate using the Engine
engine = Engine()
engine.validate(model=model, datamodule=datamodule)
# or you could also use .fit
# engine.fit(model=model, datamodule=datamodule)

# Test your test-set, compute metrics and save output results.
metrics = engine.test(model=model, datamodule=datamodule)

# Predict on your dataset 
predictions = engine.predict(model=model, data_path=<path/to/your/dataset>)