Is there any way to skip bad item in data loader?

[yangyangdotcom]

I have a dataset but some of the data is unusable (I do not know in prior). Is there any way to skip the unusable data? Thanks in advance.

[LukeLIN-web]

I don't know what loader you are using, If you iterate dataset like data = dataset[i]. you can check it and continue in loop.
Or like the following:
pytorch_geometric/examples/cluster_gcn_ppi.py

cluster_data = ClusterData(train_data, num_parts=50, recursive=False,
                           save_dir=train_dataset.processed_dir)
train_loader = ClusterLoader(cluster_data, batch_size=1, shuffle=True,
                             num_workers=12)
...
    for data in train_loader:
       if data.x satisfy some condition:
           continue
        data = data.to(device)
        optimizer.zero_grad()
        y = batch.y[:batch.batch_size]
        y_hat = model(batch.x, batch.edge_index.to(device))[:batch.batch_size] # 为什么要取前面的
        loss = F.cross_entropy(y_hat, y)
        loss.backward()
        optimizer.step()

[rusty1s]

I think this is best handled on the Dataset level, e.g., not including them as part of your examples. If you need to make this decision on the fly, take a look at IterableDataset in PyTorch.