Type 'DataLoader' doesn't have expected attributes '__iter__', 'dataset'

[anthonysirico]

Here is my dataset creation script:

data_perm = np.random.permutation(len(graphs))
index_split = int(len(graphs) * p_known)

known_indices = data_perm[:index_split]
unknown_indices = data_perm[index_split:]

known_graphs = graphs.loc[known_indices]
known_node_labels = encoded_node_labels.loc[known_indices]

unknown_graphs = graphs.loc[unknown_indices]
unknown_node_labels = encoded_node_labels.loc[unknown_indices]

known_dataset = TMSDataset(
    root='{}/baseline_known'.format(path),
    data=known_graphs,
    node_labels=known_node_labels,
)

unknown_dataset = TMSDataset(
    root='{}/baseline_unknown'.format(path),
    data=unknown_graphs,
    node_labels=unknown_node_labels,
)

training_set = known_dataset[:int(len(known_graphs) * p_train)]
validation_set = known_dataset[int(len(known_graphs) * p_train):]

training_loader = DataLoader(training_set, batch_size=32, shuffle=False)
validation_loader = DataLoader(validation_set, batch_size=32, shuffle=False)
unknown_loader = DataLoader(unknown_dataset, batch_size=1, shuffle=False)

But I'm getting the notice
Type 'DataLoader' doesn't have expected attributes '__iter__', 'dataset'

I am using InMemoryDataset, what am I missing?

[rusty1s]

Where does this issue occur? is it coming from a linter?

[anthonysirico]

yes, it is highlighted in pycharm community

[rusty1s]

Interesting. I am not totally sure why this happens, because DataLoader just inherits from torch.utils.data.DataLoader which defines a __iter__ method. Any help appreciated.

[anthonysirico]

Here is the whole output:

Type 'DataLoader' doesn't have expected attribute '__iter__' 
 Inspection info:
Reports type errors in function call expressions, targets, and return values. In a dynamically typed language, this is possible in a limited number of cases.
Types of function parameters can be specified in docstrings or in Python 3 function annotations.
Example:
def foo() -> int:
    return "abc" # Expected int, got str


a: str
a = foo() # Expected str, got int
With the quick-fix, you can modify the problematic types:
def foo() -> str:
    return "abc"


a: str
a = foo()

and if it has to do with my dataset creation, here is the code:

from abc import ABC

from scipy.io import loadmat
import pandas as pd
from torch_geometric.data import InMemoryDataset
import torch
import numpy as np
import torch_geometric
from tqdm import tqdm
from typing import Union, List
from torch import Tensor
from collections.abc import Sequence
from torch_geometric.utils import from_networkx
from sklearn.preprocessing import LabelEncoder
import networkx as nx


class TMSDataset(InMemoryDataset, ABC):
    def __init__(self, root: str, data, node_labels, transform=None, pre_transform=None,
                 pre_filter=None):
        self.data = data
        self.node_labels = node_labels
        super().__init__(root, transform, pre_transform, pre_filter)
        self.data, self.slices = torch.load(self.processed_paths[0])

    @property
    def raw_file_names(self):
        return ['/home/polar/GDL-TMS/data/graphs.mat', '/home/polar/GDL-TMS/data/nl.json']

    @property
    def processed_file_names(self) -> list[str]:
        return ['data.pt']

    def download(self):
        pass

    def process(self):

        tms_graphs = []
        for index, graph in tqdm(self.data.iterrows(), total=len(self.data)):
            networkx_graph = nx.DiGraph(self.data['A'][index])
            networkx_graph = self._get_node_features(networkx_graph, self.node_labels[index])
            pyg = from_networkx(networkx_graph, group_node_attrs=['feature'])
            pyg.y_compiled = torch.tensor(self.data['compiled'][index], dtype=torch.long)
            pyg.y_simulated = torch.tensor(self.data['simulated'][index], dtype=torch.long)
            tms_graphs.append(pyg)

        if self.pre_filter is not None:
            tms_graphs = [data for data in tms_graphs if self.pre_filter(data)]

        if self.pre_transform is not None:
            tms_graphs = [self.pre_transform(data) for data in tms_graphs]

        data_known, slices_known = self.collate(tms_graphs)

        torch.save((data_known, slices_known), self.processed_paths[0])

    def _get_node_features(self, networkx_graph, node_labels):
        nl = node_labels[0:len(networkx_graph.nodes)]
        nl = dict(enumerate(nl))
        nx.set_node_attributes(networkx_graph, nl, 'feature')
        return networkx_graph

    @property
    def num_node_features(self) -> int:
        return 1

    @property
    def num_classes(self) -> int:
        return 2

[rusty1s]

Why does it have to do with your dataset? I thought the warning appears in DataLoader?

[anthonysirico]

Just making sure it wasnt something in the code

[anthonysirico]

A pycharm thing i believe.

[Louis24]

yep its warning not error