Regrssion problem

[anthonysirico]

I am building a script that has two models:

the first model is a standard binary classification; the second is regression. My dataset contains three y values, the first two are for the binary class the second is the float for the regression. The problem is that for the binary classes containing 0, the regression value is an empty array. For instance, in the raw data, we classify something as being able to perform function x, if yes, class 1, if not, class 0. If it gets class 1, give me its regression value.

Now, if the model predicts a false 1, its going to see no regression value, or an nan etc. Any suggestions?

At the moment, my second issue is that in my dataset creation, its not giving the regression value at all because I used an if statement. I was trying to produce a dataset that where only the graphs that had J values would get a J attribute and the others not, so I need another route for this.

    def process(self):

        tms_graphs = []
        for index, graph in tqdm(self.data.iterrows(), total=len(self.data)):
            networkx_graph = nx.DiGraph(self.data['A'][index])
            networkx_graph = self._get_node_features(networkx_graph, self.node_labels[index])
            pyg = from_networkx(networkx_graph, group_node_attrs=['feature'])
            pyg.catalog = torch.tensor(self.catalog[index], dtype=torch.long)
            pyg.index = torch.tensor(index, dtype=torch.long)
            pyg.y_compiled = torch.tensor(self.data['compiled'][index], dtype=torch.long)
            pyg.y_simulated = torch.tensor(self.data['simulated'][index], dtype=torch.long)
            if not self.data['J'][index]:
                pass
            else:
                pyg.J = torch.tensor(self.data['J'][index], dtype=torch.float64)
            tms_graphs.append(pyg)

[rusty1s]

You would probably want to give this invalid regression values a float('NaN') when creating your tensor. Afterwards, I think your training loss function looks as

loss1 = F.binary_cross_entropy(...)
mask = y == 1
loss2 = F.mse_loss(out_reg[mask], y_reg[mask])

Thus, avoiding making use of NaN ground-truth values during training.

[anthonysirico]

So my original training function looked like this:

    def reg_train(epoch):
        regression_model.train()
        loss_all = 0

        for data in loader:
            data = data.to(device)
            regression_optimizer.zero_grad()
            loss = F.mse_loss(model(data), data.J)
            loss.backward()
            loss_all += loss.item() * data.num_graphs
            optimizer.step()
        return loss_all / len(loader.dataset)

Are you saying to do this:

    def reg_train(epoch):
        regression_model.train()
        loss_all = 0

        for data in loader:
            data = data.to(device)
            regression_optimizer.zero_grad()
            mask = y == 1
            loss = F.mse_loss(model(data), data.J[mask])
            loss.backward()
            loss_all += loss.item() * data.num_graphs
            optimizer.step()
        return loss_all / len(loader.dataset)

[rusty1s]

Yes, you would only want to perform regression loss on data for which you have ground-truth information.