XOR PROBLEM CHAPTER 13

[FPaoloMigl]

Hi everybody, I am reading "Machine Learning with PyTorch" and I was trying the code about the XOR problem. However I noticed that if I try to run the code in the section "Making model building more flexible with nn.Module" it doesn't learn the pattern from the data to separate it correctly.
It works if before you run the neural network with two layers using Sequential class.

Here the output I get.

Do you also have the same problem?

[FPaoloMigl]

Hi @SBanditaDas, thank you for you response.

In the XOR problem I used a NN with 2 hidden layers with dimension: 2 x 4 and 4 x 4 respectively and then the output layer: 4 x 1 as specified in the book.

This is my process to create the XOR problem:

1. First I created the database from `random.uniform
2. Next I created the batches ( 2 in this example)
3. After that I created my model and I defined the loss function and the optimizer
4. Then I trained the model
5. Finally I showed the result. However it doesn't seem enough .

I attached the notebook file
error_XOR_problem.ipynb

This is the output:

[rasbt]

Hi there. I think the neural net is just a tiny bit too small perhaps. E.g., if you change

class MyModule(nn.Module):
    def __init__(self):
        super().__init__()
        l1 = nn.Linear(2, 4)  
        a1 = nn.ReLU()
        l2 = nn.Linear(4, 4) 
        # ...

to

class MyModule(nn.Module):
    def __init__(self):
        super().__init__()
        l1 = nn.Linear(2,6)  
        a1 = nn.ReLU()
        l2 = nn.Linear(6, 4) 
        # ...

it seems to work:

[SBanditaDas]

hello @FPaoloMigl ,

Thanks for sharing your process and notebook! I noticed you're using two hidden layers with dimensions 2×4 and 4×4. One thing you might want to check is whether your activation functions are helping the model capture non-linearity effectively. Also, are you using a sufficient number of epochs or a learning rate scheduler? I’d be happy to take a look at the notebook and share some suggestions if you’d like!

[FPaoloMigl]

Hi @SBanditaDas,

I think my problem was neural network too small, because with the solution given by Mr. Raschka, I fixed the problem.