AssertionError, RGATConv mini-batch replay memory!

[Famirreza]

Dear all,
I am trying to use optimize_model() in my DQN training process for torch graph data type

`def optimize_model():
if len(memory) < BATCH_SIZE:
return
transitions = memory.sample(BATCH_SIZE)
# Transpose the batch (see https://stackoverflow.com/a/19343/3343043 for
# detailed explanation). This converts batch-array of Transitions
# to Transition of batch-arrays.
batch = Transition(*zip(*transitions))

#print("Batch is here:", batch)
# Compute a mask of non-final states and concatenate the batch elements
# (a final state would've been the one after which simulation ended)


non_final_mask = torch.tensor(tuple(map(lambda s: s is not None, batch.next_state)), device=device, dtype=torch.bool)    
non_final_next_states =  torch.cat(([s for s in batch.next_state if s is not None]))

state_batch =  torch.cat(batch.state)
action_batch =  torch.cat(batch.action)
reward_batch =  torch.cat(batch.reward)


# Compute Q(s_t, a) - the model computes Q(s_t), then we select the
# columns of actions taken. These are the actions which would've been taken
# for each batch state according to policy_net
state_action_values = policy_net(state_batch.x, state_batch.edge_index, state_batch.edge_type, state_batch.edge_attr).gather(1, action_batch)`

I faced TypeError: expected Tensor as element 0 in argument 0, but got Data.

I did not find any alternative then I changed my code the following code (Are there any other options?):

`def optimize_model():
if len(memory) < BATCH_SIZE:
return
transitions = memory.sample(BATCH_SIZE)
# Transpose the batch (see https://stackoverflow.com/a/19343/3343043 for
# detailed explanation). This converts batch-array of Transitions
# to Transition of batch-arrays.
batch = Transition(*zip(*transitions))

#print("Batch is here:", batch)
# Compute a mask of non-final states and concatenate the batch elements
# (a final state would've been the one after which simulation ended)


non_final_mask = torch.tensor(tuple(map(lambda s: s is not None, batch.next_state)), device=device, dtype=torch.bool)    

non_final_next_states_x_s = torch.cat(([s.x for s in batch.next_state if s is not None]))
non_final_next_states_edge_index_s = torch.cat(([s.edge_index for s in batch.next_state if s is not None])) 
non_final_next_states_edge_type_s = torch.cat(([s.edge_type for s in batch.next_state if s is not None])) 
non_final_next_states_edge_attr_s = torch.cat(([s.edge_attr for s in batch.next_state if s is not None])) 

x_s = torch.cat(([s.x for s in batch.state]))
edge_index_s = torch.cat(([s.edge_index for s in batch.state])) 
edge_type_s = torch.cat(([s.edge_type for s in batch.state])) 
edge_attr_s = torch.cat(([s.edge_attr for s in batch.state]))

action_batch =  torch.tensor(batch.action)
reward_batch =  torch.tensor(batch.reward)


# Compute Q(s_t, a) - the model computes Q(s_t), then we select the
# columns of actions taken. These are the actions which would've been taken
# for each batch state according to policy_net
state_action_values = policy_net(x_s, edge_index_s, edge_type_s, edge_attr_s).gather(1, action_batch)`

Then, I faced AssertionError.
Could you please explain how should I make my batch compatible with my RGATConv (or model with my batch)?

Thank you in advance for your help!

[rusty1s]

How does your policy net look like? Where does it fail concretely?

[rusty1s]

This is not a [2, num_edges] matrix which is required for message passing. Can you fix how you batch your graph together (using a diagonal stacking of indices as described in the documentation)?

[Famirreza]

I checked https://pytorch-geometric.readthedocs.io/en/latest/notes/batching.html.and https://pytorch-geometric.readthedocs.io/en/latest/notes/introduction.html#mini-batches. still I am a bit lost.
I tried to use https://pytorch-geometric.readthedocs.io/en/latest/notes/create_dataset.html to make a new Dataset of my Graph Data same as below:

data_list = [Data(...), ..., Data(...)] loader = DataLoader(data_list, batch_size = len(data_list))

Still I have error:
RuntimeError: The size of tensor a (320) must match the size of tensor b (32) at non-singleton dimension 0

[rusty1s]

What is the output of print(data_list[:5])?

[Famirreza]

[Data(edge_index=[2, 300], x=[10, 8], edge_attr=[300, 1], node_type=[10], edge_type=[300]), Data(edge_index=[2, 300], x=[10, 8], edge_attr=[300, 1], node_type=[10], edge_type=[300]), Data(edge_index=[2, 300], x=[10, 8], edge_attr=[300, 1], node_type=[10], edge_type=[300]), Data(edge_index=[2, 300], x=[10, 8], edge_attr=[300, 1], node_type=[10], edge_type=[300]), Data(edge_index=[2, 300], x=[10, 8], edge_attr=[300, 1], node_type=[10], edge_type=[300])]

[rusty1s]

Mh, so I guess Batch.from_data_list(data_list[:5]) works just fine. There needs to exist some other data object that contradicts with the expected format of any attribute. Is this possible?