Different Embedding Size

[mprithiv]

Hi

This is the HeteroData object I have
HeteroData( buyerId={ num_nodes=7813, x=[7813, 1152] }, sellerId={ num_nodes=884, x=[884, 387] }, (buyerId, txn_amount, sellerId)={ edge_index=[2, 335959] }, (buyerId, txn_date, sellerId)={ edge_index=[2, 335959] } )

I have a different embedding size for both the nodes and the edges. I am trying to understand how important it is to have same embedding size for both the nodes and the edges and impact of padding/shrinking the embedding to make it similar for all the nodes

The embeddings are created using the SentenceTransformer for alpha-numeric and LabelEncoder for numeric

class SequenceEncoder(object):
    def __init__(self, model_name="all-MiniLM-L6-v2", device=None):
        self.device = device
        self.model = SentenceTransformer(model_name, device=None)
    @torch.no_grad()
    def __call__(self, df):
        #cars = set(g for col in df.values)
        x = self.model.encode(df.values, show_progress_bar=True, convert_to_tensor=True)
        return x.cpu()

and

class LabelEncoder(object):
    def __init__(self, dtype=None):
        self.dtype = dtype
    #@torch.no_grad()
    def __call__(self, df):
        #ys = pd.read_csv(df)['cbd_total_3months']
        return torch.from_numpy(df.values).view(-1,1).to(self.dtype)#, show_progress_bar=True, convert_to_tensor=True)`

How do I set an embedding size while preparing the Data object? Is the different embedding size more of a problem in creating NodeSplit

[rusty1s]

Can you clarify on what's the concrete issue? In general, HeteroData and our heterogeneous GNNs (such as to_hetero()) are designed to be fully able to operate on differently sized embeddings, e.g., via lazy initialization. Our tutorial covers more information.

[rusty1s]

Weighted sampling is not yet supported, I'm sorry :(

I think in your case it makes more sense to have a single edge type with edge features attached to it (like amount, date). This requires that you utilize a GNN that can incorporate edge features.

You can get both edge indices as a dict by running data.edge_index_dict.

[mprithiv]

Hi. I was using the sageconv from here. Does it support incorporating edge features?

On a different note, I have two node types (buyer and seller). The node classification might happen within buyer but there might be a link classification or subgraph generation in the future. So basically, how can i ensure that both the node types(buyer and seller) are encoded along with the edge information (like "Attention Mechanism - Jointly Learning to Align and Translate in NLP"), jointly embed the multiple node and edge information?

[rusty1s]

SAGEConv does not support edge features, Here is an overview of all GNN operators that support them. All operators that support multi-dimensional edge features should guarantee that edge information is incorporated into the resulting node embeddings.

[mprithiv]

Is there an example implementation for TransformerConv

[rusty1s]

Not yet, but we are trying to integrate one with this issue #4040.