Problem Formulation

[semirelezovikj]

I am writing seeking for help in the formulation of a problem.

I am working on a research project, where I have identified that it may benefit greatly to treat the problem as a graph. Namely, for a given scene, I am given information about participating agents in that scene. However, during the training stage I am only allowed to use information about a subset of the available agents in the scene. For validation, the prediction is made on agent-basis, rather than scene-basis.

Example: Let’s say a scene (the graph) has 30 total agents that influence each other. In the training phase, only ground truth for 5 of those agents can be used. Additionally, for the testing phase - a prediction must be made on agent-basis. This means that I cannot have a list of torch_geometric.data object representing the scenes (graphs), but rather must construct a graph every time I am given a single agent. Note: I am not interested to make a prediction for the unlabeled agents at all.

Graph: Entire Scene
Node: Agent
Node Features: Img, 3 dim vector

Possible way of formulating this:

1. torch.utils.data.Dataloader (not geo Dataloader) returns for the agent_scene combo: agent_img, agent_state_vector, x, edge_index. I am returning x and edge_index, since I can’t return torch_geometric.data. I generate data using x and edge_index.

The nn.module:

Passes img through resnet50 - output 2048 features`
The 3 dimensional vector agent_state_vector: Use the information for the current agent and all surrounding agent to construct a grpah. Pass an entire graph where node_features are agent_state_vector. Pass this through, let’s say SageConv. I am only interested in the result features of just one node (the current agent node).

2. Treat as semi-supervised learning problem: however, I am not interested in coming up with predictions for the unlabeled agents at all.

Specify the labels in data.y for the 5 agents. For the unlabeled 25 agents, how would I go about omitting them from data.y? Is there an index property that states I am only interested and want to train against my subset of nodes?

[rusty1s]

This is indeed a transductive learning scenario, and therefore quite similar to our examples/gcn.py example, where you perform node-level predictions for specific nodes, but are allowed to use the inputs of all the nodes. I would implement that in a similar fashion:

1. Create a mask which represents for which node you want to learn/get predictions. In your case, that would be a bool tensor of shape [30], where 5 entries are set to True.
2. Perform message passing on the full graph: `out = model(x, edge_index)
3. Train against the predictions of training nodes: loss(out[mask], y[mask])

I don' think there is any need to use an DataLoader here. For y, you may want to set labels for omitted agents to -1 or directly save y[mask] of shape [5].

[semirelezovikj]

Excellent - that is what I ended up doing as well, I was wondering whether I was off with my conceptualization.

Thank. you for reaching out - much appreciated.