Add permutational layer to LSTM

[Takusei]

Dear @off99555

I am a student new to this problem.

I'm trying to build LSTM with input feature permutation invariant, for example, the feature is a set of position, how can I use this module to deal with LSTM?

Could you help me with this?

Best regards

[offchan42]

You mean you observe a bunch of points over time?

I suggest you clone the repo and try running the permutational_layer.py. There is an example at the bottom of the file. If you try playing with the code, printing the shape, or any info of the model as you add an extra layer, you will get it faster.

After you understand how it works, I suggest you use PermutationalModule function to create a model to encode your input. Then you feed the encoding to your LSTM layers.

[Takusei]

Thanks for your quick replay, and apology for my delay.

OK, I understand the way this code works, I suppose that it’s an encoder for the input sets.
I noticed that the input shape of data in PermutationalModule() should be the same length, for example:

PermutationalModule((4,), 3, [repeat_layers(Dense, [2, 4]), repeat_layers(Dense, [8, 16])],], last_layer_pooling=maximum)
the input data should have 3 sets, and each sets have 4 features.

I’m wonder that can this model be modified to input data with variable sets numbers? Such as , not 3 sets but input 2 sets sometimes? Could this be possible?

Thanks

[offchan42]

First of all, let define what you mean by set so that we are on the same page.
My set means a collection of objects. And each object has some features.
In the above code that you mentioned, it can be interpreted as a set of 3 points, each point having 4 features. Each feature is simply a float e.g. positionX, positionY, velocityX, velocityY.

So it means that there is only one set, not many sets.

I think that you might mean 3 objects, not 3 sets. Am I right?

[offchan42]

Also varying the number of objects is not possible. What you can do is to send features of 0 in the place of an object that is missing. E.g. if you have 2 objects, you can send 0,0,0,0 as features of the 3rd object.
This will still work fine.