Triadic Memory implementation in Odin

[222464]

Hi,

I found the Triadic Memory idea interesting, so I implemented a version of it in Odin (a C-replacement language).

It follows the C version for the most part. So far it only includes a basic test of the temporal memory, more to come.

Check it out here

[PeterOvermann]

That's really cool! Please send a pull request, and I'll include the Odin version in the main repository...

[222464]

Hi again, thanks for accepting the PR. I have some additional updates.

I found that the triadic memory itself works quite well, but the temporal memory (at least as it was in the C version) doesn't work very well. So, I made my own variant. It is available here.. It seems to predict much better overall. It doesn't OR together SDR's at all, instead it auto-encodes both the input and the context (previous hidden) into a new hidden state. The M2 module is similar to before. I also found that the "early stop" on the learning leads to decreased performance, so I removed it and capped the triadic memory values to prevent overflow. It still uses random anchor SDRs, which works for now but should probably be improved to handle local sensitivity better.

The new main.odin loads a text file and runs this new temporal memory through it, character by character. It then generates more text. It works, although not to the level of my AI system (AOgmaNeo) quite yet (which can remember 80,000 digits of pi). Still, it was fun to experiment with, and I look forward to experimenting more with it soon.

[PeterOvermann]

I think what you're doing in your variant is building some kind of a linked list. With this, you can store any sequence, only limited by memory capacity. Can it make a prediction starting somewhere in the middle of a sequence?

[222464]

I haven't tested it yet on that specifically, but I don't see why not