Sequential NeuralNetwork

[mop9047]

Adds a TimeSeries Feature so sequences of data can be used in ML5

This version extends from the Neural Network class with some overrides to allow sequences of data. It also includes some helper functions to simplify loading timeseries data.

Key Features:

• addData in the format of Array of Objects [{x: ,y: },{x: ,y: }] or Array of Arrays [[,],[,]]
• addDefaultLayers which uses tsLayers.js file to define the architectures, based on task and dataMode (still needs to be discussed).
• padCoordinates(sequence, targetLength) uses RDP algorithm to simplify coordinate data to a specified target length
• slidingWindow(data, featureKeys, targetKeys, batchLength = null) and sampleWindow(data) uses sliding Window algorithm to create batches from sequence data and take a single sample for prediction

[gohai]

Regular NeuralNetwork leaves this to the sketch to set, so I'd suggest we do the same. (Or, change NN in the same manner.)

Too long for me to recall, but there probably was some discussion whether or not to handle this transparently for the user, or not, that's worth unearthing.

[mop9047]

I agree, I'll leave this comment unresolved in this case, so it doesn't get forgotten

[shiffman]

Some additional comments along the lines of @gohai's review! Also, I see this PR still includes ml5.timeSeries(), my memory is shaky of course, but I recall a discussion around this implementation being a part of ml5.neuralNetwork() with a new "time series" task? Is that correct?

[gohai]

Some additional comments along the lines of @gohai's review! Also, I see this PR still includes ml5.timeSeries(), my memory is shaky of course, but I recall a discussion around this implementation being a part of ml5.neuralNetwork() with a new "time series" task? Is that correct?

My memory is also shaky, @shiffman - but I like the idea. Since the options are being passed to the factory method, we could keep the inheritance model that Mathew built, and have ml5.neuralNetwork() simply instantiate the specialized class whenever encounters a related task option. The alternative is to integrate Mathew's work even more tightly into the existing NeuralNetwork classes. Do you have a preference @mop9047?

[mop9047]

Some additional comments along the lines of @gohai's review! Also, I see this PR still includes ml5.timeSeries(), my memory is shaky of course, but I recall a discussion around this implementation being a part of ml5.neuralNetwork() with a new "time series" task? Is that correct?

My memory is also shaky, @shiffman - but I like the idea. Since the options are being passed to the factory method, we could keep the inheritance model that Mathew built, and have ml5.neuralNetwork() simply instantiate the specialized class whenever encounters a related task option. The alternative is to integrate Mathew's work even more tightly into the existing NeuralNetwork classes. Do you have a preference @mop9047?

Yes! I do remember having that discussion, but I don't clearly remember if we had a decision on which method to do. I definitely think its worth trying out and I have created a new branch to test it here.

However, there are some changes this method necessitates: I have tried implementing DIY time series and DIY neural network classes and the factory functions in the neural network factory function so it is all under one neuralNetwork call. However since the DIY time series class depends on an import of the DIY neural network class, it is not possible to call the DIY time series class inside of the neural network factory function as it needs to import the DIY ts class in the neural network file. This causes a circular dependency as imports of the other files are present. Thus, in the initial commit on the branch I provided, removed the neuralNetwork factory function from the neural network file and moved it all the way to the time series file to prevent this conflict. So far, all of the examples have worked with this method.

I have yet to try another method where there is a separate file for the neural network factory function, so there is no circular dependencies, however I am not sure where in the file system to put it in and a good naming convention for it. Let me know if you need any more clarification for this!

[mop9047]

@mop9047 I worked on clarifying the mouse-gesture-rdp example. Please have a look and holler if you rather do things differently. Changed it to interactively visualize the result of the RDP algorithm in real-time. Because of this, the background needs to be cleared every frame in any case - which quite simplified the overall structure (in my eyes)...

I believe we could drop the extra rdp-visualizer example - and merely link to the video explaining it.

Hi @gohai it looks pretty good! lets keep that and I'll remove the rdp-visualizer.

[gohai]

@shiffman @MOQN Very unimportant, but I thought I raise this here before I forget: was wondering if we prefer neuralNetwork-sequence-... or a (shorter) neuralSequence-...? (Feeling the former is truer to the hierarchy - but maybe too long?)