model_utils and rate cell (#70)

* Patched synapses added

* Update __init__.py

* Update patch_utils.py

patch_with_stride & patch_with_overlap functions + Create_Patches class added

* Update patchedSynapse.py

* Update hebbianPatchedSynapse.py

* Update synapse_plot.py

order added

* Create hierarchical_sc.md

1

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update sparse_coding.md

* Update sparse_coding.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Delete docs/images/hgpc_network.pdf

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create hgpc

* Delete docs/images/museum/hgpc

* Create d

* Add files via upload

* Delete docs/images/hgpc_model.png

* Delete docs/images/museum/hgpc/d

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Delete docs/images/museum/hgpc/Input_layer.png

* Add files via upload

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Update hierarchical_sc.md

* Create Generative_PC.md

* Update and rename Generative_PC.md to generative_pc.md

* Update generative_pc.md

* Update generative_pc.md

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update model_utils.py

* Update rateCell.py

* Update generative_pc.md

* Create pc-sindy.md

* Update pc-sindy.md

* Update model_utils.py

sine activation function added

* Update model_utils.py

* Update ode_utils.py

jitified

* Delete docs/museum/hierarchical_sc.md

* Delete docs/museum/generative_pc.md

* Delete ngclearn/components/synapses/patched directory

* Update __init__.py

* Add files via upload

ode with scanner added

* Update ode_solver.py

_ removed

Author: Faezehabibi

docs/images/museum/hgpc/HGPC_inputL.jpg

@@ -0,0 +1,34 @@
+# Sparse Identification of Non-linear Dynamical Systems with Predictive Coding (PC-SINDy)
+
+In this section, we teach, create, simulate, and visualize SINDy model with Predictive Coding (PC-SINDy) using NGC-Learn library components.
+
+
+
+
+
+
+
+The model **code** for this exhibit can be found [here](https://github.com/NACLab/pc_sindy.py).
+
+
+## Predictive Coding (PC)
+PC is a biological plausible learning algorithm that learns the effective representation and transformations from the data. 
+
+
+## Sparse Identification of Non-linear Dynamical Systems (SINDy)
+SINDy is a data-driven algorithm that finds the derivative of dynamical systems in terms of a symbolic equation of the system's state vector. 
+SINDy describes the derivative (linear operation acting on △t) as linear transformations
+of a manually constructed dictionary from the state vector by a coefficient matrix.
+Dictionary learning combined with LASSO (L1-norm) promotes the sparsity of the coefficient matrix
+which allows only governing terms in the dictionary stay non-zero.
+
+Learning the 
+
+
+## SINDy with Predictive Coding 
+
+<p align="center">
+  <img src="" width="550" title="">
+</p>
+
+## Predictive Coding Model Dynamics

docs/images/museum/hgpc/Input_layer.png

@@ -68,7 +68,7 @@ $$
 
 where we see that we aim to learn a two-layer generative system that specifically
 imposes a prior distribution `p(z)` over the latent feature detectors (via the 
-constraint function $\Omega\big(\mathbf{z}(t)\big)$) that we hope
+constraint function $ \Omega\big(\mathbf{z}(t)\big) $ ) that we hope
 to extract in node `z`. Note that this two-layer model (or single latent-variable layer
 model) could either be the linear generative model from <b>[1]</b> or one similar to the
 model learned through ISTA <b>[2]</b> if a (soft) thresholding function is used instead.

docs/images/museum/hgpc/hgpc_model.png

@@ -177,7 +177,12 @@ def __init__(self, name, n_units, tau_m, prior=("gaussian", 0.), act_fx="identit
         self.shape = shape
         self.n_units = n_units
         self.batch_size = batch_size
-        self.fx, self.dfx = create_function(fun_name=act_fx)
+
+        
+        omega_0 = None
+        if act_fx == "sine":
+            omega_0 = kwargs["omega_0"]
+        self.fx, self.dfx = create_function(fun_name=act_fx, args=omega_0)
 
         # compartments (state of the cell & parameters will be updated through stateless calls)
         restVals = jnp.zeros(_shape)

docs/museum/pc-sindy.md

@@ -21,6 +21,6 @@
 ## modulated synaptic components
 from .modulated.MSTDPETSynapse import MSTDPETSynapse
 ## patched synaptic components
-from .patched.patchedSynapse import PatchedSynapse
-from .patched.staticPatchedSynapse import StaticPatchedSynapse
-from .patched.hebbianPatchedSynapse import HebbianPatchedSynapse
+## from .patched.patchedSynapse import PatchedSynapse
+## from .patched.staticPatchedSynapse import StaticPatchedSynapse
+## from .patched.hebbianPatchedSynapse import HebbianPatchedSynapse