Skip to content

Commit 9373d8d

Browse files
author
Alexander Ororbia
committed
revised fn and hh-cell neurocog docs, added some refs to distribution generator
1 parent 7c56b47 commit 9373d8d

File tree

4 files changed

+57
-151
lines changed

4 files changed

+57
-151
lines changed

docs/tutorials/neurocog/dynamic_synapses.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ value matrices we might initially employ (as in synapse components such as the
2222
[DenseSynapse](ngclearn.components.synapses.denseSynapse)).
2323

2424
Building a dynamic synapse can be done by importing the [exponential synapse](ngclearn.components.synapses.exponentialSynapse),
25-
the [double-exponential synapse](ngclearn.components.synapses.doubleExpSynapse), or the [alpha synapse](ngclearn.components.synapses.alphaSynapse) from ngc-learn's in-built components and setting them up within a model context for easy analysis. Go ahead and create a Python script named `probe_synapses.py` to place
25+
the [double-exponential synapse](ngclearn.components.synapses.doubleExpSynapse), or the [alpha synapse](ngclearn.components.synapses.alphaSynapse) from ngc-learn's in-built components and setting them up within a model context for easy analysis. Go ahead and create a Python script named `probe_dynamic_synapses.py` to place
2626
the code you will write within.
2727
For the first part of this lesson, we will import all three dynamic synapse models and compare their behavior.
2828
This can be done as follows (using the meta-parameters we provide in the code block below to ensure reasonable dynamics):

docs/tutorials/neurocog/fitzhugh_nagumo_cell.md

Lines changed: 23 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,7 @@ single component system made up of the Fitzhugh-Nagumo (`F-N`) cell.
1717
from jax import numpy as jnp, random, jit
1818
import numpy as np
1919

20-
from ngcsimlib.context import Context
21-
from ngclearn.utils import JaxProcess
20+
from ngclearn import Context, MethodProcess
2221
## import model-specific mechanisms
2322
from ngclearn.components.neurons.spiking.fitzhughNagumoCell import FitzhughNagumoCell
2423

@@ -40,42 +39,21 @@ with Context("Model") as model:
4039
gamma=gamma, v0=v0, w0=w0, integration_type="euler")
4140

4241
## create and compile core simulation commands
43-
advance_process = (JaxProcess()
42+
advance_process = (MethodProcess("advance")
4443
>> cell.advance_state)
45-
model.wrap_and_add_command(jit(advance_process.pure), name="advance")
4644

47-
reset_process = (JaxProcess()
45+
reset_process = (MethodProcess("reset")
4846
>> cell.reset)
49-
model.wrap_and_add_command(jit(reset_process.pure), name="reset")
5047

51-
## set up non-compiled utility commands
52-
@Context.dynamicCommand
53-
def clamp(x):
54-
cell.j.set(x)
48+
## set up non-compiled utility commands
49+
def clamp(x):
50+
cell.j.set(x)
5551
```
5652

57-
In effect, the FitzHugh–Nagumo `F-N` two-dimensional differential
58-
equation system (developed by [1] and [2]) is
59-
a useful simplification of the more intricate Hodgkin–Huxley (H-H) squid axon
60-
model, attempting to extract some of the benefits of its more detailed modeling
61-
of the spiking cellular activation and deactivation dynamics (specifically
62-
attempting to isolate the properties related to sodium/potassium ion flow
63-
from cellular properties of excitation and propagation). Notably, the `F-N`
64-
cell models membrane potential `v` with a cubic function (which facilitates
65-
self-excitation through positive feedback) in tandem with a recovery variable `w`
66-
that provides a slower form of negative feedback. The linear dynamics that govern
67-
`w` are controlled by (dimensionless) coefficients `alpha` and `beta`, which
68-
control its shift and scale, respectively (another factor `gamma` is introduced
69-
in our implementation, which divides the cubic term in the voltage dynamics, but
70-
generally this can usually be set to either a value of `1` or `3` as in [1]).
71-
The value `tau_w` controls the time constant for the recovery variable (and,
72-
technically, ngc-learn implements `tau_m` to control the membrane potential,
73-
but this is default set to `1` since [1] and [2] typically only use a time
74-
constant for the recovery variable).
75-
76-
The initial conditions for the voltage (i.e., `v0`) and the recovery (i.e., `w0`)
77-
have been set to particular interesting values above for the demonstration
78-
purposes of this tutorial but, by default, are `0` in the `F-N` cell component.
53+
In effect, the FitzHugh–Nagumo `F-N` two-dimensional differential equation system (developed by [1] and [2]) is a useful simplification of the more intricate Hodgkin–Huxley (H-H) squid axon model, attempting to extract some of the benefits of its more detailed modeling of the spiking cellular activation and deactivation dynamics (specifically attempting to isolate the properties related to sodium/potassium ion flow from cellular properties of excitation and propagation). Notably, the `F-N` cell models membrane potential `v` with a cubic function (which facilitates self-excitation through positive feedback) in tandem with a recovery variable `w` that provides a slower form of negative feedback. The linear dynamics that govern `w` are controlled by (dimensionless) coefficients `alpha` and `beta`, which control its shift and scale, respectively (another factor `gamma` is introduced in our implementation, which divides the cubic term in the voltage dynamics, but generally this can usually be set to either a value of `1` or `3` as in [1]).
54+
The value `tau_w` controls the time constant for the recovery variable (and, technically, ngc-learn implements `tau_m` to control the membrane potential, but this is default set to `1` since [1] and [2] typically only use a time constant for the recovery variable).
55+
56+
The initial conditions for the voltage (i.e., `v0`) and the recovery (i.e., `w0`) have been set to particular interesting values above for the demonstration purposes of this tutorial but, by default, are `0` in the `F-N` cell component.
7957

8058
Formally, the core dynamics of the `F-N` can be written out as follows:
8159

@@ -84,24 +62,13 @@ $$
8462
\tau_w \frac{\partial \mathbf{w}_t}{\partial t} &= \mathbf{v}_t + a - b\mathbf{w}_t
8563
$$
8664

87-
where $a$ and $b$ are factors that drive the recovery variable's dynamics
88-
(shift and scaling, respectively), $R$ is the membrane resistance, $\tau_m$ is the
89-
membrane time constant, and $\tau_w$ is the recovery time constant ($g$ is a
90-
dividing constant meant to dampen the effects of the cubic term, but is generally
91-
set to $g = 1$ to adhere to [1] and [2])
65+
where $a$ and $b$ are factors that drive the recovery variable's dynamics (shift and scaling, respectively), $R$ is the membrane resistance, $\tau_m$ is the membrane time constant, and $\tau_w$ is the recovery time constant ($g$ is a dividing constant meant to dampen the effects of the cubic term, but is generally set to $g = 1$ to adhere to [1] and [2])
9266

9367

9468
### Simulating a FitzHugh–Nagumo Neuronal Cell
9569

96-
Given that we have a single-cell dynamical system set up as above, we can next
97-
write some code for visualizing how the `F-N` node's membrane potential and
98-
coupled recovery variable evolve with time (specifically over a period of about
99-
`200` milliseconds). We will, much as we did with the leaky integrators in
100-
prior tutorials, inject an electrical current `j` into the `F-N` cell (this time
101-
just a constant current value of `0.23` amperes) and observe how the cell
102-
produces action potentials.
103-
Specifically, we can plot the neuron's voltage `v` and recovery variable `w`
104-
as follows:
70+
Given that we have a single-cell dynamical system set up as above, we can next write some code for visualizing how the `F-N` node's membrane potential and coupled recovery variable evolve with time (specifically over a period of about `200` milliseconds). We will, much as we did with the leaky integrators in prior tutorials, inject an electrical current `j` into the `F-N` cell (this time just a constant current value of `0.23` amperes) and observe how the cell produces action potentials.
71+
Specifically, we can plot the neuron's voltage `v` and recovery variable `w` as follows:
10572

10673
```python
10774
curr_in = []
@@ -119,26 +86,26 @@ time_span = np.linspace(0, 200, num=T)
11986
dt = time_span[1] - time_span[0] # ~ 0.13342228152101404 ms
12087

12188
time_span = []
122-
model.reset()
89+
reset_process.run()
12390
t = 0.
12491
for ts in range(T):
12592
x_t = data
12693
## pass in t and dt and run step forward of simulation
127-
model.clamp(x_t)
128-
model.advance(t=t, dt=dt)
94+
clamp(x_t)
95+
advance_process.run(t=t, dt=dt)
12996
t = t + dt
13097

13198
## naively extract simple statistics at time ts and print them to I/O
132-
v = cell.v.value
133-
w = cell.w.value
134-
s = cell.s.value
99+
v = cell.v.get()
100+
w = cell.w.get()
101+
s = cell.s.get()
135102
curr_in.append(data)
136103
mem_rec.append(v)
137104
recov_rec.append(w)
138105
spk_rec.append(s)
139106
## print stats to I/O (overriding previous print-outs to reduce clutter)
140107
print("\r {}: s {} ; v {} ; w {}".format(ts, s, v, w), end="")
141-
time_span.append((ts)*dt)
108+
time_span.append(ts * dt)
142109
print()
143110

144111
import matplotlib #.pyplot as plt
@@ -169,38 +136,12 @@ plt.tight_layout()
169136
plt.savefig("{0}".format("fncell_plot.jpg"))
170137
```
171138

172-
You should get a plot that depicts the evolution of the voltage and recovery,
173-
i.e., saved as `fncell_plot.jpg` locally to disk, like the one below:
139+
You should get a plot that depicts the evolution of the voltage and recovery, i.e., saved as `fncell_plot.jpg` locally to disk, like the one below:
174140

175141
<img src="../../images/tutorials/neurocog/fncell_plot.jpg" width="400" />
176142

177-
A useful note is that the `F-N` above used Euler integration to step through its
178-
dynamics (this is the default/base routine for all cell components in ngc-learn);
179-
however, one could configure it to use the midpoint method for integration
180-
by setting its argument `integration_type = rk2` in cases where more
181-
accuracy in the dynamics is needed (at the cost of additional computational time).
182-
183-
## Optional: Setting Up The Components with a JSON Configuration
184-
185-
While you are not required to create a JSON configuration file for ngc-learn,
186-
to get rid of the warning that ngc-learn will throw at the start of your
187-
program's execution (indicating that you do not have a configuration set up yet),
188-
all you need to do is create a sub-directory for your JSON configuration
189-
inside of your project code's directory, i.e., `json_files/modules.json`.
190-
Inside the JSON file, you would write the following:
191-
192-
```json
193-
[
194-
{"absolute_path": "ngclearn.components",
195-
"attributes": [
196-
{"name": "FitzHughNagumoCell"}]
197-
},
198-
{"absolute_path": "ngcsimlib.operations",
199-
"attributes": [
200-
{"name": "overwrite"}]
201-
}
202-
]
203-
```
143+
A useful note is that the `F-N` above used Euler integration to step through its dynamics (this is the default/base routine for all cell components in ngc-learn); however, one could configure it to use the midpoint method for integration by setting its argument `integration_type = rk2` in cases where more accuracy in the dynamics is needed (at the cost of additional computational time).
144+
204145

205146
## References
206147

docs/tutorials/neurocog/hodgkin_huxley_cell.md

Lines changed: 26 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,12 @@
11
# Lecture 2E: The Hodgkin-Huxley Cell
22

3-
In this tutorial, we will study/setup one of the most important biophysical
4-
neuronal models in computational neuroscience -- the Hodgkin-Huxley (H-H) spiking
5-
cell model.
3+
In this tutorial, we will study/setup one of the most important and sophisticated biophysical neuronal models in computational neuroscience -- the Hodgkin-Huxley (H-H) spiking cell model.
64

75
## Using and Probing the H-H Cell
86

9-
Go ahead and make a new folder for this study and create a Python script,
10-
i.e., `run_hhcell.py`, to write your code for this part of the tutorial.
7+
Go ahead and make a new folder for this study and create a Python script, i.e., `run_hhcell.py`, to write your code for this part of the tutorial.
118

12-
Now let's set up the controller for this lesson's simulation and construct a
13-
single component system made up of an H-H cell.
9+
Now let's set up the controller for this lesson's simulation and construct a single component system made up of an H-H cell.
1410

1511

1612
### Instantiating the H-H Neuronal Cell
@@ -22,9 +18,7 @@ H-H cell amounts to the following:
2218
from jax import numpy as jnp, random, jit
2319
import numpy as np
2420

25-
from ngclearn.utils.model_utils import scanner
26-
from ngcsimlib.context import Context
27-
from ngclearn.utils import JaxProcess
21+
from ngclearn import Context, MethodProcess
2822
## import model-specific mechanisms
2923
from ngclearn.components.neurons.spiking.hodgkinHuxleyCell import HodgkinHuxleyCell
3024

@@ -52,18 +46,15 @@ with Context("Model") as model:
5246
)
5347

5448
## create and compile core simulation commands
55-
advance_process = (JaxProcess()
49+
advance_process = (MethodProcess("advance")
5650
>> cell.advance_state)
57-
model.wrap_and_add_command(jit(advance_process.pure), name="advance")
5851

59-
reset_process = (JaxProcess()
52+
reset_process = (MethodProcess("reset")
6053
>> cell.reset)
61-
model.wrap_and_add_command(jit(reset_process.pure), name="reset")
6254

63-
## set up non-compiled utility commands
64-
@Context.dynamicCommand
65-
def clamp(x):
66-
cell.j.set(x)
55+
## set up non-compiled utility commands
56+
def clamp(x):
57+
cell.j.set(x)
6758
```
6859

6960
Notably, the H-H model is a four-dimensional differential equation system, invented in 1952
@@ -88,15 +79,12 @@ $$
8879
\frac{\partial \mathbf{h}_t}{\partial t} &= \alpha_h(\mathbf{v}_t) * (1 - \mathbf{h}_t) - \beta_h(\mathbf{v}_t) * \mathbf{h}_t
8980
$$
9081

91-
where we observe that the above four-dimensional set of dynamics is composed of nonlinear ODEs. Notice that, in each gate or channel probability ODE, there are two generator functions (each of which is a function of the membrane potential $\mathbf{v}_t$) that produces the necessary dynamic coefficients at time $t$; $\alpha_x(\mathbf{v}_t)$ and $\beta_x(\mathbf{v}_t)$ produce different biopphysical weighting values depending on which channel $x = \{n, m, h\}$ they are related to.
82+
where we observe that the above four-dimensional set of dynamics is composed of nonlinear ODEs. Notice that, in each gate or channel probability ODE, there are two generator functions (each of which is a function of the membrane potential $\mathbf{v}_t$) that produces the necessary dynamic coefficients at time $t$; $\alpha_x(\mathbf{v}_t)$ and $\beta_x(\mathbf{v}_t)$ produce different biophysical weighting values depending on which channel $x = \{n, m, h\}$ they are related to.
9283
Note that, in ngc-learn's implementation of the H-H cell model, most of the core coefficients have been generally set according to Hodgkin and Huxley's 1952 work but can be configured by the experimenter to obtain different kinds of behavior/dynamics.
9384

9485
### Simulating the H-H Neuronal Cell
9586

96-
To see how the H-H cell works, we next write some code for visualizing how
97-
the node's membrane potential and core related gates/channels evolve with time
98-
(over a period of about `200` milliseconds). We will inject a square input pulse current
99-
into our H-H cell (specifically into its `j` compartment) and observe how the cell behaves in response.
87+
To see how the H-H cell works, we next write some code for visualizing how the node's membrane potential and core related gates/channels evolve with time (over a period of about `200` milliseconds). We will inject a square input pulse current into our H-H cell (specifically into its `j` compartment) and observe how the cell behaves in response.
10088
Specifically, we simulate the injection of this kind of current via the code below:
10189

10290
```python
@@ -112,26 +100,25 @@ v = []
112100
n = []
113101
m = []
114102
h = []
115-
model.reset()
103+
reset_process.run()
116104
for ts in range(x_seq.shape[1]):
117105
x_t = jnp.array([[x_seq[0, ts]]]) ## get data at time t
118-
model.clamp(x_t)
119-
model.run(t=ts * dt, dt=dt)
120-
outs.append(a.s.value)
121-
n.append(cell.n.value[0, 0])
122-
m.append(cell.m.value[0, 0])
123-
h.append(cell.h.value[0, 0])
124-
v.append(cell.v.value[0, 0])
125-
print(f"\r {ts} v = {cell.v.value}", end="")
106+
clamp(x_t)
107+
advance_process.run(t=ts * dt, dt=dt)
108+
outs.append(cell.s.get())
109+
n.append(cell.n.get()[0, 0])
110+
m.append(cell.m.get()[0, 0])
111+
h.append(cell.h.get()[0, 0])
112+
v.append(cell.v.get()[0, 0])
113+
print(f"\r {ts} v = {cell.v.get()}", end="")
126114
time_span.append(ts*dt)
127115
outs = jnp.concatenate(outs, axis=1)
128116
v = jnp.array(v)
129117
time_span = jnp.array(time_span)
130118
outs = jnp.squeeze(outs)
131119
```
132120

133-
and we can plot the dynamics of the neuron's voltage `v` and its three gate/channel
134-
variables, `h`, `m`, and `n`, with the following:
121+
and we can plot the dynamics of the neuron's voltage `v` and its three gate/channel variables, `h`, `m`, and `n`, with the following:
135122

136123
```python
137124
import matplotlib.pyplot as plt
@@ -161,9 +148,7 @@ plt.savefig("{0}".format("hh_plot.jpg"))
161148
plt.close()
162149
```
163150

164-
You should get a compound plot that depict the evolution of the H-H cell's voltage
165-
and channel/gate variables, i.e., saved as `hh_plot.jpg` locally to
166-
disk, like the one below:
151+
You should get a compound plot that depict the evolution of the H-H cell's voltage and channel/gate variables, i.e., saved as `hh_plot.jpg` locally to disk, like the one below:
167152

168153
```{eval-rst}
169154
.. table::
@@ -176,38 +161,11 @@ disk, like the one below:
176161
+--------------------------------------------------------+
177162
```
178163

179-
A useful note is that the H-H cell above used Euler integration to step through its
180-
dynamics (this is the default/base routine for all cell components in ngc-learn).
181-
However, one could configure the cell to use the midpoint method for integration
182-
by setting its argument `integration_type = rk2` or the Runge-Kutta fourth-order
183-
routine via `integration_type=rk4` for cases where, at the cost of increased
184-
compute time, more accurate dynamics are possible.
185-
186-
## Optional: Setting Up The Components with a JSON Configuration
187-
188-
While you are not required to create a JSON configuration file for ngc-learn,
189-
to get rid of the warning that ngc-learn will throw at the start of your
190-
program's execution (indicating that you do not have a configuration set up yet),
191-
all you need to do is create a sub-directory for your JSON configuration
192-
inside of your project code's directory, i.e., `json_files/modules.json`.
193-
Inside the JSON file, you would write the following:
194-
195-
```json
196-
[
197-
{"absolute_path": "ngclearn.components",
198-
"attributes": [
199-
{"name": "HodgkinHuxleyCell"}]
200-
},
201-
{"absolute_path": "ngcsimlib.operations",
202-
"attributes": [
203-
{"name": "overwrite"}]
204-
}
205-
]
206-
```
164+
A useful note is that the H-H cell above used Euler integration to step through its dynamics (this is the default/base routine for all cell components in ngc-learn).
165+
However, one could configure the cell to use the midpoint method for integration by setting its argument `integration_type = rk2` or the Runge-Kutta fourth-order routine via `integration_type=rk4` for cases where, at the cost of increased compute time, more accurate dynamics are possible.
166+
207167

208168
## References
209169

210-
<b>[1]</b> Hodgkin, Alan L., and Andrew F. Huxley. "A quantitative description
211-
of membrane current and its application to conduction and excitation in nerve."
212-
The Journal of physiology 117.4 (1952): 500.
170+
<b>[1]</b> Hodgkin, Alan L., and Andrew F. Huxley. "A quantitative description of membrane current and its application to conduction and excitation in nerve." The Journal of physiology 117.4 (1952): 500.
213171

ngclearn/utils/distribution_generator.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,10 @@ def fan_in_uniform(
173173
The values are sampled from a uniform distribution in the range [-limit, limit],
174174
where limit = sqrt(1 / fan_in), and fan_in is inferred from the shape.
175175
176+
| Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural
177+
| networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics.
178+
| JMLR Workshop and Conference Proceedings, 2010.
179+
176180
Args:
177181
**params: extra distribution parameters
178182
@@ -233,6 +237,9 @@ def fan_in_gaussian(
233237
The values are sampled from a normal distribution with mean 0 and stddev = sqrt(1 / fan_in),
234238
where fan_in is inferred from the shape.
235239
240+
| He, Kaiming, et al. "Delving deep into rectifiers: Surpassing human-level performance on imagenet
241+
| classification." Proceedings of the IEEE international conference on computer vision. 2015.
242+
236243
Args:
237244
**params: extra distribution parameters
238245

0 commit comments

Comments
 (0)