@@ -155,39 +155,40 @@ trainable Fourier features. Here, we underline that the choice of
155155:math: `\sigma _i` is problem dependent and typical values can be
156156:math: `1 , 10 , 100 ,` etc.
157157
158- Spatio-temporal Fourier Feature Network
159- ----------------------------------------
160-
161- For time-dependent problems, multi-scale behavior may exist not only
162- across spatial directions but also across time. The authors
163- [#wang2021eigenvector ]_ proposed another novel multi-scale
164- Fourier feature architecture to tackle multi-scale problems in
165- spatio-temporal domains. Specifically, the feed-forward pass of the
166- network is now defined as
167-
168- .. math ::
169-
170- \begin {aligned}
171- &\phi _{E}^{(x_i)}(x_i)=[\sin (2 \pi \mathbf {f}^{(x_i)} \times x_i) ; \cos (2 \pi \mathbf {f}^{(x_i)} \times \mathbf {x}_i)]^{T}, \\
172- & \phi _{E}^{(t)}(t)=[\sin (2 \pi \mathbf {f}^{(t)} \times t) ; \cos (2 \pi \mathbf {f}^{(t)} \times x_i)]^{T}, \\
173- & \mathbf {H}^{(x_i)}_1 = \sigma (\mathbf {W}_1 \cdot \phi _{E}^{(x_i)}(x_i) + \mathbf {b}_1 ),
174- \quad \text { for } i=1 , 2 , \dots , d,\\
175- & \mathbf {H}^{(t)}_1 = \sigma (\mathbf {W}_1 \cdot \phi _{E}^{(t)}(t) + \mathbf {b}_1 ),\\
176- & \mathbf {H}_{\ell }^{(x_i)} = \sigma (\mathbf {W}_\ell \cdot \mathbf {H}^{(x_i)}_{\ell -1 } + \mathbf {b}_\ell ), \quad \text { for } \ell =2 , \dots , L \text { and } i=1 ,2 , \dots , d,\\
177- & \mathbf {H}^{(t)}_{\ell } = \sigma (\mathbf {W}_\ell \cdot \mathbf {H}^{(t)}_{\ell -1 } + \mathbf {b}_\ell ), \quad \text { for } \ell =2 , \dots , L, \\
178- & \mathbf {H}_{L} = \prod _{i=1 }^d H^{(x_i)}_{L} \cdot H^{(t)}_{L} , \\
179- & \mathbf {u}_{net}(\mathbf {x}, t; {\mathbf {\theta }}) = \mathbf {W}_{L+1 } \cdot \mathbf {H}_{L} + \mathbf {b}_{L+1 },\end {aligned}
180-
181- where :math: `\phi _{E}^{(x_i)}` and :math: `\phi _{E}^{(t)}` denote spatial
182- and temporal Fourier feature mappings, respectively, and :math: `\odot `
183- represents the point-wise multiplication. Here, each entry of
184- :math: `\mathbf {f}^{(x_i)}` and :math: `\mathbf {f}^{(t)}` can be sampled
185- from different Gaussian distributions. One key difference from the
186- multi-scale Fourier feature network is that separate Fourier feature
187- embeddings are applied to spatial and temporal input coordinates before
188- passing the embedded inputs through the same fully-connected network.
189- Another key difference is that network outputs are merged using
190- point-wise multiplication and passing them through a linear layer.
158+ ..
159+ Spatio-temporal Fourier Feature Network
160+ ----------------------------------------
161+
162+ For time-dependent problems, multi-scale behavior may exist not only
163+ across spatial directions but also across time. The authors
164+ [#wang2021eigenvector]_ proposed another novel multi-scale
165+ Fourier feature architecture to tackle multi-scale problems in
166+ spatio-temporal domains. Specifically, the feed-forward pass of the
167+ network is now defined as
168+
169+ .. math::
170+
171+ \begin{aligned}
172+ &\phi_{E}^{(x_i)}(x_i)=[\sin (2 \pi \mathbf{f}^{(x_i)} \times x_i) ; \cos (2 \pi \mathbf{f}^{(x_i)} \times \mathbf{x}_i)]^{T}, \\
173+ & \phi_{E}^{(t)}(t)=[\sin (2 \pi \mathbf{f}^{(t)} \times t) ; \cos (2 \pi \mathbf{f}^{(t)} \times x_i)]^{T}, \\
174+ & \mathbf{H}^{(x_i)}_1 = \sigma(\mathbf{W}_1 \cdot \phi_{E}^{(x_i)}(x_i) + \mathbf{b}_1),
175+ \quad \text{ for } i=1, 2, \dots, d,\\
176+ & \mathbf{H}^{(t)}_1 = \sigma(\mathbf{W}_1 \cdot \phi_{E}^{(t)}(t) + \mathbf{b}_1),\\
177+ & \mathbf{H}_{\ell}^{(x_i)} = \sigma(\mathbf{W}_\ell \cdot \mathbf{H}^{(x_i)}_{\ell-1} + \mathbf{b}_\ell), \quad \text{ for } \ell=2, \dots, L \text{ and } i=1,2, \dots, d,\\
178+ & \mathbf{H}^{(t)}_{\ell} = \sigma(\mathbf{W}_\ell \cdot \mathbf{H}^{(t)}_{\ell-1} + \mathbf{b}_\ell), \quad \text{ for } \ell=2, \dots, L, \\
179+ & \mathbf{H}_{L} = \prod_{i=1}^d H^{(x_i)}_{L} \cdot H^{(t)}_{L} , \\
180+ & \mathbf{u}_{net}(\mathbf{x}, t; {\mathbf{\theta}}) = \mathbf{W}_{L+1} \cdot \mathbf{H}_{L} + \mathbf{b}_{L+1},\end{aligned}
181+
182+ where :math:`\phi_{E}^{(x_i)}` and :math:`\phi_{E}^{(t)}` denote spatial
183+ and temporal Fourier feature mappings, respectively, and :math:`\odot`
184+ represents the point-wise multiplication. Here, each entry of
185+ :math:`\mathbf{f}^{(x_i)}` and :math:`\mathbf{f}^{(t)}` can be sampled
186+ from different Gaussian distributions. One key difference from the
187+ multi-scale Fourier feature network is that separate Fourier feature
188+ embeddings are applied to spatial and temporal input coordinates before
189+ passing the embedded inputs through the same fully-connected network.
190+ Another key difference is that network outputs are merged using
191+ point-wise multiplication and passing them through a linear layer.
191192
192193.. _sirens :
193194
0 commit comments