low amplitude component problem

[AlexG9908]

Thanks for sharing the code!
I've got some questions on FBPINN to solve nonlinear problem, the u solved with PINN contains multiple frequencies. Although FBPINN can cope well with high frequencies, there is a big difference in the amplitude of the different frequency components, so the solution has a big error to the low amplitude component, e.g. the actual value of u is the 20Hz component with amplitude 1 + the 1Hz component with amplitude 0.001.
I was wondering if FBPINN could add learning rate decay to learn components with smaller amplitudes later in the training process. Or there are other ways to do it. Thanks.

[benmoseley]

I think in general this is a challenging problem as convergence behaviour across amplitudes/frequencies is complex interaction between the loss, optimiser, and network architecture. One (fairly easy to implement) approach to target specific frequencies might be sequentially learning the levels in the multilevel FBPINN domain decomposition (https://github.com/benmoseley/FBPINNs/blob/main/fbpinns/decompositions.py#L337), using subdomain scheduling https://github.com/benmoseley/FBPINNs/blob/main/fbpinns/schedulers.py. I am not sure it is easy to add a specific learning rates by solution frequency, but you could also add specific learning rates for each level in the multilevel FBPINN in the optax optimiser, as a proxy (this isn't implemented currently).