Symbolic Regression for Generic Systems- M1

[Rodolphe-Vivant]

Symbolic Regression for Generic Systems– M1

Student : Vivant Rodolphe
Supervisors : Dr Emmanuel Franck, Dr Andrea Thomann, Dr Michel Victor-Dansac
Hosting institution : Institut de Recherche Mathématique Avancée (I.R.M.A)
Time Period : June-July 2025

Context

System identification is a crucial task in the modeling of dynamical systems like in our case :

• $x(t)$: state vector
• $\dot{x}(t)$: time derivative
• $f(x)$: nonlinear dynamics

It allows us to study the properties of the system such as important parameters like Fourier frequencies or exponential decays. In our context of machine learning, system identification can be improved by the possibility of modeling complex, nonlinear systems and sometimes uncovering hidden relationships in the data.
Symbolic regression is a powerful state-of-the-art tool for discovering mathematical expressions that describe the behavior of complex systems.
The main objective of this internship of 1st year of my Master degree is to implement the SINDy and ADAM-SINDy methods in the
Scimba library in order to identify mathematical models of generic systems.

SINDy

From a set of observed data,

and a library of candidate functions:

where A, B, C, D, E, F, G are the fixed non-linear parameters $\Lambda$ of these functions.

The goal is to find a sparse vector $\theta$ of a model that can describe the system's behavior such that :

with sparsity constraint.

Adam-SINDy

This method aims to overcome the limitations of the fixed basis functions in the SINDy method by simultaneously optimizing both linear and nonlinear parameters.

• Minimize the loss function over $\Theta$
• Maximize over $\Gamma$ or $\lambda$
• Use $\Gamma$ to control each candidate function's contribution individually

GENERIC Formalism

The GENERIC framework describes both conservative and dissipative evolution:

with degeneracy constraints:

• (E): Energy
• (S): Entropy
• (L): Skew-symmetric Poisson matrix (conservative)
• (M): Symmetric semi-definite matrix (dissipative)

Equation of energy and entropy:

Final Loss:

• Minimize over $\Theta_E$, $\Lambda$, $M$ and $L$, and maximize over $\Gamma$ or $\lambda$

Results

Example 1: Harmonic Oscillator

Theoritical equation

SINDy results

• Candidate functions

  • $\Omega_1 = [x_2]$
  • $\Omega_2 = [x_1, x_2 \otimes \cos(0.75, x_1)]$

• Computed system of equations

• Plots

Comparison between theoritical and computed model with SINDy for the Harmonic Oscillator

Very good approximation, with a small error of 10−3

Adam-SINDy results

• Computed system of equation

• Plots

Comparison between theoritical and computed model with Adam for the Harmonic Oscillator

Very good approximation, with a small error of 10−3

Example 1: Damped nonlinear Oscillator

Theoritical equation

Energy:

SINDy results

• Candidate functions

  • $\Omega_1 = [p]$
  • $\Omega_2 = [p, \sin(q)]$
  • $\Omega_3 = [p^2]$

• Computed system of equations

• Plots

Comparison between theoritical and computed model with SINDy of the Damped nonlinear Oscillator

Very good approximation, with a small error of 10−3

• Energy equation trough the Generic formalism

Again, very good approximation, with a small error of 10−3

Adam-SINDy results

• Computed system of equations

• Plots

Comparison between theoritical and computed model with Adam for the Damped nonlinear Oscillator

Very good approximation of the main dynamic
Unexpected terms for the symbolic formula of p(t)

• Plot of the computed Energy equation trough Generic formalism

Comparison between theoritical and computed model for the energy

Difficulty to capture the right behaviour for the energy

Conclusion

In the future, it would be interesting to study why combining the Adam-SINDy method with the GENERIC formalism failed to recover the correct expression for the energy. This might be due to finding the appropriate training parameters or possibly an error in the implementation itself.
The next step in the development would be to incorporate noise handling for the analysis of real data.
Such methods are already implemented in Scimba, but they need to be combined with the current Symbolic Regression techniques.

Liens utiles

• 📄 [Report](lien))
• 🎤 [Presentation Slides](lien)
• 👔 [My CV](lien) (optionnel)