This repository contains the numerical examples of my Master thesis Model order reduction using group convolutional autoencoders and implements group-equivariant convolutional autoencoders combined with structure-preserving model order reduction (MOR) techniques for parametric dynamical systems. The primary application is the 2D wave equation, where rotational symmetry (C4/C8 equivariance) is exploited to improve generalization across different flow directions.
The core idea is to train a neural network autoencoder on waves propagating in one direction and generalize to perpendicular directions without retraining.
.
├── equiv_networks/ # Core library: network architectures and MOR models
│ ├── autoencoders.py # Autoencoder architecture definitions
| ├── early_stopping.py # Early stopping procedure via checkpointing
| ├── trainer.py # Training prodedure of the autoencoders
│ └── models/
| ├── general_utilities.py # Helpers that are employed in all files in the folder
│ ├── manifold_galerkin_utilities_IMR.py # manifold Galerkin quasi-Newton solver
│ ├── manifold_lspg_utilities_IMR.py # manifold LSPG quasi-Newton solver
| └── nonlinear_manifolds.py # MOR wrapper around the autoencoder
├── tests/
| ├── 45wave/
| ├── checkpoints/
| ├── mor_results
| ├── network_parameters/
| ├── scaling/
| ├── snapshots/grid/
| ├── experiment_setup.py
| ├── proj_error_AE.py
| ├── train_wave.py
└── wave_create_snapshots.py
| └─── 90wave/
| ├── AE_results/
| ├── checkpoints/
| ├── CL_results/
| ├── mor_results/
| ├── network_parameters/
| ├── pod_results/
| ├── scaling/
| ├── snapshots_grid/
| ├── scaling/
| ├── compute_cl_basis.py # Compute Cotangent Lift reduced basis
| ├── compute_pod_basis.py # Compute POD reduced basis
| ├── experiment_setup.py # Experiment configuration and FOM setup
| ├── proj_error_AE.py # Projection error: autoencoder
| ├── proj_error_cl.py # Projection error: Cotangent Lift
| ├── proj_error_pod.py # Projection error: POD
│ ├── test_wave_cl_sg.py # ROM test: CL + symplectic Galerkin projection
│ ├── test_wave_manifold_galerkin.py # ROM test: AE + (symplectic) manifold Galerkin
│ ├── test_wave_manifold_lspg.py # ROM test: AE + manifold LSPG
│ ├── test_wave_pod_galerkin.py # ROM test: POD + Galerkin projection
│ ├── train_wave.py # Autoencoder training
| └── wave_create_snapshots # Compute FOM solutions
Defines all autoencoder architectures used in the project. The key variants are:
| Name | Description |
|---|---|
CNNAutoencoder2D |
Standard (non-equivariant) CNN autoencoder baseline with transposed convolutions in the decoder architecture. Not used during the experiments, but provided as a "blueprint", as all other architectures follow this. |
UpsamplingCNNAutoencoder2D |
Standard (non-equivariant) CNN autoencoder baseline with the upsampling + convolutiona decoder architecture. Use throughout the work as "standard CNN". |
RotationUpsamplingGCNNAutoencoder2D |
Group-equivariant autoencoder with C4 or C8 rotational symmetry, using ESCNN. The encoder uses group convolutions; the decoder uses upsampling transposed group convolutions. |
TrivialUpsamplingGCNNAutoencoder2D |
GCNN autoencoder with H=C1 — equivariant in translation action (as standard CNNs) but without non-trivial group action on features. This is one of the test implementations detailed in Remark 6.4 of the thesis. |
RotationUpsamplingGCNN2D_TorchOnly |
GCNN autoencoder for H=C4, implemented using pyTorch and thus implementing group convolutions "by hand" instead of using escnn. This is one of the test implementations detailed in Remark 6.4 of the thesis. |
All autoencoders share the same encode/decode interface and are selected via the AE_REGISTRY in the test scripts. This is explained in detail in the at the end of this readme file or in the respective python scripts where this is of relevance.
Early stopping procedure, that checkpoints the best trained model via computing the current validation loss. Includes a patience parameter P, i.e., if no improvment of the validation loss for P epochs occurs, training is terminated.
Contains the complete training routine for a neural network, in this case employed only for autoencoders. Different loss functions, including the MSE, a weighted MSE, a physical aware MSE and the symplectic loss term are included here.
Implements the manifold Galerkin time integration: a quasi-Newton solver (Galerkin_quasi_newton) for implicit midpoint rule (IMR) timestepping on the reduced nonlinear manifold. The residual is formulated via Galerkin projection of the FOM onto the reduced manifold via the trained autoencoder.
Implements the manifold LSPG (Least-Squares Petrov-Galerkin) time integration: a quasi-Newton solver (LSPG_quasi_newton) for IMR timestepping. Differs from Galerkin in the test space used for projection — LSPG minimizes the full-order residual in a least-squares sense.
Implements general helpers, including a apply_decoder (applies the decoder and the respective unscaling and prolongation operator) and a get_jacobian, which computes the jacobian of the AE-decoder via pyTorches jacfwd method.
Wraps an autoencoder into a full MOR model (NonlinearManifoldsMOR2D). Handles loading/saving of network weights and interfacing with the FOM.
Projection results when using autoencoders. Contains results regarding GCNN, UpsamplingCNN and UpsamplingCNN_symplectic for the test parameters mu ∈ {0.6, 0.8}. The mean of these two is stored in seperate files (named without the mu-value at the end).
Checkpoints of the selected autoencoders for different AE setups and reduced basis sizes.
Results when using CL as projection method as well as CL-SG as reduction method. Contains projections error, reduction errors as well as the computed RB using CL (once centered and once uncentered).
Results regarding the reduction error when using different autoencoder variants. Stored as csv files, similar to results in AE_results.
Contains the network parameters that corresponds to the checkpoints in tests/90wave/checkpoints.
Results regarding experiments when using POD as projection method as well as POD-G as reduction method. Contains projections error, reduction errors as well as the computed RB using CL (once centered and once uncentered).
Contains scale.py, which performs scaling (and inverse scaling) and the reshaping (and prolongation).
Currently empty as the FOM solutions are too large to store on github. Then FOM solutions can be obtained using wave_create_snapshots.
These scripts precompute reduced bases from training snapshots. We ran them once before any ROM tests.
Loads training snapshots for mu ∈ {0.5, 0.75, 1.0}, assembles a phase-space snapshot matrix, and computes a symplectic reduced basis using pyMOR's psd_cotangent_lift. Saves the basis as a pickle file.
Loads the same training snapshots and computes a standard POD basis using pyMOR's pod function. Saves the basis as a .npy file.
Central configuration file for all experiments regarding the right moving wave. Contains:
WaveExperimentConfig— dataclass holding all experiment hyperparameters: grid size (Nx,Ny), number of timesteps (nt), timestep factor, flow direction (x_flow), and visualization flags.WaveExperiment— sets up the pyMOR full-order model (FOM) for the 2D wave equation, provides helper methods for loading initial conditions, computing reference offsets, and evaluating error metrics.get_filepath_patterns— returns a dict of standardized paths for snapshots, checkpoints, network parameters, and results directories.
These scripts evaluate how well each reduced representation can approximate test snapshots, without running a time integrator. They measure the offline approximation quality of each method.
Encodes and decodes test snapshots through the trained autoencoder and computes the relative reconstruction error for each latent dimension size.
Projects test snapshots onto the CL reduced basis using symplectic projection and computes the relative error for each basis size.
Projects test snapshots onto the POD basis via standard orthogonal projection and computes the relative error for each basis size.
These scripts run a full reduced-order model time integration and compare against reference snapshots.
Uses pyMOR's QuadraticHamiltonianRBReductor to build and solve a symplectic Galerkin ROM on the CL basis. Computes reconstruction errors against test snapshots.
Runs the manifold Galerkin ROM: the autoencoder defines the nonlinear reduced manifold, and the implicit midpoint rule is solved via quasi-Newton iteration with Galerkin projection. In particular, this contains the weakly symplectic autoencoder variant.
Same as manifold Galerkin but uses LSPG projection (minimizes the full-order residual) instead of Galerkin projection. Generally more robust but more expensive per timestep.
Uses pyMOR's InstationaryRBReductor to build and solve a standard Galerkin ROM on the POD basis.
Two files are required to perform training:
Sets all network hyperparameters and trains the autoencoders.
Before training, training data needs to be generated. This script generates the training (and also the test) data. Run this once before starting training.
Structured in a similar fashion as the tests/90wave, therefore no additional detailed description is provided here.
All test scripts that use a trained autoencoder share the same AE_REGISTRY, which maps a short name to the corresponding network class and group space:
--ae_name |
Network class | Group |
|---|---|---|
RotationUpsamplingGCNN |
RotationUpsamplingGCNNAutoencoder2D |
C4 (4 rotations) |
RotationUpsamplingGCNN_C8 |
RotationUpsamplingGCNNAutoencoder2D |
C8 (8 rotations) |
UpsamplingCNN |
UpsamplingCNNAutoencoder2D |
None (baseline) |
UpsamplingCNN_Symplectic |
UpsamplingCNNAutoencoder2D |
None (baseline) |
RotationUpsamplingGCNN_bothdir |
RotationUpsamplingGCNNAutoencoder2D |
C4 (4 rotations) |
UpsamplingCNN_bothdir |
UpsamplingGNNAutoencoder2D |
None (baseline) |
Checkpoint files are expected to follow the naming convention:
checkpoints/wave_2D_{ae_name}_p_{p_red}_{Nx}x{Ny}.pt
network_parameters/wave_2D_{ae_name}_p_{p_red}_{Nx}x{Ny}.pkl