Skip to content

Releases: PrincetonUniversity/SPECFEMPP

v0.5.0

16 Jan 14:39
@lsawade lsawade
v0.5.0
671558d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.5.0 Latest
Latest

SPECFEM++ 0.5.0

🚀 TL;DR

New Features

  • Support for non-conforming 2D meshes through a discontinuous Galerkin approach
  • 3D elastic isotropic simulation support

Improved Documentation

  • Overhaul of API documentation
    • Improved structure
    • Includes the implemented equations
  • Two new cookbooks
    • Discontinuous Galerkin for fluid-solid wave propagation with bathymetry
    • Wave propagation in the Marmousi model (adapted from SPECFEM2D)

CI

  • Nightly benchmarks for homogeneous elastic and fluid-solid simulations are run on both CPU and GPU
  • Review dashboard visualizing the nightly benchmarks and other CI information

Visualization

  • VTKHDF — Plotting in 3D is rather difficult since there are too many parameters to play with. Instead, we opted to support the writing of VTKHDF files: a single HDF5 file that stores all timesteps. Both 2D and 3D simulations support this file format.
  • Automatically cropped 4K snapshots
  • Blue-red color scheme for the output wavefields

📋 What’s New

Support for Non-Conforming Meshes

SPECFEM++ now supports non-conforming boundaries using a discontinuous Galerkin approach to the SEM. This work was contributed by Kentaro Hanson (@int-ptr-ptr).

The graphs below show results from two simulations of an acoustic-elastic layered domain meshed using:

  • A conforming mesh (top left)
  • A non-conforming mesh (top right)

The bottom graph compares traces from both simulations at a receiver within the acoustic domain. For more details, refer to the non-conforming cookbook.

image image 1 image 2

Improved Documentation

This release simplifies the API documentation by organizing it into distinct sections, with each namespace clearly separated. We have also included technical implementation details wherever possible. For instance, the API documentation for stress computation now clearly explains how each component of the stress tensor is computed. Below is a snippet of the documentation for the stress computation call for an elastic isotropic medium.

image 3

Support for 3D Elastic Isotropic Simulation

In this release, we added support for 3D elastic isotropic simulations. We include the compilation of the Fortran internal mesher, and generate the GLL databases on the fly in SPECFEM++ for an improved data layout compared to loading the Fortran databases. For a tutorial, please refer to our 3D homogeneous elastic isotropic cookbook: 3D Elastic Isotropic Cookbook. Through integration testing, we make sure that the simulation matches SPECFEM3D Cartesian sample-by-sample. To visualize the 3D wavefield in Paraview, we also included the writing of VTKHDF files.

image 4

Marmousi model simulation

We included a cookbook that walks the user through simulating wave propagation in the Marmousi model.

SPECFEM2D implemented a high-resolution Marmousi example. The original velocity model for this example was provided by Yann Capdeville, and Hom Nath Gharti created a very high-resolution CUBIT mesh from it. We included the static CUBIT mesh to create a cookbook for wave propagation in the Marmousi model. To run this cookbook, a GPU compilation is strongly recommended, as the mesh is quite fine and the code doesn't yet support MPI.

marmousi

Blue-red color scheme

Until now, we only supported magnitude plots for vector fields and absolute value for scalar fields. In particular, for pressure and rotation (Cosserat media), this felt limiting. For any scalar field, the default is now red-blue, while for vector fields, unless a component is specified, we still use magnitude by default. To pick a component, simply update the display section to include the component parameter.

display:
  format: PNG
  directory: path/to/output_dir/
  field: displacement
  component: x  # y, z, or magnitude are other possible values
  simulation-field: forward
  time-interval: 5

fluid-solid-nonconforming

Nightly Benchmarks

We run nightly benchmarks for both serial CPU and GPU simulations. At this time, the CPU simulations are run on Intel Gold 6548Y+ chips and GPU simulations on Nvidia H100s. The benchmarks are published on a static review dashboard that is updated every morning. An example of the CPU runs — a snapshot of all runs up until Jan 14 — is shown below.

newplot_(1)

📊 Statistics

analysis

New Contributors

🪵  Changelog

👨‍🔬 New physics

🚀 Performance enhancement

✚ Enhancement

Read more

Contributors

tjhei, bangerth, and 5 other contributors
Assets 2
Loading

v0.4.0

26 Jun 12:38
@lsawade lsawade
v0.4.0
8c3a12d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.4.0

🚀 TL;DR

Physics updates:

  • Support for elastic SH wave simulation (forward simulation support)
  • Support for Poro-elastic domains (forward simulation support)
  • Support for Cosserat domains (forward simulation support)

Performance Improvements:

  • Memory layout optimization. Solver time on par with SPECFEM2D.

Code maintainance:

  • New macros for better code generation
  • Introduced iterator framework to define parallelism over different solver loops.

Refactoring

  • Split examples into examples and benchmarks directories.

Configuration

  • Introduces CMAKE presets for regularly used build configurations

Maintenance

  • Updates Kokkos version to 4.6.1.
  • Adds support for AMD GPUs
  • Adds support for parallel testing

📋 What's New

Support for elastic SH wave

Wave propagation of an SH wave in an elastic medium in response to a transverse force source Ricker source time function. The domain has one horizontal interface in the vertical center of the domain splitting a slower upper from a faster lower layer.

SH wave simulation with interface

Support for Poro-elastic domains

Below wave propagation simulation in a homogeneous poroelastic medium in response to a medium central downward force in acting on.

Poro-elastic Simulation in a homogeneous medium

Support for Cosserat domains

Below a simulation of wave propagation in a Cosserat medium in response to a central downward force. On the left the norm of the displacement wavefield and on the right the rotational field around the transverse axis.

Wave propagation in a homogeneous Cosserat medium

New macros for better code generation

This feature streamlines the codebase by eliminating redundant code, making it easier to maintain. In addition to improved maintainability, this feature significantly reduces the amount of code required to implement a new medium. Below the old and new way to introduce a new data container to the solver, by the example of acoustic-isotropic media.

The old way of defining the property_container

For each medium/property combination, the previous code required the user to define the following properties container, that is used to store the properties for relevant elements.

template <>
struct properties_container<acoustic isotropic specialization> {

  using ViewType = typename Kokkos::View<type_real ***, Kokkos::LayoutLeft,
                                         Kokkos::DefaultExecutionSpace>;

  int nspec; ///< total number of acoustic spectral elements
  int ngllz; ///< number of quadrature points in z dimension
  int ngllx; ///< number of quadrature points in x dimension
  ViewType rho_inverse;
  ViewType::HostMirror h_rho_inverse;
  ViewType kappa;
  ViewType::HostMirror h_kappa;

  properties_container(const int nspec, const int ngllz, const int ngllx)
      : nspec(nspec), ngllz(ngllz), ngllx(ngllx),
        rho_inverse("specfem::compute::properties::rho_inverse", nspec, ngllz,
                    ngllx),
        h_rho_inverse(Kokkos::create_mirror_view(rho_inverse)),
        kappa("specfem::compute::properties::kappa", nspec, ngllz, ngllx),
        h_kappa(Kokkos::create_mirror_view(kappa)) {}

	template<...>
  KOKKOS_FORCEINLINE_FUNCTION void
  load_device_properties(index, point_property) const { ... }

  template<...>
  load_device_properties(index, point_property) const { ... }

  template<...> 
  inline void
  load_host_properties(index, point_property) const { ... }

  template<...>
  inline void
  load_host_properties(index, point_property) const { ... }

  void copy_to_device() {
    Kokkos::deep_copy(rho_inverse, h_rho_inverse);
    Kokkos::deep_copy(kappa, h_kappa);
  }

  void copy_to_host() {
    Kokkos::deep_copy(h_rho_inverse, rho_inverse);
    Kokkos::deep_copy(h_kappa, kappa);
  }

  template<...>
  inline void assign(index, point_property) const { ... }
};

Even after omitting quite a few code blocks with ... for clarity, leaving ~25% of the code, it is clear that this is not very user friendly, nor maintainable, and too specialized with the perspective of implementing new physics, which is why we needed a more concise, simpler way to approach new data.

The new way of defining the property_container → data_container

Using macros we can wrap all of the above medium/property specific code into a single line that takes in the variable names and generates all of the above required code during the preprocessing stage.

template <acoustic isotropic specialization>
struct data_container {
  constexpr static auto dimension = specfem::dimension::type::dim2;
  constexpr static auto medium_tag = MediumTag;
  constexpr static auto property_tag =
      specfem::element::property_tag::isotropic;

  DATA_CONTAINER(kappa, mu, rho)
};

A complete documentation for available macros can be found here

CMake Presets

Prior to this release the one always need to manually define the all flags. For example, for a CUDA release build, the user needed to

cmake3 -S . -B build/release-cuda -D CMAKE_BUILD_TYPE=Release -D BUILD_TESTS=ON -D Kokkos_ENABLE_CUDA=ON -D Kokkos_ARCH_<architecture>=ON
cmake --build build/release-cuda

to build the software. We added CMakePresets.json for a variety of “standard” compilations. Which allow for building using

cmake --preset release-cuda
cmake --build --preset release-cuda

We want to advertise this feature in particular since cmake allows for CMakeUserPresets.json which if located in the SPECFEM++ root directory takes precedence over CMakePresets.json and can be used for super custom user installation without having to populate 4 lines of the command line every time you want to build. The list of “standard” presets are:

"release"          - Default Release -- SIMD enabled
"release-nosimd"   - Release -- SIMD disabled
"release-cuda"     - Release -- CUDA enabled
"release-hip"      - Release -- HIP enabled
"release-frontier" - Release Frontier -- HIP enabled
"debug"            - Default Debug -- SIMD enabled
"debug-cuda"       - Debug -- CUDA enabled
"debug-nosimd"     - Debug -- SIMD disabled

Performance

The plots below show the performance of SPECFEM++ compared to SPECFEM2D:

Elastic simulation

Performance on CPU -- Homogeneous Elastic Performance on GPU -- Homogeneous Elastic

Acoustic Simulation

Performance on CPU -- Homogeneous Acoustic Performance on GPU -- Homogeneous Acoustic

Elastic-Acoustic coupled simulation

Performance on CPU -- Elastic-Acoustic Performance on GPU -- Elastic-Acoustic

New Contributors

Statistics

Commits and PRs since the last Release

Changelog

👨‍🔬 New physics

Read more

Contributors

lsawade, Rohit-Kakodkar, and 5 other contributors
Loading

v0.3.0

18 Feb 14:58
@icui icui
v0.3.0
4fc63c1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.3.0

What's Changed

This release contains a few major updates:

  • Python Bindings
  • Anisotropy Implementation
  • Refactor for easier implementation of new media
  • Visualization using VTK
  • New examples and cookbooks
  • meshfem3d integration (note: no IO or simulation support yet)
  • macOS support

Details for each of these are below

New Contributors

Changes

Python Bindings

Anisotropic media

PRs for the anisotropy implementation tracked by:

#257

Visualization

New Cookbooks & Examples

New features added capabilities

MacOS Support

Other Updates

  • Fixed testing with OpenMP by @lsawade in [https://github.com//pull/452]
  • Update commit template by @lsawade in [https://github.com//pull/470]
  • Restructure IO routines by @lsawade in #157
  • General CMakeLists.txt updates
    • Update for MPI_PARALLEL flag by @icui in #203
  • CI Updates
    • Updated Jenkins artifact folder by @Rohit-Kakodkadr in #184
    • Jenkins fix for GNU tests by @Rohit-Kakodkar in #219
    • Fix Boost installation to remove non-official dependency by @lsawade in #358
    • Fixing docker image creation by @lsawade in [https...
Read more

Contributors

lsawade, sirpipat, and 3 other contributors
Loading

v0.3.0-release-candidate

05 Feb 15:51
@icui icui
v0.3.0-release-candidate
ac02df2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.3.0-release-candidate Pre-release
Pre-release

What's Changed

This release contains a few major updates:

  • Python Bindings
  • Anisotropy Implementation
  • Refactor for easier implementation of new media
  • Visualization using VTK
  • New examples and cookbooks
  • meshfem3d integration (note: no IO or simulation support yet)
  • macOS support

Details for each of these are below

New Contributors

Changes

Python Bindings

Anisotropic media

PRs for the anisotropy implementation tracked by:

#257

Visualization

New Cookbooks & Examples

New features added capabilities

MacOS Support

Other Updates

  • Restructure IO routines by @lsawade in #157
  • General CMakeLists.txt updates
    • Update for MPI_PARALLEL flag by @icui in #203
  • CI Updates
    • Updated Jenkins artifact folder by @Rohit-Kakodkadr in #184
    • Jenkins fix for GNU tests by @Rohit-Kakodkar in #219
    • Fix Boost installation to remove non-official dependency by @lsawade in #358
    • Fixing docker image creation by @lsawade in #356
    • Fixed clang-format issue for header files. by @lsawade in [https://github.com/Princeto...
Read more

Contributors

lsawade, sirpipat, and 3 other contributors
Loading

v0.2.0

05 Nov 15:15
@Rohit-Kakodkar Rohit-Kakodkar
v0.2.0
f54b50f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.2.0

Description

This release adds support for forward-adjoint simulations and computing Misfit Kernels. Have a look at example 03 for user guide on how to use adjoint capabilities within SPECFEM++.

Documentation

The documentation for this release can be found here

Getting the code

git clone https://github.com/PrincetonUniversity/SPECFEMPP && \
cd SPECFEMPP && \
git checkout tags/0.2.0

What's Changed

Full Changelog: v0.1.0...0.2.0

Contributors

Rohit-Kakodkar
Loading

0.2.0-release-candidate

16 Oct 13:30
@Rohit-Kakodkar Rohit-Kakodkar
0.2.0-release-candidate
5086419
This commit was signed with the committer’s verified signature. The key has expired.
Rohit-Kakodkar
GPG key ID: 7EB917889A308210
Expired
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
0.2.0-release-candidate Pre-release
Pre-release

Description

This release adds support for forward-adjoint simulations and computing Misfit Kernels. Have a look at example 03 for user guide on how to use adjoint capabilities within SPECFEM++.

Documentation

The documentation for this release can be found here

Getting the code

git clone https://github.com/PrincetonUniversity/SPECFEMPP && \
cd SPECFEMPP && \
git checkout tags/0.2.0-release-candidate

What's Changed

Full Changelog: v0.1.0...0.2.0-release-candidate

Contributors

Rohit-Kakodkar
Loading

v0.1.0

09 Jan 21:31
@Rohit-Kakodkar Rohit-Kakodkar
v0.1.0
d199e41
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.1.0

Major Release Candidate

New Features

  • GLL Library

    • GLL library implementation to compute GLL points and weights.

  • Meshing (MESHFEM)

    • MESHFEM2D is included as the default mesher in this release
    • Implemented MESH database reader for fortran binary generated by MESHFEM

  • Sources

    • Force Source
    • Moment-tensor source

  • Source time function

    • Dirac delta source
    • Ricker (Gaussian) source

  • Receiver

    • Displacement
    • Velocity
    • Acceleration

  • Physics

    • Elastic Domain
    • Acoustic Domain
    • Elastic-Acoustic Coupling

  • Boundary Conditions

    • Free surface
    • Stacey

  • Solver

    • Newmark Time-Marching Solver

Major Changes

  • None

Minor Changes

  • None

Known Issues (Bugs)

Patches will be released soon for these bugs.

  • Memory corruption issue for Dirichlet, Stacey and Composite BC on IntelLLVM compiler 2022.2.0 when running tests. The solver does not experience these issues. Check pull request #100

New Contributors

Contributors

lsawade, sirpipat, and 2 other contributors
Loading

v0.1.0-release-candidate

03 Jan 18:01
@Rohit-Kakodkar Rohit-Kakodkar
v0.1.0-release-candidate
35d2f06
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.1.0-release-candidate Pre-release
Pre-release

Major Release Candidate

New Features

  • GLL Library

    • GLL library implementation to compute GLL points and weights.

  • Meshing (MESHFEM)

    • MESHFEM2D is included as the default mesher in this release
    • Implemented MESH database reader for fortran binary generated by MESHFEM

  • Sources

    • Force Source
    • Moment-tensor source

  • Source time function

    • Dirac delta source
    • Ricker (Gaussian) source

  • Receiver

    • Displacement
    • Velocity
    • Acceleration

  • Physics

    • Elastic Domain
    • Acoustic Domain
    • Elastic-Acoustic Coupling

  • Boundary Conditions

    • Free surface
    • Stacey

  • Solver

    • Newmark Time-Marching Solver

Major Changes

  • None

Minor Changes

  • None

Known Issues (Bugs)

Patches will be released soon for these bugs.

  • Memory corruption issue for Dirichlet, Stacey and Composite BC on IntelLLVM compiler 2022.2.0
  • Composite BC do not work using CUDA

New Contributors

Full Changelog: https://github.com/PrincetonUniversity/SPECFEMPP/commits/v0.1.0-release-candidate

Contributors

lsawade, sirpipat, and 2 other contributors
Loading

v0.0.1-beta

14 Mar 20:16
@Rohit-Kakodkar Rohit-Kakodkar
v0.0.1-beta
06b2b42
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v0.0.1-beta

Initial release of SPECFEM2D KOKKOS package

We are pleased to announce first release 0.0.1-beta for Kokkos implementation of SPECFEM2D package. This version of the package is able to simulate wave propagation through elastic homogeneous media in 2-Dimensions.

What's Changed

  • New: Added support for GLL quadrature by @Rohit-Kakodkar in #5
  • New: Read fortran mesh database files generated by xmeshfem2d by @Rohit-Kakodkar in #7
  • New: Create a Mesh struct to store the mesh (The mesh is created by reading the database binaries created by xmeshfem2d) by @Rohit-Kakodkar in #8
  • New: Implemeted a source class (Currently only Dirac sources are enabled in this version of the package) by @Rohit-Kakodkar in #20
  • New: Implemented Newmark Timescheme, Implemented domain class to store wavefields by @Rohit-Kakodkar in #21
  • New: Enabled GPU support (First working GPU implementation) by @Rohit-Kakodkar in #26
  • New: Implemented a time-marching (explicit) solver to simulate wave propagation by @Rohit-Kakodkar in #28
  • New: Enabled simulation setup using a parameter YAML file by @Rohit-Kakodkar in #31
  • New: Added support for command line parsing by @Rohit-Kakodkar in #34
  • New: Added cookbook example for simulating wave propagation through homogeneous media by @Rohit-Kakodkar in #39
  • New: Implemented routines to compute and store seismographs (Implemented a seismograph writer) by @Rohit-Kakodkar in #43
  • Thanks to @lsawade, @icui and @EtienneBachmann for their help with running the code and with changes to the documentation

New Contributors

Full Changelog: https://github.com/PrincetonUniversity/specfem2d_kokkos/commits/v0.0.1-beta

Contributors

EtienneBachmann, lsawade, and 2 other contributors
Loading