Skip to content

Add optional Ginkgo single-GPU backend, abstract PETSc calls

Terry Cojean requested to merge ginkgo_integration into master

This MR aims at integrating Ginkgo as an optional backend.

  • This is done by putting PETSc functions behind abstract classes.
  • Currently, everything should be working as expected according to my own tests.
  • We have two versions of the code, one is fully header-only including for the PETSc and Ginkgo backends, and is further in the commit history. It comes at several expenses (in terms of setup and dependency creep, notably) but might have a few inline benefits. The other version, this one, has most of slimfem still header only, but only the PETSc and Ginkgo related functions are isolated in different libraries.

Before merging, there are a few things left to discuss or to do:

  • Is there an overhead when using PETSc abstract with either version compared to master openCARP?
  • Integrate the slimfem_ginkgo and slimfem_petsc libraries into numerics directly
  • Is there any quality issues (memory leaks, bugs, style, encapsulation problems,... ?)
  • Integrate installation and configuration instruction in the docs (not exported yet).
  • Add more simple doxygen documentation.
  • Fix wrong results with the latest commits influencing the PETSc results.
  • Should parameters (openCARP.prm) be added/reviewed? Two things come to mind:
    • Currently, Ginkgo and PETSc use different solver parametrization file parameters. Is this the way to go? Do we need to unify them? If so, maybe this requires carputils adaptation as well? (Answer: both openCARP and carputils will be modified to integrate Ginkgo by @m.houillon after this is merged).
    • Ginkgo has the possibility to chose backend dynamically thanks to the executor (sequential CPU, OpenMP CPU, CUDA, HIP (AMD), DPCPP (Intel GPUs)). When using devices, a device id should be specified. Although the code isn't MPI-enabled yet when we add MPI support we need to map MPI ranks to device ids as well. What would be the best way to integrate this, does anyone have ideas or requirements? See this file.
  • Fix the autotester errors (5 out of 32 problems)
  • Describe changes in CHANGELOG

Thanks to @fritzgoebel and @m.houillon for their help and contributions!

Edited by Axel Loewe

Merge request reports