numerical.md

numerical linear algebra and more for PDE

• CFD

• BLAS + LAPACK (OpenBLAS + FLAME + ATLAS + MAGMA + KBLAS) and friends

  • C++ tensors with broadcasting and lazy computing

    • xtensor
    • The x template library

  • Benchmarking Single- and Multi-Core BLAS Implementations and GPUs for use with R

  • OpenBLAS: An optimized BLAS library

    • OpenBLAS github
    • tester for BLAS libraries including OpenBLAS and Intel MKL
    • Vector Math Library

  • BLIS (flame)

    • github repo for BLIS
    • github repo for FLAME

  • Matrix Template Library 4

  • GEMM: From Pure C to SSE Optimized Micro Kernels ulmBLAS: parent of the above link

  • GEMV kernel

  • GPU Accelerated Memory-bound Linear Algebra Kernels" KBLAS + KSPARSE

    • PhD thesis 4 KBLAS+KSPARSE
    • KBLAS: An Optimized Library for Dense Matrix-Vector Multiplication on GPU Accelerators
    • KBLAS: download
    • KSPARSE: download
    • bhSPARSE: A Sparse BLAS Library
    • fmmtl: FMM Template Library -- a structured dense matrix algorithms library, GPU accelerated, Simon Layton is a coauthor
    • LAMA -- Library for Accelerated Math Applications -- GPU accelerated
      • sourceforge project for LAMA

  • blaze-lib -- A high performance C++ math library -- claims to be the fastest, supports AVX

    • wiki link compares blaze vs. other C++ libraries

  • The Numerical Template Toolbox - C++ Scientific Computing Made Easy

    • The Numerical Template Toolbox - C++ Scientific Computing Made Easy docs

  • SIMD Vector Classes for C++

  • EVE - the Expressive Vector Engine

  • High-Performance Matrix-Matrix Multiplications of Very Small Matrices

  • AUGEM: Automatically generate high performance Dense Linear Algebra kernels on x86 CPUs

  • psBLAS -- Parallel Sparse Basic Linear Algebra Subroutines -- Fortran2003

  • Ginkgo is a high-performance linear algebra library for manycore systems, with a focus on sparse solution of linear systems. It is implemented using modern C++ (you will need at least C++14 compliant compiler to build it), with GPU kernels implemented in CUDA and HIP

    • ginkgo: getting started
      tl;dr

    git clone https://github.com/ginkgo-project/ginkgo.git
    cd ginkgo
    mkdir build
    cd build
    ccmake ..
    make -j4

  • GraphBLAS Repositories dedicated to Graphs in the Language of Linear Algebra

    • GraphBLAS Pointers
    • GraphBLAS ycombinator
    • SuiteSparse:GraphBLAS, Timothy A. Davis, (c) 2017-2023, All Rights Reserved.
    • ibmgraphblas
    • GraphBLAS Template Library (GBTL), v. 3.0
    • Rust wrappers to GraphBLAS
      • graphblas_sparse_linear_algebra
        • graphblas_sparse_linear_algebra
      • rustgraphblas

  • Elemental is a modern C++ library for distributed-memory dense and sparse-direct linear algebra and optimization.

    • Elemental.jl

  • Recommendations for a usable, fast C++ matrix library?

  • The Tensor Algebra Compiler (taco) computes sparse tensor expressions on CPUs and GPUs

    • A fast and versatile library for linear and tensor algebra

  • amgcl : C++ library for solving large sparse linear systems with algebraic multigrid method

    • amgl docs : AMGCL is a header-only C++ library for solving large sparse linear systems with algebraic multigrid (AMG) method
      • amgl issue : MPI & preconditioner+solver setup

• The Amazing Performance of C++17 Parallel Algorithms, is it Possible?

• SIMD

  • The C++ scientist

• Spectra - C++ Library For Large Scale Eigenvalue Problems

  • A header-only C++ library for large scale eigenvalue problems
    • Spectra C++ Library For Large Scale Eigenvalue Problems
  • Spectra Performance

• Dimension Reduction

  • A flexible and efficient С++ template library for dimension reduction
    • Tapkee an efficient dimension reduction library

• Wavelet and Multiscale Library

  • The Marburg Software Library is a collection of wavelet based c++ algorithms to solve multivariate partial differential equations.
  • Parallel adaptive wavelet collocation method for PDEs
  • Wavelet methods in computational fluid dynamics

• BEM - Boundary Element Library

  • Curated list of Hierarchical Matrices (H-Matrices) libraries
  • BEM++ is an open-source Galerkin boundary element library that handles Laplace, Helmholtz and Maxwell problems on bounded and unbounded domains
    • BEM++ github repo
      • Bempp-rs is an (in early development) boundary element library written in Rust with a Python interface
      • ExaFMM: a high-performance fast multipole method library with C++ and Python interfaces
      • BEM++ fork
  • Boundary Element Template Library
  • HyENA - HyENA stands for [Hy]perbolic and [E]lliptic [N]umerical [A]nalysis and is a C++ library
  • Obtaining the source code for NiHu tl;dr

  git clone -b release_2.0 git://last.hit.bme.hu/toolbox/nihu.git

  • Formulating various BEM problems with an open source C++ BEM template library

  • fmm-bem-relaxed

  • SCUFF-EM

    • PhD Thesis on SCUFF-EM

  • Hierarchical matrices

    • H2Lib is an open source software library for hierarchical matrices and H²-matrices
      • H2Lib public github repository
    • AHMED - Another software library on hierarchical matrices for elliptic differential equations github repository
      • AHMED presentation

• The C++ Standards Library for Concurrency and Parallelism http://stellar-group.org/libraries/hpx

  • HPX (High Performance ParalleX) is a general purpose C++ runtime system for parallel and distributed applications of any scale
  • HPX — рантайм-система для параллельных и распределенных вычислений

• Automatic Differentiation: mostly Julia, C++, Haskell. Scala as well

  • CasADi framework for nonlinear optimization

    • CasADi framework for nonlinear optimization
    • CasADi is a symbolic framework for numeric optimization implementing automatic differentiation in forward and reverse modes on sparse matrix-valued computational graphs
    • CasADi FAQ
    • CasADi docs
    • tipsandtricks for CasADi

  • automatic-differentiation

    • Tensors and differentiable operations (like TensorFlow) in Rust https://docs.rs/autograd/
    • Kotlin𝛁: Differentiable Functional Programming with Algebraic Data Types https://github.com/breandan/kotlingrad

  • JuliaDiff Differentiation tools in Julia

    • ForwardDiff.jl github
      • JSoC 2015 project: Automatic Differentiation in Julia with ForwardDiff.jl
      • ForwardDiff.jl docs
        • ForwardDiffNumber Types
          • ForwardDiff contributing to
      • Julia automatic differentiation with source code transformation
      • Àlex Haro Provinciale
      • Latest publications of Olivier Pironneau
    • Automatic differentiation
      • insightful intro to AD
      • /which-automatic-differentiation-tool-for-cc/
        • HyperDual Numbers: Exact numeric nth derivatives
      • Lazy Time Reversal, and Automatic Differentiation
    • Integration analog of automatic differentiation
      • Risch algorithm
        • GSoC 2013 Application Chetna Gupta: Risch algorithm for symbolic integration
        • Q: Automatic differentiation for numerical integration
        • Derivative of definite integral
          • Differentiation under the integral sign problem
      • Chebfun—numerical computing with functions
      • ApproxFun - Julia package for function approximation
        • A practical framework for infinite-dimensional linear algebra
    • Symbolic math with julia

  • Haskell Automatic Differentiation

    • What is automatic differentiation, and why does it work?

  • Why Automatic Differentiation Won’t Cure Your Calculus Blues

  • Automatic Differentiation in Computational Science : presentation

  • Reviews of AD including computational cost of FAD/BAD

    • A Hitchhiker's Guide to Automatic Differentiation - up-to-date - all you have to know about AD, Haskell focused
      • Dr Philipp Hoffmann personalia
        • National University of Ireland Maynooth Brain and Computation Lab
          • Hype: Compositional Machine Learning and Hyperparameter Optimization
    • Automatic differentiation in machine learning: a survey contains estimation for number of operations
    • AD and Sparse Matrices (Computational Complexity of Forward Mode AD)
    • Towards a Computer Algebra System with Automatic Differentiation for use with Object-Oriented modelling languages (the paper describes DAG representation for representing optimization problem)
    • Non-smooth Optimization and Automatic Differentiation (skript of the lecture)
    • PhD Thesis A Small-Perturbation Automatic-Differentiation (SPAD) Method for Evaluating Uncertainty in Computational Electromagnetics
    • Reverse-Mode AD in a Functional Framework: Lambda the Ultimate Backpropagator
      • Barak A. Pearlmutter publications
      • Jeffrey Mark Siskind's Software
    • A simple explanation of reverse-mode automatic differentiation
      • Automatic Differentiation: The most criminally underused tool in the potential machine learning toolbox?
      • Review and Unification of Methods for Computing Derivatives of Multidisciplinary Computational Models

  • Applications of AD to Real World

    • Efficient Hessian Calculations using Automatic Differentiation and the Adjoint Method
    • Automatic Differentiation Adjoint of the Reynolds-Averaged Navier–Stokes Equations with a Turbulence Model AIAA 2013-2581
    • AD for Optimization
      • Ceres Solver : can solve bounds constrained robustified non-linear least squares problems

  • AD in Haskell

  • CS267 Project Report Towards Parallel Automatic Differentiation

  • Autograd

    • autograd: Efficiently computes derivatives of numpy code
    • Autograd automatically differentiates native Torch code. Inspired by the original Python version

  • Backup

    • fast automatic differentiation jacobians by compact lu factorization

• Unum

  • What are the biggest challenges in scientific computing?

    • John Gustafson presents: Beyond Floating Point – Next Generation Computer Arithmetic + (former Unum)Create RUST posit arithmetic library based on 21 February 2017 Posit Arithmetic by John L. Gustafson - Beating Floating Point at its Own Game

      • UNUM ... a better alternative to IEEE Floating Point?
      • The Evils of Floating Point, and the Joys of Unum
      • John Gustafson (scientist)
      • Unum Computing: An Energy Efficient and Massively Parallel Approach to Numerics
        • Unum Computing slideshare
        • Unum Computing Is A Breakthrough ... A Game Changer
        • Handbook of Floating-Point Arithmetic
        • Rump's Royal Pain
        • Official Blog - SSRLabs for UNUMs implemented in hardware
      • Unums.jl
      • Julia Implementation of Unums - (seems more mature)
      • unum - Rust library for unums, a placeholder @Jan 4, 2016

      julia> f(x,y) = 333.75*y^6 + x^2*(11*x^2*y^2-y^6-121*y^4-2)+5.5*y^8+x/(2*y)
      f (generic function with 1 method)
      
      julia> f(77617,33096)
      1.64176022256015e21
      
      julia> f(Rational(77617,1),Rational(33096,1))
      ERROR: OverflowError()
       in * at rational.jl:188
       in power_by_squaring at intfuncs.jl:96
       in f at none:1
      
      julia> f(Rational(BigInt(77617),1),Rational(BigInt(33096),1))
      -8.273960599468213681411650954798162919990331157843848199178148416727096930142628e-01
      
      julia> f(77617.0,33096.0)
      -1.1805916207174113e21
      
      julia> f(BigFloat(77617),BigFloat(33096))
      -8.273960599468213681411650954798162919990331157843848199178148416727096930142628e-01

      • (more or less) accurate floating point algorithms

      • unum in D

        • Unums.jl
        • Julia Implementation of Unums - (seems more mature)
        • Experimental Unums

      • [PosIt(new Unum) in C, C++]

        • Unum & Posit
          • Toward Floating-Point Run-time Variable Precision in CPU
          • PERCIVAL: Open-Source Posit RISC-V Core with Quire Capability
        • SoftPosit
          • Berkeley SoftFloat
        • Universal Number Arithmetic in C++

  • Automatic Fortran to C++ conversion

    • Automatic Fortran to C++ conversion with FABLE

  • Promising Languages for Scientific Computing (besides Julia)

    • Polyglot Programming : review several candidate languages, namely C, C++, Fortran, Cython, Numba, Nim, Rust, D, Chapel and Julia
    • SciLua: Scientific Computing with LuaJIT
    • Chapel Quickstart Instructions tl;dr

      source util/quickstart/setchplenv.bash
      make
      make check
      chpl -o hello examples/hello.chpl

  • linear algebra library Eigen

    • Eigen Cheat sheet
    • Eigen Aliasing
    • Eigen 3.2.92 : Bug 1221: disable gcc 6 warning: ignoring attributes on template argument
    • Eigen disable GCC diagnostic ignored "-Wignored-attributes" has already merged
    • An evaluation of the Eigen linear algebra library for use in the aither CFD solver
    • A look at the performance of expression templates in C++: Eigen vs Blaze vs Fastor vs Armadillo vs XTensor
    • How should I initialize the contents of a large matrix in Eigen?

• AeroSandbox