cuBLAS
also: CUDA Basic Linear Algebra Subroutines, CUDA BLAS
cuBLAS is NVIDIA's GPU-accelerated library for Basic Linear Algebra Subprograms (BLAS) operations, enabling high-performance matrix and vector computations on CUDA-capable GPUs.
cuBLAS (CUDA Basic Linear Algebra Subroutines) is part of NVIDIA's CUDA Toolkit and provides optimized implementations of standard linear algebra operations on graphics processors. It accelerates computationally intensive tasks like matrix multiplication, vector operations, and factorizations.
Common use cases include machine learning frameworks (TensorFlow, PyTorch), scientific computing, and numerical simulations where matrix operations dominate runtime. For example, a deep neural network training loop relies heavily on cuBLAS for efficient forward and backward propagation matrix multiplications.
Applications link against cuBLAS libraries and offload work to the GPU, achieving 10-100x speedups compared to CPU-based BLAS libraries like OpenBLAS or Intel MKL for large matrices. It requires compatible NVIDIA hardware and the CUDA Toolkit to be installed.