Install the NVIDIA CUDA Toolkit on Linux

Installing CUDA on Linux is straightforward in concept but easy to break in practice. The two most common failure modes are mismatched driver versions and accidentally overwriting a working display driver. This guide covers the relationship between the display driver and the CUDA toolkit, walks through both the distro-package and runfile installation paths, and shows you how to verify a working setup with deviceQuery.

Driver vs CUDA Toolkit: What You Actually Need

The NVIDIA driver and the CUDA toolkit are separate components that must be compatible, but they are not the same thing. The driver is a kernel module (nvidia.ko) that talks to the hardware. The CUDA toolkit is a collection of compiler tools (nvcc), libraries (libcudart), and samples that sit on top of the driver.

NVIDIA publishes a CUDA/Driver compatibility table. CUDA 12.x requires driver ≥525.60.13 on Linux. Each CUDA release supports a minimum driver version; newer drivers support older CUDA runtimes through forward compatibility, but an old driver will refuse to run newer CUDA code. Always check the official release notes before you start.

• Display driver only — sufficient for gaming, Wayland compositors, and basic GPU compute through pre-built binaries.
• CUDA toolkit — required to compile CUDA code or use frameworks like PyTorch/TensorFlow with custom kernels. The toolkit bundles its own driver installer, but you can also install the toolkit against a driver you already have.

The safest installation order: install the driver first, reboot, confirm it works, then add the CUDA toolkit packages on top. Do not let the runfile installer touch your driver if you already have a working one from your distro's repository.

Before You Start

• Identify your GPU: lspci | grep -i nvidia
• Note your current kernel version: uname -r
• Disable Secure Boot or enroll the NVIDIA MOK key — the nvidia kernel module must be signed on systems with Secure Boot enabled.
• If you are on Wayland with a desktop session, confirm your compositor supports NVIDIA (GNOME 46+ and KDE Plasma 6+ work well; older versions may force Xorg fallback).

Method 1: Distro Packages (Recommended for Most Users)

Package-manager installation integrates with DKMS, handles kernel upgrades automatically, and is far easier to roll back.

Ubuntu / Debian

Ubuntu ships NVIDIA drivers through the ubuntu-drivers tool and CUDA through a separate repository.

# Install the recommended driver
sudo ubuntu-drivers install
# Or pick a specific version:
sudo apt install nvidia-driver-550

# Add the CUDA repository (Ubuntu 22.04 example)
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install cuda-toolkit-12-5

Installing cuda-toolkit-12-5 (not the meta-package cuda) avoids pulling in a second copy of the driver. The meta-package cuda installs everything including driver packages, which can conflict if you already have a driver installed.

On Debian, enable non-free and non-free-firmware components, then follow the same CUDA repo steps substituting debian12 in the repo URL.

Fedora / RHEL 9 / Rocky Linux

# Enable RPM Fusion (Fedora) for the driver
sudo dnf install https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm
sudo dnf install https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm
sudo dnf install akmod-nvidia

# Add NVIDIA's CUDA repo for Fedora 40
sudo dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/fedora40/x86_64/cuda-fedora40.repo
sudo dnf install cuda-toolkit-12-5

For RHEL 9 or Rocky 9, substitute rhel9 in the repo URL. The akmod-nvidia package rebuilds the kernel module automatically on kernel upgrades.

Arch Linux

# Driver from official repos
sudo pacman -S nvidia nvidia-utils
# CUDA toolkit
sudo pacman -S cuda

Arch ships a recent CUDA version in the extra repository. The nvidia package is tied to the current kernel; if you run linux-lts, install nvidia-lts instead.

Method 2: NVIDIA Runfile Installer

Use the runfile when you need a specific CUDA version not yet packaged by your distro, or when you are on a distro without an official CUDA repo. It installs into /usr/local/cuda-12.x and does not integrate with your package manager.

Prepare the System

# Stop the display manager so the nvidia module can be unloaded
sudo systemctl isolate multi-user.target
# Blacklist nouveau if not already done
echo 'blacklist nouveau' | sudo tee /etc/modprobe.d/blacklist-nouveau.conf
sudo dracut --force   # Fedora/RHEL
# sudo update-initramfs -u   # Debian/Ubuntu

Reboot after the initramfs update so nouveau is fully unloaded before the runfile runs.

Run the Installer

# Download from https://developer.nvidia.com/cuda-downloads
chmod +x cuda_12.5.0_555.42.02_linux.run
sudo ./cuda_12.5.0_555.42.02_linux.run

In the interactive menu, uncheck the Driver component if you already have a working driver. Only install the CUDA Toolkit, CUDA Samples, and (optionally) the Documentation. Mixing a runfile driver with a distro-managed driver is the number-one cause of broken NVIDIA setups.

Set Up Environment Variables

# Add to ~/.bashrc or /etc/profile.d/cuda.sh
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

source ~/.bashrc

Verifying the Installation with deviceQuery

NVIDIA ships CUDA samples separately since CUDA 11.6. Clone the samples repository, build deviceQuery, and run it.

git clone https://github.com/NVIDIA/cuda-samples.git
cd cuda-samples/Samples/1_Utilities/deviceQuery
make
./deviceQuery

A successful run ends with Result = PASS and prints your GPU name, compute capability, global memory size, and driver/runtime versions. If you see a mismatch between the CUDA Driver Version and the CUDA Runtime Version, your driver is too old for the installed toolkit.

# Also confirm the driver is loaded
nvidia-smi

nvidia-smi output shows the driver version in the top-right corner and the maximum CUDA version the driver supports. This is the ceiling; the toolkit version must be at or below it.

Troubleshooting

Module Not Loading After Reboot

Check dmesg | grep -i nvidia for errors. On systems with Secure Boot, the unsigned module will be silently skipped. Either disable Secure Boot or use mokutil --import to enroll the key generated during DKMS build.

sudo dkms status
sudo dkms autoinstall

nvcc Not Found

The toolkit is installed but nvcc is not in your PATH. For distro packages on Debian/Ubuntu, nvcc lives at /usr/local/cuda/bin/nvcc via a symlink managed by update-alternatives. Run:

sudo update-alternatives --config cuda
nvcc --version

Black Screen After Reboot

Boot to a TTY (Ctrl+Alt+F2), remove the package that installed the display driver, and reinstall a known-good version. On Ubuntu: sudo apt purge nvidia-* then reinstall. Never mix packages from the Ubuntu repo and the CUDA repo for the driver component.

Wrong libcuda.so Linked

If deviceQuery fails with a driver/runtime version mismatch, check which libcuda.so is being resolved:

ldconfig -p | grep libcuda
# Ensure /usr/local/cuda/lib64 is not overriding the system driver stub