Skip to main content

PyTorch CUDA compatibility issues

Overview​

On a GPU VM, sometimes nvidia-smi works fine but PyTorch still can't see the GPU. There are two common causes:

  • PyTorch / CUDA version mismatch: the installed PyTorch requires a newer CUDA than the VM's driver supports
  • CUDA driver initialization failure (Error 802): nvidia-smi works but the CUDA runtime isn't initialized

Symptom 1: PyTorch / CUDA version mismatch​

Error​

CUDA initialization: The NVIDIA driver on your system is too old (found version 12080).
RuntimeError: The NVIDIA driver on your system is too old

Cause​

pip install torch installs the latest PyTorch by default. If that PyTorch requires a newer CUDA than what the VM driver supports, you get the error above.

Example: the VM driver supports CUDA 12.8, but pip installed a PyTorch built for CUDA 13.0.

Fix​

Step 1: Check the VM's CUDA version

nvidia-smi

The top right of the output shows it as CUDA Version: 12.8.

Step 2: Install a PyTorch that matches the driver

Pick the command for your CUDA version:

CUDA versionInstall command
12.8pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu128
12.6pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu126
12.4pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu124
Full compatibility list

The PyTorch official install page lists the install command for each CUDA version.

Step 3: Verify GPU detection

python3 -c "import torch; print(torch.cuda.is_available())"
# True means it's working

Symptom 2: CUDA Error 802 (system not yet initialized)​

Error​

CUDA: False
RuntimeError: Unexpected error from cudaGetDeviceCount(). Error 802: system not yet initialized

Cause​

nvidia-smi uses the NVML interface and works fine, but the CUDA runtime uses a separate driver initialization path. If you run a CUDA workload before that path finishes initializing right after a VM restart, you see this error.

Note

This isn't a permissions issue. sudo produces the same error.

Fix​

Restart the VM, then wait 1–2 minutes and try again.

# Confirm GPU initialization
python3 -c "import torch; torch.cuda.init(); print('GPU OK')"

If the error repeats after a restart, contact support. Include the VM ID and full error text for faster resolution.

When pip3 isn't found​

If you see pip3: command not found, install pip first:

sudo apt-get update && sudo apt-get install -y python3-pip

Automating setup at VM creation (on_init_script)​

Instead of installing manually every time, put it in on_init_script at VM creation, and it runs automatically on first boot.

#!/bin/bash
apt-get update
apt-get install -y python3-pip
pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu128
Where to set on_init_script

VM creation wizard → Basic Info step → Init script field. The script runs only once on the VM's first boot, not on subsequent reboots.

Next steps​