PyTorch is a widely known Deep Learning framework and installs the newest CUDA by default, but what about CUDA 10.1? If you have not updated NVidia driver or are unable to update CUDA due to lack of root access, you may need to settle down with an outdated version such as CUDA 10.1. Which means you can’t use GPU by default in your PyTorch models though. How do I solve it?

Contents

Prerequisite

This tutorial assumes you have CUDA 10.1 installed and you can run python and a package manager like pip or conda. Miniconda and Anaconda are both fine, but Miniconda is lightweight. We wrote an article on how to install Miniconda.

This tutorial assumes that you have CUDA 10.1 installed and that you can run python and a package manager like pip or conda.Miniconda and Anaconda are both good, but Miniconda is lightweight. We wrote an article about how to install Miniconda.

In A Nutshell

First see if CUDA 10.1 is installed
cat /usr/local/cuda/version.txt
[For pip] Run pip3 install with specified version and -f. Here we will install 1.7.0. You can also install 1.3.0, 1.3.1, 1.4.0, 1.5.0, 1.5.1, 1.6.0.
pip3 install torch==1.7.0 torchvision==0.8.1 -f https://download.pytorch.org/whl/cu101/torch_stable.html

Use pip if you are using Python 2.

Note: PyTorch currently supports CUDA 10.1 up to the latest version (Search torch- in https://download.pytorch.org/whl/cu101/torch_stable.html).
[For conda] Run conda install with cudatoolkit
conda install pytorch torchvision cudatoolkit=10.1 -c pytorch
Verify PyTorch is installed
Run Python with

import torch x = torch.rand(5, 3) print(x)
Verify PyTorch is using CUDA 10.1
import torch torch.cuda.is_available()

Verify PyTorch is installed

To ensure that PyTorch has been set up properly, we will validate the installation by running a sample PyTorch script. Here we are going to create a randomly initialized tensor.

import torch
print(torch.rand(5, 3))

The following output will be printed. Yours will be similar.

tensor([[0.3380, 0.3845, 0.3217],
[0.8337, 0.9050, 0.2650],
[0.2979, 0.7141, 0.9069],
[0.1449, 0.1132, 0.1375],
[0.4675, 0.3947, 0.1426]])

Verify if CUDA is available to PyTorch

To check if your GPU driver and CUDA are accessible by PyTorch, use the following Python code to decide if or not the CUDA driver is enabled:

import torch
torch.cuda.is_available()

In the case of people who are interested, the following two parts introduce PyTorch and CUDA.

What is PyTorch?

PyTorch is an open-source Deep Learning framework that is scalable and versatile for testing, reliable and supportive for deployment. It allows for quick, modular experimentation via an autograding component designed for fast and python-like execution. With the introduction of PyTorch 1.0, the framework now has graph-based execution, a hybrid front-end that allows for smooth mode switching, collaborative testing, and effective and secure deployment on mobile platforms.

PyTorch has 4 key features according to its homepage.

PyTorch is production-ready: TorchScript smoothly toggles between eager and graph modes. TorchServe speeds up the production process.
PyTorch support distributed training: The torch.collaborative interface allows for efficient distributed training and performance optimization in research and development.
PyTorch has a robust ecosystem: It has an expansive ecosystem of tools and libraries to support applications such as computer vision and NLP.
PyTorch has native cloud support: It is well recognized for its zero-friction development and fast scaling on key cloud providers.

What is CUDA?

CUDA is a general parallel computation architecture and programming model developed for NVIDIA graphical processing units (GPUs). Using CUDA, developers can significantly improve the speed of their computer programs by utilizing GPU resources.

In GPU-accelerated code, the sequential part of the task runs on the CPU for optimized single-threaded performance, the compute-intensive section, such as PyTorch code, runs on thousands of GPU cores in parallel through CUDA. Developers can code in common languages such as C, C++, Python while using CUDA, and implement parallelism via extensions in the form of a few simple keywords.

NVIDIA’s CUDA Toolkit includes everything you need to build GPU-accelerated software, including GPU-accelerated modules, a parser, programming resources, and the CUDA runtime.

You can learn more about CUDA in CUDA zone and download it here: https://developer.nvidia.com/cuda-downloads.

Reference: https://pytorch.org/get-started/locally/