Installing Tensorflow-GPU with NVIDIA CUDA on a Google Cloud Platform VM instance

As a software engineer and part of Analytics and Machine Learning team at Searce, I tried to build a project with Tensorflow-GPU and NVIDIA CUDA configured VM instance on Google Cloud Platform. During the setup, I faced many challenges and I couldn’t find a proper source which consolidates each step to achieve the goal. So, here I am writing this post which will describe each step in detail. I hope this will be helpful to those who tried something similar on GCP.

Steps in detail:

1. Setup a Google Cloud Instance: Create an instance with your choice of name and region(Although not all regions provide GPUs).I have chosen us-west1 as region. In machine type, select 4 vCPUs. Customise your machine type by selecting number of GPUs.

Instance Configuration

In Boot Disk option, change OS image to Ubuntu16.04 LTS. You can increase the Boot Disk size also. By default, it’s 10 GB. I have increased it to 50 GB.Allow HTTP and HTTPS traffic for your instance.After that click on Management, disks, networking, SSH keys option. You need to add the startup script which will automatically download the required NVIDIA GPU drivers for the graphics card. The latest version of the script can be found here or you can type “gpu script” in help option in GCP console itself. Select script for ubuntu-16.04 LTS and CUDA-9. Paste the following script in Startup Script option.You can either add this script in Startup Script option or install CUDA drivers from the site but I recommend to use the following script.

2. Connecting to SSH Server : Once the instance is set up, hit the SSH button to connect with SSH server. To stay up-to-date with the SSH server, hit the command.

sudo apt-get update

Once it’s done, you can go to the official Tensorflow site for GPU installation. The 1st and 2nd instructions are already satisfied in our case. To get the copy of cuDNN library for CUDA, we are required to get NVIDIA membership. You can become a NVIDIA developer for free using their official website. Once you are done, you need to download a proper version of cuDNN for your instance. You are given different options for different operating system. In our case, we will be using cuDNN v7.1.4 Library for Linux. This is the most important step as there are many dependencies on this cuDNN version. This cuDNN version is for CUDA 9.0 version.

NVIDIA cuDNN download

3. Upload and Unpack cuDNN tar file: To upload cuDNN tar folder to your instance, run the following google cloud command,

gcloud compute scp /Users/meghadesai/Desktop/cudnn-9.0-linux-x64-v7.1.tgz <username>@test-gpu:~/

To check whether the file is uploaded to a proper destination, use ls command. You should be able to find .tar file. To unpack this file, use the following command.

tar -xzvf cudnn-9.0-linux-x64-v7.1.tgz

Once the file is unpacked, use the following command to copy this to cuda directory.

sudo cp cuda/include/cudnn.h /usr/local/cuda/include
sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*

4. Installing all necessary python packages: While cuDNN library is getting downloaded, we can install all necessary python packages which might be used in future. Install ipython and pip using following commands.Press ‘y’ for each yes/no question to install. Make sure that you are getting pip for python3.

sudo apt-get install ipython
sudo apt-get install python3-pip

Some more python packages which I will be using in future are numpy, pandas, scipy. You can install these packages according to your need but I recommend to install the basic python packages. Use following command to install.

sudo pip3 install numpy scipy matplotlib ipython jupyter pandas sympy nose

It may take some time to install all these packages. Meanwhile, you can read more about NVIDIA cuDNN library and its features here.

After installing all above packages, run the following command. This command is required because we will be using Tensorflow from the source.

sudo apt-get install openjdk-8-jdk git python-dev python3-dev python-numpy python3-numpy python-six python3-six build-essential python-pip python3-pip python-virtualenv swig python-wheel python3-wheel libcurl3-dev libcupti-dev

Now, run the following command.

nano ~/.bashrc

Add these two following lines of code, to make sure that our VM instance knows where the CUDA file is which we uploaded and unpacked in the previous step.

export LD_LIBRARY_PATH=”$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64" 
export CUDA_HOME=/usr/local/cuda

Then, use the source command to reload the file.

source ~/.bashrc

Now, we are adding repository of bazel for google cloud apis.

echo “deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8” | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub... | sudo apt-key add -
sudo apt-get install bazel
sudo apt-get upgrade bazel

5. Install Tensorflow on instance : Now, clone Tensorflow’s git directory to your instance using following command. It will install the latest version of tensorflow.

git clone https://github.com/tensorflow/tensorflow
cd ~/tensorflow
./configure

Now, run the following commands in this particular sequence.

/usr/bin/python3.5

After running this command, press enter until you get to “Please input the desired Python library path to use”. Hit the following command.

/usr/local/lib/python3.5/dist-packages

Press enter until you get to “Do you wish to build TensorFlow with CUDA support?” Press ‘y’ as a reply and hit enter.Then, run these following commands to copy the files to the tensorflow temp folder.

bazel build — config=opt — config=cuda //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

This might take some time to run but remember patience is the key! :))

Once the commands for bazel file are implemented successfully, run the following command and exit the SSH window.

sudo pip3 install /tmp/tensorflow_pkg/tensorflow [PRESS TAB TO COMPLETE FILENAME]

Now, Again connect to SSH server window and try out the following code to check if the GPU is set and being utilised.Type python3.It will open python interpreter and implement the following code.

import tensorflow as tf
with tf.device('/cpu:0'):
    a_c = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a-cpu')
    b_c = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b-cpu')
    c_c = tf.matmul(a_c, b_c, name='c-cpu')
with tf.device('/gpu:0'):
    a_g = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a-gpu')
    b_g = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b-gpu')
    c_g = tf.matmul(a_g, b_g, name='c-gpu')
with tf.Session(config=tf.ConfigProto(log_device_placement=True)) as sess:
    print (sess.run(c_c))
    print (sess.run(c_g))
print 'DONE!'

Output of the code

If you get the similar output then you have setup the GPU successfully and you are done!!!

Thank you for reading this and I hope you installed CUDA successfully on your VM instance for your next super-duper next-gen project. Feel free to check out more interesting articles on our Searce Engineering Blog.

Thank you!