I’m working on a research project for my bachelor’s degree with two teammates: Akesh and Tharusha. We needed to train large language models (LLMs), which requires powerful computers and our personal computers just weren't enough for this task. Thankfully, Dr. Surangika and Dr. Nisansa: our project su helped us get access to Google Cloud Platform, where we can use more powerful machines. We tried setting up a computer with a GPU on Google Cloud by following various online guides, but none of them worked out at first.
After many hours of trying different methods, I finally set it up successfully. I wanted to share how I did it in this blog, hoping it can save time for anyone else who’s stuck like we were.
Installing NVIDIA drivers
sudo apt-get install build-essential linux-headers-$(uname -r)
select Y
gcc --version
Run following command to download the package
wget https://us.download.nvidia.com/XFree86/Linux-x86_64/535.104.05/NVIDIA-Linux-x86_64-535.104.05.run
Make the downloaded package executable using following command
chmod +x NVIDIA-Linux-x86_64-535.104.05.run
Run the following command to install the package. You need to follow on screen instructions and you may get few warnings mid-way, but you can just ignore them.
sudo ./NVIDIA-Linux-x86_64-535.104.05.run
You can verify your nividia driver installation by executing following command. You may see a table of GPUs if your installation is successfull.
nvidia-smi
Installing CUDA
First make sure you have CUDA capable graphic driver
lspci | grep -i nvidia
Make sure you have a supported version of linux
uname -m && cat /etc/*release
Verify the system has gcc installed
gcc --version
Run following commands
wget https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
I’m not sure what exactly happens here but you need to add a repository to your sources list. To access to sources list you can use the following command.
sudo nano /etc/apt/sources.list
Then add follwing two lines there. Make sure you add the codename of your distribution. (bookworm in my case since I have Debian 12)
deb http://deb.debian.org/debian/ bookworm main contrib non-free
deb-src http://deb.debian.org/debian/ bookworm main contrib non-free
Save and exit from the editor.
Then run following commands
sudo add-apt-repository contrib
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-5
sudo apt-get install -y cuda-drivers
Now you have successfully installed CUDA but it’s not finished yet. You have to add CUDA to the PATH. Execute following commands to do that.
export PATH=/usr/local/cuda-12.5/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.5/lib64:$LD_LIBRARY_PATH
source ~/.bashrc
Well done! Now you can verify your CUDA installation by executing below command.
nvcc --version
Installing Jupyter Notebook
Download the package using below command
cd ~ && curl -O https://repo.anaconda.com/archive/Anaconda3-2021.11-Linux-x86_64.sh
Run following command to install it.
bash Anaconda3-2021.11-Linux-x86_64.sh
Now you need to generate the config file so that you can save some configurations to make your life easier when running it. But first you need to reboot your VM instance.
jupyter notebook --generate-config
Run following command to edit the config file. Make sure to replace the <user_name>.
nano /home/<user_name>/.jupyter/jupyter_notebook_config.py
Add following lines and save the file.
c.NotebookApp.token = ''
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.port = 8888
Finally, you are good to use jupyter notebook. Use following command to run.
jupyter notebook
If you need your jupyter notebook to run even when the terminal is closed add following flags when running the above command
jupyter notebook & disown
&: Puts it into the background of the currently running shell
disown: Removes the job from the background of the running shell and makes it run independently of the shell so that you may close it
Comments