Late last year I got a couple of ASUS 2U server’s off eBay which are now racked in my home lab. They came with some Nvidia Tesla K10 GPU compute cards in them, 4 in each, and I wanted to get them setup so that I could use CUDA & potentially rendering with either Blender or Davinci Resolve as a render farm.
As I installed Ubuntu 20.04 in them, I thought it was going to be as simple as just installing CUDA – which it was with a minor setback. The current version of CUDA is 11 which Nvidia decided to remove support for compute level 3.0 devices including the K10’s I have, so I had to install the earlier version.
The Tesla K10 card is a double height compute GPU card. They were launched in May 2012, so are 9 years old, which is why I’m not surprised Nvidia have removed support from the latest version of CUDA.
The main difference between a Compute & a Graphics card is that a compute card has no output ports. Each card consists of 2 GPU’s each with 4Gb of memory, 1536 shading units, 128 texture mapping units on each. Processing wise each GPU can handle 2.289 TFLOPS 32 bit floating point & 95.36 GFLOPS at 64 bit floating point.
Looking at reviews when they launched the RRP for these cards was US$5099 each, & the 2 servers I got off Ebay had 4 of them in each. Won’t say exactly how much I paid for them but it was around 2% of what they must have cost originally.
The servers themselves are ASUS ESC 4000 G2’s with dual 8-core Xeon’s, 32 CPU threads total & 64Gb of system RAM. The My PlayHouse channel on YouTube has one of these servers but he’s only got 1 GPU running, but mine has 4 but it don’t sound as loud as his does.
Back to getting CUDA working. Now as I had installed the current version of CUDA, I had to first remove the existing version:
sudo rm /etc/apt/sources.list.d/cuda* sudo vi /etc/apt/sources.list sudo apt remove --autoremove nvidia-cuda-toolkit sudo apt remove --autoremove nvidia-* sudo apt-get purge nvidia* sudo apt-get autoremove -y sudo apt-get autoclean -y
Next I had to add the correct NVIDIA ppa & then install cuda-10-1:
sudo apt update sudo add-apt-repository ppa:graphics-drivers sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub sudo bash -c 'echo "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/cuda.list' sudo bash -c 'echo "deb http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/cuda_learn.list' sudo apt update sudo apt install cuda-10-1 libcudnn7 nvtop
Next I had to install gcc-8. I’m not 100% certain if CUDA 10 also requires gcc-8 but Ubuntu 20.04 has a newer version by default & blender complained it couldn’t compile the CUDA driver until I installed gcc-8 & made it the default:
sudo apt install gcc-8 g++-8 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 110 --slave /usr/bin/g++ g++ /usr/bin/g++-8 --slave /usr/bin/gcov gcov /usr/bin/gcov-8
Once this was done, a reboot to make certain it was all running I was able to get Blender to use CUDA using all 4 cards installed,each card has 2 GPU’s with 4GB of ram each.
How I got blender to work with all 8 GPU’s will be a separate article.