Getting Blender to use CUDA in a render farm

In an earlier article I covered how I got the NVIDIA Tesla K10 GPU cards working under Ubuntu 20.04 with CUDA. In this article I will cover how I got Blender working to allow me to use them.

As a reminder I have a pair of ASUS ESC 4000 2U servers, each dual 8 core Xeon (32 CPU threads), 64GB of system memory & each containing 4 NVIDIA Tesla K10 GPU cards. Each GPU card contains 2 GPUS with 4GB of memory each, totalling 8 GPUS & 32GB of memory just for graphics.

Note: The very first time you run Blender it will compile the CUDA driver it needs. This will take a few minutes before it can do any rendering.

Now Blender has all of it’s config within each individual .blend file, so you have to set it up to use GPU’s within the UI – which isn’t possible on a headless system installed in a rack.

To get around this we have the .blend file configured to use just the CPU & we then use a pair of shell and Python wrappers to use the GPU’s.

Firs the shell script. This one renders the first frame in the .blend file:

#!/bin/sh

RENDER_DIR=$(pwd)/render

mkdir -p ${RENDER_DIR}

blender -b intro.blend \
	-o ${RENDER_DIR}/frame_##### -F PNG -x 1 \
	-P gpurender.py \
	-E CYCLES \
	-t 0 \
	-f 1 \
	-- \
	--cycles-device CUDA \
	--cycles-print-stats

Here it’s set to render the first frame (-f 1) from the file intro.blend.

It will render the frame into the render directory (which it will create) as a png image. This is setup with the -o and -F parameters.

The other parameters tell it to use the CYCLES render engine and the -P refers to a python script. This script tells blender to use the GPU’s:

import bpy

bpy.data.scenes[0].render.engine = "CYCLES"

# Set the device_type
bpy.context.preferences.addons[
    "cycles"
].preferences.compute_device_type = "CUDA"

# Set the device and feature set
bpy.context.scene.cycles.device = "GPU"

# get_devices() to let Blender detects GPU device
bpy.context.preferences.addons["cycles"].preferences.get_devices()
print(bpy.context.preferences.addons["cycles"].preferences.compute_device_type)
for d in bpy.context.preferences.addons["cycles"].preferences.devices:
    d["use"] = 0
    if d["name"][:6] == 'NVIDIA':
        d["use"] = 1
    print(d["name"], d["use"])

What the script does is ensures the scene uses CYCLES, then selects all NVIDIA GPU’s for rendering but disables the use of the CPU.

When run you should then see the frame being rendered, something like the following:

If you have installed nvtop you can also see the GPU’s in action during the render:

Now the python script works if the .blend file has a single scene. It could be reworked to support multiple scenes but for now it does what I need.

To render the entire animation in one go you just need to change the shell script from rendering a single frame (-f 1) to all frames (-a):

#!/bin/sh

RENDER_DIR=$(pwd)/render

mkdir -p ${RENDER_DIR}

blender -b intro.blend \
	-o ${RENDER_DIR}/frame_##### -F PNG -x 1 \
	-P gpurender.py \
	-E CYCLES \
	-t 0 \
	-a \
	-- \
	--cycles-device CUDA \
	--cycles-print-stats

Just make certain that the render directory is empty, run that script & come back in a few hours. Once done you should see that directory full of png images, one per frame, ready to be formed into an animation.