GPU Job Basics

Last updated on 2026-08-24 | Edit this page

Overview

Questions

  • When should I use a GPU?
  • How do I request GPUs in SLURM?
  • Which GPU type should I choose?
  • How do I write a GPU batch script?

Objectives

  • Understand when GPUs accelerate your work and when they do not
  • Master --gres=gpu syntax for GPU requests
  • Match GPU requirements to available hardware
  • Write and submit GPU batch scripts

When Should You Use a GPU?

Workloads That Benefit

  • Deep Learning: Training neural networks (PyTorch, TensorFlow)
  • Matrix Math: Large matrix multiplications, dense linear algebra
  • GPU-Accelerated Libraries: RAPIDS, CuPy, Dask-CUDA
  • Molecular Dynamics: GPU-accelerated GROMACS, NAMD
  • Custom CUDA Code: Parallel kernels

Workloads That Do NOT Benefit

  • Single-threaded serial code
  • I/O-bound jobs (waiting for disk/network)
  • Light computation (simple statistics, data cleaning)
  • Memory-bound sequential code
Callout

Don’t Waste GPU Resources

Before submitting a GPU job:

  1. Test on CPU first
  2. Run with nvidia-smi and check utilization
  3. If GPU utilization is below 50%, run on CPU instead

GPU nodes are expensive and limited. Wasting them slows everyone else down.

Requesting GPUs

Syntax

BASH 

#SBATCH --gres=gpu:N              # N GPUs, any type
#SBATCH --gres=gpu:TYPE:N         # N GPUs of specific type

Examples

BASH 

#SBATCH --gres=gpu:1              # 1 GPU, any type (fastest to allocate)
#SBATCH --gres=gpu:a100:1         # 1 A100 (80 GB, production)
#SBATCH --gres=gpu:l40s:1         # 1 L40S (48 GB, prototyping)
#SBATCH --gres=gpu:l40s:1         # 1 L40S (48 GB, inference)
#SBATCH --gres=gpu:2              # 2 GPUs, any type

Which GPU to Choose

  • Starting a project: gpu:1 (let SLURM choose)
  • Need lots of GPU memory: gpu:a100:1 (80 GB)
  • Production training: gpu:a100:1 (fastest)
  • Prototyping: gpu:l40s:1 (most available)

GPU Batch Script

BASH 

#!/bin/bash -l
#SBATCH --job-name=train_model
#SBATCH --partition=gpu
#SBATCH --gres=gpu:a100:1
#SBATCH --cpus-per-task=8
#SBATCH --mem=64G
#SBATCH --time=12:00:00
#SBATCH --output=train_%j.out

module load cuda miniconda3
source ~/ml_env/bin/activate

python train.py --epochs 100 --batch_size 128

Key points:

  • --partition=gpu: Must use the gpu partition
  • --gres=gpu:a100:1: Request the GPU
  • --cpus-per-task=8: GPUs still need CPU support threads
  • module load cuda: Required for CUDA libraries

Interactive GPU Testing

Test GPU code before submitting batch jobs:

BASH 

srun --partition=gpu --gres=gpu:l40s:1 \
     --cpus-per-task=4 --mem=16G --time=00:30:00 \
     --pty bash -l

Inside the session:

BASH 

module load cuda miniconda3
python -c "import torch; print(torch.cuda.is_available())"
Challenge

Challenge 1: GPU Test Job

Create and submit a GPU test script:

BASH 

#!/bin/bash -l
#SBATCH --job-name=gpu_test
#SBATCH --partition=gpu
#SBATCH --gres=gpu:1
#SBATCH --cpus-per-task=2
#SBATCH --mem=8G
#SBATCH --time=00:05:00
#SBATCH --output=gpu_%j.out

module load cuda miniconda3

python << 'EOF'
import torch
print(f"CUDA available: {torch.cuda.is_available()}")
if torch.cuda.is_available():
    print(f"GPU: {torch.cuda.get_device_name(0)}")
    x = torch.randn(1000, 1000).cuda()
    y = torch.mm(x, x)
    print(f"GPU computation result shape: {y.shape}")
EOF

Submit with sbatch and check the output.

Output should show CUDA available: True, the GPU model name, and a torch.Size([1000, 1000]) result. This confirms your GPU job is configured correctly and the GPU is being used.

Key Points
  • GPUs accelerate parallel computation: deep learning, matrix math, CUDA code
  • Do not use GPUs for serial, I/O-bound, or light computational tasks
  • Request GPUs with --gres=gpu:TYPE:N (e.g., --gres=gpu:a100:1)
  • Always use --partition=gpu and module load cuda for GPU jobs
  • A100 for production, L40S for prototyping and inference, RTX PRO 6000 for large-memory work
  • Test GPU code interactively before submitting long batch jobs