Skip to main content
This tutorial shows how to create custom Pod templates by extending official Runpod base images with additional Python dependencies and pre-baked ML models. You’ll learn the complete workflow from Dockerfile creation to deployment and testing. Custom templates allow you to package your specific dependencies, models, and configurations into reusable Docker images that can be deployed as Pods. This approach saves time during Pod initialization and ensures consistent environments across deployments.

​
What you’ll learn

In this tutorial, you’ll learn how to:
  • Create a Dockerfile that extends a Runpod base image.
  • Add Python dependencies to an existing base image.
  • Pre-package ML models into your custom template.
  • Build and push Docker images with the correct platform settings.
  • Deploy and test your custom template as a Pod.

​
Requirements

Before you begin, you’ll need:
  • A Runpod account.
  • Docker installed on your local machine.
  • A Docker Hub account for hosting your custom images.
  • At least $5 in Runpod credits for testing.
  • Basic familiarity with Docker and command-line operations.

​
Step 1: Create a custom Dockerfile

First, you’ll create a Dockerfile that extends a Runpod base image with additional dependencies:
  1. Create a new directory for your custom template:
mkdir my-custom-template
cd my-custom-template
  1. Create a new Dockerfile:
touch Dockerfile
  1. Open the Dockerfile in your preferred text editor and add the following content:
# Use the specified base image
FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04

# Install additional Python dependencies
RUN pip install --no-cache-dir \
    transformers \
    accelerate

# Don't specify a CMD here to use the base image's CMD
This extends the official Runpod PyTorch 2.8.0 base image, and installs two additional Python packages. This means that these packages will be automatically installed every time the Pod starts, so you won’t need to run pip install again after Pod restarts.
When building custom templates, always start with a Runpod base image that matches your CUDA requirements. The base image includes essential components like the /start.sh script that handles Pod initialization.
To maintain access to packaged services (like JupyterLab and SSH over TCP), we avoid specifying a CMD or ENTRYPOINT in the Dockerfile. Runpod base images include a carefully configured startup script (/start.sh) that handles Pod initialization, SSH setup, and service startup. Overriding this can break Pod functionality.

​
Step 2: Add system dependencies

If your application requires system-level packages, add them before the Python dependencies: 
# Use the specified base image
FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04

# Update package list and install system dependencies
RUN apt-get update && apt-get install -y \
    git \
    wget \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Install additional Python dependencies
RUN pip install --no-cache-dir \
    transformers \
    accelerate \
    datasets \
    torch-audio

# Don't specify a CMD here to use the base image's CMD
Always clean up package lists with rm -rf /var/lib/apt/lists/* after installing system packages to reduce image size.

​
Step 3: Pre-bake ML models

To reduce Pod setup overhead, you can pre-download models during the Pod initialization process. Here are two approaches:

​
Method 1: Simple model download script

Create a Python script that downloads your model:
  1. Create a file named download_model.py in the same directory as your Dockerfile:
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Download and cache the model
model_name = "microsoft/DialoGPT-medium"
print(f"Downloading {model_name}...")

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

print("Model downloaded and cached successfully!")
  1. Update your Dockerfile to include and run this script:
# Use the specified base image
FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04

# Install additional Python dependencies
RUN pip install --no-cache-dir \
    transformers \
    accelerate

# Copy and run model download script
COPY download_model.py /tmp/download_model.py
RUN python /tmp/download_model.py && rm /tmp/download_model.py

# Don't specify a CMD here to use the base image's CMD

​
Method 2: Using the Hugging Face CLI

For more control over model downloads, use the Hugging Face CLI: 
# Use the specified base image
FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04

# Install additional Python dependencies
RUN pip install --no-cache-dir \
    transformers \
    accelerate \
    huggingface_hub

# Pre-download specific model files
RUN python -c "from huggingface_hub import snapshot_download; snapshot_download('microsoft/DialoGPT-medium', cache_dir='/root/.cache/huggingface')"

# Don't specify a CMD here to use the base image's CMD
Pre-baking large models will significantly increase your Docker image size and build time. Consider whether the faster Pod startup time justifies the larger image size for your use case.

​
Step 4: Build and push your Docker image

Now you’re ready to build your custom image and push it to Docker Hub:
  1. Build your Docker image with the correct platform specification:
docker build --platform=linux/amd64 -t my-custom-template:latest .
The --platform=linux/amd64 flag is crucial for Runpod compatibility. Runpod’s infrastructure requires AMD64 architecture images.
  1. Tag your image for Docker Hub (replace YOUR_USERNAME with your Docker Hub username):
docker tag my-custom-template:latest YOUR_USERNAME/my-custom-template:latest
  1. Push the image to Docker Hub:
docker push YOUR_USERNAME/my-custom-template:latest
If you haven’t logged into Docker Hub from your command line, run docker login first and enter your Docker Hub credentials.

​
Step 5: Create a Pod template in Runpod

Next, create a Pod template using your custom Docker image:
  1. Navigate to the Templates page in the Runpod console.
  2. Click New Template.
  3. Configure your template with these settings:
    • Name: Give your template a descriptive name (e.g., “My Custom PyTorch Template”).
    • Container Image: Enter your Docker Hub image name (e.g., YOUR_USERNAME/my-custom-template:latest).
    • Container Disk: Set to at least 20 GB to accommodate your custom dependencies.
    • Volume Disk: Set according to your storage needs (e.g., 20 GB).
    • Volume Mount Path: Keep the default /workspace.
    • Expose HTTP Ports: Add 8888 for JupyterLab access.
    • Expose TCP Ports: Add 22 if you need SSH access.
  4. Click Save Template.

​
Step 6: Deploy and test your custom template

Now you’re ready to deploy a Pod using your custom template to verify everything works correctly:
  1. Go to the Pods page in the Runpod console.
  2. Click Deploy.
  3. Choose an appropriate GPU (make sure it meets the CUDA version requirements of your base image).
  4. Click Change Template and select your custom template under Your Pod Templates.
  5. Fill out the rest of the settings as desired, then click Deploy On Demand.
  6. Wait for your Pod to initialize (this may take 5-10 minutes for the first deployment).

​
Step 7: Verify your custom template

Once your Pod is running, verify that your customizations work correctly:
  1. Find your Pod on the Pods page and click on it to open the connection menu. Click Jupyter Lab under HTTP Services to open JupyterLab.
  2. Create a new Python notebook and test your pre-installed dependencies:
# Test that your custom packages are installed
import transformers
import accelerate
print(f"Transformers version: {transformers.__version__}")
print(f"Accelerate version: {accelerate.__version__}")

# If you pre-baked a model, test loading it
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "microsoft/DialoGPT-medium"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

print("Model loaded successfully!")
  1. Run the cell to confirm everything is working as expected.

​
Advanced customization options

​
Setting environment variables

You can set default environment variables in your template configuration:
  1. In the template creation form, scroll to Environment Variables.
  2. Add key-value pairs for any environment variables your application needs:
    • Key: HUGGINGFACE_HUB_CACHE
    • Value: /workspace/hf_cache

​
Adding startup scripts

To run custom initialization code when your Pod starts, create a startup script:
  1. Create a startup.sh file in your project directory:
#!/bin/bash
echo "Running custom startup script..."
mkdir -p /workspace/models
echo "Custom startup complete!"
  1. Add it to your Dockerfile:
# Copy startup script
COPY startup.sh /usr/local/bin/startup.sh
RUN chmod +x /usr/local/bin/startup.sh

# Modify the start script to run our custom startup
RUN echo '/usr/local/bin/startup.sh' >> /start.sh

​
Using multi-stage builds

For complex applications, use multi-stage builds to reduce final image size: 
# Build stage
FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04 as builder

# Install build dependencies
RUN apt-get update && apt-get install -y build-essential
RUN pip install --no-cache-dir some-package-that-needs-compilation

# Final stage
FROM runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04

# Copy only the necessary files from builder
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages

# Install runtime dependencies
RUN pip install --no-cache-dir transformers accelerate

​
Troubleshooting

Here are solutions to common issues when creating custom templates:

​
Build failures

  • Platform mismatch: Always use --platform=linux/amd64 when building.
  • Base image not found: Verify the base image tag exists on Docker Hub.
  • Package installation fails: Check that package names are correct and available for the Python version in your base image.

​
Pod deployment issues

  • Pod fails to start: Check the Pod logs in the Runpod console for error messages.
  • Services not accessible: Ensure you’ve exposed the correct ports in your template configuration.
  • CUDA version mismatch: Make sure your base image CUDA version is compatible with your chosen GPU.

​
Performance issues

  • Slow startup: Consider pre-baking more dependencies or using a smaller base image.
  • Out of memory: Increase container disk size or choose a GPU with more VRAM.
  • Model loading errors: Verify that pre-baked models are in the expected cache directories.

​
Best practices

Follow these best practices when creating custom templates:

​
Image optimization

  • Use .dockerignore to exclude unnecessary files from your build context.
  • Combine RUN commands to reduce image layers.
  • Clean up package caches and temporary files.
  • Use specific version tags for dependencies to ensure reproducibility.

​
Security considerations

  • Don’t include sensitive information like API keys in your Docker image.
  • Use Runpod Secrets for sensitive configuration.
  • Regularly update base images to get security patches.

​
Version management

  • Tag your images with version numbers (e.g., v1.0.0) instead of just latest.
  • Keep a changelog of what changes between versions.
  • Test new versions thoroughly before updating production templates.

​
Next steps

Now that you have a working custom template, consider these next steps:
  • Automate builds: Set up GitHub Actions or similar CI/CD to automatically build and push new versions of your template.
  • Share with team: If you’re using a team account, share your template with team members.
  • Create variations: Build specialized versions of your template for different use cases (development vs. production).
  • Monitor usage: Track how your custom templates perform in production and optimize accordingly.
For more advanced template management, see the Template Management API documentation to programmatically create and update templates.
⌘I