Deploy Ollama

Get up and running with Llama 3, Mistral, Gemma, and other large language models locally.

⭐ 60.0k stars📜 MIT License🔴 Advanced⏱ ~20 minutes

What You’ll Get

A fully working Ollama instance running on your server. Your data stays on your hardware — no third-party access, no usage limits, no surprise invoices.

Prerequisites

A server with Docker and Docker Compose installed (setup guide)
A domain name pointed to your server (optional but recommended)
Basic terminal access (SSH)

The Config

Create a directory for Ollama and add this docker-compose.yml:


# -------------------------------------------------------------------------
# 🚀 Created and distributed by The AltStack
# 🌍 https://thealtstack.com
# -------------------------------------------------------------------------
 
# Docker Compose for Ollama
version: '3.8'
 
services:
  ollama:
    image: ollama/ollama:latest # Official image is highly recommended for GPU support
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    # For GPU support (NVIDIA), uncomment the following:
    # deploy:
    #   resources:
    #     reservations:
    #       devices:
    #         - driver: nvidia
    #           count: all
    #           capabilities: [gpu]
    networks:
      - ollama_net
    healthcheck:
      test: [ "CMD", "curl", "-f", "http://localhost:11434/api/tags" ]
      interval: 10s
      timeout: 5s
      retries: 5
 
networks:
  ollama_net:
    driver: bridge
 
volumes:
  ollama_data:
    name: ollama_data

Let’s Ship It


# Create a directory
mkdir -p /opt/ollama && cd /opt/ollama
 
# Create the docker-compose.yml (paste the config above)
nano docker-compose.yml
 
# Pull images and start
docker compose up -d
 
# Watch the logs
docker compose logs -f

Post-Deployment Checklist

Service is accessible on the configured port
Admin account created (if applicable)
Reverse proxy configured (Caddy guide)
SSL/HTTPS working
Backup script set up (backup guide)
Uptime monitor added (Uptime Kuma)

The “I Broke It” Section

Container won’t start?


docker compose logs ollama | tail -50

Port already in use?


# Find what's using the port
lsof -i :PORT_NUMBER

Need to start fresh?


docker compose down -v  # ⚠️ This deletes volumes/data!
docker compose up -d