Configuring Database Access for Django and Celery-Beat in Docker

Your issue stems from the different networking contexts between your containerized services and your non-containerized Django application. Let me explain the problem and provide a solution.

The Problem

1. Inside Docker containers (celery-beat): Services need to use the service name postgres to connect to the PostgreSQL database.
2. Outside Docker containers (Django): Your application needs to use localhost to connect to the database.

Solution: Multiple Database Configurations

You can configure Django to use different database connection settings based on the environment. Here's how to implement this:

1. Update your Django settings.py

Replace your single database configuration with a setup that detects the environment:

PYTHONCopied
import os

# Default database configuration
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'postgres',
        'USER': 'postgres',
        'PASSWORD': '1234',
        'PORT': '5432',
    }
}

# Determine if running in Docker
IN_DOCKER = os.environ.get('IN_DOCKER', False)

# Set the appropriate host based on environment
if IN_DOCKER:
    DATABASES['default']['HOST'] = 'postgres'  # Use service name in Docker
else:
    DATABASES['default']['HOST'] = 'localhost'  # Use localhost outside Docker

2. Update your Docker Compose file

Add the environment variable to your celery-beat service:

YAMLCopied
celery-beat:
  build: .
  command: celery -A recinto worker --beat --scheduler django --loglevel=info
  volumes:
    - ./:/code
  depends_on:
    - rabbitmq
    - postgres  # Add dependency on postgres
  environment:
    - IN_DOCKER=True  # Add this environment variable
  networks:
    - recinto_network

Also add the same environment variable to your celery worker:

YAMLCopied
celery:
  build: .
  command: celery -A recinto worker -l info
  volumes:
    - ./:/code
  depends_on:
    - rabbitmq
    - postgres  # Add dependency on postgres
  environment:
    - IN_DOCKER=True  # Add this environment variable
  networks:
    - recinto_network

3. Remove DATABASE_URL from .env

Since you're now handling the database configuration directly in settings.py, you can remove the DATABASE_URL from your .env file to avoid conflicts.

Alternative: Using Multiple Database Configurations

If you prefer to keep using environment variables, you can set up multiple database configurations:

PYTHONCopied
# settings.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'postgres',
        'USER': 'postgres',
        'PASSWORD': '1234',
        'HOST': os.environ.get('DB_HOST', 'localhost'),
        'PORT': '5432',
    }
}

Then in your Docker Compose file:

YAMLCopied
celery-beat:
  # other settings...
  environment:
    - DB_HOST=postgres

And your Django application would use the default 'localhost' value.

Additional Recommendations

1. Add a health check for the PostgreSQL service to ensure it's fully ready before celery-beat tries to connect:

YAMLCopied
postgres:
  # existing configuration...
  healthcheck:
    test: ["CMD", "pg_isready", "-U", "postgres"]
    interval: 10s
    timeout: 5s
    retries: 5

celery-beat:
  # existing configuration...
  depends_on:
    postgres:
      condition: service_healthy

2. Make sure your celery-beat service also depends on the postgres service:

YAMLCopied
celery-beat:
  # existing configuration...
  depends_on:
    - rabbitmq
    - postgres

This approach allows your containerized services to use the Docker network service name while your non-containerized Django application uses localhost, solving your connection issue.