n8n queue mode Docker Compose – workers stuck in Queued and not processing

n8n Queue Mode Errors / Leave a Comment
Step by Step Guide to solve n8n queue mode docker compose issues
Step by Step Guide to solve n8n queue mode docker compose issues

Who this is for: Developers and DevOps engineers deploying n8n with Docker‑Compose who need reliable queue mode operation. We cover this in detail in the n8n Queue Mode Errors Guide.

Quick Diagnosis

# Symptom Immediate Fix
1 Error: Could not connect to Redis Verify REDIS_HOST, REDIS_PORT, REDIS_PASSWORD; ensure the Redis container is healthy.
2 Queue mode disabled – missing “queueMode” flag Add -e EXECUTIONS_PROCESS=queue (or -e QUEUE_MODE=1 for legacy images) to both n8n and n8n‑worker.
3 docker compose up exits with “service … failed to start” Add a healthcheck for Redis and use depends_on with condition: service_healthy.
4 Jobs stay pending Confirm the n8n‑worker container runs with the same queue env vars as the UI container.
5 “connection reset by peer” Raise Redis timeout and increase Docker‑Compose memory limits; avoid CPU/memory starvation.

Apply the first fix that matches your logs; each step below explains the why and how.

1. Verify Redis Connectivity: The Core of Queue Mode

If you encounter any n8n queue mode ssl misconfiguration resolve them before continuing with the setup.

Queue mode stores execution payloads in a single Redis instance (or cluster). If n8n cannot reach Redis it silently falls back to the default “process” mode.

1.1 Essential environment variables

Variable Recommended value Description
EXECUTIONS_PROCESS queue Switches n8n from the default main process to queue mode.
REDIS_HOST redis (service name) Must match the Redis service name in the same compose file.
REDIS_PORT 6379 Default Redis port; change only if you expose a non‑standard port.
REDIS_PASSWORD (optional) Set when the Redis container uses requirepass.
REDIS_TLS true/false Enable for TLS‑enabled Redis.

EEFA note – In production always set REDIS_PASSWORD and enable TLS (REDIS_TLS=true). Missing these can cause silent connection failures that are hard to debug.

1.2 Minimal working Docker‑Compose – Redis service

services:
  redis:
    image: redis:7-alpine
    restart: unless-stopped
    command: ["redis-server", "--appendonly", "yes"]
    ports:
      - "6379:6379"
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5

The Redis container runs with AOF persistence and a healthcheck that returns PONG when ready.

1.3 Minimal working Docker‑Compose: n8n UI container

  n8n:
    image: n8nio/n8n:latest
    restart: unless-stopped
    ports:
      - "5678:5678"
    environment:
      - EXECUTIONS_PROCESS=queue
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - REDIS_PASSWORD=${REDIS_PASSWORD:-}
    depends_on:
      redis:
        condition: service_healthy
    volumes:
      - n8n_data:/home/node/.n8n
    command: ["n8n", "start"]

*The UI container is configured to use queue mode and waits for a healthy Redis service.*

1.4 Minimal working Docker‑Compose – n8n worker container

  n8n-worker:
    image: n8nio/n8n:latest
    restart: unless-stopped
    environment:
      - EXECUTIONS_PROCESS=queue
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - REDIS_PASSWORD=${REDIS_PASSWORD:-}
    depends_on:
      redis:
        condition: service_healthy
    command: ["n8n", "worker"]
    volumes:
      - n8n_data:/home/node/.n8n

Both UI and worker share identical Redis settings; the worker processes queued jobs.

1.5 Connectivity checklist

  • Run docker compose exec redis redis-cli ping → PONG.
  • Search n8n logs for “Connected” with docker compose logs n8n | grep -i redis.
  • Ensure no AUTH errors appear.

If any check fails, adjust REDIS_HOST, REDIS_PORT, or REDIS_PASSWORD and redeploy.

2. Run Both UI and Worker in Queue Mode

Launching only the UI container in queue mode leaves jobs stranded.

2.1 Dual‑service pattern

Service Required env vars Critical flag
n8n (Web UI) EXECUTIONS_PROCESS=queue + Redis vars Must expose port 5678
n8n-worker Same as UI Executes queued jobs

Both containers must exactly match the Redis connection details.

2.2 Quick log verification

docker compose logs n8n | grep "Queue mode"
docker compose logs n8n-worker | grep "Queue mode"

You should see lines such as:

[2026-01-02 12:34:56] INFO  Queue mode enabled – using Redis at redis:6379

If the worker log lacks this line, the EXECUTIONS_PROCESS=queue variable is missing or overridden. If you encounter any n8n queue mode kubernetes deployment errors resolve them before continuing with the setup.

3. Service Dependency & Healthcheck Best Practices

Docker Compose may start n8n before Redis is ready, causing a transient fallback to process mode.

3.1 Proper depends_on with healthcheck

depends_on:
  redis:
    condition: service_healthy

*Compose now waits until the Redis healthcheck (`PING` → **PONG**) succeeds.*

3.2 Common failure patterns

Symptom Root cause Fix
Connection refused on start‑up Redis not ready Add healthcheck + depends_on.
Service redis failed to start Host‑volume permission error Ensure the host directory is chmod 700 and owned by UID 1000 (node).
Redis out of memory No maxmemory limit, large payloads Add command: ["redis-server", "--maxmemory", "256mb", "--maxmemory-policy", "allkeys-lru"].

EEFA warning – Using restart: always without healthchecks can create a restart loop where n8n repeatedly attempts to connect to a dead Redis. Prefer restart: unless-stopped combined with healthchecks.

4. Connecting to an External Redis Instance

When Redis runs outside the Compose network (e.g., managed AWS ElastiCache), you must expose the endpoint correctly.

4.1 External Redis configuration snippet

services:
  n8n:
    ...
    environment:
      - REDIS_HOST=redis-prod.mycompany.com
      - REDIS_PORT=6379
      - REDIS_PASSWORD=superSecret
    extra_hosts:
      - "redis-prod.mycompany.com:10.0.5.12"

extra_hosts provides a static IP mapping for DNS‑less environments.*

4.2 Connectivity debugging

docker compose exec n8n nslookup redis-prod.mycompany.com
docker compose exec n8n bash -c "echo > /dev/tcp/redis-prod.mycompany.com/6379"

If DNS resolution fails, add the host entry to extra_hosts or attach the services to a custom network that has proper DNS resolution. Avoid network_mode: host in production.

5. Resource Limits & Queue Back‑Pressure

High‑throughput queues can exhaust CPU or memory, leaving executions stuck in pending.

5.1 Recommended limits – UI container

  n8n:
    deploy:
      resources:
        limits:
          cpus: "2"
          memory: "2G"

5.2 Recommended limits – Worker container

  n8n-worker:
    deploy:
      resources:
        limits:
          cpus: "2"
          memory: "2G"

Both containers receive the same CPU and memory caps to avoid bottlenecks.

5.3 Monitoring checklist

Metric Tool Threshold
Redis connected_clients redis-cli info clients ≤ 1000
n8n‑worker CPU usage docker stats n8n-worker ≤ 80 %
n8n‑worker memory RSS docker stats n8n-worker ≤ 1.8 GB (90 % of limit)
Queue length (n8n_queue) redis-cli LLEN n8n_queue < 5000 (adjust per workload)

If thresholds are exceeded, increase the limits or scale out additional workers (see the child page Scaling n8n Queue Mode with Docker‑Swarm).

6. Advanced Debugging: Capture Full Redis Traffic

When logs are inconclusive, capture the Redis protocol exchange.

6.1 Enable verbose Redis logging (development only)

    command: ["redis-server", "--appendonly", "yes", "--loglevel", "debug"]

Debug logging is I/O‑heavy; disable it in production.

6.2 Record traffic with tcpdump inside the n8n container

docker compose exec n8n apt-get update && apt-get install -y tcpdump
docker compose exec n8n tcpdump -i any -nn -s 0 -w /tmp/redis.pcap port 6379

Open redis.pcap in Wireshark and filter for redis to spot AUTH failures or malformed commands.

7. Common Pitfalls & Preventive Actions

Pitfall Why it breaks queue mode Preventive action
Missing EXECUTIONS_PROCESS=queue on one container Worker never picks jobs Store env vars in a shared .env and reference it via env_file: .env for both services.
Running n8n with --prisma while Redis mis‑configured Prisma falls back to SQLite, bypassing queue Remove --prisma unless you need a dedicated DB; queue mode does not depend on Prisma.
Host volume mounted with wrong UID/GID n8n cannot write queue state, causing silent errors Ensure the volume owner matches node UID 1000: docker compose run --rm n8n chown -R 1000:1000 /home/node/.n8n.
Using network_mode: host on macOS Host DNS is bypassed, breaking REDIS_HOST resolution Stick to bridge networking; add static entries via extra_hosts if needed.
Redis persistence disabled & container restarts Queue data lost → jobs disappear mid‑execution Enable AOF (--appendonly yes) or mount a persistent volume at /data.

Recap

  1. Add EXECUTIONS_PROCESS=queue and matching Redis vars to both n8n and n8n‑worker.
  2. Ensure Redis is healthy (depends_on + healthcheck).
  3. Verify connectivity (docker compose exec n8n redis-cli ping).
  4. Confirm both containers share the same REDIS_HOST/REDIS_PORT.
  5. Raise CPU/memory limits if jobs stay “pending”.

Apply these steps in order; the first that resolves the log error will bring queue mode back online.

Must Read

Leave a Comment

Your email address will not be published. Required fields are marked *