n8n queue mode duplicate execution – why workflows run twice and how to fix it

Step by Step Guide to solve n8n queue mode duplicate execution
Step by Step Guide to solve n8n queue mode duplicate execution


Who this is for: n8n administrators and workflow engineers who see the same workflow fire multiple times for a single trigger and need a reliable, production‑grade solution. We cover this in detail in the n8n Queue Mode Errors Guide.


Quick Diagnosis

Symptom: One trigger generates two or more identical workflow runs, causing duplicate records or API calls.

Root cause: The queue worker processes the same executionId more than once because the queue is mis‑configured (e.g., queueMode set to parallel with too many concurrent workers, or the Redis lock TTL is too short).

One‑line fix: Switch to queueMode: "fifo", use a short execution lock (executionTimeout: 30s), limit concurrency, and add a deduplication guard that checks the executionId before proceeding.


1. n8n Queue Mode Basics

If you encounter any n8n queue mode job stuck resolve them before continuing with the setup.

Setting Recommended for deduplication
queueMode fifo – guarantees first‑in‑first‑out ordering, eliminating race conditions.
maxConcurrentExecutions 1‑2 per workflow – prevents multiple workers from grabbing the same job.
executionTimeout 30‑45s – keeps the Redis lock alive for the whole run.
redisLockKeyPrefix Keep default (n8n:queue) but ensure it’s unique per instance.

EEFA note: In production always use a dedicated Redis instance with persistence. An in‑memory store loses locks on restart, instantly spawning duplicates.


2. Why Duplicate Execution Happens

  1. Lock expiration – If a workflow runs longer than executionTimeout, Redis releases the lock and another worker re‑processes the same executionId.
  2. Parallel dequeue – With queueMode: "parallel" and several concurrent workers, multiple workers may read the same pending job before the lock is written.
  3. Improper Redis configuration – Sharing the default localhost:6379 across clustered nodes creates lock‑key collisions.

3. Step‑by‑Step Fix

3.1 Update the Queue Configuration

Queue settings (JSON) – place these in config.json or the equivalent environment file.

{
  "queueMode": "fifo",
  "maxConcurrentExecutions": 1,
  "executionTimeout": 30
}

Redis connection (JSON) – keep the lock prefix default.

{
  "redis": {
    "host": "redis-prod.mycompany.com",
    "port": 6379,
    "password": "••••••••",
    "tls": true
  }
}

Docker / K8s environment variables – preferred for containerised deployments.

N8N_QUEUE_MODE=fifo
N8N_MAX_CONCURRENT_EXECUTIONS=1
N8N_EXECUTION_TIMEOUT=30
N8N_REDIS_HOST=redis-prod.mycompany.com
N8N_REDIS_TLS=true

EEFA: Restart the n8n service after any change and verify with n8n config:get queueMode.

3.2 Insert a Deduplication Guard

Set node – capture the execution ID

{
  "name": "Set executionId",
  "type": "n8n-nodes-base.set",
  "parameters": {
    "values": [
      {
        "name": "executionId",
        "value": "={{$json[\"$executionId\"]}}"
      }
    ]
  }
}

IF node – abort if the ID was already processed

// Returns true when the ID is *not* in Redis
return await $redis.get(`executed:${$json.executionId}`) !== '1';

If the condition fails, route the flow to a **No‑Op** node that simply ends the workflow.

Function node – mark the ID as completed

await $redis.set(`executed:${$json.executionId}`, '1', 'EX', 86400); // keep for 24 h
return {};

3.3 Clean‑up on Successful Completion

Add the Function node above as the final step of the workflow. It guarantees the execution ID is stored for a day, preventing re‑processing if a lock were to expire later. If you encounter any n8n queue mode missing worker process resolve them before continuing with the setup.


4. Monitoring & Preventive Checklist

Check Verification Method
Queue mode is fifo n8n config:get queueModefifo
maxConcurrentExecutions ≤ 2 Inspect config.json or env vars
Execution timeout ≤ 45 s n8n config:get executionTimeout
Redis lock TTL matches timeout redis-cli TTL n8n:queue:<executionId> after a run
No duplicate executionId in DB SELECT COUNT(*) FROM executions WHERE execution_id = ?
Alert on > 1 execution per trigger Grafana/Prometheus alert on n8n_queue_duplicate_total metric

EEFA tip: Enable telemetry (N8N_TELEMETRY_ENABLED=true) only in staging; production telemetry can expose execution IDs.


5. Advanced: External Idempotency Keys

When calling APIs that support idempotency (e.g., Stripe, Salesforce), forward the n8n executionId as the idempotency key:

POST https://api.stripe.com/v1/charges
Idempotency-Key: {{ $json.executionId }}

This adds a second line of defense— the external service itself will reject duplicates.


6. Conclusion

Duplicate execution in n8n occurs when the queue worker re‑processes the same executionId, typically because queueMode is parallel and the lock TTL (executionTimeout) expires under load. Fix it by switching to queueMode: "fifo", limiting maxConcurrentExecutions to 1‑2, and setting executionTimeout to 30 seconds. Add a deduplication guard at the workflow start that aborts if the executionId already exists in Redis, and mark the ID as processed at the end. Monitor lock TTLs and execution counts to catch regressions early.

Leave a Comment

Your email address will not be published. Required fields are marked *