How to Use n8n with Custom APIs – Step‑by‑Step Guide, Code Samples & Troubleshooting

Who this is for: Developers who need to call a private or non‑standard REST API from an n8n workflow without creating a custom node. In practice this often occurs when the API requires a non‑standard auth header or an unusual pagination scheme. We cover this in detail in the n8n Architectural Decision Making Guide.

---

Overview

1. Add an HTTP Request node – set URL, method, and authentication (Bearer/OAuth2).
2. Define headers & query parameters – use UI fields or expressions.
3. Implement pagination – pick the pattern (offset/limit, cursor, or Link header) and use Loop / SplitInBatches.
4. Enable retries & error handling – configure retry options and attach an Error Trigger.
5. Transform the response – flatten JSON with Set or Function nodes.
6. Test → stage → production – use environment‑specific credentials and monitoring.

---

Quick Diagnosis

If you encounter any n8n webhooks for developers resolve them before continuing with the setup.

A private or non‑standard REST API is not covered by n8n’s built‑in integrations. The objective is to call the API, manage authentication, pagination, and errors, and pass the response to downstream nodes without creating a custom node.

---

1. Preparing Your API – What n8n Needs to Know

If you encounter any n8n for internal tools vs core systems resolve them before continuing with the setup.

API Characteristic	n8n Setting	Typical Value
Base URL	HTTP Request → URL	https://api.example.com/v1
Auth Type	Authentication dropdown	None / Basic / Bearer / OAuth2
Custom Headers	Headers table	X-API-Key: abc123
Pagination	Query Parameters or Loop	page=1&limit=100
Rate Limits	–	60 req/min

Note – Use HTTPS. n8n rejects self‑signed certificates unless rejectUnauthorized is disabled, which is not recommended for production. Teams typically discover this after the first failed call.

---

2. First HTTP Request Node

If you encounter any designing human in the loop n8n resolve them before continuing with the setup.

The following JSON shows a minimal node definition, containing only required fields.

{
  "parameters": {
    "url": "https://api.example.com/v1/items",
    "method": "GET",
    "authentication": "bearer",
    "bearerToken": "={{$env.API_TOKEN}}",
    "queryParametersUi": {
      "parameter": [
        { "name": "page", "value": "1" },
        { "name": "limit", "value": "100" }
      ]
    },
    "headerParametersUi": {
      "parameter": [
        { "name": "X-Client-ID", "value": "my-client-id" }
      ]
    },
    "responseFormat": "json",
    "jsonParameters": true,
    "options": {
      "fullResponse": false,
      "ignoreHttpStatusCodes": false
    }
  },
  "name": "Fetch Items",
  "type": "n8n-nodes-base.httpRequest",
  "typeVersion": 2,
  "position": [250, 300]
}

Why this matters

• {{$env.API_TOKEN}} reads the token from an environment variable, the safest way to store secrets.
• jsonParameters: true allows a later switch to POST/PUT without redesigning the node.
• ignoreHttpStatusCodes: false forces the workflow to stop on 4xx/5xx unless an Error Trigger is added.

Tip – If the API returns CSV, set Response Format to *File* and parse the CSV downstream. In practice, adjusting the node is faster than building a full custom integration.

---

3. Dynamic OAuth2 Authentication

Many SaaS APIs require OAuth2. Create an OAuth2 API credential (Settings → Credentials → OAuth2 API) and reference it in the HTTP Request node.

{
  "parameters": {
    "url": "https://api.example.com/v2/reports",
    "method": "POST",
    "authentication": "oAuth2",
    "credentialType": "OAuth2 API",
    "options": { "fullResponse": true },
    "bodyParametersUi": {
      "parameter": [
        { "name": "date_from", "value": "={{$json[\"startDate\"]}}" },
        { "name": "date_to",   "value": "={{$json[\"endDate\"]}}" }
      ]
    }
  },
  "name": "Create Report",
  "type": "n8n-nodes-base.httpRequest",
  "typeVersion": 2,
  "position": [500, 300]
}

Key points

• n8n stores the refresh token and automatically renews the access token, preventing mid‑run 401 Unauthorized.
• fullResponse: true returns response headers (e.g., Location for async report URLs).

Extra – Some providers need a custom scope or resource. Add them under Additional Fields → Scope.

---

4. Pagination Strategies

APIs usually expose pagination in one of three patterns. Choose the one that matches the endpoint and implement the corresponding n8n construct.

Pagination Type	Example Response	n8n Implementation
Offset/Limit	{ "items": [...], "next_offset": 200 }	Loop node that updates offset until next_offset is null.
Cursor	{ "data": [...], "cursor": "abc123" }	Store the cursor in a Set node, then feed it back via an expression.
Link Header (RFC 5988)	Link: <https://api.example.com/v1?page=2>; rel="next"	Function node parses response.headers.link and extracts the next URL.

Quick heads‑up: pagination bugs typically surface after several runs, not on the first execution.

Cursor‑Based Loop Example

First, the HTTP Request that reads the current page:

{
  "parameters": {
    "url": "https://api.example.com/v1/items",
    "method": "GET",
    "queryParametersUi": {
      "parameter": [
        { "name": "cursor", "value": "={{$json[\"cursor\"] || \"\"}}" },
        { "name": "limit",  "value": "100" }
      ]
    },
    "responseFormat": "json"
  },
  "name": "Get Page",
  "type": "n8n-nodes-base.httpRequest",
  "typeVersion": 2,
  "position": [250, 500]
}

Next, a tiny Function node that extracts the cursor for the next iteration:

return [{ json: { cursor: $json["cursor"] } }];

Finally, connect the nodes in a loop; the UI creates the circular connection. The loop ends automatically when cursor is empty.

Performance tip – Enable Continue On Fail and add a Write Binary File node to capture raw payloads for audit when processing very large data sets.

---

5. Error Handling & Retries

Transient failures (429, 502, etc.) are inevitable. n8n provides three complementary mechanisms:

1. Retry on Failure – configure count, delay, and HTTP codes.
2. Error Trigger – catches any node failure and can route alerts to Slack, Email, etc.
3. Conditional Branch – use an If node to inspect the response and decide whether to back‑off.

Retry Options (JSON snippet)

{
  "options": {
    "retryOnFail": true,
    "retryCount": 5,
    "retryDelay": 2000,
    "retryOnHttpCodes": [429, 502, 503, 504]
  }
}

When a 429 includes a Retry-After header, n8n respects it automatically.

Caution – Avoid infinite retry loops in production; add a guard that aborts after a reasonable duration (e.g., 10 minutes). A modest retry count (3‑5) reduces noise without hiding real problems.

---

6. Transforming the API Response

Most custom APIs return nested JSON. Use a Set node for simple renames or a Function node for more complex flattening.

// Input: $json = { data: [{ id:1, meta:{ name:"A" } }, …] }
const flat = $json.data.map(item => ({
  id: item.id,
  name: item.meta.name,
}));
return [{ json: flat }];

The output is a clean array ([{id:1, name:"A"}, …]) ready for downstream nodes such as Google Sheets → Append.

---

7. Deploying the Workflow – From Test to Production

Stage	Action	Checklist
Development	Run manually, enable Execution Log	✔ Verify auth token ✔ Validate pagination ✔ Confirm data shape
Staging	Duplicate workflow, set Cron Trigger (e.g., every 15 min)	✔ Use separate env vars (STAGING_API_TOKEN) ✔ Verify rate‑limit handling
Production	Activate workflow, add Error Trigger → Slack	✔ Monitor Execution Statistics ✔ Set Max Execution Time to avoid runaway loops

Scalability – For high‑throughput pipelines, run the workflow on **n8n Cloud** with a dedicated **Worker** to isolate API keys from other projects.

---

8. Real‑World Pitfalls & Fixes

Symptom	Likely Cause	Fix
401 Unauthorized after a few successful calls	Access token expired, refresh token missing	Ensure OAuth2 credential includes a Refresh Token and enable Auto Refresh.
Empty response body but status 200	Missing Content-Type: application/json header	Add header Content-Type: application/json.
Workflow hangs on pagination	Loop condition never false (e.g., missing cursor)	Add a Check node that breaks when cursor is null or undefined.
Rate‑limit errors despite retries	Retry delay too short, ignoring Retry-After	Enable **Retry on Fail** and let n8n respect Retry-After; optionally add a **Wait** node before retry.
Binary file not saved	responseFormat set to json instead of file	Switch **Response Format** to *File* and map binary output to a **Write Binary File** node.

---

Architectural Diagram 1 – High‑Level Flow

---

Pagination Loop Diagram – Cursor Flow

---

Conclusion

Configuring a single HTTP Request node, handling authentication (Bearer or OAuth2), applying the appropriate pagination pattern, and enabling built‑in retry/error mechanisms makes it possible to integrate any custom API into n8n. Flattening the response with Set/Function nodes prepares the data for downstream services, while a staged deployment (dev → staging → production) ensures safety and observability. The approach follows production‑ready practices: secret management via environment variables, automatic token refresh, respect for rate limits, and clear error‑handling pathways. Deploy it today and let n8n handle the heavy lifting for bespoke integrations.