How to Avoid API Rate Limits in n8n: Batching, Delays, and Retry Strategies
The moment your n8n workflow processes more than a handful of items, you meet the 429: Too Many Requests. Every serious API — OpenAI, HubSpot, Shopify, Google, LinkedIn scrapers — enforces a rate limit, and blasting them in a tight loop gets you throttled, temporarily banned, or silently dropped.
The fix isn't to slow everything down blindly. It's to understand the four levers that control request volume in n8n and combine them so your workflow runs as fast as the API allows and no faster. Here's how.
Lever 1: Batch with Split In Batches
The Split In Batches node (aka Loop Over Items) is your primary control. Instead of firing 500 requests at once, process 10–25 at a time, complete the batch, then continue. This alone resolves the majority of rate-limit incidents because most limits are 'requests per second', and batching flattens your spikes.
Lever 2: Insert deliberate delays
Between batches, add a Wait node. If an API allows 60 requests per minute and you batch by 10, a one-second wait between batches keeps you comfortably under. Do the math from the API's documented limit rather than guessing — a Wait that's too long wastes hours on big jobs.
Lever 3: Turn on node-level retries
Every HTTP Request node has a Retry On Fail setting. Enable it, set 3–5 attempts, and add a wait between tries. This handles transient 429s and 503s automatically without failing the whole run. Pair it with 'Continue On Fail' on non-critical steps so one bad item doesn't kill the batch.
⚡ Skip the build — grab the ready-made template
The AI web scraper template already bakes in throttling and retries — a good reference build for how large-volume n8n workflows should handle rate limits.
Get "AI Web Scraper for Market Intelligence" on Gumroad — $79 →
Lever 4: Exponential backoff for hard limits
For strict APIs, a fixed retry delay isn't enough. Implement exponential backoff: wait 1s, then 2s, then 4s, then 8s between retries. In n8n you can drive this with a small Code node that reads the retry count and returns an increasing wait time, or by respecting the Retry-After header the API returns in its 429 response — always honor that header when present.
Putting it together
A resilient high-volume workflow looks like: Split In Batches → HTTP Request (Retry On Fail + Continue On Fail) → Wait → loop. Add a Code node that reads Retry-After and you have a workflow that self-throttles to whatever the API tells it. This is exactly the pattern that separates a demo from something you can run against a real catalog or lead list.
Frequently asked questions
How do I know an API's rate limit?
Check the provider's docs (usually 'requests per minute/second') and inspect the response headers — many APIs return X-RateLimit-Remaining and Retry-After that tell you exactly how much room you have left.
Should I just add a huge Wait everywhere?
No — that wastes time and still risks bursts. Batch first, delay to match the documented limit, and let retries handle the occasional spike. Tune to the API, don't over-throttle.
Ready to automate this?
You can wire this together yourself with the steps above, or install a battle-tested version in minutes. The AI Web Scraper for Market Intelligence template ships with the full n8n workflow JSON, setup notes, and sample credentials so you can import, plug in your keys, and go live today.
