Handling Rate Limits When Querying Anthropic Models

Handling Rate Limits When Querying Anthropic Models

What You’ll Need

Table of Contents

Understanding Anthropic Rate Limit Headers and Error Codes

When your application makes heavy calls to Claude 3.5 Sonnet, Claude 3 Opus, or Claude 3 Haiku, Anthropic enforces rate limits across several dimensions. These limits depend on your usage tier and include Requests Per Minute (RPM), Tokens Per Minute (TPM), and Tokens Per Day (TPD). If you hit any of these ceilings, Anthropic responds with an HTTP status code 429 (Too Many Requests).

Understanding how to inspect the response headers allows your application to pause execution before triggering errors, or to wait for the exact duration specified by the server. Anthropic sends back explicit tracking headers with every API call:

  • anthropic-ratelimit-requests-limit: Total request capacity per minute.
  • anthropic-ratelimit-requests-remaining: Remaining requests available in the current window.
  • anthropic-ratelimit-requests-reset: ISO 8601 timestamp indicating when your request quota resets.
  • anthropic-ratelimit-tokens-limit: Total token allowance per minute.
  • anthropic-ratelimit-tokens-remaining: Remaining token count for the current minute.
  • anthropic-ratelimit-tokens-reset: Timestamp when the token counter resets.
  • retry-after: Duration in seconds (or an HTTP date) indicating how long you must pause requests.

Parsing these headers correctly inside your API client prevents your services from burning through retries while the rate limit window is closed. Similar to secure authentication workflows like Implementing JWT Refresh Token Rotation In Express, managing state between asynchronous requests is essential to prevent system lockouts and unhandled failures.

{
  "error": {
    "type": "rate_limit_error",
    "message": "Number of request tokens has exceeded your daily limit. Please try again later."
  }
}

When handling raw HTTP requests, you must check for rate_limit_error inside the error response payload and extract the retry-after header value. If retry-after is missing, you should fallback to an exponential backoff formula.

💡 Fast-Track Your Project: Don’t want to configure this yourself? I build custom n8n pipelines and bots. Message me with code SYS3-HUGO.

Building an Exponential Backoff and Jitter Strategy in TypeScript

Relying on standard fixed delay intervals when retrying failed requests causes secondary rate limit spikes called thundering herds. If fifty concurrent requests fail at once and all retry after exactly two seconds, they will slam the server again simultaneously.

To solve this, we implement full jitter along with exponential backoff. Full jitter randomizes the delay period between zero and the calculated exponential ceiling.

Below is a complete, runnable Node.js/TypeScript script using the official @anthropic-ai/sdk package that handles rate limits safely. If you host this application on a server like a Hetzner VPS, you can run thousands of text extractions continuously without crashing your server threads.

import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY || "",
});

interface ExecutionOptions {
  maxRetries: number;
  baseDelayMs: number;
  maxDelayMs: number;
}

const DEFAULT_OPTIONS: ExecutionOptions = {
  maxRetries: 5,
  baseDelayMs: 1000,
  maxDelayMs: 60000,
};

function calculateJitterDelay(attempt: number, baseDelay: number, maxDelay: number): number {
  const exponentialDelay = Math.min(maxDelay, baseDelay * Math.pow(2, attempt));
  return Math.floor(Math.random() * exponentialDelay);
}

function sleep(ms: number): Promise<void> {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

export async function generateTextWithRetry(
  prompt: string,
  options: ExecutionOptions = DEFAULT_OPTIONS
): Promise<string> {
  let attempt = 0;

  while (attempt <= options.maxRetries) {
    try {
      const response = await anthropic.messages.create({
        model: "claude-3-5-sonnet-20241022",
        max_tokens: 1024,
        messages: [{ role: "user", content: prompt }],
      });

      const textBlock = response.content.find((block) => block.type === "text");
      return textBlock ? textBlock.text : "";
    } catch (error: any) {
      attempt++;

      if (attempt > options.maxRetries) {
        throw new Error(
          `Max retries reached (${options.maxRetries}). Failure reason: ${error.message}`
        );
      }

      let waitTime = calculateJitterDelay(attempt, options.baseDelayMs, options.maxDelayMs);

      if (error?.status === 429) {
        const retryAfterHeader = error?.headers?.["retry-after"];
        if (retryAfterHeader) {
          const parsedSeconds = parseFloat(retryAfterHeader);
          if (!isNaN(parsedSeconds)) {
            waitTime = Math.ceil(parsedSeconds * 1000);
          }
        }
        console.warn(
          `Rate limited by Anthropic API (429). Attempt ${attempt}/${options.maxRetries}. Retrying in ${waitTime}ms.`
        );
      } else if (error?.status >= 500) {
        console.warn(
          `Anthropic server error (${error.status}). Attempt ${attempt}/${options.maxRetries}. Retrying in ${waitTime}ms.`
        );
      } else {
        throw error;
      }

      await sleep(waitTime);
    }
  }

  throw new Error("Unexpected end of execution loop.");
}

async function main() {
  try {
    const output = await generateTextWithRetry("Explain rate limits in distributed systems.");
    console.log("Success Result:", output);
  } catch (err) {
    console.error("Execution failed:", err);
  }
}

main();

This code catches the 429 status code explicitly, reads the retry-after header when available, and dynamically applies full jitter backoff if header values are missing.

Implementing a Redis Queue to Throttle Requests with BullMQ

Client-side retries help when occasional spikes occur, but high-throughput systems require proactive queue management. Instead of sending burst traffic directly to Anthropic, you should place prompt jobs into a queue that enforces strict rate limits per minute.

If you are running your application inside Docker containers, read our guide on Optimizing Container Volume Performance On Hetzner VPS to make sure your Redis persistence layer does not create IO bottlenecks under load.

Below is a complete implementation using bullmq and ioredis to construct an Anthropic job worker that automatically restricts outgoing jobs to 40 requests per minute.

import { Queue, Worker, Job } from "bullmq";
import Redis from "ioredis";
import Anthropic from "@anthropic-ai/sdk";

const redisConnection = new Redis({
  host: process.env.REDIS_HOST || "127.0.0.1",
  port: parseInt(process.env.REDIS_PORT || "6379", 10),
  maxRetriesPerRequest: null,
});

const anthropic = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY || "",
});

export interface AnthropicJobData {
  prompt: string;
  jobId: string;
}

export const anthropicQueue = new Queue<AnthropicJobData>("anthropic-requests", {
  connection: redisConnection,
  defaultJobOptions: {
    attempts: 5,
    backoff: {
      type: "exponential",
      delay: 2000,
    },
    removeOnComplete: true,
    removeOnFail: false,
  },
});

const worker = new Worker<AnthropicJobData>(
  "anthropic-requests",
  async (job: Job<AnthropicJobData>) => {
    console.log(`Processing job ${job.id} for prompt ID: ${job.data.jobId}`);

    try {
      const response = await anthropic.messages.create({
        model: "claude-3-5-sonnet-20241022",
        max_tokens: 500,
        messages: [{ role: "user", content: job.data.prompt }],
      });

      const textBlock = response.content.find((block) => block.type === "text");
      return { result: textBlock ? textBlock.text : "" };
    } catch (error: any) {
      if (error?.status === 429) {
        console.error(`Rate limit exceeded for job ${job.id}. Delaying job retry.`);
      }
      throw error;
    }
  },
  {
    connection: redisConnection,
    limiter: {
      max: 40,
      duration: 60000,
    },
    concurrency: 2,
  }
);

worker.on("completed", (job, returnvalue) => {
  console.log(`Job ${job.id} completed successfully.`);
});

worker.on("failed", (job, err) => {
  console.error(`Job ${job?.id} failed with error: ${err.message}`);
});

async function enqueueSampleJobs() {
  for (let i = 1; i <= 5; i++) {
    await anthropicQueue.add(`request-${i}`, {
      prompt: `Summarize technical specification item number ${i}`,
      jobId: `ID-${i}`,
    });
  }
}

enqueueSampleJobs();

Using BullMQ’s limiter configuration ensures your system never sends more requests than Anthropic allows within a 60-second window, eliminating 429 errors altogether.

Handling Anthropic Rate Limits inside n8n Workflows

If you prefer visually orchestrated workflows instead of writing standalone TypeScript services, n8n Cloud or self-hosted n8n provides built-in mechanisms to handle API rate limits gracefully. You can also deploy open-source automation tools locally as shown in our step-by-step guide on How to Self-Host Activepieces on Cloud Servers.

In n8n Cloud, when querying the Anthropic REST API using the HTTP Request Node, you should configure automatic retries directly inside the node settings.

Step-by-Step Node Configuration in n8n

  1. Add the HTTP Request node to your canvas.
  2. Set Method to POST.
  3. Set URL to https://api.anthropic.com/v1/messages.
  4. Add Headers:
    • x-api-key: ={{ $env.ANTHROPIC_API_KEY }}
    • anthropic-version: 2023-06-01
    • content-type: application/json
  5. Inside the Body JSON parameter, set your prompt configuration:
{
  "model": "claude-3-5-sonnet-20241022",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": "={{ $json.queryPrompt }}"
    }
  ]
}
  1. Open the Settings tab of the HTTP Request node.
  2. Enable On Error and choose Continue (Using Error Output) or toggle Retry On Fail.
  3. Set Max Tries to 5.
  4. Set Wait Between Tries (ms) to 3000.

To handle items in batches without blowing past token per minute (TPM) limits, place a Loop Over Items (Split In Batches) node before the HTTP Request node. Combine it with a Wait node set to pause for 2 seconds between batch executions.

{
  "nodes": [
    {
      "parameters": {
        "batchSize": 1,
        "options": {}
      },
      "id": "batch-loop",
      "name": "Loop Over Items",
      "type": "n8n-nodes-base.splitInBatches",
      "typeVersion": 3,
      "position": [450, 300]
    },
    {
      "parameters": {
        "amount": 2,
        "unit": "seconds"
      },
      "id": "rate-limit-delay",
      "name": "Wait Delay",
      "type": "n8n-nodes-base.wait",
      "typeVersion": 1,
      "position": [670, 300]
    },
    {
      "parameters": {
        "method": "POST",
        "url": "https://api.anthropic.com/v1/messages",
        "sendHeaders": true,
        "headerParameters": {
          "parameters": [
            {
              "name": "x-api-key",
              "value": "={{ $env.ANTHROPIC_API_KEY }}"
            },
            {
              "name": "anthropic-version",
              "value": "2023-06-01"
            },
            {
              "name": "content-type",
              "value": "application/json"
            }
          ]
        },
        "sendBody": true,
        "specifyBody": "json",
        "jsonBody": "={\n  \"model\": \"claude-3-5-sonnet-20241022\",\n  \"max_tokens\": 500,\n  \"messages\": [\n    {\n      \"role\": \"user\",\n      \"content\": \"={{ $json.prompt }}\"\n    }\n  ]\n}",
        "options": {
          "batching": {
            "batch": {
              "batchSize": 1,
              "batchInterval": 2000
            }
          }
        }
      },
      "onError": "continueErrorOutput",
      "id": "anthropic-http-request",
      "name": "Anthropic API Call",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [890, 300]
    }
  ],
  "connections": {
    "Loop Over Items": {
      "main": [
        [],
        [
          {
            "node": "Wait Delay",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Wait Delay": {
      "main": [
        [
          {
            "node": "Anthropic API Call",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Anthropic API Call": {
      "main": [
        [
          {
            "node": "Loop Over Items",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

This workflow structure enforces a hard pause of 2 seconds between individual items, keeping overall requests under 30 per minute and preventing unexpected rate limit errors.

Getting Started

To implement these rate-limiting strategies across your enterprise applications:

  1. Provision an infrastructure base using a reliable platform like a Hetzner VPS or Contabo VPS. If you want an alternative cloud setup, launch your instance on DigitalOcean.
  2. Register a domain on Namecheap for hosting your custom webhooks and worker dashboards safely over SSL.
  3. Deploy an n8n Cloud workspace or self-host n8n to manage prompt pipelines visual-first.
  4. Implement TypeScript queues with Redis for heavy data processing workloads to stay under Anthropic limits.

Outsource Your Automation

Don’t have time? I build production n8n workflows, WhatsApp bots, and fully automated YouTube Shorts pipelines. Hire me on Fiverr, mention SYS3-HUGO for priority. Or DM at chasebot.online.

Want to automate this yourself?

Start with n8n Cloud (free tier available) or self-host on a Hetzner VPS for full control.

Want this engine running on your own VPS?

This blog publishes itself — daily, unattended, on free API tiers. The full engine, Hugo theme, and setup guide are available as System 3.

Get System 3
system online