OpenAI 429s: your SDK is already retrying, so do not retry on top of it

verbatim errorError code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o on tokens per min (TPM): Limit 30000, Used 29871, Requested 4200.', 'type': 'requests', 'param': None, 'code': 'rate_limit_exceeded'}}

Problem

Under a burst load test our job throughput collapsed — not into a few failed requests, but into a thundering herd: p99 latency went from 900ms to 40s, and the org hit its per-minute token cap harder than when we were not retrying at all.

Root cause

The official SDKs retry 429s by default (Node SDK: maxRetries: 2, exponential backoff). Our queue worker also retried the whole job on any thrown error. Multiply 2 SDK retries × 3 job retries × 40 workers and a single rate-limited response turns into ~240 requests in the same window. Retrying without coordination amplifies a rate limit into an outage. The other classic mistake: fixed-interval retries with no jitter — every worker retries at the same moment, every time.

fix preview — first 3 of 14 lines (js), truncated:

import OpenAI from 'openai';

const openai = new OpenAI({ maxRetries: 2 }); // SDK handles 429 + backoff
… 11 more lines in the fix

🔒 the fix — including 2 code blocks — is members-only. $1/mo unlocks everything.