Overview

OpenAI API Rate Limits restrict the number of requests and tokens per time period. When exceeded, the API returns 429 errors. This article covers error causes, troubleshooting, and retry strategies.

Rate Limit Types

Identifying Rate Limit Errors

import openai

try:
    response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[{"role": "user", "content": "Hello"}]
    )
except openai.error.RateLimitError as e:
    print(f"Rate Limit Error: {e}")

Exponential Backoff Retry

from tenacity import retry, stop_after_attempt, wait_exponential

@retry(
    stop=stop_after_attempt(5),
    wait=wait_exponential(multiplier=1, min=2, max=60)
)
def call_with_retry(messages):
    return openai.ChatCompletion.create(
        model="gpt-4",
        messages=messages
    )

Multi-Key Rotation

API_KEYS = ["key1", "key2", "key3"]
current_key_index = 0

def call_with_rotation(messages):
    global current_key_index
    for _ in range(len(API_KEYS)):
        openai.api_key = API_KEYS[current_key_index]
        try:
            return openai.ChatCompletion.create(messages=messages)
        except openai.error.RateLimitError:
            current_key_index = (current_key_index + 1) % len(API_KEYS)
    raise Exception("All keys exhausted")

Prevention

1. Request Batching: Merge multiple small requests
2. Response Caching: Cache similar requests
3. Rate Limiting Middleware: TokenBucket or LeakyBucket
4. Monitoring: Set TPM usage alerts

References

• OpenAI Rate Limits
• Tenacity Library