Budget-cap recovery
When a request does not fit your token budget, the API lowers max_tokens and runs the request if it can. It returns 429 budget_exceeded only when the prompt alone is longer than the budget left, or when the budget left has no room for one output token per completion. Requests near the cap gives the rule for each budget and what clears a refusal.
curl
curl -i "$ACS_API_BASE/completions" \
-H "Authorization: Bearer $ACS_API_KEY" -H "Content-Type: application/json" \
-d '{"model": "llama-8b", "prompt": "Hi", "max_tokens": 8}'
Python
import os
from openai import APIStatusError, OpenAI
client = OpenAI(
base_url=os.environ["ACS_API_BASE"],
api_key=os.environ["ACS_API_KEY"],
max_retries=0,
)
long_prompt = open("long_prompt.txt").read() # longer than the budget left
try:
resp = client.completions.create(model="llama-8b", prompt=long_prompt, max_tokens=8)
except APIStatusError as e:
# e.body is the inner error object, and e.code is its code.
# e.message is the SDK's own summary ("Error code: 429 - {...}").
if e.status_code == 429 and e.code == "budget_exceeded":
reason = e.body.get("message") if isinstance(e.body, dict) else e.message
raise SystemExit(f"Budget exhausted: {reason}")
raise
The 429 budget_exceeded response has neither Retry-After nor X-Should-Retry: false, so with the default max_retries=2 the OpenAI SDK sends the request two more times before your except block runs. Because a budget does not refill before its reset, those retries fail until your active runs release their holds. Each retry also counts against your rate limit.
max_retries=0 stops those retries, along with the SDK's retries of every other error from this client.
Monthly budgets reset on the first of the month, and the daily budget resets at midnight, both in UTC.
Example response
The curl request needs about 10 tokens and runs while the key has that much left. The refusal below came from a longer prompt, sent with a key that had 1,200 tokens left on its monthly budget:
HTTP/1.1 429 Too Many Requests
{"error": {"code": "budget_exceeded", "type": "invalid_request_error",
"message": "Not enough key monthly token budget for this request (1,200 remaining, up to 4,096 needed, including active runs). Reduce the prompt, maximum tokens or number of branches, or wait for active runs."},
"request_id": "req_..."}
The message text names the budget that refused the request, with the names that Budgets uses. needed is this request's own worst case. The phrase "including active runs" means that remaining already subtracts the tokens held by your active runs.
The API keys page shows the tokens held by active runs next to the account total, while its per-key rows show settled usage only.
A request that fits after the API lowers max_tokens returns 200 with the X-Acs-Max-Tokens-Clamped header, which curl -i prints in this form:
HTTP/1.1 200 OK
X-Acs-Max-Tokens-Clamped: requested=N,applied=M,reason=budget,source=client
Budget refusal versus full queue
When a key has 16 requests running and 64 requests queued, the next request gets 429 queue_full with Retry-After: 5. Retry a queue_full request after 5 seconds. Tell it apart from 429 budget_exceeded by error.code.