ACS Infrapreview
Research preview — apply for access. We review each application and email you when it's approved, with your API key ready on the dashboard.

Completion jobs

Submit ordinary text completion work once, then close the connection and poll for its result. This is an opt-in alternative to keeping a synchronous /v1/completions connection open. Existing synchronous clients keep working.

Python example

Install httpx, then download the runnable Python example. Set ACS_API_KEY and run:

python python_completion_jobs.py \
  --model llama-8b --prompt 'Hello' --max-tokens 128 \
  --idempotency-key my-experiment-item-001 --deadline 900

Use your chosen model ID from the models page. Keep the same idempotency key, API key, and request parameters when restarting the script after a lost connection. The client bounds its wait, respects Retry-After, closes its HTTP client, and checks the result's status. Reaching the local deadline stops polling; it does not cancel the server's work.

HTTP contract

Send POST /v1/completion-jobs with the usual bearer key, a required Idempotency-Key header (1–128 visible ASCII characters), and completion JSON:

{"model":"llama-8b","prompt":"Hello","max_tokens":128}

A new job returns 202 with its id, status_url, and result_url. Repeating the same key and request returns 200 with the existing job. A different request under that key returns 409. Jobs belong to the exact submitting API key; another key on the same account cannot retrieve or cancel them.

  • GET /v1/completion-jobs/{id} returns 200 metadata, including status, usage_pending, and expiry information. This does not mean completion succeeded.
  • GET /v1/completion-jobs/{id}/result returns 202 while pending, with Retry-After: 1. On completion it returns the completion JSON and actual logical HTTP status. For example, a late quota rejection is 429, upstream failure normally 502, and interrupted worker ownership 503.
  • POST /v1/completion-jobs/{id}/cancel cancels queued work immediately and requests cancellation of running work. Cancelled results return 409.

Use the result endpoint's HTTP status before reading choices. Authentication must remain active, but exhausting the token budget does not block retrieval of an already-paid result. Job responses are private and never cached.

Supported requests and limits

The initial profile requires an explicit model and max_tokens. It supports ordinary non-streaming completion JSON, including bounded batches, n, seeds, and text echo. The sum of requested output tokens (max_tokens × n × batch size) cannot exceed 4096. Request JSON is limited to 64 KiB and the result to 2 MiB. Oversized or malformed upstream results fail explicitly.

Streaming, stream_options, activation/steering parameters, prompt_logprobs, and positive logprobs are rejected before inference. Use the synchronous API for those modes. Job submission shares the public completion rate limit; execution uses the same key concurrency, model limits, and token reservations. There may be at most 8 active jobs per key, 32 per account, and 128 globally. Stored jobs, including tombstones, are capped at 100 per account and 1000 globally.

Retention and interrupted work

Results and prompts expire 24 hours after a terminal result; subsequent result retrieval returns 410. A small idempotency tombstone remains for seven days after completion. Once it is deleted, reusing its idempotency key can create new work. Do not recycle keys for different experiment items.

If the worker loses ownership, the job becomes interrupted; the server never replays claimed work. usage_pending: true means a reservation is still held, possibly because upstream work or accounting could not be confirmed. Such holds keep the tombstone beyond seven days and prevent idempotency-key reuse until reconciled. Large content still expires. Contact the operator with the job ID and request ID before submitting a new invocation for uncertain work.