Completion jobs

A completion job runs on the server after you close the connection, so you submit it once and poll for the result. Use a job for a plain text completion that must keep running after your connection drops, and a synchronous /v1/completions request for streaming, logprobs, or activations.

Run the example script

The script needs Python 3.11 or later and httpx. Download the example script, set ACS_API_KEY, and run it. The script calls the production API unless ACS_API_BASE names another deployment. Submit and poll a completion job lists its steps.

python python_completion_jobs.py \
  --model llama-8b --prompt 'Hello' --max-tokens 128 \
  --idempotency-key my-experiment-item-001 --deadline 900

To resume after a lost connection, run the script again with the same idempotency key, API key, and request parameters. The --deadline limit stops the polling but does not cancel the job on the server.

Submit a job

Send POST /v1/completion-jobs with your bearer key, an Idempotency-Key header, and a completion request as JSON:

{"model":"llama-8b","prompt":"Hello","max_tokens":128}

An idempotency key is 1 to 128 visible ASCII characters long, and each API key has its own set of idempotency keys.

Only the submitting API key can read or cancel a job, so another key on the same account gets 404 job_not_found.

Poll for the result

GET /v1/completion-jobs/{id} returns 200 with the job's metadata, in this format:

{
  "id": "...",
  "request_id": "...",
  "status": "...",
  "status_url": "...",
  "result_url": "...",
  "usage_pending": ...,
  "expires_at": ...,
  "result_expired": ...,
  ...
}

A 200 status from this endpoint does not mean that the completion succeeded.

GET /v1/completion-jobs/{id}/result returns 202 with Retry-After: 1 while the job waits in the queue or runs. Once the job ends, the endpoint returns the stored body with the status that /v1/completions would have given. A finished job can return these statuses, among others:

Check the HTTP status of the result before you read choices.

You can still fetch a result after you use up your token budget, but not with a paused or revoked key.

Cancel a job

POST /v1/completion-jobs/{id}/cancel cancels a queued job at once. A running job gets cancel_requested and ends as cancelled unless the completion had already finished.

Reference

A job needs an explicit model and max_tokens. It takes non-streaming completion JSON, including batches of prompts, n, seed, and echo.

We run at most four jobs at once across all users, so eight jobs from one key do not all run in parallel.

The submit endpoint refuses a job with these errors:

• 429 job_capacity when the new job would exceed a job cap.
• 400 unsupported_job_mode when the body includes stream: true, stream_options, an activation or steering field, prompt_logprobs, or logprobs with any value, including 0. Use the synchronous API for those modes.
• 400 invalid_job_request, instead of unsupported_job_mode, when the body also fails the completion schema, such as stream_options without stream: true.

A job counts against the same per-key limits, model limits, and budgets as a synchronous request. Near a budget cap, the API shortens the job's max_tokens. If synchronous requests fill the 16 request slots and the queue of 64 requests, a job can end with 429 queue_full.

Every job endpoint sends Cache-Control: no-store, private.

Retention and interrupted work

We delete a job's request and result 24 hours after the job ends, and /result then returns 410. For seven days after the job ends, we keep a tombstone that ties the job's idempotency key to the job. After those seven days, every endpoint for the job returns 404 job_not_found, and a request with the same idempotency key creates new work.

Do not reuse idempotency keys across experiment items.

A job that the worker loses becomes interrupted and does not run again.

While a job runs, usage_pending: true means that its token reservation is still open. On an ended job, it means that the server failed to confirm the upstream work or its accounting.

An open reservation on an ended job keeps the tombstone and blocks the idempotency key past seven days until the server or our team reconciles it. The server reconciles it automatically about an hour after the reservation goes stale. The request and result still expire after 24 hours.

If an ended job still shows usage_pending: true, email us at infra@acsresearch.org with the job ID and request ID before you submit that work again.

Next steps

• Wait out a cold boot on the synchronous endpoint
• Recover from a budget refusal