ACS Infrapreview
Research preview. Apply for access. We review each application and email you when it's approved, with your API key ready on the API keys page.

Using the base-model API

We serve base models through an OpenAI-compatible /v1/completions endpoint. A base model has no instruction tuning, and we do not apply a chat template, so it continues whatever text you send. The API returns logprobs for generated and prompt tokens and rejects an unknown field instead of ignoring it.

To give an AI coding assistant every docs page in one Markdown document, point it at https://infra.acsresearch.org/llms.txt.

What is this?

The API serves base models for research. A base model predicts the next token from raw text.

  • Models. We serve a small model for quick tests and larger models for real runs. Models lists them with their warm policy, because some stay warm and others start on first use.
  • Next-token access. Send any text as a prefill and read its continuation. logprobs and prompt_logprobs return token probabilities for likelihood, surprisal, and interpretability work. echo returns the prompt with the completion, and streaming sends the completion as server-sent events.
  • Activations. Capture the residual stream in the same response as a completion, steer a completion with your own vector, or harvest activations for a whole corpus.
  • Sampling controls. Set temperature, top_p, top_k, min_p, the penalties, and a seed, which repeats the output of a request sent on its own. A field that you leave out gets its neutral value, such as 1.0 for temperature and top_p.
  • An OpenAI-compatible endpoint with strict validation. A misspelled parameter returns an error instead of silently taking a default, and every error from /v1/completions is a JSON object with a code and a message.
  • A browser Workbench. Try prompts in the Workbench, branch them in the Loom, and compare models side by side, without writing code.
  • Per-key budgets and usage. We cap tokens per key and per account and track usage, as Account describes.

Prerequisites

  • You need an account that an administrator has approved, which you get by applying for access.
  • You need an API key. After approval, your first key is on the API keys page, as Account explains. Copy it on your first visit, because the page shows it only once.

Quick start

Export the key and the API base URL for curl and the Python client:

export ACS_API_KEY="acs-bm-..."
export ACS_API_BASE="https://infra.acsresearch.org/v1"

Send a first request with curl:

curl -s "$ACS_API_BASE/completions" \
  -H "Authorization: Bearer $ACS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "llama-8b", "prompt": "The capital of France is", "max_tokens": 1, "logprobs": 5}'

Or install the OpenAI Python SDK with pip install openai and send it from Python:

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["ACS_API_BASE"],
    api_key=os.environ["ACS_API_KEY"],
)

resp = client.completions.create(
    model="llama-8b",
    prompt="The capital of France is",
    max_tokens=1,
    logprobs=5,
)
print(resp.choices[0].text)

Both requests return a text_completion object, such as this recorded response:

{
  "id": "cmpl-91ea94c6ed0e5b75",
  "object": "text_completion",
  "model": "meta-llama/Llama-3.1-8B",
  "choices": [
    {
      "index": 0,
      "text": " a",
      "logprobs": {
        "text_offset": [0],
        "tokens": [" a"],
        "token_logprobs": [-1.7437232732772827],
        "top_logprobs": [
          {
            " a": -1.7437232732772827,
            " Paris": -1.9937232732772827,
            " one": -2.5562233924865723,
            " the": -2.5562233924865723,
            " also": -3.1187233924865723
          }
        ]
      },
      "finish_reason": "length"
    }
  ],
  "usage": {"prompt_tokens": 6, "completion_tokens": 1, "total_tokens": 7}
}

The model continues with " a" in choices[0].text, and top_logprobs shows " Paris" with the second-highest logprob. logprobs explains the alternatives.

Both requests leave temperature and top_p unset, so both fields take their neutral value of 1.0.

Next steps