ACS Infrapreview
Research preview. Apply for access. We review each application and email you when it's approved, with your API key ready on the API keys page.

Workbench

The Workbench lets you prompt base models in your browser without writing code. You can work with a prompt in three ways:

  • Single mode, the default view, continues one prompt.
  • Compare mode runs one prompt on different models, or on one model at different settings, side by side. To switch to it, select Compare in the toolbar under the session title.
  • The Loom branches a prompt into a tree of continuations. You can return to any node and read per-token logprobs.

Your sessions and compare runs are private to you.

Generate a completion

  1. Sign in and open the Workbench.
  2. Pick a model and an API key in the Settings panel on the right, which you can fold to the side. A new session starts on the model with the most GPUs among models we keep warm and those in an enabled warm window. Existing sessions keep their model.
  3. Type a raw prompt under Prompt to complete.
  4. Select Continue. The completion streams in at the end of your prompt.

The session list shows pinned sessions first, then recent ones. A row's ⋮ menu pins, renames, or deletes the session. The download button beside the mode toggle exports the session as a .txt file, as a .jsonl file with every snapshot, or every session in one .jsonl file.

Sampling defaults

Single mode, every Compare lane, and the Loom start from the neutral sampling values, which are the same for every model: temperature 1.0, top_p 1.0, top_k -1, min_p 0, presence and frequency penalties of 0, and a repetition penalty of 1.0. The Loom has a field only for temperature, so it always sends the neutral value of the other fields. An API request that leaves a sampling field out gets the same neutral value, as the API reference explains.

Single mode limits

Use the API for anything past these limits:

  • The Continue button generates one completion each time you select it.
  • The Stop sequence field accepts one string.
  • The Top logprobs shown field accepts a value from 1 to 20. The Workbench records top logprobs only when Heatmap is on, and a new session starts with the heatmap off unless you turn on the heatmap default in Settings.
  • The Max tokens field accepts up to 4000 tokens.
  • Single mode has no field for n, prompt_logprobs, echo, or add_special_tokens.

Read the heatmap

Turn on Heatmap in the Settings panel and set Top logprobs shown to the number of alternatives to record, from 1 to 20. After a run, the Workbench colors each generated token by its probability, where a darker green means a higher probability, as the Loom heatmap explains. Hover over a token to open its Token probabilities dropdown, and select the token to keep the dropdown open. Select an alternative in the dropdown to swap it in. The Workbench then replaces that token and deletes the text after it. Select Continue to generate from the swapped token.

Model picker and startup banner

The Workbench model picker labels each model "Always on," "Scheduled warm window," or "Needs startup" from its configuration, without a check for a running container. The "Needs startup" label includes a static time estimate, but an on-demand model used within its idle window shows a recent-use hint instead.

While a Workbench request waits for startup, a banner shows the elapsed time and a status message:

  1. Startup stages arrive after about 150 seconds. Until then, the banner says that progress details are unavailable.
  2. After that, the banner shows "Waiting for Modal to allocate GPUs" as an inferred stage.
  3. When the container reports a stage, the banner shows that stage.

For a kept-warm model, the banner appears only when a stage arrives. Other tabs on the same session also show the progress, but tabs on a different session do not.

Next steps