ACS Infrapreview
Research preview. Apply for access. We review each application and email you when it's approved, with your API key ready on the API keys page.

Compare mode

Compare mode runs one prompt in up to 6 lanes at once and streams the results side by side. Each lane has its own model and sampling settings. Use it to compare models, or one model at two settings such as temperature 0.2 and 1.2.

To switch views, select Compare or Single in the toolbar under the title of a Workbench session. An old /workbench/<session>/compare link opens that session with Compare mode on.

Set up the lanes

  1. Type the prompt once under Shared prompt. Every lane uses it.
  2. Pick a model in each lane. Open the lane's Settings for max_tokens, temperature, top_p, top_k, min_p, the three penalties, seed, and stop.
  3. To run one model at different settings, select the Duplicate lane icon button in the lane's header. To add another model, select Add lane.

A lane sends every sampling value and uses the same defaults as Single mode.

Run every lane at once

Select Run all, or use the Ctrl+Enter shortcut, which is Cmd+Enter on a Mac, to start every lane. Each lane has its own Stop button, and Stop all cancels every lane in flight. Each lane streams on its own, so a lane that waits for its model to start does not delay the others. That lane's status shows the startup stage.

Compare runs leave the session's Single mode history unchanged.

Restore a saved run

The Workbench saves each run that you start with Run all to the saved comparisons list below the lanes, newest first. Each row shows the time, the shared prompt, and the number of lanes. Expand a row to see each lane's model.

Select Restore and confirm to reload the saved prompt and lanes with their outputs. We keep only each session's 20 most recent runs.

Read the heatmap

Turn on Heatmap in a lane's Settings. Then set Top logprobs shown to the number of alternatives that you want to record, from 1 to 20. Token colors follow the Loom heatmap, where a darker green means a higher probability. Hover over a token to open its Token probabilities dropdown with the top alternatives, and select the token to keep the dropdown open. The same dropdown opens in a restored run. A new lane starts with the heatmap off unless you turn on the heatmap default in Settings.

Billing

All lanes bill the API key shown in the key picker next to the Run all button. Each lane is a separate generation with its own budget reservation, so a 4-lane run counts as 4 generations against your usage and budget. When you are near your budget cap, the lanes that fit start. Each other lane shows a budget error, as Requests near the cap describes.

Next steps