The unit is measured, the measurement is checked, and the decision is acted on.

berth

the estimator

Predicts what a workload costs and how fast it runs on a given accelerator, before you rent it.

You describe the workload: model, concurrency, prompt and output lengths, and the latency bound you have to hold. berth returns dollars per million served tokens, time to first token, and time per output token, for every placement it knows about.

• A placement that misses your bound is excluded, not ranked cheaply. Its cost per served token is not high, it is undefined.
• It is a page of physics. A closed-form roofline plus a queueing term. No learned components, no training data, every number auditable by hand.
• Nothing leaves your machine. It runs in your environment.

$ berth place --model llama3-8b --slo-ms 800

sounding

the harness

Measures the real thing and checks the prediction.

It drives a live endpoint, records what actually happened, and produces the traces that say whether the prediction was right.

• It never imports the estimator. Enforced by test. Evidence that cannot be separated from the thing it evaluates is not evidence.
• Run conditions are captured, not assumed. Hardware identity, the served model, precision, and cache behaviour are all read from the machine and recorded, or recorded as unverifiable.
• Only physical impossibility refuses a file. Everything else is printed and passed. A checker that rejects correct data is the more dangerous failure, because it is silent.

pilot

the agent

Watches for change, adapts the placement, and proves what the change saved.

A placement is not a decision, it is a position that decays. A model version ships. A price moves. Traffic shifts. Most teams decide once and never check, because checking means re-measuring and nothing says when it is worth doing.

• It never touches a request. No proxy, no gateway, no traffic. Your own infrastructure moves the workload.
• It proposes, you dispose. A change arrives as a pull request against your deployment config, with the diff and the evidence attached. It cannot merge and cannot write to a default branch.
• Most passes produce nothing. That is the design. An agent that proposes every week gets muted, and a muted agent is worse than none because it looks like coverage.
• A saving is proven, not claimed. Under the Holdout Protocol a declared slice of traffic stays on the old placement so the difference can be measured.

The code is open source and free, like everything else here. Holding a class correct over time, and proving what it saved, is what we are paid for.