The unit comes first. Everything that makes a market comes after.

Open problems

Stating what we cannot yet model is rarer than publishing what we can, and it is the more useful half. These are open, and we would rather they were solved by someone else than not at all.

• Where the closed form stops describing the machineactive

  A page of physics predicts most of inference well. It does not describe everything, and the interesting question is the boundary: disaggregated serving, speculative decoding, and the regimes where a scheduler rather than the silicon sets the answer. Mapping that boundary is what makes a learned residual tractable rather than a replacement for understanding.

• Generalization across architecturesactive

  A constant fitted on one accelerator transfers to another it has never seen, within ten percent, across different memory technology. Whether that survives a different compiler and a different memory system is the measurement that turns a useful result into a general one.

• Portfolio placement across a classopen

  Workloads with different deadlines share capacity. Solving each in isolation leaves the headroom that a tight-deadline class reserves unused by the loose one. Whether solving jointly beats solving independently by more than the uncertainty is an open question with a pre-registered kill criterion.

• Mechanism design for heterogeneous computeopen

  Once output is priced in a common unit, allocation becomes a market design problem rather than a scheduling one. What contracts, what settlement, and what happens to depreciation schedules when an external unit prices what the asset produces.

How we work

A measurement organisation is only worth what its method is worth. These are the rules, and they are the reason to believe anything above.

• Pre-registration. The success criterion is written and published before the measurement is taken. A bar set afterwards is not a bar.
• The instrument never imports the model it evaluates. Enforced by test. Evidence that cannot be separated from the thing it evaluates is not evidence.
• Unfavourable results publish. Including the ones that contradict something we already said in public.
• Every number carries its provenance. Measured, fitted, configured, or simulated, stated on the face of the output. A number labelled measured that contains a simulated input is the one unrecoverable error.
• The failures of the instrument are published too. Eleven so far, each with its mechanism, how it was caught, and the test that fails if it returns. A measurement tool that has never been caught lying has not been used hard enough.
• Hardware access never buys editorial input. No provider pays to be measured, ranked, or placed, and every data sale is disclosed.

Writing

Essays and technical notes, on the unit, on placement, and on what the corpus turns up.