Stop Paying Generation Prices for Decisions

TL;DR. A model that returns only typed answers - a choice, a score, a yes/no probability - shipped on 15 September at $0.042 per million input tokens with free output. Its lesson for an online store is a sorting test, not a speed multiplier: most of the AI calls a store makes through a chat API are decisions billed as writing. Below: four rungs for placing those calls, an audit of my own extension against them, a cost model on three workloads, and the boundary. The money path stays on a deterministic engine, as argued in the previous article; this one is about the judgments that have no rules.

The bill is mostly decisions

On 15 September TypeSafe AI came out of stealth with a model that cannot write a sentence. Jev takes a block of state - a ticket, an order, a product record - and a set of typed questions, and answers all of them in one parallel pass: pick one of these options, place this on a scale, is this statement true. Every answer carries a probability, and choices and scores carry a confidence figure derived from how that probability is spread. Input is priced at $0.042 per million tokens; output tokens are free because there is no token-by-token generation to meter. The company calls the class "System One models", after Kahneman's fast intuitive thinking, and named the model after Jevons: when a resource gets radically cheaper, you use radically more of it.

Strip away the launch noise and the interesting claim is not the speed multiplier. It is a classification of work. Look at what a store actually sends to a chat API and most of it is not writing. It is deciding: which category, which queue, is this urgent, does this look like fraud, is this review abusive, is this listing ready to go live. Where those decisions run through a chat completion - and in the integrations I audit, that is the common case - the store is renting a writer and paying writer prices, per output token and at generation latency, to get back a single word that code then has to parse.

I made the same point from the other side three months ago, about the money path: for tax, promo eligibility and discount stacking a probabilistic model should not decide at all; it should write the rules and a deterministic engine should execute them. That article left a second class of decisions unaddressed. A store is full of judgments that have no spec - "is this customer angry", "does this product belong in Outdoor or Fitness" - and cannot be compiled into a ruleset. Those still need a model. The question this piece answers is which model, at what price, and how you tell the two classes apart.

Four rungs, one test per rung

For the store processes this article is about, every AI call fits one of four rungs. The rung is set by the shape of the question, not by the tooling you happen to have wired up. It is a working scheme for those processes, not a taxonomy of every AI call: generating code, or pulling an arbitrary string out of free text, needs a generative model even though a program consumes the result, and both belong on rung four.

1. Code computes it exactly. Length limits, checksums, date arithmetic, a VAT number's format, whether a cart total crosses a threshold. If a function can return the answer, a model must not be asked. TypeSafe's own manifesto draws the line the same way: use code for what it is best at, exact computation, and invoke a model for semantic judgment.
2. A compiled rules engine executes it. The rule is specifiable and finite, but it was written in prose - promo terms, tax tables, eligibility logic. An LLM formalizes it once into a declarative contract; a sound engine runs it per instance at zero marginal tokens. This is the pattern from the previous article, and it is where the money path lives.
3. A calibrated decision model judges it. There is no spec, only criteria a competent person could apply in a few seconds: which team, which category, how frustrated, is this spam. The answer is a choice, a score or a yes/no with a probability, and code decides what to do at each confidence level. This is the rung System One models are built for. Where it runs on a model today it usually runs through a chat completion forced into JSON, unless the team has trained a classifier of its own.
4. A generative model writes it. The output is text a human will read: the reply, the description, the summary, the suggested correction. This is the only rung where you pay generation prices for something that is actually generation.

The mistake most stores make is not choosing the wrong model on rung four. It is running rungs one, two and three on rung-four infrastructure, because it was the only API in the room.

An online store through this lens

Where do a store's decisions actually sit? The same surfaces come up in every audit.

Two things stand out. Rung three accounts for the most rows, and in the stores I see most of those rows run on no model at all, or on one chat completion bolted onto a single surface. Search vendors have shipped query classifiers for years; a per-event judgment on every SKU or every ticket at chat-model prices and latencies is what rarely survives the budget conversation. And the money-path rows stay on rung two no matter how good a calibrated 0.93 looks. "Does this cart qualify" has a correct answer defined by rules you wrote; a probability is the wrong output type for it.

My own extension, audited

I sell a Magento extension that validates personalization-field input - product custom options - against a plain-language rule: "a valid Italian Codice Fiscale", "a gift message under 80 characters with no profanity". It sends the merchant's rule and the customer's input to a chat model through an LLM gateway and asks for a JSON verdict with four fields: valid, a written reason, a suggested correction, and a confidence figure. A per-rule confidence threshold decides whether a strict rule blocks add-to-cart. Because reasoning-class models bill their hidden reasoning into the output budget, the call caps output at two thousand tokens - a ceiling, not the typical spend - so that one boolean, one number and two short strings come back intact.

Put that through the four rungs and it splits cleanly. The 80-character limit and the Codice Fiscale checksum are rung one; they should never have reached a model. "Contains profanity" and "reads as a gift message" are rung three: yes/no judgments with a probability. The reason and the suggested correction are rung four - both are generated text, and the reason could just as well be a template keyed to which check failed - and both are only needed when the answer was no. One generative call is doing the work of three rungs, and the confidence number it returns is the one figure you cannot trust from a chat model: prompted-for confidence is overconfident and inconsistent, which is exactly the failure TypeSafe says it trained against.

I am not writing this to disown the extension. It works, it caches every unique input so the model is called once, and the gateway caps daily spend. I am writing it because the gateway already routes each consuming module to a model by capability, and a decision model is a new capability, not a new module. The slot exists. What changes is that the boolean and the probability come from something built to produce them, and the generative call runs only on the failures, where a human will actually read its output.

What the numbers look like

A cost model, not a benchmark. Jev is early access and I have not run it at production volume. The figures below put TypeSafe's published rate against Anthropic's list prices as of 2026-09-24 - Claude Sonnet 5 at $2 per million input tokens and $10 per million output, Claude Fable 5.1 at $10 and $50 - with no prompt caching and no batch discount on either side, and an output estimate per scenario for the chat side. What the comparison shows is the gap between a typed answer and a chat completion on the same decision. It does not prove that nobody could afford these workloads before; it shows what each way costs.

Three shapes of decision a mid-market store actually has:

A decision on every search. Classify the intent of a query - navigational, product, support, size-or-brand - on 1.5 million searches a month at about 400 tokens per call, with the chat model returning one label and a probability, about 40 tokens. That is 600 million input tokens. At $0.042 per million the decision model costs about $25 a month; Sonnet 5 lands near $1,800 and Fable 5.1 near $9,000. The more important difference is not on the invoice. The 70-to-500-millisecond range TypeSafe documents is at least the right order of magnitude for a storefront request; whether it fits yours is a measurement, not a given. A multi-second chat completion is not, which is why a per-search judgment today is either a dedicated classifier or nothing.

A decision on every SKU, every import. Eight questions per product - category fit, attribute normalization, completeness, a few flags - over 200,000 SKUs at 900 tokens per call is 180 million tokens per full pass, with the chat model returning eight answers, about 150 tokens. About $8 on the decision model; about $660 on Sonnet 5 and $3,300 on Fable 5.1, per pass. At $8 the pass is cheap enough to attach to every import. At $660 or $3,300 it becomes a scheduled job with a budget owner, which is how most catalog-quality passes I have seen end up.

A decision on every ticket. Route, detect a refund request, read the tone, on 20,000 support and return messages a month at 600 tokens each, three answers back, about 80 tokens: 12 million input tokens. Fifty cents on the decision model, about $40 on Sonnet 5, $200 on Fable 5.1. Be honest about this one: at that volume the bill is a rounding error either way. The case here is latency and calibration, not dollars.

The ratios survive halving the chat-model side for batch pricing, and they shrink but do not close on a cheaper chat model. The pattern across the three is the one the model's name predicts: two of the three are workloads that, in my experience, do not get built at chat-model prices and latencies, and the objection goes away when a decision costs a hundredth of a cent. TypeSafe's own workflow evaluations, which the company says are the high end of real-world gains, place Jev on the cost-accuracy frontier: not the most accurate point on their chart, but the cheapest point that is still competitive. For high-volume decisions, that is the point that matters.

Calibration is the part that changes the architecture

The price gets the attention. The probabilities are what change how you build, and two numbers need separating first. Every answer carries a probability. For a yes/no question that is the whole answer, so a Noul of 0.05 is a confident no, not an uncertain one. Choice and Score answers add a second figure, confidence, which TypeSafe derives from the shape of the distribution - peaked or flat - as a convenience for thresholding. Calibration is a claim about the probabilities: across many answers at 0.9, roughly nine in ten should be right. It is a property of a population, never a guarantee about one answer, and it is a claim you verify on your own labels before you automate anything. The Texas State study is the example to keep in mind: Jev's raw probabilities were usable, but a recalibration step fitted to the authors' own labelled sample cut the calibration error by more than three times, and they note that such a map should be refitted, not transported to another dataset.

Once you have checked it, calibration lets you do something a chat model's self-reported confidence never let you do safely: gate each action on its own threshold, scaled to what a wrong answer costs. The numbers that follow are illustrative. Routing a ticket at 0.6 is fine; the worst case is a transfer. Holding an order for fraud review at 0.6 is not; that wants 0.9 and a human on the other side. Below the floor the system does not act at all - it asks, escalates, or falls back to a rule - and that is the only sane way to run a decision unattended.

Two design consequences follow from the parallel evaluation. First, ask every question your code might need in one call, including speculative ones; a tenth question costs tokens and almost no time, and code ignores what turned out to be irrelevant. Second, decompose. "Is this listing good enough to publish" hides five judgments; ask five questions and combine them with weights you control in code, so that changing a priority is a coefficient change and not a re-prompt. Both patterns are in TypeSafe's documentation, and both are the opposite of how a chat-model integration is usually written.

The boundary, stated plainly

• Early access, launch pricing. Jev shipped on 2026-09-15 in early access - a waitlist at launch, with developers reporting admission within days - and the company says it cannot prove the price is not subsidized and expects it to go down, not up. Rate limits are documented as adjusting dynamically. Plan around the category, not the rate card.
• "Cannot hallucinate" means cannot make a type error. The output is constrained to your schema. It can still be confidently wrong, which the founder conceded on the launch thread. Calibration is the mitigation, not immunity.
• Vendor evaluations, plus a first wave of outside ones. The published workflow evals use the average of two frontier chat models as the reference answer and were built by TypeSafe's own team. There is no general independent benchmark yet, but domain studies started appearing within a week. The Texas State preprint coded 195,857 police crash narratives with a 27-question schema and audited 2,416 answers against blinded human labels: Jev reached an F1 of 0.91 where Fable 5.1 reached 0.97, at a cost governed by schema size rather than text length, and with probabilities that needed recalibration on the authors' own data. Narrow, but real. Measure on your own data before you believe either the headline or the skeptics.
• It picks; it does not extract or write. Give it candidates. A colour value, a category, a queue: it chooses from the list you send, up to 255 options, and you should add "other". If the value has to be produced from text, that is rung four, or a regex first.
• Text only, 64,000 tokens per request. State plus all questions fit in 64k; state plus the longest single question in 32k. Images, audio and long documents need a step before it. English is the primary training language and where the documentation says accuracy is currently best; other languages are handled but not equally well, and TypeSafe's own advice is to test on your content and lean on the confidence signal when routing.
• One HTTP endpoint, no PHP SDK. Python and TypeScript clients exist; from any other stack it is a plain POST, which is how a store backend talks to every provider anyway.
• Not for the money path. Promo eligibility, tax, discount stacking and pricing are not judgments; each has one right answer that follows from a document you can point to. Compile that document into a rules engine and let the engine answer. A confidence score, however well calibrated, adds nothing there except a new way to be wrong. This article and the previous one are two halves of one position.
• It is a classifier. The driest comment on the launch thread was "so... a classifier model?", and it is fair. Teams with labelled data have trained these for a decade. What is new is zero-shot judgment over natural-language criteria, calibrated, at infrastructure pricing, from one endpoint - and the fact that the category now exists in the open: TypeSafe published an adapter that makes any chat model answer in the same typed shape, and open reimplementations appeared within a week.

One audit question for every AI call

The audit I now run on every AI feature in a store is one question: is this a decision or a deliverable? A deliverable is text a person will read; pay generation prices, use the best model you can afford, and call it as rarely as the workflow allows. A decision is a value code will branch on. If code can compute it, compute it. If it follows from rules you wrote, compile the rules and run the engine. If it is a judgment, ask a model built to return a typed answer with a probability, and gate the action on that probability.

Put a store's AI calls through that question and most of them move down a rung or two. The bill follows, but the bill was never the interesting part. The interesting part is the decisions you start making because they finally cost little enough to run at scale.