What are decision models?

Chat models generate text in response to a prompt. Decision models take an input and a question with fixed allowed answers, such as "is this support ticket urgent?" or "which team should handle it?", and return a probability for each answer.

The model catalogue lists public TurinTech models, built with Artemis: eight decision models and four chat models. The Compressed decision models and Compressed LLMs collections link to the current cards, files and results. This post explains what decision models are and why testing them differs from testing chat models.

A question with allowed answers

A request to a decision model has two parts. The state is the thing to decide about: a customer message, an invoice, an agent's log, sometimes an image. The questions each have a type and a set of allowed answers. The common types are yes or no, pick one option from a list, and a score on an ordered scale.

The shape of a request and its answers. The probabilities here are illustrative.

The model reads the state once and scores every allowed answer in a single pass. Its only possible outputs are the options in the request, so there is nothing to parse and it cannot answer outside the list. The probability attached to each answer is what makes it useful: a program can act on a confident answer straight away and send an uncertain one to a person.

Where they are useful

They fit any step where software has to choose from a known set of options. A support queue can use one to route tickets and flag the urgent ones, a trust and safety team to triage reports, a finance team to check invoices against rules. Agent builders use them to ask "should the agent call a tool here, and which one?" before spending a full chat-model call on it. Cloudflare's own example is classifying websites by category for its threat intelligence team (Cloudflare blog).

These jobs used to need a classifier trained for each label set. A decision model takes the labels as part of the request, so adding a category means editing the request, with no new data or retraining. Because it scores the options in one pass instead of writing an answer token by token, each decision is cheap to compute.

A short history, mostly from the last month

TypeSafe AI introduced the idea as "System One models" with its hosted model Jev on 15 September 2026, along with the /v1/systemone request format that the open models now share (TypeSafe blog). Open models followed quickly. On 1 October, Cloudflare released Clef and Clef-Flash under Apache 2.0, Perplexity released pplx-decider, and Jared Palmer released Kev 1.0, a family of open decision models anyone can train. A day later llama.cpp added a /v1/systemone endpoint, so these models run locally with the same server people already use for chat models (Hugging Face blog). The open-source vLLM Semantic Router project then published Decision 3.0, a family of six decision models, and on 12 October llama.cpp added support for the two largest, d3 and d3-flash.

curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "My Stripe integration keeps failing. Please help ASAP.",
  "questions": {"urgency": {"type": "noul", "instructions": "Does this message express urgency?"}}
}'

noul is the format's name for a yes or no question.

Why compressing them is a different problem

The originals are large: Clef and Perplexity's Decider are each about 54 GB at full precision. People shrink models like these by storing the weights at lower precision, and a 4-bit file is usually the first thing anyone downloads.

When compression damages a chat model, the damage usually shows in the text and someone notices. A decision model's output is one option and a number. A compressed file that picks "billing" where the original picked "technical" looks just as plausible as one that agrees, and nothing in the answer says it changed.

The confidence matters too. Teams set thresholds on it, for example acting automatically above 90% and asking a person below that. A file that keeps the same top answer but shifts the probabilities changes how many cases reach a person and how many mistakes get through, without any visible sign.

So we test a compressed decision model on how often it gives the same answer as the original across many real decisions, and on whether it ever overturns an answer the original was sure of. Benchmark accuracy alone can hide changes, because a file can get different questions wrong and still score the same.

How we checked every file

Every decision-model file was compared with its full-precision original, and with the most common public files where they exist, at identical llama.cpp settings:

• 2,000 decisions from the test split of the public typed-decisions dataset (400 cases with about five questions each),
• 1,000 additional decisions, historically called held-out, whose independence from calibration and file selection has not been verified,
• a 300-item sample of the public Decision Index benchmarks,
• 330 image questions for the models that read images,
• shuffled answer options and repeat runs.

The 2,000-decision panel informed file selection. The headline number in every article is agreement: the share of the 2,000 test decisions on which a file gives the same answer as the original. It is not accuracy. We also count how many answers the original was more than 80% sure of that a file changed. The tests ran on an NVIDIA A100, an RTX 5090, a GB10 (DGX Spark), an RTX 3060 and a desktop CPU; each article says which. The d3 files were tested one request at a time, because of an open llama.cpp issue with parallel requests that the d3 article explains.

The Gemma models are chat models, so they were checked differently: how often each file predicts the same next token as the original, how closely its first 32 generated tokens match, and a 310-item task sample.

The searchable model catalogue lists the current releases and recommended downloads.

Why we did this

Artemis is a code optimization platform. Teams define an objective, a representative benchmark and validation checks, then use Artemis to search for and test candidate changes. TurinTech models, built with Artemis, show measured trade-offs on specific workloads. The results and test settings are on the model cards.

The results are mixed in places, and the cards say so. The Kev-4B, Kev-9B, Clef, Clef-Flash and d3 default files are 16 to 26% smaller than the public Q4_K_M comparisons and agree with the original 0.2 to 2.3 points less often. Kev-27B has a different trade-off, documented on its card. At about the same size, ours keep more of the original's decisions: the public IQ3_M files of Clef and Clef-Flash agree 87.6% and 89.5% of the time, against 95.6% and 95.0% for ours. For Perplexity's Decider, ours are the only public files that run on stock llama.cpp (checked on 13 October 2026), and they are smaller and closer to the original than a plain 4-bit file made from it. Each article has the full comparison.

These are independent conversions. They are not affiliated with or endorsed by Cloudflare, Perplexity, Google, the Kev authors or the vLLM Semantic Router Team, and the credit for the models belongs to them.

The models

Decision models (collection):

Agreement is on 2,000 test decisions per model, measured on the hardware in brackets. On an NVIDIA A100 the default Clef and Decider files need 15.0 and 16.6 GiB of GPU memory, against 49.9 and 50.8 GiB for the originals, so about a third. The default d3 file needs 16.4 GiB on an RTX 5090, against 54.6 GiB for the original.

Compressed LLMs (collection): the first-choice file for Gemma 4 E4B is 4.22 GB, down from 14.9 GB, and for Gemma 4 E2B 2.31 GB, down from 9.3 GB. Each repository also has smaller files, down to 3.56 GB and 2.05 GB. Both models are covered in one article.

The catalogue also includes Qwen3.5 0.8B, with a 0.569 GB default download, and GLM-5.2 MaxMin108, whose default download requires six shards. Each card documents its quality limits.

Teams can compare original and smaller files on representative data, then use Artemis to explore improvements under their own validation checks. Explore Artemis.