Jev decides in milliseconds. Here is what it needs to decide well

A new kind of model came out this month, and it is not another chatbot.

Jev, from TypeSafe AI, does not write sentences. TypeSafe describes it as "unstructured state in, typed probabilistic decisions out." You give it a situation and a question whose answers you fixed in advance, and it gives back a decision — with a probability for every possible answer.

That is a genuinely useful shape. Most steps in most AI workflows are decisions, not paragraphs: which queue, is this allowed, how urgent, does this apply. Using a model that writes prose to produce one word has always been the expensive way to do them.

This post is about the part of Jev's description that is easiest to skip: you give it the situation.

What Jev is, from TypeSafe's own sources

Everything below comes from TypeSafe's launch post and the Cloudflare model page.

• Three question types: yes-or-no, choose one of these options, and give this a score. Each returns the answer plus probabilities and a confidence.
• It gives up text generation on purpose. It does not write replies and is not for chat.
• Answers are fixed in advance, so it cannot return something outside your list. Up to 255 options per question.
• Context window: 32,000 tokens.
• Status: early access, via TypeSafe's console and on Cloudflare as typesafe/jev.

TypeSafe also reports large speed and cost advantages over language models. In their words, workflow evaluations showed "193.6x faster, 444.6x cheaper" — and, to their credit, they add "we expect that these are on the higher end of real world gains." Those are TypeSafe's figures, not ours. Measure on your own workload before you plan around them.

The guarantee, and what it does not cover

Fixed answers give you a real guarantee: the output is always one of your options, correctly typed. No parsing failures, no invented categories.

But it guarantees the answer is one of your options. It does not guarantee the option is right. Those are different promises, and the gap between them is where systems built on Jev will go wrong.

It decides over whatever it is handed

Hand Jev the current refund policy and it decides about refunds under that policy, well. Hand it last year's refund policy and it decides about refunds under last year's policy — equally well, just as fast, with a confident probability attached.

That is worse than a chatbot getting it wrong, not better. A rambling wrong answer looks suspicious. A crisp, well-typed decision with a high confidence score looks finished, and nobody questions it.

And you cannot solve this by handing Jev everything. Thirty-two thousand tokens is plenty for a situation and nowhere near enough for a company's documents. Something has to choose what goes in — and that choice is now the most important decision in your system, made before Jev sees anything.

Choosing by similarity reintroduces the old problem

The obvious way to choose is similarity search: find passages that look like the question. It brings back the old version of a policy as readily as the current one, because both are about the same thing. It brings back a paragraph saying a rule does not apply, because it is about the rule. And it misses the exception a few sections away, because the exception does not repeat the question's words.

So you end up with a very fast, very precise decision on top of a guess.

Choosing by walking a map

FastMemory chooses differently. It builds a map from the structure your documents already have — headings, sections, and the references people wrote between them — and answers by walking it: from the policy to the rule, the rule to the limit, the limit to the role, and on to the exception that modifies it.

The state you hand Jev is then the chain a person would have assembled, and it arrives with the route it took. The build reads a folder with no model call per document, and the map lives in a graph store you likely already run — Postgres with Apache AGE is the plain version.

cargo install fastmemory      # or: pip install fastmemory
fastmemory build ./policies
fastmemory query "Can this customer get a refund on a discounted order?"

The division of labour

Put the route and the probabilities side by side and you have something neither tool gives you alone: a decision with its evidence and its confidence attached. An auditor can check the evidence. A risk team can set a threshold on the confidence. Both are real controls.

Three honest caveats

1. There is no official integration. FastMemory and Jev are separate tools; you combine them yourself, and it is not much work.
2. Jev is days old and in early access. Build a prototype on it before you build a business on it.
3. Set confidence thresholds from your own data. "Calibrated" is a claim about averages. Your threshold is a claim about your documents. Test the second.

Where this is the wrong choice

When the output a person needs is a sentence — an explanation, a draft — Jev deliberately cannot write one. When a question has open-ended answers, or would need more than 255 options, it is probably a search rather than a decision. And if your documents have no structure worth walking, similarity search is fine.

Try it

Find one decision your system makes today with a language model. Write down its possible answers. Then write down, by hand and in order, which documents that decision actually depends on. If the answers fit on a list and the documents form a chain, that is your first candidate.

FastMemory is free and open source: github.com/FastBuilderAI/memory. How it reads, in plain words: fastbuilder.ai/fastmemory/architecture.