When searching for an apartment, you can check the “pets allowed” box. Or you can type into the search bar:
Looking for an apartment where I can move in right away with my dog. I work from home, so a studio won’t work.
To a human, the connection is obvious, and the catalog already has the right filters:
- pets allowed;
- number of rooms;
- move-in date.
From there, regular code filters listings in the database. We don’t need a long text reply from a model here — the service only needs to translate a person’s free-form request into a set of available search filters. The same task comes up when matching jobs, equipment, or contractors: the user describes the goal in their own words, while the system searches by strict fields.
Jev, a model by TypeSafe AI for fast structured decisions, is built around this exact task. What interests me in this story is what actually stands behind the release: a new architecture, a training method, or the way we read answers from the model. I want to figure out what is truly new here and what explains the claimed speed and low cost.
How This Was Solved Before Jev — and Why It Is Expensive
The apartment search example is not an abstraction. Starting in 2023, companies massively built this kind of “human text to system parameters” translation using standard generative LLMs:
- Catalog search (Zillow, Booking.com, Instacart). A user writes a query in their own words, and an LLM uses Function Calling or Structured Outputs to build a JSON object with strict filters (
property_type,amenities,price_max), which is then sent to a regular search engine. - Data and log queries (Honeycomb Query Assistant). An engineer describes an issue in natural language, and the model turns the phrase into a structured query with
WHERE,GROUP BY, and a time window from the available telemetry schema. - High-volume routing and labeling (LinkedIn, Intercom, DoorDash). In support tools and internal agents, the model solves a narrow classification task at every step: pick a ticket topic, estimate complaint urgency, or decide which tool to call next.
In all of these cases, the industry hit the same contradiction: the application needs a short decision from a closed list, but gets it through word-by-word text generation.
A standard LLM generates its answer token by token. Even when the format is strictly constrained by a schema, the model spends time and compute printing brackets, keys, and punctuation marks, while the application spends time parsing the output. A telling example is a report from LinkedIn engineers on building their AI assistant: they had to replace JSON with YAML in model outputs simply so the model would generate fewer brackets and quotes — saving tokens and reducing latency. At high volume, other teams had to drop runtime LLM calls altogether and train separate BERT-based classifiers for every single filter.
With Jev, the shape of the answer is defined upfront: a selected option, a probability, or a score on a scale. Instead of generating text word by word, the model reads the input context once and evaluates all passed questions in a single pass. The program immediately gets numbers and selected keys, and dropping autoregression sharply cuts both latency and compute cost. TypeSafe’s overview of the approach.
This gives us two very different workflows:
Standard LLM: query + schema → token-by-token JSON generation → response parsing → condition in code
Decision model: context + list of questions → single pass (option probabilities) → condition in code
In an independent benchmark by the authors of the open-source project Rev (more on it below), on a request with 20 questions about a single text, the difference looks like this:
| Metric | Text-generating LLM | Jev |
|---|---|---|
| Response time | 5.10 s | 0.27 s |
| Cost per decision, relative to Jev | 15× | 1× |
In this test, Jev turned out to be roughly 19 times faster and 15 times cheaper.
How to Use Jev in Code
Jev has three question types:
- Noul — estimates the probability of a “yes” answer to a given question.
- Choice — picks one option from a predefined list. For example, it identifies a support ticket topic: “billing”, “delivery”, or “refund”. It returns the winning option along with the probability of each option.
- Score — gives a numeric rating on a scale with predefined levels. For example, customer frustration: 0 — calmly describes the issue; 1 — annoyed, but polite; 2 — writes an angry complaint. It returns the probability of each level and the weighted average score.
You don’t have to pass all three types: each question has its own type, and you can mix any of them in a single request. For Choice, you define the answer options; for Score, the scale levels; for Noul, the question itself is enough. API primitives documentation.
Let’s go back to the apartment search example. This is my illustration of the task rather than a benchmark of Jev, but it clearly shows the main engineering requirement: for the model to properly match phrases like “a studio won’t work” or “moving in with my dog” to system filters, raw technical field names (pets_allowed) are not enough — every parameter needs a clear description. For simple binary flags (“pets allowed”, “move-in ready”), Noul fits well, while Choice works for selecting the number of rooms.
Below is a Python snippet using the TypeSafe SDK. The client is already configured, while catalog and filter_policy represent our application’s catalog and business rules.
Notice how roles are split in the code: the model only returns probabilities for each question, while the threshold logic is defined by
filter_policyon the application side (for example: above0.8a filter is applied automatically, in the0.4–0.8gray zone the app asks the user to clarify, and below0.4the condition is ignored).
from typesafe_sdk import Choice, Noul
# The message the user typed into the search bar.
user_text = "Looking for an apartment where I can move in right away with my dog. I work from home, so a studio won't work."
# Available search filters: key for the program on the left, description for the model on the right.
filter_descriptions = {
"pets_allowed": "Pets are allowed in the apartment",
"move_in_ready": "The apartment is vacant and ready for immediate move-in",
"furnished": "The apartment comes with furniture",
"parking": "Includes a parking spot",
"near_metro": "Within walking distance of a metro station",
}
# Ask the same question for each filter, changing only its description.
questions = {}
for filter_id in filter_descriptions:
description = filter_descriptions[filter_id]
question = "Should this filter be applied based on the user's request? Filter: " + description
questions[filter_id] = Noul(instructions=question)
# For the number of rooms, provide predefined options.
questions["rooms"] = Choice(
instructions="What room count requirement did the user specify?",
criteria={
"any": "Number of rooms is not specified",
"studio": "Studio",
"1": "One room",
"2": "Two rooms",
"3_plus": "Three or more rooms",
"non_studio": "Any number of rooms except a studio",
"clarify": "Other requirement — needs clarification",
},
)
# Send the user's text and all questions in a single call to Jev.
response = client.system_one(
state=user_text,
questions=questions,
)
# Application rules decide whether the answers are clear enough to run the search.
if filter_policy.needs_clarification(response):
ask_user_to_clarify(response)
else:
# Select filters, search listings, and show results to the user.
selected_filters = filter_policy.select(response)
listings = catalog.search(selected_filters)
show_results(listings)
All questions are sent in a single call. response.nouls contains scores for the binary conditions, while response.choices["rooms"] holds the probability distribution across room options.
The model matches the text against available filters, while the application sets the selection and clarification thresholds, queries the database, and displays the results. If a new filter is added to the catalog, it joins the same batch check without rewriting the prompt for specific phrasing.
Is There a Breakthrough? How Open Source Beat Jev in Days
In the Jev announcement, TypeSafe talks about a new architecture, parallel answer evaluation, and a training method called Reinforcement Learning for Calibrated Decisions (RLCD). Yet shortly after, developers started getting competitive results in much simpler ways: taking off-the-shelf open models, changing how answers are read, and fine-tuning them when needed.
The most telling example for me is Rev. In the first version of its README from September 22, the author writes that one person and two coding agents built the project in about three days. It is based on an open Qwen model with LoRA fine-tuning and a small extra network layer (a decision head) that scores the provided options.
The current Rev report publishes the following results on 975 held-out questions:
| Model | Accuracy | Median latency |
|---|---|---|
| Hosted Jev | 85.0% | 142 ms |
| Rev on Qwen3.5-4B | 87.7% | 37 ms |
| Rev on Qwen3.5-9B | 88.7% | 40 ms |
| Rev on Qwen3.8-27B | 92.0% | 75 ms |
These are measurements by the Rev authors, not my own experiment. The authors consider the 4B model’s accuracy lead borderline given evaluation noise. Training took 63 minutes for the 4B model and 33 minutes for the 9B model on a single B200; they achieved lower serving costs under full GPU utilization. On Jev’s own published workflow questions, their initial report still left Jev in the lead.
Still, the main takeaway here is very concrete: compact open models beat Jev on the chosen benchmark, and a working alternative was put together in just a few days.
SemIf shows an even simpler path: reading option scores directly from an off-the-shelf Qwen3.5-4B with no fine-tuning at all. It is an accessible starting point for your own implementation. SemIf repository.
On a separate benchmark, JevBench v1.6.1, the open models Quyet-1.0-Large and deck-31B also score higher than Jev 1.13.0 on Capability Score: 81.69 and 77.59 versus 77.13. This is a composite quality metric rather than raw accuracy. The benchmark belongs to Benchmark Heaven, and on its other summary metric, Composite, Jev remains ahead of both models. Published JevBench snapshot.
The next step is making the training of such models a routine engineering tool. Unsloth already offers a ready-made workflow for Qwen and Gemma: a lightweight network layer for scoring options is attached to the language model, which is then fine-tuned on examples formatted as “state → questions → correct answers”.
In Unsloth’s published results, Qwen3.5-0.8B accuracy on typed-decisions jumped from 36% to 73%, and Qwen3.5-2B from 33% to 78%. For Gemma 4 E4B, the authors separately report 77% test accuracy and 49 minutes of training. This table doesn’t give a direct comparison with Jev, but it shows that useful results are achievable even on tiny models, and the training method itself is not tied to a single model family. Training decision models in Unsloth.
This gives a direct answer to the question from the beginning of the article: the main speed gain comes not from a secret architecture built from scratch, but from switching from word-by-word generation to scoring options in a single pass — while a compact decision head and fine-tuning are needed for accuracy and calibrated probabilities.
I will cover how this works under the hood in a separate detailed post. There we will break down direct logit reading, how the decision head works, and how to fine-tune Qwen and Gemma for your own domain. What matters to me here is the big picture: Jev was beaten in specific comparisons, and reproducing the approach itself proved accessible to small teams and solo developers.
Microsoft Also Releases a Decision Model
On October 9, 2026, Microsoft introduced Microsoft-Decision-1. Under the hood is Qwen3.5-9B, fine-tuned to evaluate predefined options in a single pass. The model is available via Foundry and OpenRouter. Microsoft claims the highest accuracy in its own comparison across 36 benchmarks covering nearly 150,000 questions. These are company figures rather than an independent ranking. Microsoft-Decision-1 announcement.
The parallel with Jev is direct: the application passes data and allowed answers, the model returns scores, and code decides what to do next. The similarity with Unsloth and Rev is different: to build a specialized solution, they take an existing open language model and fine-tune it for the target behavior.
Microsoft demonstrates the business case on its own teams. Xbox Research used Decision-1 to route more than 10,000 pieces of feedback into predefined topics. According to the company, quality matched GPT-6 Sol, while processing was more than 14 times faster and 200 times cheaper. These multipliers apply to that specific internal comparison. Xbox Research example in the announcement.
To me, this is a signal that a commercially important category is forming around such models. Businesses have plenty of repetitive, narrow tasks: routing tickets, labeling feedback, screening documents, or picking a path for a request. Saving even a little on one small decision adds up when you make millions of them.
Three paths converge here: a managed service like Jev, your own model built with open tools, and offerings from major cloud providers like Microsoft. Microsoft’s release strengthens the economic case for the approach, though the actual payoff for an application depends on its traffic and the cost of errors. So the next question is practical: where does a solution like this pay off for us?
Conclusion
What I like about Jev is the idea of making a model’s role in a program small, predictable, and explicit. It has a specific question, a closed list of options, and a clear place in the execution flow.
In the apartment search example, the model helps translate human words into catalog filters. The rules for enabling filters, querying the database, and handling empty results stay visible in regular code. This lets you discuss, test, and fix each part separately.
The results from open projects like Rev and Unsloth, along with the release of Microsoft-Decision-1, make this story bigger than Jev alone. You now have a choice between a ready-made cloud API and a small model you can fine-tune on your own data in an hour on a single GPU. Either way, that choice still has to start with designing the questions themselves and a way to verify the quality of the answers.
The model evaluates meaning. The program decides what to do with the evaluation.
This boundary does not eliminate model errors, but it makes their place and impact in the system much easier to understand.
References
- System One and Jev announcement by TypeSafe AI and primitives documentation (
Noul,Choice,Score) - Rev repository — open-source alternative built on Qwen and LoRA
- SemIf repository — reading option scores from an off-the-shelf model without fine-tuning
- JevBench benchmark on Hugging Face
- Unsloth guide to training decision models
- Microsoft-Decision-1 announcement
