Models

TypeSafe AI's Jev Sparks a Decision-Model Gold Rush at 150ms Latency

TypeSafe AI's Jev answers in 150ms, costs $0.042 per million input tokens, and can't hallucinate. Now Cloudflare, OpenAI, Amazon and Microsoft have shipped copycats — but accuracy gaps remain.

AI Companies Develop Decision Models to Avoid Hallucinations
AI Companies Develop Decision Models to Avoid HallucinationsAI-generated
By Elena Vasquez6 min read

Why it matters

  • TypeSafe AI released Jev on 15 September and raised $870 million at a $7.5 billion valuation.
  • Independent testing found Jev's median latency was 150 milliseconds versus 1.89 seconds for OpenAI's GPT-6 Astra.
  • Jev costs $0.042 per million input tokens with free output tokens, versus $10/$50 per million for GPT-6 Astra.
  • Cloudflare, Amazon, OpenAI and Microsoft have all released decision models imitating Jev.
  • A hybrid of Jev and GPT-6 slightly beat GPT-6's accuracy at 41 percent of the cost.

On 15 September, a startup called TypeSafe AI released a model named Jev and made a claim that borders on heresy in AI circles: the model is cheaper, faster, and it can't hallucinate. Within six or seven weeks, Cloudflare, Amazon, OpenAI, and Microsoft had all shipped copycats. Startups, academic researchers, and independent hobbyists released dozens more. The company behind the disruption recently raised $870 million at a $7.5 billion valuation, according to its own announcement.

The implications reach beyond one startup's marketing. If a class of models can judge, classify, and route at a fraction of LLM cost, whole categories of work that never made economic sense for generative AI — moderating a firehose of social posts, screening phishing emails, guarding against prompt injection — suddenly become practical.

What exactly is a 'decision model'?

Most researchers call Jev a decision model, or decider model. TypeSafe AI brands it a "System One model," borrowing from psychologist Daniel Kahneman's book Thinking, Fast and Slow. Kahneman drew a line between fast, instinctive thought (System One) and slow, deliberate thought (System Two). TypeSafe AI argues modern LLMs deliver the second but are too slow to deliver the first.

The speed numbers are the story. TypeSafe AI claims Jev delivers end-to-end response latency between 70 and 500 milliseconds. LLMs typically need at least several seconds, and complex tasks like coding software from scratch can take hours.

Independent testing backs the company up. A pre-print co-authored by Ramayya Krishnan, a professor of information systems at Carnegie Mellon University, found Jev delivered a median latency of 150 milliseconds across benchmarks designed to gauge decision accuracy, including RewardBench and JudgeBench. OpenAI's most capable model at the time, GPT-6 Astra, delivered a median latency of 1.89 seconds on the same tests.

How cheap is it?

A faster reply is usually a cheaper reply, because it burns less compute time, and TypeSafe AI's pricing reflects that. The startup charges $0.042 per million input tokens, with output tokens free. ChatGPT charges $10 per million input tokens and $50 per million output tokens for GPT-6 Astra. Even GPT-6 Luna, OpenAI's fast, inexpensive inference variant, costs $0.10 per million input tokens and $0.50 per million output tokens.

That gap opens tasks LLMs currently can't handle, says Matt Mastracci, an independent developer who has trained and released several decision models, including Djev, which Cloudflare cited as an inspiration for its own decision models. "You can think of content moderation, such as moderating a fire hose of social media posts, or an extremely busy chat room," Mastracci said.

Michelle Chen, a group product manager at Cloudflare — which released its own decision models, Clef and Claric — puts a number on the opportunity: "You can probably offload 80 percent of your typical LLM tasks to a System One model that is going to be so much faster and so much cheaper."

How do decision models work under the hood?

TypeSafe AI has offered few details on how Jev achieves its speed, but its open-weights imitators are more transparent.

Cloudflare's Clef, free to download on Hugging Face, illustrates the architecture. "With an LLM, when you're generating an output token, it looks at all the previous tokens, and predicts the next output token," Chen explains. That happens in sequence, token after token, until an end token appears. "Our decision models don't do that. They just do a single forward pass, to generate one read of all the inputs, and then generate one output. That's why it's a lot faster."

Decision models based on an LLM, such as Clef (built on Alibaba's Qwen LLM), keep the neurons and transformer architecture. They still take tokens in and push tokens out. What changes is the output layer: the parts of the model that turn tokens into text are constrained.

"Jev can only answer questions with a predefined, structured answer format," says Zhongyu Wei, a professor at Fudan University and co-author of a study that benchmarks decision models and introduces an open-weight alternative, SII-Decision. "We give it a question and options such as A, B and C, and it must choose from those options, returning probabilities alongside its choice."

In one sense, this is old wine. Decision models are a return to classifiers, the workhorse models behind spam filters and face detection. "These kinds of models are not new," Chen says. "It's almost like a rebranding."

The upgrade over prior classifiers is generalization. Classifiers traditionally required domain-specific training. Decision models retain an LLM's general intelligence, which makes them applicable across many situations. "You can pick it up off the shelf, you point it at your problem, and it works," Mastracci says.

Cloudflare is already putting that to work. Before decision models, teams wanting a classifier had to "get a lot of data, stand up a machine learning team, and fine-tune" a model, Chen says. Now one decision model covers many domains. Cloudflare's deployments include:

  • Flagging spam pull requests in its GitHub documentation repository
  • Catching spam and phishing emails
  • Spotting phishing domains
  • Detecting prompt-injection attacks on LLMs
  • Reading internet backbone outage emails and judging whether they affect Cloudflare's services

Do decision models really stop hallucinations?

The name TypeSafe AI nods to type-safe programming languages, which reject operations that mix incompatible data types — adding a number to a string, say — before the program runs. The company builds on that analogy to claim Jev will never hallucinate.

Strictly speaking, that's true. LLMs hallucinate because they calculate token probabilities repeatedly across a huge vocabulary and output many tokens, leaving room for less probable tokens to lead a response astray. Decision models only calculate probabilities among pre-determined options and pick the most probable one. Hallucination, by that mechanism, is eliminated.

But "can't hallucinate" is not the same as "decides correctly." Krishnan cautions that "eliminating hallucinations in the output does not eliminate errors in decision-making." He and his co-authors tested Jev on judging whether answers to questions were correct. Jev performed well when the judgment could be reached by reading the prompt, but performance dropped when the task required math or involved a logic puzzle.

Wei and his co-authors report similar results. The best decision models slightly beat LLM accuracy on document understanding questions and score close to LLMs on tool-use questions, such as choosing the correct software tool to call. But decision models fall behind in domains requiring topic expertise: medicine, finance, and law.

"I think reasoning is a key factor," Wei says. Reasoning lets an LLM work through a response before answering. But reasoning alone doesn't explain every gap — some LLMs performed well without it — so Wei points to training methods as another factor, and used decision-specific training for SII-Decision.

What comes next: hybrids, not replacements

Decision models and LLMs don't have to compete. Krishnan and his colleagues tested passing Jev's low-confidence answers to GPT-6 for additional consideration. The combination slightly beat the accuracy of GPT-6 alone, and did so at 41 percent of the cost.

Wei expects this "hybrid architecture" to become the norm. "The decision model can call an LLM to give a more detailed explanation. And an LLM can call a decision model for a quick judgment. I think that's very flexible."

For now, the imitators keep coming — dozens of models in six or seven weeks, by Mastracci's count — and the benchmark race between decision models and LLMs will hinge on whether the newcomers can close their gaps in math, logic, and expert domains without giving up the speed that made them interesting in the first place.

Original: typesafe.ai

Share this article:

More from Elena Vasquez

Elena Vasquez

Show full bio

Market editor covering technology and product news in artificial intelligence and machine learning at AI In Context.

708 articles

Related articles

  1. Decision AI Models Emerge as New Category: Jev, GLiDE and Open Weights
  2. Jev reaches 35% of Vercel's paid teams within two weeks of launch
  3. OpenAI's Decisions API chases TypeSafe's Jev in fast agent control
  4. TypeSafe AI Hits $7.5B Valuation Weeks After Jev's Release
  5. Amazon open-sources Strands Decider 2B as decision models multiply

« Previous articleNext article »