TLDR
- Jev is a System One decision model by TypeSafe. It returns typed answers with calibrated probabilities instead of text.
- In our tests, Jev answers in roughly 100 to 200 milliseconds, against 2 to 10 seconds for the large language models compared.
- Thunders already makes repeat runs deterministic by caching selectors. Jev covers only the steps the cache cannot answer: first runs, changed pages, invalidated steps.
- Jev's limits today: text only, a 32k state budget, no published EU region, no third-party benchmark, and it cannot count, calculate or compare dates.
- European customers keep Thunders' EU-resident inference path. Providers without an EU region are gated per customer.
Jev is a System One decision model released by TypeSafe in September 2026, and it does not write text. It reads a state, answers typed questions about it, and returns a choice or a probability.
At Thunders, we tested it, and adopted it where it made sense.
In our tests, Jev answers in roughly 100 to 200 milliseconds, against 2 to 10 seconds for the large language models in the same comparison.
For AI Test Automation, where every test step waits on a model, that is a change in the shape of the problem, not a tuning win.
This article covers what a System One model is, how deterministic your runs are on each path, what a faster model does and does not fix if you build your own, and where Jev's limits sit today for Web, Mobile, API, Desktop and SAP testing.
What is a System One model, and how is it different from an LLM?
A System One model is a decision model that returns a typed answer and a calibrated probability instead of text. You give it a state and a question with a fixed answer shape. It returns the answer and its confidence. It never writes a sentence, so nothing has to be parsed, retried or repaired.
A large language model does the opposite job. It generates tokens one at a time, so the length of the answer sets the length of the wait. That is right when you need a plan, some code, or a test written in natural language. It is expensive when all you needed was yes or no.
TypeSafe trained Jev with RLCD, reinforcement learning for calibrated decisions, rather than RLHF. Ask the same question twice and TypeSafe reports near-identical probabilities, where the models it compared disagreed with themselves even at temperature 0. TypeSafe also states Jev holds SOC 2 Type II and does not train on customer data.
The limits decide where it is useful. Jev is text only, takes no image input, and has a 64k token context with a 32k state budget. A screen only a vision model can read is not a System One case.
Why does "do it yourself" AI Test Automation spend so much time waiting on models?
Because a test run is made of many small decisions, and each one costs a full model round trip. Which action is this step. Which element does it mean. Did the assertion pass. None of those needs prose. All of them have been paying prose prices.
Our own measurements say how much.
When you do it yourself, using a naive approach, a simple step classification averages a little over 3 seconds. Element selection averages around 4 seconds and passes 10 at the 95th percentile. Assertion evaluation averages over 3 seconds. A full selector resolution through the language and vision stages costs around 8 seconds per attempt.
Multiply any of those by the steps in a real suite and you have the reason AI Test Automation feels slow to teams promised the opposite.
How does Thunders already make repeat runs deterministic and fast?
Simple: by not calling a model on most steps.
This is not new, and it is not something a System One model made possible. A Thunders test resolves an element with AI on the first run, then caches a deterministic selector. Later runs replay it directly. No model call, no token cost, no variance.
On a stable suite, repeat runs are deterministic and one to two orders of magnitude cheaper than sending every step to a large language model.
So Jev is not a rescue. It is a new tool that fits an architecture we already built, and it answers what the cache legitimately cannot: the first run, a changed page (5% of runs on average), a step the cache correctly invalidated.
That is why we love Jev. It is the next step in the overall efficiency game.
How deterministic are your runs, really?
It depends on which path answers the step, and the spread between the three is wider than the speed gap.
| Path | Variance across identical runs | What that means for your suite |
|---|---|---|
| Thunders cached replay | Deterministic by design. The cached selector replays and no model is consulted | The same step resolves the same way every time, so a red run is a real regression, not a model changing its mind |
| System One decision (Jev) | Low variance. TypeSafe reports near-identical probabilities on repeated questions, and every answer carries a calibrated confidence | A borderline decision surfaces as a low score you can threshold on, instead of a confident wrong answer |
| Large language model call per step | High variance. TypeSafe reports the models it compared disagreed with themselves even at temperature 0 | The same suite can pass and fail on the same build, and your team spends the week separating real failures from model noise |
That ordering matters more than the latency ordering, and most comparisons skip it. A regression suite exists to tell you whether today's build broke something yesterday's did not. A suite that answers differently on identical input cannot do that job, however fast it answers.
So between a quicker suite and a repeatable one, we take repeatable. That is why the cache stays the first stage and a System One model slots in behind it. The steadiest model call is still the one you never make.
Low variance is not no variance. Jev narrows the spread on decisions that genuinely need a model. It does not turn them into cached replay.
What are Jev's limitations today?
Text only, a 32k state budget, no EU region, rate limits TypeSafe says can move without notice, and no third-party benchmark we have seen. Which of those hurts depends on what you test.
United States only. TypeSafe publishes no EU region for Jev. For European customers that decides the question on its own, which is why Jev sits behind the per-customer gate described below.
A 32k state budget, against a DOM that is usually larger. TypeSafe documents 64k tokens per request, of which 32k covers the state plus the longest question. The 54 KB document in its batching demonstration fits comfortably. A raw DOM from a dense data grid, a long form or an SAP screen often does not. In our measurements, 25% of enterprise applications comfortably cross that 32k boundary in their DOM.
No visual input. TypeSafe states Jev accepts text only, with no image, audio or video. Canvas applications, streamed desktop sessions and any screen whose meaning lives in pixels stay with a vision model. A System One decision cannot replace that stage, only the stages in front of it.
No independent benchmark, and capacity still settling. Every speed and calibration figure published so far comes from TypeSafe or from its users, ourselves included. We have seen no third-party benchmark. TypeSafe's own documentation says its rate limits are adjusting dynamically because it is serving a very large volume of demand, and that they can change without notice. A regression fleet is bursty by nature, so for a testing platform that is a capacity question rather than a footnote.
It cannot count, calculate or compare dates. TypeSafe lists all three as known failure modes: Jev is not a calculator, it does not count reliably, and it reads dates as text rather than as ordered quantities. Test assertions do exactly these things, constantly. Rows in a grid, a total on an invoice, an expiry in the future. Those comparisons belong in code, with the model asked only the judgment part.
Three more that matter for a suite rather than a demo. TypeSafe says calibration is measured across groups of predictions and does not guarantee any individual answer is correct, so a confidence score tells you how to route, not whether this answer is right. It returns no explanation of its reasoning, so a wrong resolution gives you a number and no why. And the jev-latest alias moves when a new release ships, so answers can change without a change on your side. Pin the version if you have tuned thresholds against it. For a regression suite, an unpinned model is a silent variable, which is precisely the thing a regression suite exists to eliminate.
Will Jev solve every concern about building your own test automation?
No. A faster decision model removes one line from the cost of building your own AI Test Automation and leaves the heavier lines untouched.
Auditable self-healing. Any model can recompute a broken selector. Your release manager asks a different question: what changed, why, and who signed it off. Thunders records the selector before and after, the stage that resolved it, the screenshot and the step recap, so a heal is reviewable and reversible. A faster model repairs quicker. It does not produce the evidence, and self-healing without evidence is a suite that quietly edits itself between releases.
Infrastructure. Runs need browsers, real devices and emulators, desktop sessions, parallel capacity, retries, artifact storage, and a queue that survives the whole team pushing at once. None of that gets cheaper because a decision takes 150 milliseconds instead of 3 seconds.
Collaboration. A suite is a shared asset, not a script. Test cases, test sets and test plans, permissions, review of generated steps, searchable run history, links into Jira, Linear or Xray, CI triggers, dashboards a product manager can read. This is product work, which is why an in-house harness never quite finishes it.
Evaluation and routing. Each decision wants a different model, and the right answer changes every few months. More on that below.
Security and residency. Where inference runs, and which providers are allowed per customer, are architectural constraints rather than settings you add later.
Jev takes a real bite out of one of those five lines. It does not touch the other four.
What is Thunders integrating, and what is the GDPR boundary for European customers?
We are integrating System One decisions into the product, and gating the provider per customer until data residency exists. Both halves are true at once, and you should read them together.
Thunders runs its own AI inference inside the EU. Any provider without an EU region is gated per customer, and European customers keep the EU-resident path. That is a deliberate choice about where your test data sits, and it holds regardless of how fast a model outside the boundary happens to be.
Thunders holds ISO 27001, SOC 2 and GDPR commitments, and residency is part of that set rather than a setting. When an EU region exists for a provider, the gate opens. Until then it stays shut, and the speed work continues on models we can already run inside the boundary.
Buy or build: why should a testing vendor own the model choice?
Because the work is continuous, and it is not the work your team is paid for. One integration against one model is a week. Keeping a testing platform correct across models is permanent.
You route each decision to a different model, because the best model for element selection is not the best for assertion evaluation. You keep a golden set per decision. You run evaluations on every prompt and model change, because a prompt that improves one decision routinely degrades another. Then a better provider appears and you re-run all of it.
Thunders has migrated models across its pipeline more than once on the strength of those evaluations, each one justified by golden-set numbers rather than a launch announcement. A team building AI Test Automation in-house pays that tax forever, on top of the product they actually sell.
So a customer buying Thunders is buying the swapping, not one model. The model underneath will change. The evaluation harness that decides when is the durable part.
Vendors adapt, and that is the product
The best model for a given decision changes every few months. Any architecture that assumes otherwise is already dated.
A platform whose value is a provider abstraction and an evaluation harness gets faster and cheaper without you doing anything. You do not rewrite a test or re-record a flow. The routing changes underneath your existing suite, and your Web, Mobile, API, Desktop and SAP tests run quicker on the steps you already wrote.
That is the outcome to hold vendors to. Not which model they use today. Whether a better model next quarter reaches you as a speed improvement or as a migration project.
Where Thunders fits in
Thunders is a no-code AI Test Automation platform that covers Web, Mobile, API and Desktop applications, including SAP. You describe a journey in natural language, and Thunders generates the test, runs it, and keeps it working as your interface changes.
The architecture in this article is why that works at scale:
- First run uses AI, later runs replay a cached deterministic selector, so a stable suite is deterministic and one to two orders of magnitude cheaper than calling a large language model per step.
- The runner tries deterministic, then a language model, then a vision model, so you pay for the stage each step needs.
- Every heal is recorded and reviewable, so a self-healing suite still leaves an audit trail.
- Model choice is ours to change, not yours, which makes a new model class like Jev an integration for us rather than a project for you.
Thunders covers SAP web interfaces today, and is extending to native SAP GUI.
Create a free Thunders account to see how your suite behaves under this architecture, or book a demo to walk through the surfaces we cover, the EU-resident inference path, and model routing on your tests.
Sources
- TypeSafe, System One: what a System One model is, and the typed answer shapes it returns.
- TypeSafe, Models: context limits, rate limits and pricing for Jev.
- TypeSafe, Jev 1.13 jaggedness: the failure modes TypeSafe documents itself.
