There is a new AI in town, and unlike everything that has dominated the last few years, it doesn't hold conversations, doesn't write prose and has no opinions. It just decides.
I recently got early access to TypeSafe AI's Jev and have been testing it in our own production-critical pipelines. Instead of sentences, Jev answers with typed choices and probabilities: yes/no, a score, a pick from a fixed list. No parsing, no hallucinated paragraphs, no personality.
That sounds like a downgrade until you realise how much of the daily infrastructure work was never a conversation in the first place. "Should this Ansible change be rolled out automatically, or should a human look at it?" doesn't need an essay back. It needs a fast, calibrated verdict.
The chatbot pattern is the wrong tool for most pipeline decisions
Most of us have already solved this task, just with the wrong tool: you ask an LLM to "answer with JSON in this format", parse the result and hope it doesn't wrap the answer in an explanation or a markdown block you didn't ask for.
It usually works. But it is a conversational model pressed into a role it was never built for: a function, not a dialogue partner. You generate words you are not going to read, only to pull three bits of information out of them: safe, doubtful, dangerous.
In our plan/apply workflow, that is exactly what happens in the step where an AI instance reviews a proposed Ansible or Terraform diff and judges whether it should be passed on to a human. It is a classification dressed up as a conversation.
What a "System One" model is
Jev calls itself a "System One" model, a reference to Kahneman'sDaniel Kahneman is an Israeli-American psychologist and Nobel laureate in economics (2002), awarded for his work on behavioural economics together with Amos Tversky. His best-known book is Thinking, Fast and Slow (2011), which describes two "systems" of human thinking:System 1: fast, automatic, intuitive, emotional. It is what recognises a face in a split second, or instantly "knows" that 2+2=4.System 2: slow, deliberate, analytical. It is what works out a complex maths problem, or weighs a difficult choice against another. fast, intuitive thinking as opposed to the slow, reasoning kind. You don't send a prompt and wait for text. You send a state (in our case, a diff) and a set of typed questions up front, and you get typed answers back, with probabilities on each option.
The difference from asking GPT or Claude to return JSON is structural, not stylistic. The answer is constrained to the options you defined yourself. There is no JSON that can fail parsing and no risk of the model "forgetting" the format halfway through an answer. And because the model evaluates all questions in parallel instead of generating text token by token, it is markedly faster and cheaper for the kind of narrow, repetitive decisions a pipeline makes thousands of times a day.
It is a small component you drop into one place in an otherwise ordinary codebase, where the code needs a single verdict.
Here is what we actually send, and what we actually get back, for the decision I use in our deploy gate. The input is split into two fields, as in TypeSafe's own playground:
State (the diff only):
{
"state": "--- before: logging.yml\n+++ after: logging.yml\n@@ -4,7 +4,7 @@\n handlers:\n - type: rotating_file\n- retention_days: 14\n+ retention_days: 30"
}Questions (defined up front, independent of the state):
{
"risk_level": {
"type": "choice",
"instructions": "How risky is this change to roll out to production?",
"criteria": {
"safe": "Additive or low-risk change",
"needs_review": "Plausible, but with a real blast radius",
"risky": "Could cause downtime or a security hole"
}
},
"touches_security_boundary": {
"type": "noul",
"instructions": "Does the diff change a firewall rule or a security group?"
}
}Output (the actual response from our run in the playground):
{
"model": "jev-1.13.0",
"answers": {
"risk_level": {
"type": "choice",
"choice": "safe",
"confidence": 0.66,
"probabilities": { "risky": 0, "needs_review": 0.22, "safe": 0.78 },
"stats": {}
},
"touches_security_boundary": { "type": "noul", "noul": 0.01, "stats": {} }
},
"usage": { "input_tokens": 441, "output_tokens": 62 },
"request_id": "playground_1e9560b8a94c95743b684bfe47c032eb660",
"evaluation_time_ms": 96.33424999992712
}answers.risk_level.choice is either "safe", "needs_review" or "risky", nothing else, because those are exactly the three keys we defined ourselves under criteria in the Questions block above. Jev doesn't invent the options. It chooses between the ones we already fixed, so the code can branch directly on the value without handling a fourth, unexpected answer. Compare that with an ordinary LLM, where you have to write your own parsing code to rescue the situation when the model wraps the answer in an explanation anyway.
One clarification: this JSON format is TypeSafe's own, a contract for their API, not an open standard. It sits at a different level from MCP, which I have written about before, where the whole point is vendor independence. Jev's format solves a different problem (typed decisions instead of text), and the two don't compete; a Jev assessment could easily sit behind an MCP tool call as the decision logic itself.
How I rebuilt our Ansible deploy gate around Jev
I took our existing AI review step and built a parallel version around Jev, to see whether the same step could be done faster and cheaper without compromising the quality of the verdict.
The setup: each diff is sent as the state together with two questions, a choice between "safe", "needs review" and "risky", and a yes/no on whether the change touches a firewall rule or a security group. Jev answers with a choice and a probability for each option, not just a label.
I ran the same classification through both paths, Jev and our current LLM-based gate, on a set of example diffs, and measured response time and price. The difference in speed is marked, in the order of magnitude TypeSafe advertises themselves. The interesting part is not the number itself, but that a narrow decision now costs a fraction of what it did when you no longer pay for prose you discard anyway.
Worth being clear about: TypeSafe itself reports 70-500 ms end-to-end and large speed and price advantages in their own workflow evaluations, numbers that by their own account probably sit at the optimistic end of what you can expect in practice. Those are the vendor's own numbers, so I don't use them as the answer. Our own baseline is a direct JSON classification with no reasoning, and that is the comparison I am actually interested in.
"70 ms versus several seconds" is not apples to apples if the LLM baseline does full chain-of-thought along the way. Many demos compare Jev against a model that reasons its way through several steps. Our own baseline doesn't: it is asked to answer directly with JSON, same task, same format, no reasoning along the way. So the difference we measure is a fair comparison rather than marketing.
Why I run it in shadow mode first
I have not given Jev the keys to anything yet.
The model has been publicly available for barely a week. Giving a model with no track record access to a production apply button is how you end up in an incident report.
Instead, Jev runs in shadow mode: it evaluates every diff in parallel with our existing gate, but its answers are logged, not acted on. Over the coming weeks I will compare its assessments with what actually happened (was the plan approved, was it rejected, did something go wrong) before I consider letting it influence the decision itself. It is the same discipline the rest of our workflow is already built on: a new component never moves straight to production, but through a step where someone can see what it would have done before it is allowed to actually do it.
Where I still insist on a human, whatever the confidence number says
Even when Jev one day moves from shadow mode to having real influence, it doesn't change the rule: the AI never proposes direct action. It proposes an assessment of a diff, and that assessment is still read by a human before anything hits production, whether the assessment comes from a conversational model that spent several sentences thinking it over, or a model that answered with 85% probability in 100 milliseconds.
High confidence is not the same as full context. Jev can probably judge quickly and accurately whether a diff looks like something that is usually safe. It has no feel for why things look the way they do at our place specifically, or which hidden dependencies exist between our systems. A fast, calibrated model makes the decision cheaper to ask. It doesn't make it the last word.
TypeSafe advertises "zero hallucination", and that is true in a narrow sense: Jev doesn't invent an answer outside the options you defined. But a calibrated model is not the same as an infallible one. If a model is calibrated on a given population so that its 95% level corresponds to roughly 95% correct verdicts, about 5% will still be wrong, and the model doesn't raise its hand on those, because it was, after all, 95% sure by its own calibration. A confident, wrong answer is fully compatible with good calibration. That applies to any model that returns a probability, and it is exactly why the confidence number should never stand alone as a decision. It is a signal, not a guarantee.
It is interesting that the Danish Agency for Digital Government's current bill on public authorities' use of AI defines "decision support" as AI-generated proposals or recommendations that a natural person includes in the overall professional assessment, without the AI system itself making the decision or initiating the measure. The bill also gives no basis for fully automated decisions under GDPR Article 22. That is, legally speaking, exactly the same distinction I am trying to hold on to here: a model, however well calibrated, delivers a proposal for a decision, it doesn't make it itself.
In our concrete example, Jev gives "safe" a probability of 78%, while the answer has a confidence value of 66%. The two numbers are not the same thing. TypeSafe describes confidence as a calibrated measure, where higher confidence corresponds to higher accuracy, but it is not necessarily identical to "there is a 66% chance the answer is correct". I won't use either number as a direct guarantee of correctness, only as a signal for when a human needs to step in and when it isn't necessary to spend their time. A model like Jev thus doesn't decide whether the human is removed from the decision, but when the human actually has to be involved and when it is just noise.
A serious attempt at AI that talks to software
The chatbot era made AI good at talking to people. This is something else: a serious attempt at a model that mainly talks to software, not to us. Infrastructure automation may be exactly where it pays off first, not because the decisions are more important than the ones a chatbot makes, but because there are so damn many of them, and each one is so small that it never made sense to pay for an essay to get a yes or no.
I will update when the shadow mode data starts to say something. Until then: if you are sitting with a pipeline full of narrow, repetitive decisions dressed up as chat calls, it may be worth asking whether a conversation was ever what you needed.