Can We Make Software Smarter? First Look at Jev.

We can ask a coding agent to build a feature, work through a bug, or explain an unfamiliar codebase. That changes how we work. But there's another question worth asking: what can the software do once we've finished writing it?

Can it recognize which request needs attention? Choose a useful next step? Handle an ordinary case without someone checking every decision?

That distinction stood out to me in Diogo Almeida's talk about what comes after RLHF. He describes an ambition that goes beyond faster development:

“We used to think that software would get a lot smarter, not just cheaper to write.” - Diogo Almeida

Almeida helped develop ChatGPT and later founded TypeSafe. His vision is AI built for direct use by software, not humans. We define the questions and allowed answers in code (hence the name TypeSafe.ai), and the model returns structured decisions our code can act on. The interface is designed for machines and does not require a conversational exchange with a person.

"Jev is a new class of models where code is meant to be the consumer" - Diogo Almeida

Assistance and automation ask different things of us

An assistant can be useful while we're still checking its work. We can clarify a request, reject a suggestion, or catch a mistake before it reaches someone else.

When a workflow runs in the background, those opportunities change. We have to decide in advance what the system may do, how we'll detect mistakes, and what happens when it can't make a reliable decision.

Almeida uses this distinction to explain the gap between impressive AI demonstrations and dependable automation. His broader claim that human-preference training explains that gap is an argument, not a complete account of every model or deployment. Reliability also depends on the task, available information, tests, permissions, and surrounding software.

For us as engineers, the useful question is concrete: which decision are we asking the model to make, and what would a wrong answer cost?

Start with the right task

One of Almeida's slides puts “doing the right task” at the base of a pyramid, underneath data, compute, and algorithms. That's the part that most closely matches how I approach software.

Before choosing a model, we need to understand the problem. Who needs help? What happens today? Which improvement would be worth the effort?

In the previous post on software factories, we looked at choosing and shaping work before handing it to agents. The same thinking applies inside the software we build.

“Automate customer support” leaves too many decisions unresolved. “Route incoming messages to billing, technical support, or a review queue” gives us something we can test.

We can gather representative messages, agree on the right destinations, and compare the model with the current process. We can also set a budget for the experiment, including the work of labeling examples, reviewing mistakes, and operating the result.

Where Jev fits

Jev returns structured decisions rather than generated prose. Its primitives let us choose among options, score against ordered levels, or estimate whether a statement is true. The application supplies the context and questions, then uses the answers in code.

The difference starts with what the model is trained to do. TypeSafe calls its approach reinforcement learning for calibrated decisions, or RLCD. The aim is to return useful decisions with probabilities that reflect how often those predictions hold up.

We can still supply ordinary text, such as a support message, or organize that context in JSON. The questions can use natural-language instructions too. The input does not have to be source code. The machine-oriented design is in the typed questions and answers, and in how our application uses the result.

For a support message, we might ask which team should handle it, whether it describes an outage, and whether it requests a refund. The application can combine those answers with ordinary rules.

Recognizing a refund request doesn't authorize a refund. Account permissions, eligibility, and payment execution still belong in the application.

Jev could replace an LLM call whose entire job is classification or scoring. It doesn't replace the model that writes a thoughtful response, investigates an unfamiliar problem, or generates code. LangChain's integration guide illustrates that division with model routing and checks on agent tool calls.

We can use different tools for different parts of the same workflow. A decision model routes the message. An LLM drafts a reply. Code enforces permissions. A person handles an exception.

Laya gives us another way to try it

A second video about Jev and Laya makes the software-building angle easier to picture. These small decisions become parts we can combine into a larger application.

Laya adds an open-source option. We can inspect its implementation, run it ourselves, and adapt it to a particular task. That makes it interesting when we want control over deployment and have examples of the decisions we need it to make.

Its own model card offers a useful qualification:

“Laya is a fast base to specialise, not a zero-shot decision engine.”

The stronger typed-decision results come from a checkpoint fine-tuned on that benchmark's training split. We should expect to evaluate which checkpoint fits our work and whether further adaptation is needed.

The headline comparisons also need care. The repository says its authors used published Jev results rather than testing Jev themselves, with differences in prompts and sample sizes. Those numbers don't establish that Laya will beat Jev in our application.

Running a model ourselves also brings costs: hardware, deployment, monitoring, and maintenance. An open-source license gives us options. It doesn't make the whole system free to operate.

We don't have to automate every case

The accuracy-versus-coverage chart in the second video's screenshots points to a practical approach: let the system handle a subset of cases and send the rest for review.

The question becomes how much work we can handle at an acceptable error rate. We also need to count the effort left in the review queue. Automating easy cases may leave people with a smaller but harder workload.

Calibration gives those probabilities a practical meaning. Across many predictions assigned an 80% probability, the corresponding outcome should occur about 80% of the time. TypeSafe's primer describes this as a property of groups of predictions, not a guarantee about an individual answer. We still need to check how well it holds on our own data.

For Choice and Score answers, Jev also returns a confidence field that summarizes how concentrated the probability distribution is. Noul returns the probability of a yes answer without a separate confidence field. The confidence statistic is not automatically the chance that a decision is correct on our data. TypeSafe recommends checking thresholds against the actual use case.

We should check confident mistakes as carefully as uncertain answers. A model that confidently chooses the wrong destination won't be caught by a rule that only escalates low-confidence cases.

Is it ready for production?

My assessment is that these models are worth bounded trials. We haven't tested either one ourselves beyond POCs, and the sources discussed here don't establish reliability for our workload.

For an inbox-routing pilot, we'd start by recording predictions alongside the existing process. We'd compare them with reviewed decisions, look at missed urgent messages and incorrect routes, and measure response time and total cost. Only then would we let a well-understood category run automatically, with a fallback when the service fails or the input doesn't fit.

That's how we can turn an interesting model into useful software. We choose a decision that matters, make its boundaries clear, and gather evidence that it works.

Writing software faster gives us more opportunities to try. Making it smarter means doing the work to find out which decisions it can actually carry for us.